Linux “Awk” Command: What It Is and How to Use It

In the landscape of programming languages dedicated to text-based data processing, “AWK” holds a prominent position due to its power and versatility. Originally designed for data manipulation and report generation, AWK has proven over time that it can extend its functionalities well beyond its original purpose, especially when combined with other Linux commands.

In this article, we aim to explore in detail the features and functionalities of AWK. We will start with an overview of the installation process of AWK, which is pre-installed on many Unix-based operating systems, such as Linux and MacOS. Then, we will delve into the basic syntax of AWK, illustrating the fundamental structure of AWK commands and the rules governing the use of patterns and actions.

We will continue with a series of practical examples of using AWK, demonstrating how this language can be used to manipulate data and generate reports. These examples will cover a wide range of scenarios, from pattern-based text filtering to tabular data processing.

Furthermore, we will explore the use of AWK in writing more complex scripts, highlighting how the power and flexibility of AWK can be leveraged to automate and simplify tasks of significant data processing complexity.

Finally, we will discuss how AWK can be combined with other Linux commands to perform advanced tasks, such as process termination based on name or data stream processing in pipelines. This section will also include a series of additional examples illustrating the wide range of AWK applications, from splitting a file into multiple files to converting uppercase characters to lowercase.

The goal of this article is to provide an overview of AWK, illustrating its capabilities through practical examples and providing the foundation for an effective and informed use of this powerful tool. Whether you are IT professionals looking for new skills or researchers in need of an effective means for data processing, we hope this article can serve as an introduction to your learning journey with AWK.

AWK Installation

AWK is pre-installed on many Unix-based operating systems, such as Linux and MacOS. You can check if AWK is installed on your system by typing `awk –version` in the terminal.

Basic Syntax of AWK

The basic syntax of an AWK command is as follows:

awk '/pattern/ {azione}' file

– `pattern` is a regular expression. AWK will perform the action on every line of the file that matches the pattern.
– `action` is an action that AWK will perform on every line that matches the pattern. If no action is specified, the default action is to print the line.
– `file` is the input file on which AWK will execute the command.

Example of Using AWK

Suppose we have a file called `students.txt` with the following content:

John Doe 23 Mathematics
Jane Doe 22 Physics

If we wanted to print only the lines that contain “Doe”, we could use the following AWK command:

awk '/Doe/ {print $0}' students.txt

The result would be:

John Doe 23 Mathematics
Jane Doe 22 Physics

If we wanted to print only the name and age of the students, we could use, for example:

awk '{print $1, $2}' students.txt

This command would print:

John 23
Jane 22

AWK Scripts

Beyond its use from the command line, AWK can also be used to write more complex scripts. An example of an AWK script is the following:

BEGIN {
print "Start of the script"
}

/Doe/ {
print $1, $2
}

END {
print "End of the script"
}

This script would print the name and age of each student with the surname “Doe,” in addition to printing a message at the beginning and end of the script.

It’s also possible to use awk in a pipeline with other shell commands. For example, if you wanted to kill all processes with the same name, such as “myprocess,” you could use a command that combines `ps`, `awk`, and `kill`:

ps aux | grep myprocess | awk '{print $2}' | xargs kill -9

In this command, `ps aux` lists all the processes running on your system, while `grep myprocess` filters the output to only include lines that contain “myprocess”. `awk ‘{print $2}’` extracts the second field of each line, which is the process ID (PID). Finally, `xargs kill -9` passes these PIDs to the `kill` command to terminate the processes.

The `-9` option forces the process to terminate and can cause unsaved data loss.

Example 2: How to terminate all processes of a specific user

Suppose now you want to terminate all processes of the user “myuser”. You can do something very similar to the previous example:

ps aux | grep myuser | awk '{print $2}' | xargs kill -9

Beyond managing processes, Awk can be used in a number of other ways. Here are some additional examples:

1. Counting lines in a file

awk 'END { print NR }' filename

This command counts the number of lines in the specified file and prints the result.

2. Printing lines from a file that match a pattern

awk '/pattern/ {print}' filename

This command prints only the lines that contain the specified pattern.

3. Splitting a file into multiple files

awk '{print > ("file" NR ".txt")}' filename

This command splits the input file into multiple files, with one line per file.

4. Sum of values in a column

awk '{sum+=$1} END {print sum}' filename

This command calculates the sum of the values in the first column of a file.

5. Converting uppercase characters to lowercase

echo "HELLO WORLD" | awk '{print tolower($0)}'

This command converts all uppercase letters to lowercase.

6. Printing the nth field of a line

echo "field1 field2 field3 field4" | awk '{print $3}'

This command prints the third field of a line.

In any case, it’s possible to delve deeper into all the capabilities of awk by using the reference manual: https://man7.org/linux/man-pages/man1/awk.1p.html