How To Maximize On Your Usage Of Awk

Abbreviated from the names of its developers – Aho, Weinberger, and Kernighan – Awk is a  scripting language used in Unix or Linux environments for manipulating data and generating reports.

 

The awk command programming language requires no compiling, and allows the user to employ variables, numeric functions, string functions, and logical operators. In this article, we’ll be looking at some examples demonstrating its many and diverse uses.

Some Awk Basics

In essence, the awk command scans a file line by line, splits each input line into fields, compares input lines or fields to a specified pattern, then performs one or more actions on the matched lines.

 

Awk operations are especially useful in transforming data files, and producing formatted reports.

Selecting Specific Fields

The easiest and most common usage of awk is selecting specific fields from files, or from data that is fed into the awk command line. You use white space (any sequence of blanks and tabs) as a field separator by default, and a simple example at the Linux $ prompt would be:

 

$ echo one two three four five | awk ‘{print $4}’

four

$ who | awk ‘{print $1}’

jdoe

fhenry

 

In the example above, awk is extracting just the fourth and first fields from the data provided.

Using Awk To Pull Text From Files

By adding the name of the file after the awk command, you can use the mechanism to pull specific text from files. For example, to pick out the first, fifth and last words in the single line of text from the file “HelenKellerQuote”:

 

$ awk ‘{print $1,$5,$NF}’ HelenKellerQuote

 

The beautiful heart.

 

Specifying $NF in the command pulls out the last piece of text on each line. Here, NF represents the number of fields in the line (23) while $NF represents the value of that field (“heart.”). The period is included because it’s part of the final text string.

Using Awk To Print Text From Linux Files

The awk command may be used to print fields from a file in any specified order. In the example below, awk is rearranging the fields in a date command output:

 

$ date | awk ‘{print $4,$3,$2}’

 

2019 Nov 22

 

Note that if you leave out the commas between the field designators in an awk command, the output will be pushed into a single string.

 

$ date | awk ‘{print $4 $3 $2}’

 

2019Nov21

 

If you happen to replace the commas with hyphens, awk will attempt to subtract one field from another:

 

$ date | awk ‘{print $4-$3-$2}’

 

1997

 

Here, awk is subtracting 22 (the day of the month) from the year (2019) and simply ignoring “Nov”.

 

If you want the output to be separated by something other than white space, you can specify an output separator with the OFS (output field separator) modifier like this:

 

$ date | awk ‘{OFS=”-“; print $4,$3,$2}’

 

2019-Nov-22

 

As part of a script, you can use awk to simply display some relevant text. In this example, we’re using awk to display a line of text to label a set of data so that you can identify what you’re looking at:

 

$ who | awk ‘BEGIN {print “Current logins:”} {print $1}’

 

Current logins:

 

shs

 

nemo

Specifying Field Separators

If your text is separated by some character other than white space (commas, colons, semicolons, etc.), you can inform awk by using the -F (input separator) option as shown here:

 

$ cat testfile

 

a:b:c,d:e

 

$ awk -F : ‘{print $2,$3}’ testfile

 

b c,d

 

To specify a separator and pull a field from the colon-separated /etc/passwd file, you could use a syntax like this:

 

(Image source: NetworkWorld.com)

Evaluating Fields Using Awk

You can also use the awk command to take a deeper look into the contents of a data file. If for example, you want to list only the user accounts in /etc/passwd, you can include a test for the 3rd field. In the screenshot we’re only interested in any UIDs that are 1000 and above:

 

 

(Image source: NetworkWorld.com)

 

To add a title for the listing, you can add a BEGIN clause:

 

 

(Image source: NetworkWorld.com)

 

You can use the “\n” (newline characters) modifier to separate your intended output lines, if you want more than one line in your title:

 

 

(Image source: NetworkWorld.com)

Using Awk For Math Calculations

The awk command includes several mathematical capabilities, and is able to calculate square roots, logs, tangents, and other functions. So for example:

 

$ awk ‘BEGIN {print sqrt(2019)}’

 

44.9333

 

This returns the square root of 2019. The syntax below calculates the logarithm of 2019:

 

$ awk ‘BEGIN {print log(2019)}’

 

7.61036

 

Creating Scripts With Awk

It’s possible to use awk for creating standalone scripts. The example below builds on the user accounts case given earlier, but also counts the number of users with accounts on the system.

 

 

(Image source: NetworkWorld.com)

 

The BEGIN section is run only when the script starts, and provides a heading, dictates the field separator, and sets up a counter to start with 0. The script also includes an END section which only runs after all the lines in the text provided to the script have been processed. It displays the final count of lines which meet the specification in the middle section.

 

The awk command has a long tradition in Unix, and as we’ve demonstrated, it still provides a range of uses and functionality for Linux environments.