When you want to process a text file using the command line, awk, sed and cut are the most used programs. Here you will learn the basic use of each one.

Table of Contents

awk

This powerful tool can do simple and complex text processing tasks.

  • Print (show) all contents of a file.
    awk '{print}' text.txt
    #or
    awk '{print $0}' text.txt
    
  • You can pipe the output of another command to awk.
  • Print first column of a tab-separator file (or command).
    df -h | awk '{print $1}'
    
  • Print last column
    awk '{print $NF}' text.txt
    
  • Print several columns.
    df -h | awk '{print $1,$3}'
    
  • Export the output to a file.
    df -h | awk '{print $1,$3}' > output.txt
    
  • Using a diferent separator.
    awk -F ":" '{print $1}' /etc/passwd
    
  • Using a condition.
    awk -F ":" '/sshd/ {print $1,$7}' /etc/passwd
    
  • Print the total number of lines (records).
    awk -F ":" 'END {print NR}' /etc/passwd
    

sed

Use the ‘Streaming EDitor’ to transform a text.

  • Substitute “word1” for “word2”.
    sed 's/word1/word2/' text.txt
    

    This command will not change the file, only show the results. You can export the output to a file with > or use -i to edit the original file.

    sed 's/word1/word2/' text.txt > newtext.txt
    
    sed -i 's/word1/word2/' text.txt
    

    Also, it only changes the first occurence in each line. To change all ocurrences:

    sed 's/word1/word2/g' text.txt
    
  • Delete word1.
    sed 's/word1//g' text.txt
    
  • Replace “o” for “O” only on lines that match a pattern.
    sed '/root/s/o/O/g' /etc/passwd
    
  • Delete lines matching a pattern.
    sed '/root/d' /etc/passwd
    
  • You can use RegExp when looking for a pattern. This command will delete any empty line.
    sed '/^$/d' text.txt
    

cut

cut is a simpler version of awk. You can use it to separate a text in columns and show a specific column or several columns.

  • Print first column on a “:” delimiter file.
    cut -d ":" -f1 /etc/passwd
    
  • Print two columns.
    cut -d ":" -f1,7 /etc/passwd
    
  • By default, cut uses the delimiter as a separator in the output, but you can change it with --output-delimiter=DELIMITER.
    cut -d ":" -f1,7 --output-delimiter=" " /etc/passwd