Searching and Filtering Text: grep, cut, sort, uniq

Answer real questions from logs and CSV files with grep, cut, sort, uniq, wc, tr, head, tail, paste and column, building up to a complete access-log report.

By The Dark DebuggerUpdated 31 min read

This is the first article in the text-processing part of the Bash path. It covers the small filters you will combine every day: grep in depth (including regular expressions), cut, sort, uniq, wc, tr, head, tail, paste and column. You should be comfortable with redirection and pipes; everything here is used inside pipelines. The next two articles cover the heavier tools, sed and awk.

Why these tools still matter

Picture a shop's checkout failing intermittently. Someone asks: is it one client, is it a bot, when did it start, which endpoint? You have SSH access to the web server and its access log, and no dashboard. Every one of those questions is a one-line pipeline of the tools in this article, and they run in seconds on logs with millions of lines, because each tool streams its input instead of loading it.

They are also everywhere: in Docker images, CI runners, recovery shells and every Linux server you will ever log into. Learning them once pays off for decades.

Sample data

Every example uses three files. Create them in an empty directory so you can follow along. First, an nginx access log in the standard "combined" format:

cat > access.log <<'EOF'
203.0.113.42 - - [09/Nov/2026:08:01:12 +0000] "GET / HTTP/1.1" 200 5120 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0"
203.0.113.42 - - [09/Nov/2026:08:01:13 +0000] "GET /css/main.css HTTP/1.1" 200 1834 "https://shop.example.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0"
10.0.0.5 - - [09/Nov/2026:08:02:00 +0000] "GET /healthz HTTP/1.1" 200 2 "-" "kube-probe/1.30"
198.51.100.7 - - [09/Nov/2026:08:02:40 +0000] "GET /products/42 HTTP/1.1" 200 7342 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/130.0.0.0 Safari/537.36"
198.51.100.7 - - [09/Nov/2026:08:02:41 +0000] "GET /img/products/42.jpg HTTP/1.1" 304 0 "https://shop.example.com/products/42" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/130.0.0.0 Safari/537.36"
192.0.2.15 - - [09/Nov/2026:08:05:02 +0000] "GET /wp-login.php HTTP/1.1" 404 162 "-" "python-requests/2.31.0"
192.0.2.15 - - [09/Nov/2026:08:05:02 +0000] "GET /.env HTTP/1.1" 404 162 "-" "python-requests/2.31.0"
192.0.2.15 - - [09/Nov/2026:08:05:03 +0000] "GET /.git/config HTTP/1.1" 404 162 "-" "python-requests/2.31.0"
192.0.2.15 - - [09/Nov/2026:08:05:03 +0000] "GET /admin/config.php HTTP/1.1" 404 162 "-" "python-requests/2.31.0"
192.0.2.15 - - [09/Nov/2026:08:05:04 +0000] "GET /phpmyadmin/ HTTP/1.1" 404 162 "-" "python-requests/2.31.0"
192.0.2.15 - - [09/Nov/2026:08:05:04 +0000] "GET /xmlrpc.php HTTP/1.1" 404 162 "-" "python-requests/2.31.0"
192.0.2.15 - - [09/Nov/2026:08:05:05 +0000] "GET /wp-admin/ HTTP/1.1" 404 162 "-" "python-requests/2.31.0"
192.0.2.15 - - [09/Nov/2026:08:05:05 +0000] "GET /wp-login.php HTTP/1.1" 404 162 "-" "python-requests/2.31.0"
203.0.113.42 - - [09/Nov/2026:08:07:55 +0000] "GET /products?page=2 HTTP/1.1" 200 15003 "https://shop.example.com/" "Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0"
10.0.0.5 - - [09/Nov/2026:08:12:00 +0000] "GET /healthz HTTP/1.1" 200 2 "-" "kube-probe/1.30"
198.51.100.23 - - [09/Nov/2026:08:15:21 +0000] "POST /api/orders HTTP/1.1" 201 412 "-" "curl/8.5.0"
198.51.100.23 - - [09/Nov/2026:08:15:58 +0000] "POST /api/orders HTTP/1.1" 500 89 "-" "curl/8.5.0"
203.0.113.9 - - [09/Nov/2026:08:20:13 +0000] "GET /robots.txt HTTP/1.1" 200 67 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
203.0.113.9 - - [09/Nov/2026:08:20:14 +0000] "GET /products/42 HTTP/1.1" 200 7342 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
10.0.0.5 - - [09/Nov/2026:08:22:00 +0000] "GET /healthz HTTP/1.1" 200 2 "-" "kube-probe/1.30"
203.0.113.42 - - [09/Nov/2026:09:03:37 +0000] "GET /cart HTTP/1.1" 200 3921 "https://shop.example.com/products?page=2" "Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0"
203.0.113.42 - - [09/Nov/2026:09:04:10 +0000] "POST /api/orders HTTP/1.1" 201 398 "https://shop.example.com/cart" "Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0"
198.51.100.23 - - [09/Nov/2026:09:10:44 +0000] "POST /api/orders HTTP/1.1" 502 157 "-" "curl/8.5.0"
198.51.100.23 - - [09/Nov/2026:09:11:02 +0000] "POST /api/orders HTTP/1.1" 201 405 "-" "curl/8.5.0"
198.51.100.7 - - [09/Nov/2026:09:12:09 +0000] "GET /account HTTP/1.1" 301 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/130.0.0.0 Safari/537.36"
198.51.100.7 - - [09/Nov/2026:09:12:10 +0000] "GET /account/ HTTP/1.1" 200 4410 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/130.0.0.0 Safari/537.36"
10.0.0.5 - - [09/Nov/2026:09:32:00 +0000] "GET /healthz HTTP/1.1" 200 2 "-" "kube-probe/1.30"
192.0.2.77 - - [09/Nov/2026:09:40:31 +0000] "GET /favicon.ico HTTP/1.1" 404 162 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 18_1 like Mac OS X) Mobile/15E148 Safari/604.1"
198.51.100.7 - - [09/Nov/2026:10:01:15 +0000] "GET /img/logo.png HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/130.0.0.0 Safari/537.36"
198.51.100.7 - - [09/Nov/2026:10:01:16 +0000] "GET /products/42 HTTP/1.1" 200 7342 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/130.0.0.0 Safari/537.36"
203.0.113.42 - - [09/Nov/2026:10:05:49 +0000] "GET /orders/1042 HTTP/1.1" 200 2876 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0"
198.51.100.23 - - [09/Nov/2026:10:06:30 +0000] "GET /api/orders/1042 HTTP/1.1" 200 1500 "-" "curl/8.5.0"
203.0.113.42 - - [09/Nov/2026:10:09:02 +0000] "GET /logout HTTP/1.1" 302 0 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0"
EOF

Split on spaces, each line has these fields. The numbers matter, because cut and awk address fields by position:

FieldExampleMeaning
1203.0.113.42client IP
4[09/Nov/2026:08:01:12timestamp (with the bracket)
6"GETmethod (with the quote)
7/products/42path
9200HTTP status
105120response size in bytes

Second, an application log with two Java stack traces:

cat > app.log <<'EOF'
2026-11-09 08:15:57 INFO  [http-8080-3] POST /api/orders received (order 1043)
2026-11-09 08:15:57 INFO  [http-8080-3] reserving stock for order 1043
2026-11-09 08:15:58 WARN  [http-8080-3] payment gateway slow: 2950 ms
2026-11-09 08:15:58 ERROR [http-8080-3] order 1043 failed
java.net.SocketTimeoutException: Read timed out
    at com.example.orders.PaymentClient.charge(PaymentClient.java:88)
    at com.example.orders.OrderService.place(OrderService.java:142)
2026-11-09 08:15:58 INFO  [http-8080-3] stock released for order 1043
2026-11-09 09:04:10 INFO  [http-8080-1] POST /api/orders received (order 1044)
2026-11-09 09:04:10 INFO  [http-8080-1] order 1044 placed
2026-11-09 09:10:44 ERROR [http-8080-2] order 1045 failed
java.sql.SQLTransientConnectionException: orders-pool - Connection is not available
    at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:696)
2026-11-09 09:11:02 INFO  [http-8080-2] order 1046 placed
EOF

Third, a CSV export of orders:

cat > orders.csv <<'EOF'
order_id,date,region,product,qty,unit_price
1001,2026-11-01,north,keyboard,2,49.99
1002,2026-11-01,south,monitor,1,189.00
1003,2026-11-02,west,mouse,5,19.50
1004,2026-11-02,north,monitor,2,189.00
1005,2026-11-03,east,headset,1,79.00
1006,2026-11-03,south,keyboard,3,49.99
1007,2026-11-04,west,monitor,1,189.00
1008,2026-11-04,north,mouse,10,19.50
1009,2026-11-05,east,keyboard,1,49.99
1010,2026-11-05,south,headset,2,79.00
1011,2026-11-06,north,headset,1,79.00
1012,2026-11-06,west,keyboard,4,49.99
EOF

grep: find the lines you care about

grep PATTERN FILE... prints every line that contains a match for the pattern:

grep 'POST' access.log
198.51.100.23 - - [09/Nov/2026:08:15:21 +0000] "POST /api/orders HTTP/1.1" 201 412 "-" "curl/8.5.0"
198.51.100.23 - - [09/Nov/2026:08:15:58 +0000] "POST /api/orders HTTP/1.1" 500 89 "-" "curl/8.5.0"
203.0.113.42 - - [09/Nov/2026:09:04:10 +0000] "POST /api/orders HTTP/1.1" 201 398 "https://shop.example.com/cart" "Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0"
198.51.100.23 - - [09/Nov/2026:09:10:44 +0000] "POST /api/orders HTTP/1.1" 502 157 "-" "curl/8.5.0"
198.51.100.23 - - [09/Nov/2026:09:11:02 +0000] "POST /api/orders HTTP/1.1" 201 405 "-" "curl/8.5.0"

Always put the pattern in single quotes. Patterns are full of characters the shell would otherwise interpret (*, $, [, |, spaces), and single quotes pass them to grep untouched.

The options you'll use every day

Count matching lines with -c, show line numbers with -n, and invert the match (lines that don't match) with -v:

grep -c 'POST' access.log
grep -n 'ERROR' app.log
5
4:2026-11-09 08:15:58 ERROR [http-8080-3] order 1043 failed
11:2026-11-09 09:10:44 ERROR [http-8080-2] order 1045 failed
grep -v 'INFO' app.log
2026-11-09 08:15:58 WARN  [http-8080-3] payment gateway slow: 2950 ms
2026-11-09 08:15:58 ERROR [http-8080-3] order 1043 failed
java.net.SocketTimeoutException: Read timed out
    at com.example.orders.PaymentClient.charge(PaymentClient.java:88)
    at com.example.orders.OrderService.place(OrderService.java:142)
2026-11-09 09:10:44 ERROR [http-8080-2] order 1045 failed
java.sql.SQLTransientConnectionException: orders-pool - Connection is not available
    at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:696)

-i ignores case. Here the user agent says Googlebot but we typed it in lowercase (the cut at the end just keeps the IP and path; it is explained below):

grep -i 'googlebot' access.log | cut -d' ' -f1,7
203.0.113.9 /robots.txt
203.0.113.9 /products/42

Whole words: -w

How many requests failed with HTTP 500? The obvious command gives the wrong answer:

grep -c 500 access.log
grep 500 access.log | cut -d' ' -f9,10
3
200 15003
500 89
200 1500

grep matches text, not fields: 15003 and 1500 (response sizes) both contain 500. -w only matches where the pattern is a whole word, with no letter, digit or underscore on either side:

grep -cw 500 access.log
1

-w fixed this one, but a size of exactly 500 bytes would still match. The robust approach is to anchor the pattern to its position in the line; in this log format the status always follows the closing quote of the request, so '" 500 ' is precise. We'll use that below.

Only the matching part: -o

-o prints just the text that matched, one match per line, instead of the whole line. Combined with a counting pipeline it extracts values:

grep -o 'python-requests/[0-9.]*' access.log | sort | uniq -c
      8 python-requests/2.31.0

Context: -A, -B, -C

An error line is rarely enough; you want the stack trace after it or the warning before it. -A N prints N lines after each match, -B N before, and -C N both:

grep -A 3 'ERROR' app.log
2026-11-09 08:15:58 ERROR [http-8080-3] order 1043 failed
java.net.SocketTimeoutException: Read timed out
    at com.example.orders.PaymentClient.charge(PaymentClient.java:88)
    at com.example.orders.OrderService.place(OrderService.java:142)
--
2026-11-09 09:10:44 ERROR [http-8080-2] order 1045 failed
java.sql.SQLTransientConnectionException: orders-pool - Connection is not available
    at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:696)
2026-11-09 09:11:02 INFO  [http-8080-2] order 1046 placed
grep -B 1 'ERROR' app.log
2026-11-09 08:15:58 WARN  [http-8080-3] payment gateway slow: 2950 ms
2026-11-09 08:15:58 ERROR [http-8080-3] order 1043 failed
--
2026-11-09 09:04:10 INFO  [http-8080-1] order 1044 placed
2026-11-09 09:10:44 ERROR [http-8080-2] order 1045 failed

The -- lines separate non-adjacent groups. -B 1 shows the slow-payment warning right before the first failure, which is exactly the clue you would want at 2am.

-m N stops after N matches, which is useful on huge files when you only need to know the first occurrence:

grep -m 1 'ERROR' app.log
2026-11-09 08:15:58 ERROR [http-8080-3] order 1043 failed

Searching many files: -r, -l, -L, -h

Say the database host db01.internal is being retired and you need every config that still mentions it. A conf.d directory holds orders.conf, cache.conf, reports.conf and a stray backup, env.sh.bak. -r searches recursively, and with -n you get file, line number and line:

grep -rn 'db01.internal' conf.d/
conf.d/orders.conf:2:host = db01.internal
conf.d/reports.conf:2:dsn = postgres://report@db01.internal:5432/orders
conf.d/env.sh.bak:1:DB_HOST=db01.internal

-l lists only the names of files that match (ideal for feeding to another command), --include restricts which files are searched, and -L lists the files that don't match:

grep -rl 'db01.internal' conf.d/
grep -rl --include='*.conf' 'db01.internal' conf.d/
grep -L 'db01.internal' conf.d/*
conf.d/orders.conf
conf.d/reports.conf
conf.d/env.sh.bak
conf.d/orders.conf
conf.d/reports.conf
conf.d/cache.conf

When you give grep more than one file, it prefixes each line with the file name; -h turns that off and -H forces it on:

grep 'order 1043' app.log access.log
app.log:2026-11-09 08:15:57 INFO  [http-8080-3] POST /api/orders received (order 1043)
app.log:2026-11-09 08:15:57 INFO  [http-8080-3] reserving stock for order 1043
app.log:2026-11-09 08:15:58 ERROR [http-8080-3] order 1043 failed
app.log:2026-11-09 08:15:58 INFO  [http-8080-3] stock released for order 1043

Finally, -q prints nothing and only sets the exit status (0 for a match, 1 for none), which is what you use in if statements; see exit codes.

Regular expressions for grep

The pattern is a regular expression: a small language for describing text. You only need a handful of pieces to cover most real searches.

PatternMatches
.any single character
*zero or more of the previous item
^ / $start / end of the line
[abc], [0-9]one character from the set or range
[^0-9]one character not in the set
[[:digit:]], [[:alpha:]], [[:space:]], [[:upper:]]POSIX character classes
\b, \<, \>word boundary, start of word, end of word (GNU)
+, ?, {n,m}, |, ( )one or more, optional, repetition, alternation, grouping (ERE)

Basic vs extended: grep vs grep -E

grep uses basic regular expressions (BRE) by default, where + ? { } | ( ) are ordinary characters unless you put a backslash in front of them. grep -E uses extended regular expressions (ERE), where they are special without backslashes. Server errors (status 5xx), three ways:

grep -E '" (500|502|503) ' access.log | cut -d' ' -f1,7,9
198.51.100.23 /api/orders 500
198.51.100.23 /api/orders 502
grep '" 5[0-9]\{2\} ' access.log | cut -d' ' -f1,7,9
grep '" 5[0-9]{2} ' access.log | cut -d' ' -f1,7,9
grep -E '" 5[0-9]{2} ' access.log | cut -d' ' -f1,7,9
198.51.100.23 /api/orders 500
198.51.100.23 /api/orders 502
198.51.100.23 /api/orders 500
198.51.100.23 /api/orders 502

The first (BRE with \{2\}) and third (ERE with {2}) found both lines. The second printed nothing: in BRE, {2} means the literal characters {2}. No error, just no output. My advice is to use grep -E whenever the pattern has any of + ? { } | ( ); it saves you a whole class of silent mistakes.

An IP address extractor shows several pieces together. ^ anchors to the start of the line, [0-9]{1,3} is one to three digits, and (\.[0-9]{1,3}){3} is "a dot and one to three digits", three times:

grep -oE '^[0-9]{1,3}(\.[0-9]{1,3}){3}' access.log | sort -u
10.0.0.5
192.0.2.15
192.0.2.77
198.51.100.23
198.51.100.7
203.0.113.42
203.0.113.9

Anchors and escaping dots

. matches any character, and an unanchored pattern matches anywhere in the line. Both bite when you search for an IP address. Looking for client 192.0.2.1:

grep -c '192.0.2.1' access.log
grep -c '^192\.0\.2\.1 ' access.log
8
0

The first pattern matched the eight requests from 192.0.2.15, because nothing said the address had to end after the 1. The second escapes the dots (\. is a literal dot), anchors to the start of the line, and requires the space that follows the IP. Client 192.0.2.1 made no requests at all.

Anchors and classes also find structure. Every normal log line starts with a date, so lines that don't start with one are stack-trace continuation lines:

grep -vE '^[0-9]{4}-[0-9]{2}-[0-9]{2} ' app.log
java.net.SocketTimeoutException: Read timed out
    at com.example.orders.PaymentClient.charge(PaymentClient.java:88)
    at com.example.orders.OrderService.place(OrderService.java:142)
java.sql.SQLTransientConnectionException: orders-pool - Connection is not available
    at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:696)

And alternation with anchors pulls the section headers and port settings out of a config file (we'll use this app.conf again in the sed article):

grep -E '^\[|^port' app.conf
[server]
port = 8080
[database]
port = 5432
[cache]
port = 6379
[metrics]
port = 9100

Fixed strings: -F

When the thing you are searching for is literal text full of regex characters, such as a thread name in brackets, a version number or an IP, use -F and nothing is special:

grep -c '[http-8080-3]' app.log
grep: Invalid range end
grep -cF '[http-8080-3]' app.log
5

Brackets make a character set, and inside it p-8 is a range from p to 8, which is backwards. That one failed loudly. The quiet version is worse:

grep -c '[ERROR]' app.log
grep -c 'ERROR' app.log
13
2

[ERROR] means "any one of the letters E, R or O", which nearly every line contains.

GNU grep also has -P for Perl-compatible regular expressions, which adds shortcuts like \d (digit) and lookarounds:

grep -P '\d{4} ms' app.log
2026-11-09 08:15:58 WARN  [http-8080-3] payment gateway slow: 2950 ms

-P is not available in BSD/macOS grep, so keep it out of scripts that must be portable.

cut: pick out columns

cut extracts fields separated by a single delimiter character (-d), or character positions (-c):

cut -d' ' -f1 access.log | head -n 3
cut -d' ' -f1,9 access.log | head -n 3
203.0.113.42
203.0.113.42
10.0.0.5
203.0.113.42 200
203.0.113.42 200
10.0.0.5 200

Field lists accept ranges: -f2- is "field 2 to the end", -f1-3 the first three:

head -n 4 orders.csv | cut -d, -f3,5
head -n 3 orders.csv | cut -d, -f2-
region,qty
north,2
south,1
west,5
date,region,product,qty,unit_price
2026-11-01,north,keyboard,2,49.99
2026-11-01,south,monitor,1,189.00

GNU cut adds --complement (everything except the listed fields) and --output-delimiter:

head -n 3 orders.csv | cut -d, --complement -f2,6
head -n 3 orders.csv | cut -d, -f1,4 --output-delimiter=' -> '
order_id,region,product,qty
1001,north,keyboard,2
1002,south,monitor,1
order_id -> product
1001 -> keyboard
1002 -> monitor

-c cuts by character position, handy for fixed-width data like the timestamp at the start of each app.log line:

cut -c1-19 app.log | head -n 2
2026-11-09 08:15:57
2026-11-09 08:15:57

Where cut stops being the right tool

cut has three limits worth knowing before they surprise you:

  1. Output order is file order. -f9,1 prints field 1 then field 9; you can't reorder columns with cut.

    head -n 2 access.log | cut -d' ' -f9,1
    203.0.113.42 200
    203.0.113.42 200
  2. Every delimiter counts. Command output aligned with runs of spaces, like df, produces empty fields:

    df -P /
    df -P / | cut -d' ' -f5
    Filesystem     1024-blocks    Used Available Capacity Mounted on
    /dev/sdd        1055762868 4529796 997529600       1% /
     
     

    Two empty lines: field 5 is an empty string between two of the padding spaces. Squeeze the spaces first with tr -s ' ' (below), or use awk, which splits on runs of whitespace by default.

  3. It knows nothing about quoting. A CSV field like "Smith, Jane" contains the delimiter; see Common mistakes.

sort: put lines in order

sort sorts lines as text, character by character. That is not what you want for numbers:

printf '%s\n' 10 9 100 25 | sort
printf '%s\n' 10 9 100 25 | sort -n
10
100
25
9
9
10
25
100

-n compares numerically, -r reverses. For human-readable sizes like 27M and 580K, use -h. Here are the four largest directories under /usr/share, first correctly and then with -n, which ignores the units:

du -sh /usr/share/* 2> /dev/null | sort -h | tail -n 4
27M	/usr/share/locale
31M	/usr/share/python-babel-localedata
42M	/usr/share/vim
46M	/usr/share/icons
du -sh /usr/share/* 2> /dev/null | sort -n | tail -n 4
580K	/usr/share/libthai
612K	/usr/share/lintian
708K	/usr/share/perl5
1008K	/usr/share/dbus-1

For version numbers, -V understands that 1.10 comes after 1.9:

printf '%s\n' v1.10.0 v1.9.2 v1.2.0 | sort
printf '%s\n' v1.10.0 v1.9.2 v1.2.0 | sort -V
v1.10.0
v1.2.0
v1.9.2
v1.2.0
v1.9.2
v1.10.0

Sorting by a column: -t and -k

-t sets the field separator and -k chooses the key. Largest quantity first (skipping the header with tail -n +2, covered below):

tail -n +2 orders.csv | sort -t, -k5,5nr | head -n 3
1008,2026-11-04,north,mouse,10,19.50
1003,2026-11-02,west,mouse,5,19.50
1012,2026-11-06,west,keyboard,4,49.99

-k5,5nr reads as "the key starts at field 5 and ends at field 5; compare it numerically, reversed". The ,5 matters. -k5 alone means "from field 5 to the end of the line". That doesn't change a single-key sort much, but it breaks multi-key sorts. Sort by region, then by quantity, largest first:

tail -n +2 orders.csv | sort -t, -k3 -k5nr
1005,2026-11-03,east,headset,1,79.00
1009,2026-11-05,east,keyboard,1,49.99
1011,2026-11-06,north,headset,1,79.00
1001,2026-11-01,north,keyboard,2,49.99
1004,2026-11-02,north,monitor,2,189.00
1008,2026-11-04,north,mouse,10,19.50
1010,2026-11-05,south,headset,2,79.00
1006,2026-11-03,south,keyboard,3,49.99
1002,2026-11-01,south,monitor,1,189.00
1012,2026-11-06,west,keyboard,4,49.99
1007,2026-11-04,west,monitor,1,189.00
1003,2026-11-02,west,mouse,5,19.50

Wrong: within north, the order with quantity 10 is last. -k3 compared north,headset,... against north,mouse,... all the way to the end of the line, so those keys were never equal and the second key was never used. With both keys bounded:

tail -n +2 orders.csv | sort -t, -k3,3 -k5,5nr
1005,2026-11-03,east,headset,1,79.00
1009,2026-11-05,east,keyboard,1,49.99
1008,2026-11-04,north,mouse,10,19.50
1001,2026-11-01,north,keyboard,2,49.99
1004,2026-11-02,north,monitor,2,189.00
1011,2026-11-06,north,headset,1,79.00
1006,2026-11-03,south,keyboard,3,49.99
1010,2026-11-05,south,headset,2,79.00
1002,2026-11-01,south,monitor,1,189.00
1003,2026-11-02,west,mouse,5,19.50
1012,2026-11-06,west,keyboard,4,49.99
1007,2026-11-04,west,monitor,1,189.00

Always write keys as -kN,N.

Stable sorts: -s

When two lines have equal keys, sort breaks the tie by comparing the whole line, and -r reverses that last-resort comparison too. -s (stable) turns the tiebreak off, so equal lines keep their input order. Compare the two mouse orders here:

tail -n +2 orders.csv | sort -t, -r -k4,4 | head -n 2
tail -n +2 orders.csv | sort -t, -r -s -k4,4 | head -n 2
1008,2026-11-04,north,mouse,10,19.50
1003,2026-11-02,west,mouse,5,19.50
1003,2026-11-02,west,mouse,5,19.50
1008,2026-11-04,north,mouse,10,19.50

Without -s, the tie was broken by the whole line in reverse, so 1008 came first. With -s, they stayed in file (date) order. Use -s when the input order means something, such as time order in a log.

Other sort options worth knowing

  • -u outputs only the first of each run of equal lines: a sorted list of distinct values.
  • -f folds case. In the C and C.UTF-8 locales, sorting compares byte values, so every uppercase letter comes before every lowercase one (Apple Cherry apple banana, below). Locales such as en_US.UTF-8 use dictionary-style collation instead, so the same input sorts differently on a machine with another locale. If a script's output order must be identical everywhere, run sort with LC_ALL=C.
  • -o file writes the result to a file, and is safe even when that file is the input.
  • -c checks whether input is already sorted, and reports the first line out of order.
tail -n +2 orders.csv | cut -d, -f3 | sort -u
printf '%s\n' banana Apple apple Cherry | sort
east
north
south
west
Apple
Cherry
apple
banana

uniq: collapse and count duplicates

uniq removes adjacent duplicate lines, and uniq -c prefixes each line with how many times it repeated. The word "adjacent" is the entire lesson of this section. Here is uniq -c on the client IPs, straight from the log:

cut -d' ' -f1 access.log | uniq -c
      2 203.0.113.42
      1 10.0.0.5
      2 198.51.100.7
      8 192.0.2.15
      1 203.0.113.42
      1 10.0.0.5
      2 198.51.100.23
      2 203.0.113.9
      1 10.0.0.5
      2 203.0.113.42
      2 198.51.100.23
      2 198.51.100.7
      1 10.0.0.5
      1 192.0.2.77
      2 198.51.100.7
      1 203.0.113.42
      1 198.51.100.23
      1 203.0.113.42

Useless: every IP appears several times because uniq only merges lines that are next to each other. Sort first, so identical lines become adjacent:

cut -d' ' -f1 access.log | sort | uniq -c
      4 10.0.0.5
      8 192.0.2.15
      1 192.0.2.77
      5 198.51.100.23
      6 198.51.100.7
      7 203.0.113.42
      2 203.0.113.9

Then sort the counts, numerically and largest first. This four-stage pattern, sort | uniq -c | sort -rn, is the single most useful pipeline in this article. It answers "what are the most common X?" for any X you can cut out of a line:

cut -d' ' -f1 access.log | sort | uniq -c | sort -rn
      8 192.0.2.15
      7 203.0.113.42
      6 198.51.100.7
      5 198.51.100.23
      4 10.0.0.5
      2 203.0.113.9
      1 192.0.2.77

uniq -d prints only lines that occur more than once (paths requested repeatedly), and uniq -u only lines that occur exactly once. uniq -i ignores case.

cut -d' ' -f7 access.log | sort | uniq -d
/api/orders
/healthz
/products/42
/wp-login.php

wc: count lines, words and bytes

wc access.log
wc -l access.log app.log
wc -l < access.log
  33  497 4472 access.log
  33 access.log
  14 app.log
  47 total
33

Plain wc prints lines, words and bytes. -l, -w and -c select one. Reading from stdin (< file) leaves the file name off, which is what you want inside $( ).

-c counts bytes and -m counts characters, which differ for anything outside ASCII:

printf 'café\n' | wc -c
printf 'café\n' | wc -m
6
5

é is two bytes in UTF-8.

tr: translate, squeeze and delete characters

tr works on single characters and only reads stdin. It maps characters in the first set to the corresponding ones in the second:

tr 'a-z' 'A-Z' <<< "post /api/orders"
head -n 3 orders.csv | tr ',' '\t'
POST /API/ORDERS
order_id	date	region	product	qty	unit_price
1001	2026-11-01	north	keyboard	2	49.99
1002	2026-11-01	south	monitor	1	189.00

-s squeezes runs of a repeated character into one, which is the fix for cut on space-aligned output:

echo "web01    up     12d" | tr -s ' '
df -P / | tr -s ' ' | cut -d' ' -f5
web01 up 12d
Capacity
1%

-d deletes characters. Its most common job is stripping the carriage returns from files created on Windows (see Common mistakes): tr -d '\r'. Add -c to complement the set, so tr -cd '[:print:]\n' deletes everything that isn't printable.

head and tail: the start and end of a file

head -n N prints the first N lines and tail -n N the last N:

head -n 3 orders.csv
tail -n 2 app.log
order_id,date,region,product,qty,unit_price
1001,2026-11-01,north,keyboard,2,49.99
1002,2026-11-01,south,monitor,1,189.00
    at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:696)
2026-11-09 09:11:02 INFO  [http-8080-2] order 1046 placed

tail -n +N means "starting at line N", so tail -n +2 skips a header line. You have seen it throughout this article:

tail -n +2 orders.csv | head -n 2
1001,2026-11-01,north,keyboard,2,49.99
1002,2026-11-01,south,monitor,1,189.00

head -c N takes bytes instead of lines.

Following a growing file: tail -f

tail -f prints the end of a file and then keeps waiting, printing new lines as they are appended. It is how you watch a log while reproducing a bug. Here a background job appends a line one second after tail starts, and timeout 3 stops tail after three seconds (normally you press Ctrl-C):

cp app.log live.log
(sleep 1; echo "2026-11-09 10:20:00 ERROR [http-8080-1] order 1047 failed" >> live.log) &
timeout 3 tail -n 2 -f live.log
    at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:696)
2026-11-09 09:11:02 INFO  [http-8080-2] order 1046 placed
2026-11-09 10:20:00 ERROR [http-8080-1] order 1047 failed

Use tail -F (capital F) for logs that get rotated: it notices when the file is renamed and replaced, and reopens the new one. tail -f log | grep --line-buffered ERROR filters live output; --line-buffered makes grep pass each line on immediately instead of in blocks.

paste: join lines side by side

paste merges files line by line, separated by tabs:

printf '%s\n' web01 web02 db01 > hosts.txt
printf '%s\n' 10.0.1.11 10.0.1.12 10.0.2.21 > ips.txt
paste hosts.txt ips.txt
paste -d' ' ips.txt hosts.txt
web01	10.0.1.11
web02	10.0.1.12
db01	10.0.2.21
10.0.1.11 web01
10.0.1.12 web02
10.0.2.21 db01

The second form is a ready-made /etc/hosts fragment. With -s (serial), paste joins all lines of one input into a single line, which turns a column into a comma-separated list or an arithmetic expression:

paste -sd, hosts.txt
tail -n +2 orders.csv | cut -d, -f5 | paste -sd+
tail -n +2 orders.csv | cut -d, -f5 | paste -sd+ | bc
web01,web02,db01
2+1+5+2+1+3+1+10+1+2+1+4
33

bc is a calculator that reads an expression from stdin; 33 units were ordered in total. (In the awk article the same sum is one short command.) Each - argument reads one line from stdin, so paste - - - - reassembles every four lines into one row:

printf '%s\n' host web01 status up host db01 status down | paste - - - -
host	web01	status	up
host	db01	status	down

column: align output into a table

column -t lines up whitespace-separated columns; -s sets the input separator:

head -n 5 orders.csv | column -t -s,
order_id  date        region  product   qty  unit_price
1001      2026-11-01  north   keyboard  2    49.99
1002      2026-11-01  south   monitor   1    189.00
1003      2026-11-02  west    mouse     5    19.50
1004      2026-11-02  north   monitor   2    189.00

It is the last stage of many report pipelines. Requests per client and status code:

cut -d' ' -f1,9 access.log | sort | uniq -c | sort -rn | head -n 6 | column -t
8  192.0.2.15    404
5  203.0.113.42  200
4  10.0.0.5      200
3  198.51.100.7  200
2  203.0.113.9   200
2  198.51.100.7  304

Where you'll use this

These filters do most of the "quick answer" work in operations and data handling:

  • Incident response: who is hitting the server, which endpoint is failing, when it started. The scanner at 192.0.2.15 (eight 404s in three seconds probing for /wp-login.php and /.env) is exactly what these pipelines expose.
  • Migrations and audits: grep -rn across a config tree before you rename a host, a database or an environment variable, so nothing is missed.
  • Data checks: distinct values in a CSV column, duplicate IDs (sort | uniq -d), row counts before and after an import.
  • CI and scripts: grep -q as a test, wc -l as a threshold, sort -u to normalise lists before comparing them.

Here is a complete report script built from the pipelines above. Every section is one pipeline; the script adds headers and argument checking:

#!/usr/bin/env bash
# access-report.sh - quick traffic summary from an nginx/Apache combined-format log.
set -o pipefail
log=${1:?usage: access-report.sh ACCESS_LOG}
[[ -r $log ]] || { echo "cannot read $log" >&2; exit 1; }
 
total=$(wc -l < "$log")
clients=$(cut -d' ' -f1 "$log" | sort -u | wc -l)
echo "== $log: $total requests from $clients clients"
 
echo; echo "== Top 5 clients"
cut -d' ' -f1 "$log" | sort | uniq -c | sort -rn | head -n 5
 
echo; echo "== Status codes"
cut -d' ' -f9 "$log" | sort | uniq -c | sort -rn
 
echo; echo "== Requests per hour"
cut -d: -f2 "$log" | sort | uniq -c
 
echo; echo "== Top 404 paths"
grep '" 404 ' "$log" | cut -d' ' -f7 | sort | uniq -c | sort -rn | head -n 5
 
echo; echo "== Server errors (5xx)"
grep -E '" 5[0-9]{2} ' "$log" | cut -d' ' -f1,4,7,9 | tr -d '[' | column -t

The "requests per hour" line uses a trick worth stealing: with : as the delimiter, field 2 of ...[09/Nov/2026:08:01:12 +0000]... is the hour.

./access-report.sh access.log
== access.log: 33 requests from 7 clients
 
== Top 5 clients
      8 192.0.2.15
      7 203.0.113.42
      6 198.51.100.7
      5 198.51.100.23
      4 10.0.0.5
 
== Status codes
     15 200
      9 404
      3 201
      2 304
      1 502
      1 500
      1 302
      1 301
 
== Requests per hour
     20 08
      8 09
      5 10
 
== Top 404 paths
      2 /wp-login.php
      1 /xmlrpc.php
      1 /wp-admin/
      1 /phpmyadmin/
      1 /favicon.ico
 
== Server errors (5xx)
198.51.100.23  09/Nov/2026:08:15:58  /api/orders  500
198.51.100.23  09/Nov/2026:09:10:44  /api/orders  502

Reading it: a quarter of all traffic is a scanner, and both server errors are POST /api/orders from the same API client, 55 minutes apart. That is a lead for the payment gateway timeout we saw in app.log. The ${1:?...} expansion makes the script refuse to run without an argument:

./access-report.sh
./access-report.sh nope.log
./access-report.sh: line 4: 1: usage: access-report.sh ACCESS_LOG
cannot read nope.log

When a report like this grows conditions ("only count 5xx between 08:00 and 09:00", "sum bytes per client"), switch to awk; the awk article rewrites this report in one program. Scheduling it daily is covered in real-world scripts.

Common mistakes

Windows line endings break $ anchors

A CSV exported from a Windows tool ends every line with \r\n. The \r is invisible, but it sits between your text and the end of the line:

printf 'order_id,region\r\n1001,north\r\n1002,south\r\n' > export.csv
grep -c 'north$' export.csv
cat -A export.csv
0
order_id,region^M$
1001,north^M$
1002,south^M$

cat -A shows the carriage return as ^M. Strip it and the anchor works:

tr -d '\r' < export.csv > export-unix.csv
grep -c 'north$' export-unix.csv
1

The same \r makes sort -u treat north and north\r as different values and makes numeric comparisons fail. When results look impossible, cat -A is the first thing to run.

cut on CSV with quoted fields

cat > customers.csv <<'EOF'
id,name,city
17,"Smith, Jane",Leeds
18,Raj Patel,Bristol
EOF
cut -d, -f3 customers.csv
city
 Jane"
Bristol

cut doesn't understand quotes, so the comma inside "Smith, Jane" shifted every field on that line. If a CSV can contain quoted fields, use a real CSV parser. Python's standard library has one and is installed almost everywhere:

python3 -c 'import csv, sys; [print(row[2]) for row in csv.reader(sys.stdin)]' < customers.csv
city
Leeds
Bristol

GNU awk can also handle simple quoted CSV (see awk).

Counting without sorting first

uniq -c without sort gave us eighteen lines of partial counts for seven IPs earlier. The output looks plausible, which is what makes it dangerous. If the input is not already grouped, the pattern is always sort | uniq -c.

Sorting numbers as text, and unbounded keys

sort without -n puts 100 before 25, and -k3 without ,3 silently ignores every later key. Both were shown above with real output; the habit that prevents both is writing every key as -kN,N plus a type letter: -k5,5n.

Matching inside other fields

grep 500 counted response sizes 15003 and 1500 as server errors. Anchor patterns to the structure of the line ('" 500 '), use -w, or extract the field first with cut and match on that.

Summary

  • grep options to know cold: -i -v -n -c -l -L -r -w -o -m -q, context with -A/-B/-C, -E for extended regex, -F for literal text.
  • Basic regex treats + ? { } | ( ) literally unless escaped; grep -E makes them special. Escape dots (\.) and anchor (^, $, trailing space) when matching IPs and codes.
  • cut -d X -fN extracts fields split on a single character; it can't reorder fields, handle runs of spaces (tr -s ' ' first) or quoted CSV.
  • sort -n for numbers, -h for sizes, -V for versions, -t, -kN,N for columns (always bound the key), -s to keep input order for ties, -u for distinct values.
  • uniq only merges adjacent lines: sort | uniq -c | sort -rn is the frequency-count pipeline.
  • wc -l < file gives a bare count; tr translates, squeezes (-s) and deletes (-d '\r'); tail -n +2 skips a header; tail -F follows rotated logs.
  • paste -sd, joins lines, paste a b merges files side by side, column -t aligns the final output.

Exercises

Exercise 1: Profile the scannereasy

How many requests came from 192.0.2.15, and which paths did it request, most frequent first? Make sure your pattern can't match another address that merely starts with the same digits.

Show Solution
grep -c '^192\.0\.2\.15 ' access.log
grep '^192\.0\.2\.15 ' access.log | cut -d' ' -f7 | sort | uniq -c | sort -rn
8
      2 /wp-login.php
      1 /xmlrpc.php
      1 /wp-admin/
      1 /phpmyadmin/
      1 /favicon.ico
      1 /admin/config.php
      1 /.git/config
      1 /.env

The ^ and the trailing space pin the match to the whole first field; \. makes the dots literal. Paths like /.env and /.git/config are classic probes for leaked secrets.

Exercise 2: Count the user agentsmedium

The user agent is the last quoted string on each line and contains spaces, so splitting on spaces won't extract it. Count requests per user agent, most frequent first, using only cut, sort and uniq.

Hint

Which character surrounds the user agent? Count how many of them come before it.

Show Solution
cut -d'"' -f6 access.log | sort | uniq -c | sort -rn
      8 python-requests/2.31.0
      7 Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0
      6 Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/130.0.0.0 Safari/537.36
      5 curl/8.5.0
      4 kube-probe/1.30
      2 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
      1 Mozilla/5.0 (iPhone; CPU iPhone OS 18_1 like Mac OS X) Mobile/15E148 Safari/604.1

With " as the delimiter, field 2 is the request, field 4 the referrer and field 6 the user agent. Choosing the delimiter cleverly often saves a regex, the same trick the report script uses with : to get the hour.

Exercise 3: Best-selling products by order countmedium

From orders.csv, list how many orders each product appeared in, highest count first, with ties in alphabetical order. The header line must not be counted.

Show Solution
tail -n +2 orders.csv | cut -d, -f4 | sort | uniq -c | sort -k1,1nr -k2,2
      4 keyboard
      3 headset
      3 monitor
      2 mouse

The final sort has two keys: field 1 (the count) numerically reversed, then field 2 (the name) as text. uniq -c output is separated by spaces, which is sort's default separator, so no -t is needed.

Exercise 4: An error table per clienthard

Produce an aligned table of how many 4xx and 5xx responses each client received, broken down by status code, largest count first.

Show Solution
grep -E '" [45][0-9]{2} ' access.log | cut -d' ' -f1,9 | sort | uniq -c | sort -rn | column -t
8  192.0.2.15     404
1  198.51.100.23  502
1  198.51.100.23  500
1  192.0.2.77     404

'" [45][0-9]{2} ' matches a status from 400 to 599 in the right position, cut keeps IP and status, sort | uniq -c counts each pair, sort -rn orders by count, and column -t aligns the result.


Tested with: GNU Bash 5.2.21 on Ubuntu 24.04 (WSL2), November 2026, with GNU grep 3.11, GNU coreutils 9.4 (cut, sort, uniq, wc, tr, head, tail, paste) and column from util-linux. Every command was run against the sample files above and the output pasted unchanged; the shell's locale was C.UTF-8, which is why uppercase sorts before lowercase. The /usr/share sizes come from that machine and will differ on yours.

Was this page helpful?
The Dark Debugger

Written by The Dark Debugger

Engineer documenting the things that took me too long to learn, so they take you less time. Every Bash tutorial here is tested end to end before publishing.