Unit 3: Searching and pipes · Lesson 1 of 6 · 18 min

Filtering with grep

Read one access log line field by field, then keep only the lines you care about.

mission

You are: You are the analyst asked about yesterday's web traffic.
Situation: Monitoring flagged a burst of 404 errors on 14 October. The access log has 1378 lines.
Goal: Count the errors and find the automated clients behind them.
Hand in: The number of 404 responses and the scanner user agents.

Try grep on real lines

$ grep '' access.log

192.0.2.58 - - [14/Oct/2026:00:02:30 +0000] "GET /pricing.html HTTP/1.1" 404 156

1 of 8 lines. grep -c would print 1.

The first eight lines of the access log. Type any text to filter them.

Count by status

Put spaces around the code. Without them, 304 would also match a size of 3045 bytes.

learner@root-lab

learner@root-lab:/var/log/nginx$ grep -c ' 404 ' access.log
59
learner@root-lab:/var/log/nginx$ grep -c ' 304 ' access.log
136

Follow a user agent

learner@root-lab

learner@root-lab:/var/log/nginx$ grep 'python-requests' access.log | head -n 2
192.0.2.200 - - [14/Oct/2026:14:41:45 +0000] "GET /.env HTTP/1.1" 404 158 "-" "python-requests/2.31.0"
192.0.2.200 - - [14/Oct/2026:14:41:49 +0000] "GET /wp-login.php HTTP/1.1" 404 168 "-" "python-requests/2.31.0"
learner@root-lab:/var/log/nginx$ grep -c 'scan-bot' access.log
40

you should see Browsers do not ask for /.env or /wp-login.php on a shop with no WordPress. Scripts do.

defender tip -i ignores case, -v keeps lines that do not match. grep -v ' 200 ' shows everything that went wrong.

Command	What it does
`grep text file`	Lines that contain text
`grep -c text file`	Count them
`grep -v text file`	Lines that do not contain it
`grep -i text file`	Ignore case
`grep -n text file`	Show line numbers

Ignore case, show line numbers, invert

-i ignores case, -n adds line numbers, -m1 stops at the first hit, -v keeps lines that do not match.

learner@root-lab

learner@root-lab:/var/log/nginx$ grep -ic 'SCAN-BOT' access.log
40
learner@root-lab:/var/log/nginx$ grep -m1 -n '/.env' access.log
129:203.0.113.45 - - [14/Oct/2026:02:13:08 +0000] "GET /.env HTTP/1.1" 404 184 "-" "Mozilla/5.0 (compatible; scan-bot/1.2)"
learner@root-lab:/var/log/nginx$ grep -vc ' 200 ' access.log
195

you should see The first probe for .env sits on line 129, and 195 requests did not get a 200.

What a defender takes away

1grep keeps lines, -c counts them.
2Anchor your pattern with spaces or it matches too much.
3User agents reveal scripts, even though they can be faked.

Questions

0 of 2 solved

Answer each one from what you find on your lab machine.

1How many requests got a 404 response?
Show hint
grep -c ' 404 '
2How many requests came from the scan-bot user agent?
Show hint
grep -c scan-bot