Unit 3: Searching and pipes · Lesson 2 of 6 · 20 min
Pipes and rankings
Join small commands into a pipeline that ranks clients and finds what the scanner got.
mission
- You are
- You are ranking clients for the incident ticket.
- Situation
- The 404s came from a few addresses. You need to name the noisiest one and check whether any probe succeeded.
- Goal
- Build a ranking pipeline and filter the scanner's successful requests.
- Hand in
- The top 404 source and any sensitive path that returned 200.
One stage at a time
192.0.2.89 - - [14/Oct/2026:00:00:20 +0000] "GET / HTTP/1.1" 200 31932 192.0.2.35 - - [14/Oct/2026:00:00:42 +0000] "GET / HTTP/1.1" 200 14564 192.0.2.58 - - [14/Oct/2026:00:02:30 +0000] "GET /pricing.html HTTP/1.1" 404 156 192.0.2.35 - - [14/Oct/2026:00:04:04 +0000] "GET /about HTTP/1.1" 200 24374 198.51.100.81 - - [14/Oct/2026:00:04:57 +0000] "GET / HTTP/1.1" 200 14812 198.51.100.209 - - [14/Oct/2026:00:05:19 +0000] "GET /assets/js/app.js HTTP/1.1" 200 30762 192.0.2.100 - - [14/Oct/2026:00:07:08 +0000] "GET /blog/ HTTP/1.1" 200 30650 192.0.2.58 - - [14/Oct/2026:00:08:01 +0000] "GET /assets/js/app.js HTTP/1.1" 200 26528
8 lines. head -n 8 access.log, before any filter
cut- Keeps only chosen columns or characters from each line. Built into Linux and macOS.
sort- Sorts lines of text. Built into Linux and macOS.
uniq- Collapses repeated neighbouring lines. With -c it counts them. Built into Linux and macOS.
Rank every client
cut -d' ' -f1 access.log | sort | uniq -c | sort -rn | head -n 3
52 192.0.2.171
51 192.0.2.87
49 198.51.100.138
you should see Busy customers. Volume alone does not mean an attack.
head- Prints the first lines of a file or of piped text. Built into Linux and macOS.
Rank only the errors
grep ' 404 ' access.log | cut -d' ' -f1 | sort | uniq -c | sort -rn | head -n 3
37 203.0.113.45
6 192.0.2.200
2 198.51.100.140
you should see One address dominates the 404s.
grep- Prints the lines of text that match a pattern. Built into Linux and macOS.
Did any probe succeed?
A scan full of 404s found nothing. The lines that matter are the probes that came back 200.
grep 203.0.113.45 access.log | grep ' 200 ' | cut -d' ' -f7,9
/.git/config 200
/robots.txt 200
/sitemap.xml 200
you should see robots.txt and sitemap.xml are public on purpose. /.git/config is not: the site's Git metadata is downloadable.
defender tip Fix it in the web server config by denying every path that starts with /., then grep the log again tomorrow.
What a defender takes away
- 1cut, sort, uniq -c, sort -rn is the ranking pattern you will reuse daily.
- 2Filter first, then rank.
- 3A scan is noise. A 200 on a sensitive path is a finding.
Questions
0 of 2 solved
Answer each one from what you find on your lab machine.