Unit 1: Requests and responses · Lesson 4 of 5 · 22 min
Project: map the app
Turn a list of paths into a status map you can keep, sort and hand over.
mission
- You are
- The analyst preparing the shop review.
- Situation
- The review starts tomorrow. Your lead wants a map of the app the whole team can work from.
- Goal
- Build a file that lists every known path with its status code and where redirects go.
- Hand in
- map.txt, sorted by status code.
From paths to a map
- paths.txtone path per line
- while readone request each
- map.txtpath, code, redirect
- sortgroup by code
Write the path list
Start from what the app links to and what the README mentions.
printf '%s\n' / /health /shop /login /account /logout /admin /home /old-shop /api/status /api/products /api/orders /search > paths.txt && wc -l paths.txt
13 paths.txt
you should see 13 paths.
printf- Prints formatted text. Built into Linux and macOS.
wc- Counts lines, words and bytes. Built into Linux and macOS.
Request every path
%{redirect_url} adds where a redirect points, so the map shows the whole route.
while read p; do curl -s -o /dev/null -w "$p %{http_code} %{redirect_url}\n" http://127.0.0.1:8000$p; done < paths.txt > map.txt && cat map.txt
/ 200
/health 200
/shop 200
/login 200
/account 302 http://127.0.0.1:8000/login
/logout 303 http://127.0.0.1:8000/
/admin 403
/home 301 http://127.0.0.1:8000/
/old-shop 301 http://127.0.0.1:8000/shop
/api/status 200
/api/products 200
/api/orders 401
/search 200
you should see One line per path. Redirects show their target.
curl- Sends web requests and prints the response. Built into Linux, macOS and Windows 10 and later.
cat- Prints the contents of one or more files. Built into Linux and macOS.
Group by status
sort -k2 map.txt / 200 /api/products 200 /api/status 200 /health 200 /login 200 /search 200 /shop 200 /home 301 http://127.0.0.1:8000/ /old-shop 301 http://127.0.0.1:8000/shop /account 302 http://127.0.0.1:8000/login /logout 303 http://127.0.0.1:8000/ /api/orders 401 /admin 403 cut -d' ' -f2 map.txt | sort | uniq -c 7 200 2 301 1 302 1 303 1 401 1 403
you should see Most paths are 200. Four redirect, and two refuse you.
defender tip Keep the map in the review folder and rerun it after every fix. A changed status code is the fastest sign that something moved.
sort- Sorts lines of text. Built into Linux and macOS.
cut- Keeps only chosen columns or characters from each line. Built into Linux and macOS.
uniq- Collapses repeated neighbouring lines. With -c it counts them. Built into Linux and macOS.
Note the paths to look at first
awk '$2 >= 400' map.txt
/admin 403
/api/orders 401
you should see The pages that refuse you: these guard something.
awk- Splits each line into fields and prints or counts the ones you ask for. Built into Linux and macOS.
What a defender takes away
- 1A status map is the first artifact of any web review.
- 2Store it as text so you can sort, diff and rerun it.
- 3Redirect targets belong on the map; they show how the app routes people.
Questions
0 of 3 solved
Answer each one from what you find on your lab machine.