Unit 8: APIs and how modern apps talk · Lesson 3 of 5 · 18 min
Rate limits
Check how many requests the API allows, what it says when you run out and how clients should react.
mission
- You are
- The analyst reviewing the shop's API.
- Situation
- A price comparison site has been pulling the catalogue so often that the API slows down for real customers.
- Goal
- Measure the product endpoint's limit and read the headers it uses to announce it.
- Hand in
- The limit, the window and the headers a well-behaved client should honour.
A rate limit as a token bucket
Read the limit headers
curl -sI http://127.0.0.1:8000/api/v1/products | grep -i ratelimit
RateLimit-Limit: 10
RateLimit-Remaining: 9
you should see The limit and how many requests are left in the current window.
Watch the budget drain
for i in $(seq 1 11); do curl -s -o /dev/null -D - http://127.0.0.1:8000/api/v1/products | grep -iE '^(HTTP|ratelimit-remaining)' | tr -d '\r' | paste -sd' '; done
HTTP/1.1 200 OK RateLimit-Remaining: 8
HTTP/1.1 200 OK RateLimit-Remaining: 7
HTTP/1.1 200 OK RateLimit-Remaining: 6
HTTP/1.1 200 OK RateLimit-Remaining: 5
HTTP/1.1 200 OK RateLimit-Remaining: 4
HTTP/1.1 200 OK RateLimit-Remaining: 3
HTTP/1.1 200 OK RateLimit-Remaining: 2
HTTP/1.1 200 OK RateLimit-Remaining: 1
HTTP/1.1 200 OK RateLimit-Remaining: 0
HTTP/1.1 429 Too Many Requests RateLimit-Remaining: 0
HTTP/1.1 429 Too Many Requests RateLimit-Remaining: 0
you should see The remaining count falls to zero, then the API answers 429.
Read the refusal
curl -si http://127.0.0.1:8000/api/v1/products | grep -iE '^(HTTP|retry-after)'
HTTP/1.1 429 Too Many Requests
Retry-After: 60
you should see 429 and a Retry-After header.
defender tip Good clients back off for Retry-After seconds. A client that keeps hammering through 429s is a scraper or a bug, and both show clearly in the access log.
Client behaviour
while true; do get(url); done
Ignores 429 and keeps the server busy.
r = get(url)
if r.status == 429:
sleep(int(r.headers["Retry-After"]))Waits as told.
What a defender takes away
- 1Rate limits protect availability for everyone.
- 2Announce limits with RateLimit headers and refuse with 429 and Retry-After.
- 3Limit per token or per address, whichever the client cannot easily change.
Questions
0 of 2 solved
Answer each one from what you find on your lab machine.