Unit 1: Python for defenders · Lesson 6 of 6 · 22 min
Case file: one bad library version
The morning a backdoored library is announced, answer which of 36 hosts run it, and avoid the version comparison trap.
mission
- You are
- the engineer on call when the xz backdoor news breaks
- Situation
- An advisory says liblzma 5.6.0 and 5.6.1 carry a backdoor. Operations exported the package list from all 36 hosts to
data/fleet/packages.csv. - Goal
- List every affected host within minutes, then check the OpenSSL baseline without falling for string comparison.
- Hand in
- A sorted list of affected hosts and a correct count of hosts below baseline.
case file
xz Utils Backdoor
xz Utils open source project · 2024
On March 29, 2024, engineer Andres Freund disclosed that xz/liblzma versions 5.6.0 and 5.6.1 contained a backdoor (CVE-2024-3094) that could affect sshd on Linux systems where sshd links liblzma through libsystemd. He noticed it while investigating SSH logins using unusual CPU and valgrind errors on test systems. The malicious build step existed only in the release tarballs, not in the git repository, and was introduced by a project committer; it was caught before reaching most stable distributions.
What went wrong
A long-trusted contributor to an under-resourced project inserted obfuscated code, with part of it present only in release tarballs that differed from the source repository.
What would have stopped it
- Build packages from the source repository (or verify release tarballs match the tagged source) instead of trusting maintainer-generated tarballs.
- Use reproducible builds and diff release artifacts against source to catch injected build scripts.
- Stage new upstream versions in testing/rolling channels with performance and behavior monitoring before promoting to stable.
- Support and review critical open source dependencies (maintainer health, multiple reviewers).
What defenders could have seen
- Performance regressions such as sshd authentication taking noticeably more CPU/time.
- Memory-checker (valgrind) errors appearing in a library after an upgrade.
- Inventory query showing xz/liblzma 5.6.0 or 5.6.1 installed.
sourcesseclists.orgcisa.gov
From export to answer
- Package exporthost, os, package, version
- Upstream version5.6.1-1 becomes 5.6.1
- Compareset membership or numeric tuples
- Affected hostssorted, ready for the ticket
Look at the export
Know your columns before writing code. Debian style versions add a packaging suffix after the dash.
head -4 data/fleet/packages.csv host,os,package,version fleet-01,ubuntu-22.04,liblzma5,5.4.5-0.3 fleet-01,ubuntu-22.04,openssl,3.0.13-0ubuntu3.4 fleet-01,ubuntu-22.04,openssh-server,9.2p1-2+deb12u3 cat data/fleet/baseline.txt openssl minimum 3.0.13 liblzma5 never 5.6.0 5.6.1 grep -c liblzma5 data/fleet/packages.csv 36
you should see One liblzma5 row per host, 36 in all.
Find the backdoored versions
The bad versions are an exact list, so a set lookup on the upstream part is all you need.
cat > scripts/xzcheck.py <<'EOF' import csv BAD = {"5.6.0", "5.6.1"} with open("data/fleet/packages.csv") as f: for row in csv.DictReader(f): if row["package"] != "liblzma5": continue upstream = row["version"].split("-")[0] if upstream in BAD: print(row["host"], row["os"], row["version"]) EOF python3 scripts/xzcheck.py fleet-07 debian-testing 5.6.1-1 fleet-19 debian-testing 5.6.0-0.2 fleet-28 debian-testing 5.6.1-1
you should see Three hosts, all on the testing distribution that picks up new upstream releases first.
defender tip Stable distributions shipped older xz releases and were never affected. Knowing which channel each host follows is part of the inventory.
See the comparison trap
Versions are not text. Compared as strings, 3.0.2 looks newer than 3.0.13 because the character 2 sorts after 1.
python3 -c 'print("3.0.2" < "3.0.13", (3, 0, 2) < (3, 0, 13))'
False True
you should see False for text, True for tuples. Only the tuple answer is right.
Check the OpenSSL baseline both ways
cat > scripts/baseline.py <<'EOF' import csv MINIMUM = "3.0.13" def parse(version): return tuple(int(part) for part in version.split("-")[0].split(".")) as_text = as_numbers = 0 with open("data/fleet/packages.csv") as f: for row in csv.DictReader(f): if row["package"] != "openssl": continue upstream = row["version"].split("-")[0] if upstream < MINIMUM: as_text += 1 if parse(upstream) < parse(MINIMUM): as_numbers += 1 print("compared as text: ", as_text, "hosts below", MINIMUM) print("compared as numbers:", as_numbers, "hosts below", MINIMUM) EOF python3 scripts/baseline.py compared as text: 4 hosts below 3.0.13 compared as numbers: 16 hosts below 3.0.13
you should see The text comparison misses hosts that the numeric comparison catches.
defender tip For real packages use the distribution's own comparison, such as dpkg --compare-versions, which also understands epochs and suffixes.
Hand over a clean list
mkdir -p reports && python3 scripts/xzcheck.py | sort > reports/xz-affected.txt cat reports/xz-affected.txt fleet-07 debian-testing 5.6.1-1 fleet-19 debian-testing 5.6.0-0.2 fleet-28 debian-testing 5.6.1-1
you should see Three lines, sorted, ready to paste into the incident ticket.
Comparing versions
if upstream < "3.0.13":
3.0.2 slips through and the host is reported as fine.
if parse(upstream) < parse("3.0.13"):Each part is compared as an integer, the way versions work.
What a defender takes away
- 1An inventory you can query turns a scary headline into a five minute answer.
- 2Exact bad versions call for a set lookup.
- 3Never compare versions as strings.
- 4Know which hosts follow testing or rolling channels.
Questions
0 of 3 solved
Answer each one from what you find on your lab machine.