Unit 1: Python for defenders · Lesson 5 of 6 · 22 min

Virtual environments and trusted packages

Install third party code into a venv from pinned, hashed requirements and catch a tampered package.

mission

You are
the engineer who owns the team's tooling
Situation
The ipcheck tool needs the requests library. Package registries get poisoned, so the team pins every package to a version and a SHA-256 hash.
Goal
Install the requirements into a virtual environment, then show that a tampered copy is refused.
Hand in
The installed requests version and the name of the package that failed its hash check.

The hash gate at the loading dock

01

Read the requirements

Each package is pinned to an exact version and the hash of the exact file you reviewed.

learner@py-lab
cat projects/ipcheck/requirements.txt
requests==2.32.3 \
    --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6
idna==3.10 \
    --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3
urllib3==2.2.3 \
    --hash=sha256:ca899ca043dcb1bafa3e262d73aa25c465bfb49e0bd9dd5d59f1d0acba2f8fac
...
02

Create and enter a virtual environment

A venv is a private copy of Python for one project. Nothing you install there touches the system Python.

learner@py-lab
python3 -m venv ~/venvs/ipcheck
source ~/venvs/ipcheck/bin/activate && which python && pip --version | cut -d' ' -f1-2
/home/learner/venvs/ipcheck/bin/python
pip 24.0

you should see python now points inside the venv.

03

Install with hashes enforced

This machine has no internet, so pip installs from the team's local wheelhouse. --require-hashes makes pip refuse any file that does not match.

learner@py-lab
source ~/venvs/ipcheck/bin/activate && pip install --no-index --find-links /opt/wheelhouse --require-hashes -r projects/ipcheck/requirements.txt
...
Successfully installed certifi-2024.8.30 charset-normalizer-3.4.0 idna-3.10 requests-2.32.3 urllib3-2.2.3
source ~/venvs/ipcheck/bin/activate && pip list --format=freeze
certifi==2024.8.30
charset-normalizer==3.4.0
idna==3.10
pip==24.0
requests==2.32.3
urllib3==2.2.3
04

Try a mirror someone else set up

A colleague suggests a faster internal mirror at /opt/mirror. Same file names, same versions.

learner@py-lab
python3 -m venv /tmp/v2
/tmp/v2/bin/pip install --no-index --find-links /opt/mirror --require-hashes -r projects/ipcheck/requirements.txt
...
Processing /opt/mirror/requests-2.32.3-py3-none-any.whl (from -r projects/ipcheck/requirements.txt (line 1))
Processing /opt/mirror/idna-3.10-py3-none-any.whl (from -r projects/ipcheck/requirements.txt (line 3))
ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    idna==3.10 from file:///opt/mirror/idna-3.10-py3-none-any.whl (from -r projects/ipcheck/requirements.txt (line 3)):
        Expected sha256 946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3
             Got        ad3da620d8f871a08ca427585bd2c600451d9c1c50ac696bb05010c9faf1aa4f

you should see pip refuses to install and names the package whose hash does not match.

defender tip Without --require-hashes this install succeeds silently. The hash check is the only thing between you and a modified library.

05

See what changed

A wheel is a zip file. Compare the file lists of the two copies.

learner@py-lab
diff <(unzip -Z1 /opt/wheelhouse/idna-3.10-py3-none-any.whl | sort) <(unzip -Z1 /opt/mirror/idna-3.10-py3-none-any.whl | sort)
5a6
> idna/_telemetry.py
unzip -p /opt/mirror/idna-3.10-py3-none-any.whl idna/_telemetry.py
import os
ENDPOINT = 'http://198.51.100.88/collect'

you should see An extra file that points at an outside address. In a real incident you would report the mirror and preserve the file.

Trusting
requests
idna

Any version, any file. Whatever the index serves today gets installed.

Pinned and hashed
idna==3.10 \
    --hash=sha256:...

One reviewed file. A swapped file fails the install.

What a defender takes away

  1. 1Give every tool its own virtual environment.
  2. 2Pin versions and hashes, and install with --require-hashes.
  3. 3A hash mismatch is an incident to report, not an error to work around.

Questions

0 of 3 solved

Answer each one from what you find on your lab machine.

  1. Show hint

    pip list or pip show requests

  2. Show hint

    Read the block pip prints under the hash error.

  3. Show hint

    Compare the two file lists with unzip -Z1.