pytest-http-cache

Record real HTTP traffic the first time your tests run, replay it from disk every time after. Tests run offline, and a third party changing their API can't break your suite until you decide to re-record.

Works with httpx, requests, and aiohttp — and doesn't depend on any of them. Traffic is captured by a local record/replay proxy, so requests go through each library's real stack and its own proxy support instead of being mocked.

Install

pip install pytest-http-cache

The plugin registers itself; there is nothing to enable.

Use

def test_users():
    users = httpx.get("https://api.example.com/v1/users").json()
    assert users[0]["name"] == "Ada"

First run: the request goes out and the response is written to .http-cache/. Every later run: the response is served from disk, no network involved. No fixture, decorator, or import is needed — installing the plugin is enough, and http_cache_mode = off opts out.

The cache mirrors the URL, so you can read and diff it:

.http-cache/
  api.example.com/
    v1/users/
      GET.response.http          # raw wire format: status line, headers, blank line, body
      cacheconfig.json           # optional, see "Varying the key"

Configure

# pytest.ini / pyproject.toml
[pytest]
http_cache_dir = .http-cache      # where responses live
http_cache_mode = auto            # auto | record | replay | refresh | off
http_cache_local = false          # cache localhost / 127.0.0.1 / 0.0.0.0 traffic too
http_cache_exclude_hosts =        # hosts to never cache

Mode	Behaviour
auto (default)	Replay what's cached, record what isn't
record	Always call upstream, always overwrite
replay	Replay only; a miss fails the test (use in CI to prove the suite is offline)
refresh	Drop the entry, then re-record
off	Pass everything through, store nothing

Anything can be set on the command line, and per test:

pytest --http-cache-mode=refresh
pytest --http-cache-clear                          # wipe the cache first
pytest --http-cache-clear=https://api.example.com  # wipe one subtree first

@pytest.mark.http_cache(mode="record", local=True)
def test_against_a_live_api(): ...

Local requests

localhost, 127.0.0.1, 0.0.0.0, and ::1 are passed straight through and never cached — your own test server should stay live. Set http_cache_local = true (or the marker) when you do want them recorded.

Clearing the cache

The store is plain files with no index, so all of these are equivalent and safe:

rm -rf .http-cache                          # everything
rm -rf .http-cache/api.example.com/v1/users # one endpoint
pytest --http-cache-clear

def test_fresh(http_cache):
    http_cache.clear("https://api.example.com/v1/users")

Anything deleted is simply a miss on the next run.

Varying the key

By default the key is method + host + path — query parameters and headers are ignored, so ?page=1 and ?page=2 share one entry. When an endpoint really does vary, opt in per directory:

def test_paged(http_cache):
    http_cache.vary("https://api.example.com/v1/users", query=["page"])

or write it by hand after the first run:

{"vary": {"query": ["page"], "headers": ["Accept-Language"]}}

Each combination is then stored as GET.<hash>.response.http beside the default entry, and cacheconfig.json records what each hash stands for.

Certificate authority

HTTPS is intercepted with a CA generated on first use in ~/.cache/pytest-http-cache/ca, reused from then on. Point PYTEST_HTTP_CACHE_CA_DIR somewhere else for hermetic or read-only-home environments (CI containers, sandboxes).

The http_cache fixture

Member	Purpose
clear(url_pattern=None)	Delete everything, or a URL prefix / glob subtree
vary(url, query=[...], headers=[...])	Write vary rules for a URL's directory
has(url, method="GET")	Is this response cached?
path_for(url, method="GET")	Where the entry lives
hits / misses	Counters for the session
root / mode / settings	Resolved configuration

How it works

A mitmproxy instance starts once per session on an ephemeral loopback port. The session's proxy and CA environment variables point client libraries at it; aiohttp additionally gets a small shim because ClientSession ignores the environment by default (trust_env=False). On a hit the proxy answers from disk without opening an upstream connection; on a miss it forwards the request and writes the response. See SPEC.md for the design.

Dependencies

pytest and mitmproxy (>=11, tested against 12.2.3). Client libraries are never imported unless you use them.

Tested on CPython 3.12–3.13, Linux and macOS. Windows is not supported: cache directories are named host:port, and : is not a legal path character there.

Status

Working end to end: httpx, requests, and aiohttp all record and replay over both HTTP and HTTPS. See SPEC.md for the design. Streaming responses (server-sent events) and WebSocket traffic pass through uncached.