Move fixture cache from FixtureDef to SetupState by seifertm · Pull Request #14770 · pytest-dev/pytest

seifertm · 2026-07-23T10:16:40Z

This PR removes FixtureDef.cached_result in favor of a cache that is attached to SetupState. FixtureRequest received methods to manipulate the fixture cache.

The PR is easiest reviewed commit by commit.

Open issues

Previously, fixture values were cached as part of FixtureDef objects and the cache entries contained a cache_key. The cache key is either None for non-parametrized fixtures or SubRequest.param. This allows invalidating cache entries when fixture parameters change.

Hoisting up the fixture cache to SetupState technically makes FixtureDef part of the cache key. This means we should use a two-dimensional index (e.g. tuple[FixtureDef, param]) to store entries in the fixture cache. Unfortunately, this is tricky, because TopRequest doesn't know about the current fixture param and therefore doesn't know the cache key.

This work has been done as part of the pytest sprint 2026, sponsored by Omicron.

RonnyPfannschmidt · 2026-07-23T11:16:41Z

Good find, thanks @RonnyPfannschmidt !

I pushed 5 additional commits. The tracking of the currently active request parameter for --setup-show is no longer attached to FixtureDef. Instead, --setup-show uses the existing SetupState.fixture_cache, which already contains the active fixture param as a cache key. The variable naming was changed from "cache_key" to "param" or "active_param".

In order to distinguish between "no param" and "param=None", I introduced a sentinel value. This prevents --setup-show from printing my_fixture[None] for unparametrized fixtures.

bluetech · 2026-07-23T11:32:13Z

Previous stalled PR you might want to use as a reference/comparison: #14104

seifertm · 2026-07-23T16:00:52Z

Thanks @bluetech! This topic was raised during the pytest sprint and I wasn't aware that there have been previous attempts at this. I'll check out the PR.

seifertm · 2026-07-24T09:51:06Z

Previous stalled PR you might want to use as a reference/comparison: #14104

The linked PR appears to rework fixture evaluation logic and move that to runner.py. This PR reduces the mutability of FixtureDef by moving the fixture cache from FixtureDef to SetupState. On first sight, both PRs seem to be complementary rather than competing with each other.

nicoddemus · 2026-07-24T14:09:47Z

-        # If the fixture was executed, the current value of the fixture.
-        # Can change if the fixture is executed with different parameters.
-        self.cached_result: _FixtureCachedResult[FixtureValue] | None = None
        self._finalizers: Final[list[Callable[[], object]]] = []


Unrelated to the PR, more of a question (@RonnyPfannschmidt): what about the finalizers? We likely would also want finalizers to be hosted per-thread?

finlizers should also move to setupstate

nicoddemus · 2026-07-24T14:11:39Z

+        # Importing the appropriate types from the fixtures module lead to circular
+        # imports, so we leave the cache untyped for now
+        self.fixture_cache = {}  # type: ignore[var-annotated]


I think having the type checking here is important, let's move the type definitions we need to a new fixtures_types module, containing only the type definitions, which allows us to import them here without cycles.

I agree that it would be important to allow type-safe use of the fixture cache. I noticed that we cannot simply type the fixture cache as dict[FixtureDef[FixtureValue], FixtureValue], because FixtureValue is only bound once to the whole dict, but it can be different for each entry.

Therefore, I ended up creating a FixtureCache class that provides a type-safe API for its entries. The class lives in a dedicated fixture_cache module. I could simply move the cache accessors from FixtureRequest to the new class, so the change doesn't even introduce an extra layer of indirection.

see commit 9e2a891

Let me know what you think of that.

nicoddemus · 2026-07-24T14:12:34Z

    )


+def test_suppress_capturing(pytester: Pytester) -> None:


Why is this test defined here?

It was added to make the CI pass. The codecov job seems to fail, if the coverage of the PR is less than 100%. Apparently this also applies to files that are touched by the PR, but didn't have full coverage. This test specifically covers the negative branch of this if statement.

INTODAN · 2026-07-28T15:05:21Z

Hello!
Can you add which existing issues this PR might solve? For example the stalled PR #14104 specifies multiple existing issues it solves, such as #9287

Thanks!

seifertm · 2026-08-04T18:30:25Z

I haven't forgotten about this PR and will try to address the remaining comments this week.

…of FixtureDef.

Use _invalidate_fixture_cache instead.

… to FixtureRequest.

…Def to SubRequest

…the result is successful or an exception.

…he responsibility of FixtureRequest.

… traceback

…he_value and _cache_exception, respectively

…of SetupState, instead of FixtureDef.

…it about the contents of the cache key.

…hing fixture results to distinguish between *no param* and *None*.

…arams for --setup-show.

…ty in pytest_fixture_post_finalizer.

…ather than fetching it from the cache.

for more information, see https://pre-commit.ci

Co-authored-by: Bruno Oliveira <bruno@soliv.dev>

This allows typed use of SetupState.fixture_cache without import cycles. All of the methods of the new FixtureCache class were already present in FixtureRequest, so this refactoring makes FixtureRequest more manageable without introducing and extra layer of indirection.

seifertm · 2026-08-09T18:00:28Z

Hello! Can you add which existing issues this PR might solve? For example the stalled PR #14104 specifies multiple existing issues it solves, such as #9287

Thanks!

@INTODAN This PR doesn't resolve an existing issue. It is a refactoring towards enabling multi-threaded runs in pytest. The scope of this PR has been limited deliberately to allow a proper review. I guess the best issue to reference is #13768
The approach has been discussed with some of the core devs at this year's pytest sprint. The larger idea is to move mutable state into SetupState and give each thread their own SetupState.

seifertm · 2026-08-09T18:01:01Z

Added the FixtureCache class and rebased to latest main.

RonnyPfannschmidt · 2026-08-19T21:24:31Z

Looking at this in the context of the long-standing invocation-scoped fixtures request (#1681, POC in #12816), since that feature is blocked on exactly the state this PR touches. Two notes.

On the "Open issues" section: key by owner, not by param.

You want tuple[FixtureDef, param] and are blocked because TopRequest doesn't know the current param. Consider making the second dimension the owner of the instance instead:

for the node-anchored scopes the owner is the scope node, which SubRequest already computes in __init__,
and TopRequest knows its own node, so the lookup is always answerable.

param then stays inside the entry for miss detection, exactly as you have it now. That gets you the two-dimensional index without the thing that blocked you, and it happens to be the dimension a future "invocation" scope needs as well — there the owner is the requesting request rather than a collection node, because such an instance is not anchored on the tree at all.

Complementarity with #14104.

#14104 moves the other half of the per-instance state: FixtureDef._finalizers plus the teardown ordering graph, into SetupState. This PR drops cached_result and keeps _finalizers; #14104 drops _finalizers and keeps cached_result. Together they would empty FixtureDef of instance state entirely, which is the actual precondition for any scope that can have more than one live instance per definition. They will conflict textually, so we should pick an order rather than let them race. Note also that #14104's SetupState._active_fixtures is keyed by FixtureDef alone and asserts on re-entry, so the same owner dimension applies there.

For the record: this PR on its own does not yet unblock #12816. I ported the invocation-scope patch onto this branch and it still hits assert not self._finalizers in FixtureDef.execute, because the finalizer list stays on FixtureDef and FixtureCache remains one entry per fixturedef. That is not a complaint about the scope of this PR — just a note that the owner-keyed index above is what turns it from a relocation into the foundation.

Thanks for picking this up at the sprint, the direction is right.

psf-chronographer Bot added the bot:chronographer:provided (automation) changelog entry is part of PR label Jul 23, 2026

seifertm force-pushed the move_fixture_cache_from_fixturedef_to_setupstate branch from c54967f to 6678e5d Compare July 23, 2026 10:19

seifertm marked this pull request as ready for review July 23, 2026 10:19

RonnyPfannschmidt requested changes Jul 23, 2026

View reviewed changes

The-Compiler force-pushed the main branch from cb2ee5b to ced9022 Compare July 24, 2026 09:22

seifertm force-pushed the move_fixture_cache_from_fixturedef_to_setupstate branch from cf3854d to 61286bd Compare July 24, 2026 09:34

seifertm commented Jul 24, 2026

View reviewed changes

Comment thread src/_pytest/fixtures.py Outdated

nicoddemus requested changes Jul 24, 2026

View reviewed changes

seifertm force-pushed the move_fixture_cache_from_fixturedef_to_setupstate branch from 1c6a7d2 to 9720a75 Compare July 24, 2026 14:39

seifertm added 16 commits August 9, 2026 17:54

fixtures, runner: Store cached fixture values in SetupState, instead …

1d92a15

…of FixtureDef.

fixtures: Convert fixture results to NamedTuples

e19500b

fixtures: Extracted method to invalidate the cached result of a fixture.

41fad39

fixtures: Don't allow setting the cached fixture value to None.

99f6bc2

Use _invalidate_fixture_cache instead.

fixtures: RequestFixtureDef.cache_key returns the correct cache key.

7cdb9ab

fixtures: Move methods for manipulating fixture cache from FixtureDef…

e6b41d3

… to FixtureRequest.

fixtures: Move determination of fixture value cache keys from Fixture…

ae3df0d

…Def to SubRequest

fixtures: Split up methods for cache population depending on whether …

82e61b3

…the result is successful or an exception.

fixtures: Selecting the correct cache key for fixture values is now t…

c93b9f4

…he responsibility of FixtureRequest.

fixtures: FixtureRequest._set_cached_exception no longer requires the…

310ec4e

… traceback

fixtures: Rename _set_cached_result and _set_cached_exception to _cac…

bf6ed1c

…he_value and _cache_exception, respectively

fixtures: Convert FixtureRequest._cache_key into a property

059dbdd

docs: Add news fragment

383a280

setuponly, runner: Move track currently active fixture param as part …

9010d17

…of SetupState, instead of FixtureDef.

fixtures: Rename cache_key to active_param or param to be more explic…

9253de0

…it about the contents of the cache key.

fixtures: Use sentinel value for non-existant fixture params when cac…

1d1c807

…hing fixture results to distinguish between *no param* and *None*.

seifertm and others added 10 commits August 9, 2026 17:54

fixtures, setuponly: Reuse SetupState.fixture_cache to track active p…

4fd8b2c

…arams for --setup-show.

fixtures: Address typing error for generic namedtuples in Python 3.10

230df16

fixtures: Replace mentions of FixtureDef.cached_result in the docs

776c973

setuponly: Add test for suppressing capture.

35526b0

setuponly: Adds assertion that the cached fixture value cannot be emp…

49cf935

…ty in pytest_fixture_post_finalizer.

setuponly: _show_fixture_action directly receives the fixture param r…

51f3857

…ather than fetching it from the cache.

[pre-commit.ci] auto fixes from pre-commit.com hooks

426890a

for more information, see https://pre-commit.ci

changelog: Clarify wording in news fragment

ba644a8

Co-authored-by: Bruno Oliveira <bruno@soliv.dev>

pyproject: Ignore additional TYPE_CHECKING expressions from coverage.

5dc60c7

seifertm force-pushed the move_fixture_cache_from_fixturedef_to_setupstate branch from 9720a75 to 9e2a891 Compare August 9, 2026 17:44

seifertm requested review from RonnyPfannschmidt and nicoddemus August 9, 2026 18:01

Uh oh!

Uh oh!

Conversation

seifertm commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Open issues

Uh oh!

RonnyPfannschmidt Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

seifertm Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bluetech commented Jul 23, 2026

Uh oh!

seifertm commented Jul 23, 2026

Uh oh!

seifertm commented Jul 24, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nicoddemus Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

RonnyPfannschmidt Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

nicoddemus Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

seifertm Aug 9, 2026

Choose a reason for hiding this comment

Uh oh!

nicoddemus Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

seifertm Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

INTODAN commented Jul 28, 2026

Uh oh!

seifertm commented Aug 4, 2026

Uh oh!

seifertm commented Aug 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

seifertm commented Aug 9, 2026

Uh oh!

RonnyPfannschmidt commented Aug 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

seifertm commented Jul 23, 2026 •

edited

Loading

seifertm Jul 24, 2026 •

edited

Loading

seifertm Jul 24, 2026 •

edited

Loading

seifertm commented Aug 9, 2026 •

edited

Loading