Skip to content

tests/test_ruleset_hint_tracking_branch.py: the re-run the hint offers is dated later than the first run, so a leg's seed made again, even in the same second, is another commit and fails "kept at its commit" where it passed by the clock (#922) - #943

Merged
brettheap merged 1 commit into
mainfrom
l3-922-seed-commit-date
Oct 18, 2026

Conversation

@brettheap

@brettheap brettheap commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Closes #922.

What changed, and why

One file, a test: tests/test_ruleset_hint_tracking_branch.py. The tools are untouched.

U007, "kept at its commit" cannot see a same-second commit. test_the_hint_names_one_exit_and_taking_it_finishes_the_run compares each leg's tip after the re-run with the first run's (tip(...) == before[repo]), and the re-run set no committer date. A seed commit made again in the same second, with the same tree, identity and message, is the same object, so a re-run that made a new seed for a leg it should have kept pushed the object the leg already held, and the comparison passed by the clock.

The re-run (operator) now carries GIT_COMMITTER_DATE 2031-01-01, from a new LATER constant beside SCAFFOLDED, so a seed made again has another id. The module docstring and the comment above the comparison say so.

Decisions the Direction leaves open:

  • The committer date only, no author date. The issue asks for a later GIT_COMMITTER_DATE "and an author date if it matters". It does not: the committer date is part of the commit id, so it alone gives the new commit another one. The mutant runs below pin the author date to the same value in both runs and the mutants still fail, which shows it.
  • A fixed 2031 date, as the dedicated reuse test does (tests/test_scaffold_reuse_seeded_legs.py:160-161), not a date read from the first run's commit. It is later than any first run until 2031, and a reader sees the same device in both tests.
  • The date is on every operator run of the test, not only the scaffold's. The family and adoption re-runs make no comparison with a first run, so it changes nothing for them; both pass with it (23 passed, no skips, git-filter-repo present). One operator dict is simpler than a branch on the case.

Left out of scope

The issue has no "Needs a ruling" or "Documented limit" section, so nothing was left out under that rule. Nothing outside the one test file was touched, and no other test file needed a change: tests/test_scaffold_reuse_seeded_legs.py and tests/test_dry_run_asks_remotes.py import refusing and template from this file and pass unchanged.

Tests

Each run under a stub gh first on PATH that exits 97, GH_TOKEN=bogus, an empty GH_CONFIG_DIR, one pytest process at a time, -p no:cacheprovider -o tmp_path_retention_policy=failed, wrapped in timeout 2400.

tests/test_ruleset_hint_tracking_branch.py (before the change, origin/main)  23 passed in 35.43s
tests/test_ruleset_hint_tracking_branch.py                                   23 passed in 36.85s
tests/test_scaffold_reuse_seeded_legs.py                                     30 passed in 40.52s
tests/test_dry_run_asks_remotes.py                                           32 passed in 12.76s
tests/test_agents_md_codes.py                                                47 passed in 1.93s
tests/test_cli_reference.py                                                  23 passed in 32.84s
tests/test_update_add_codes_documented.py                                     6 passed in 0.44s
tests/test_repo_hygiene.py                                                   81 passed in 2.47s
tests/test_no_auto_maintenance.py                                            60 passed in 12.16s

tests/test_shape_doctor.py was not run: no shape-doctor.py and no helper it imports changed.

Mutants

Both are made in a scratch clone of this branch's base, in scaffold-project.py's _seed_legs, and neither is in this PR. The failing run is -k "test_the_hint_names_one_exit and scaffold", six cases: scaffold-spec holds no seed after its first run (the first leg's push is the one refused), so the two spec cases cannot tell the mutants from the tool.

To fix "the same second" and not leave it to the clock, a run marked "pinned" exports GIT_AUTHOR_DATE and GIT_COMMITTER_DATE as the same instant to the whole pytest process, so the first run and the re-run commit at the same time to the second; run_script passes the process environment through. The new test's own LATER overrides the committer date of the re-run only.

Mutant A: reuse of a held seed switched off. if reused.get(role): becomes if False and reused.get(role):, so _take_held_seed is never taken and the leg's seed is made again and pushed as it was.

test dates result
old pinned (same second) 6 passed: the mutant passes
old not pinned 3 failed, 3 passed: the two spec cases and scaffold-code-trunk passed; scaffold-code-main and both scaffold-root cases made the seed again in another second and failed
new pinned (same second, author date equal too) 4 failed, 2 passed
new not pinned 4 failed, 2 passed

The new test fails at assert again.returncode == 0, again.stderr + again.stdout (line 427): the seed made again is another commit, its push over the held one is refused (! [rejected] HEAD -> main (fetch first)), and the tool exits 2 with the ruleset hint.

Mutant B: the seed made again and forced over the held one. Mutant A, and the leg's push becomes +HEAD:refs/heads/<branch>, so the push is accepted and the leg's tip moves.

test dates result
old pinned (same second) 6 passed: the mutant passes
old not pinned 2 failed, 4 passed: the two spec cases and the two scaffold-code cases passed; the two scaffold-root cases made the seed again in another second and failed
new pinned (same second, author date equal too) 4 failed, 2 passed
new not pinned 4 failed, 2 passed

The new test fails at assert tip(remotes / repo, tracking) == before[repo], repo (line 443), the "kept at its commit" assertion, on Atlas-spec.git: the tip is a different commit from the first run's. This is the assertion #922 is about. The rev-list --count --all assertion just above it passes under this mutant, since a forced push leaves one commit, so the tip comparison is the only one that sees it.

Fix mutant: the fix is the one line operator = {**refusing(template), **LATER}; without LATER the test is the old test, and the "old" rows above are its result.

Merge results

  • git merge-tree --write-tree HEAD origin/main (origin/main 64464d31064c062cc3b9dd6900493ef3d99ec533): clean, tree 938ca5e3392ffdde60be1eb626f6e3354e8a32a4.
  • Open pull requests that touch tests/test_ruleset_hint_tracking_branch.py (gh pr list --state open --limit 100 --json number,headRefOid,files, 7 open): none, so there is no other head to merge against.

Lane: openreposhape-3 (openRepoShape-3)

🤖 Generated with Claude Code

Summary by Sourcery

Ensure hint-tracking tests distinguish regenerated seed commits from reused commits by dating reruns later than the initial run.

Bug Fixes:

  • Make the hint-tracking test reliably detect regenerated seed commits during reruns, including when both runs occur within the same second.

Enhancements:

  • Clarify the test’s commit-identity assumptions and ensure rerun commits use a later committer date so unchanged legs are verified by commit identity rather than timing.

Tests:

  • Strengthen the ruleset hint tracking coverage for legs that should retain their original commits after taking the offered exit.

…s is dated later than the first run, so a leg's seed made again, even in the same second, is another commit and fails "kept at its commit" where it passed by the clock (#922)

`tip(...) == before[repo]` compared a leg's tip with the first run's, and the
second run set no committer date. A seed commit made again in the same second,
with the same tree, identity and message, is the same object, so with
`_take_held_seed` disabled the scaffold-code case still passed. The second
run now carries GIT_COMMITTER_DATE 2031-01-01, as the dedicated reuse test
does, so a re-made seed has another id.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Oct 18, 2026

Copy link
Copy Markdown
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

This test-only change makes the operator re-run use a fixed later GIT_COMMITTER_DATE, ensuring any incorrectly recreated seed has a different commit ID and causing the existing push or tip-preservation assertions to catch regressions that could previously pass when commits were created in the same second.

File-Level Changes

Change Details Files
Make the scaffold re-run produce distinct commit IDs so the test reliably detects unintended seed recreation.
  • Add a fixed later committer date for the operator re-run.
  • Apply the date to the existing operator environment without changing production tools or other tests.
  • Document why only the committer date is required and how it protects the tip comparison.
tests/test_ruleset_hint_tracking_branch.py
Clarify the test’s behavioral contract around preserving previously pushed leg commits.
  • Update the module documentation to explain the same-second commit collision addressed by the test.
  • Annotate the re-run setup and final tip assertion with the later-date rationale.
tests/test_ruleset_hint_tracking_branch.py

Assessment against linked issues

Issue Objective Addressed Explanation
#922 Make the end-to-end ruleset-hint test detect when a re-run creates a new seed commit instead of reusing the held commit, even if both commits would otherwise be made in the same second. ✅

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sonarqubecloud

Copy link
Copy Markdown

@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-3, 2026-10-18T16:37:31Z) RV943's independent review of #943 at 101cbe9f: READY. No MUST, no SHOULD. RV943 is lane 3's opus reviewer, not the writer.

Mutants. Re-made in RV943's own scratch copy of the head, with GIT_AUTHOR_DATE and GIT_COMMITTER_DATE pinned to one instant for the whole pytest process, and -k "test_the_hint_names_one_exit and scaffold":

  • A (if False and reused.get(role):): the old test gives 6 passed, so the mutant survives. The new test gives 4 failed, 2 passed, all four at :427 (assert 2 == 0, ! [rejected] HEAD -> trunk (fetch first)).
  • B (A plus +HEAD:refs/heads/...): the old test gives 6 passed, so the mutant survives. The new test gives 4 failed, 2 passed, all four at :443 (AssertionError: Atlas-spec.git: the tip differs from the first run's).
  • The two passes are the scaffold-spec cases, which hold no seed.
  • The author date stayed pinned, so the committer date alone is enough.
  • tests/test_scaffold_reuse_seeded_legs.py kills both mutants too: 1 failed, 29 passed each.
  • Control, the unmutated head with the dates pinned: 12 passed.

The 2031 date cannot make an assertion pass without testing anything.

  • No assertion reads a date: tip() reads the branch by name, and rev-list --count counts commits.
  • The tool matches a held seed by its tree only (scaffold-project.py:518).
  • Nothing in the repository exports GIT_COMMITTER_DATE.
  • The family and adoption re-runs take the date but assert nothing that reads it, and they pass with it and without it.

Tests, run one at a time under a stub gh that exits 97, GH_TOKEN=bogus and an empty GH_CONFIG_DIR:

  • tests/test_ruleset_hint_tracking_branch.py: 23 passed (no skips)
  • tests/test_scaffold_reuse_seeded_legs.py: 30 passed
  • tests/test_dry_run_asks_remotes.py: 32 passed

Merge: git merge-tree with origin/main 64464d31 is clean, tree 938ca5e3. No other open PR touches this file, its two importers, scaffold-project.py or tests/conftest.py.

NITs, none blocking:

  1. :326-328: "however fast" is misplaced. Only a re-run in the same second pushes the held object; a slower one is refused at :427 anyway.
  2. :324, :331: "LATER" holds only until 2031-01-01. The check still works, and the reuse test uses the same date.
  3. Outside this PR: scaffold-project.py:518-519's docstring says "two runs never make the same commit", which tests/test_ruleset_hint_tracking_branch.py: the "kept at its commit" check cannot catch a new seed commit made in the same second #922 disproves for runs in the same second. Wording only, for a follow-up.
  4. At landing, the explicit squash message carries the Lane line and the Sonnet Co-Authored-By line.

It lands under Brett Heap's v1.2.0 merge word once tests is green on the head.

Lane: openreposhape-3 (openRepoShape-3)

@sourcery-ai

sourcery-ai Bot commented Oct 18, 2026

Copy link
Copy Markdown

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 1 day and 6 hours by commenting @sourcery-ai review. Upgrade to get a review now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tests/test_ruleset_hint_tracking_branch.py: the "kept at its commit" check cannot catch a new seed commit made in the same second

1 participant