Skip to content

fix(reports): freeze time in the stale-retry boundary test to stop a CI race - #44586

Closed
rusackas wants to merge 1 commit into
masterfrom
fix/execution-claim-stale-retry-race
Closed

rusackas wants to merge 1 commit into
masterfrom
fix/execution-claim-stale-retry-race

Conversation

@rusackas

Copy link
Copy Markdown
Member

SUMMARY

test_stale_retry_recovers_at_max_delay intermittently fails on master's Python-Unit CI, most recently as test_stale_retry_recovers_at_max_delay[Alert-sqlite-3599-False].

The test asserts on either side of the 3600s stale-retry threshold: age=3599 should still be within the window (not eligible for reclaim), age=3601 should be past it (eligible). It captures now once and derives last_eval_dttm from it, but claim_execution → _claim_execution (superset/commands/report/execution_claim.py:172) compares that against a fresh datetime.utcnow() call internally, not the now the test passed in. Sitting exactly 1 second inside the boundary, any real delay between the test's setup and that internal check — plausible under a loaded CI run with pytest -n auto — pushes the effective age past 3600s and flips the assertion.

This is a test-only fix: datetime.utcnow() inside _claim_execution is the correct production behavior (staleness has to be measured against real wall-clock time, not a caller-supplied value), so the fix freezes time for the test instead of changing what the code compares against.

TESTING INSTRUCTIONS

pytest tests/unit_tests/commands/report/execution_claim_test.py -k test_stale_retry_recovers_at_max_delay -v

All 4 non-skipped parametrizations pass locally, including the exact case that flaked in CI (Alert-sqlite-3599-False).

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

🤖 Generated with Claude Code

…CI race

test_stale_retry_recovers_at_max_delay asserts on either side of the
3600s stale-retry threshold, but claim_execution compares against a
fresh datetime.utcnow() internally rather than the `now` passed into
it. Under load, any real delay between the test's setup and that
internal check pushes the effective age across the boundary and
flips the assertion — this is what flaked as
test_stale_retry_recovers_at_max_delay[Alert-sqlite-3599-False] on
master's Python-Unit CI. Freeze time for the test instead of relying
on the real clock staying still.

Co-Authored-By: Evan Rusackas <evan@rusackas.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@bito-code-review

bito-code-review Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Agent Run #935ce2

Actionable Suggestions - 0
Review Details
  • Files reviewed - 1 · Commit Range: e216123..e216123
    • tests/unit_tests/commands/report/execution_claim_test.py
  • Files skipped - 0
  • Tools
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers an incremental AI Review.

  • /review full - Manually triggers a full AI Review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@codecov

codecov Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.06%. Comparing base (390bf29) to head (e216123).

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #44586      +/-   ##
==========================================
- Coverage   81.06%   81.06%   -0.01%     
==========================================
  Files        2955     2955              
  Lines      178325   178325              
  Branches    41312    41312              
==========================================
- Hits       144565   144563       -2     
- Misses      31050    31052       +2     
  Partials     2710     2710              
Flag Coverage Δ
hive 36.79% <ø> (ø)
mysql 55.98% <ø> (+<0.01%) ⬆️
postgres 55.99% <ø> (+<0.01%) ⬆️
presto 38.69% <ø> (ø)
python 85.37% <ø> (-0.01%) ⬇️
sqlite 55.71% <ø> (ø)
unit 77.64% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@hainenber

Copy link
Copy Markdown
Contributor

This was fixed in #44651 so I'll be closing this PR off. If anything, this one has more comments :D

@hainenber hainenber closed this Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants