Skip to content

test(reports): freeze the clock in the stale retry boundary test - #44651

Merged
EnxDev merged 1 commit into
apache:masterfrom
EnxDev:enxdev/fix/flaky-stale-retry-claim-test
Sep 25, 2026
Merged

EnxDev merged 1 commit into
apache:masterfrom
EnxDev:enxdev/fix/flaky-stale-retry-claim-test

Conversation

@EnxDev

@EnxDev EnxDev commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

SUMMARY

test_stale_retry_recovers_at_max_delay[*-sqlite-3599-False] fails intermittently in CI, for example on #44650 (run):

AssertionError: assert (ExecutionClaim(initial_state='Retrying') is not None) is False

The test builds last_eval_dttm from utc_now().replace(microsecond=0), which can be up to a second behind the real clock. _claim_execution then measures the row's age with datetime.utcnow(). The effective age of the "3599 seconds" case is therefore 3599 s, plus the dropped fraction, plus however long the test takes. On a slow runner that reaches the 3600 s delay, the claim goes through, and the test fails.

The fix freezes the clock at the test's timestamp for the test body, so both sides read the same time. Only the test changes.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A

TESTING INSTRUCTIONS

pytest tests/unit_tests/commands/report/execution_claim_test.py

To reproduce the flake, add time.sleep(1.1) just before the claim_execution(...) call in the test. On master the two 3599-False cases fail with the CI error above. With this change they still pass.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

🤖 Generated with Claude Code

test_stale_retry_recovers_at_max_delay sets last_eval_dttm from a
microsecond-truncated timestamp, while the claim measures its age with
datetime.utcnow(). The truncation plus the test's own runtime could push
the 3599 second case past the 3600 second delay on a slow runner, so it
was claimed and the test failed intermittently in CI. Freeze the clock at
the test's timestamp so both sides agree.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@bito-code-review

bito-code-review Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Agent Run #cc31f0

Actionable Suggestions - 0
Additional Suggestions - 1
  • tests/unit_tests/commands/report/execution_claim_test.py - 1
    • Frozen clock vs utcnow · Line 397-397
      This test now relies on `freeze_time` governing how `_claim_execution` reads the clock, but the callee uses the deprecated `datetime.utcnow()` (superset/commands/report/execution_claim.py:172). freezegun 1.5.1 still patches `utcnow`, so this works today; as CPython removes `utcnow`, this coupling can silently un-freeze the staleness check and make the test flaky again. Prefer a timezone-aware clock read in the callee.
Review Details
  • Files reviewed - 1 · Commit Range: ab760d8..ab760d8
    • tests/unit_tests/commands/report/execution_claim_test.py
  • Files skipped - 0
  • Tools
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers an incremental AI Review.

  • /review full - Manually triggers a full AI Review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@codecov

codecov Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.10%. Comparing base (3bd6e10) to head (ab760d8).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #44651      +/-   ##
==========================================
- Coverage   81.10%   81.10%   -0.01%     
==========================================
  Files        2955     2955              
  Lines      178385   178385              
  Branches    41336    41336              
==========================================
- Hits       144680   144678       -2     
- Misses      30996    30998       +2     
  Partials     2709     2709              
Flag Coverage Δ
hive 36.80% <ø> (ø)
mysql 56.00% <ø> (ø)
postgres 56.00% <ø> (ø)
presto 38.70% <ø> (ø)
python 85.38% <ø> (-0.01%) ⬇️
sqlite 55.72% <ø> (ø)
unit 77.67% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@EnxDev
EnxDev merged commit 83731ce into apache:master Sep 25, 2026
81 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants