Skip to content

Warn when db clean skips only some rows of a batch - #73568

Merged
vincbeck merged 4 commits into
apache:mainfrom
aws-mwaa:ferruzzi/db-clean/compare-archived
Oct 5, 2026
Merged

vincbeck merged 4 commits into
apache:mainfrom
aws-mwaa:ferruzzi/db-clean/compare-archived

Conversation

@ferruzzi

Copy link
Copy Markdown
Contributor

The guarded DELETE added in #66350 logged a warning only when deleted == 0, which fires when an entire batch is skipped by the skip_if_referenced guard. The partial case is both the likely one and the harmful one: the archive has already committed a copy of every row the SELECT found, so a row the guard skips is archived and still live, and export-archived will emit a phantom copy of it.

The count is gated on skip_if_referenced so tables that cannot skip rows don't pay the extra COUNT(*) per batch. That also drops a latent wrong message, since an unguarded table reaching deleted == 0 previously claimed its rows were still referenced by another table when nothing referenced them.

Raised by @ramitkataria while reviewing #66350.
Blocks #73417 and should be merged first


Was generative AI tooling used to co-author this PR?
  • [ x ] Yes (please specify the tool below)

Test case generated by: [Claude Sonnet 5] following the guidelines

The guarded DELETE added in apache#66350 logged a warning only when ``deleted == 0``, which fires when an entire batch is skipped by the skip_if_referenced guard.  The partial case is both the likely one and the harmful one: the archive has already committed a copy of every row the SELECT found, so a row the guard skips is archived and still live, and export-archived will emit a phantom copy of it.
@ferruzzi
ferruzzi requested a review from potiuk September 24, 2026 22:24
@vincbeck
vincbeck merged commit 747f69b into apache:main Oct 5, 2026
303 of 304 checks passed
@vincbeck
vincbeck deleted the ferruzzi/db-clean/compare-archived branch October 5, 2026 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants