fix(core): reject a DELETE or an UPDATE whose WHERE clause cannot reach the provider - #24657
Open
michaelsembwever wants to merge 3 commits into
Open
fix(core): reject a DELETE or an UPDATE whose WHERE clause cannot reach the provider#24657michaelsembwever wants to merge 3 commits into
michaelsembwever wants to merge 3 commits into
Conversation
github-actions
Bot
added
core
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #24657 +/- ##
==========================================
+ Coverage 81.36% 81.70% +0.33%
==========================================
Files 1117 1127 +10
Lines 397872 415707 +17835
Branches 397872 415707 +17835
==========================================
+ Hits 323725 339648 +15923
- Misses 55229 56112 +883
- Partials 18918 19947 +1029 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
martin-g
reviewed
Aug 27, 2026
michaelsembwever
force-pushed
the
fix/dml-where-subquery-not-supported
branch
from
September 7, 2026 09:16
38b7a33 to
648f444
Compare
…ch the provider A DELETE or an UPDATE whose WHERE clause holds an IN or an EXISTS subquery changed every row of the target table, and reported the whole table as affected. The optimizer rewrites the subquery into a semi join, so the condition leaves the `Filter` nodes that `extract_dml_filters()` reads. The provider then received an empty filter list, which is the encoding for "no WHERE clause", and applied the statement to all rows. An always-false WHERE clause reached the provider the same way. The simplifier folds the predicate into an empty relation, so again no filter survived, and `DELETE FROM t WHERE false` emptied the table. Add `classify_dml_input()`, which walks the input plan of a DELETE or an UPDATE before the provider hook runs. An empty relation means that no row matches, so the statement reports a count of 0. A join, a predicate on another table, or any other node that restricts or multiplies rows raises a "not implemented" error. The hook stays untouched in both cases, so a provider that writes to durable storage cannot lose rows. `classify_dml_input()` and `extract_dml_filters()` both need the target table and its aliases, so `collect_dml_target_refs()` collects that set once and each DML arm passes it to both. `classify_dml_input()` also states the LIMIT gap that it leaves open: `DELETE FROM t LIMIT n` deletes every matching row, because a `Limit` carries no predicate and reaches the provider as no filter. `UPDATE ... LIMIT` is already rejected by the SQL planner. The sqllogictest cases run a second time with `datafusion.optimizer.max_passes = 0`. No rule rewrites the subquery on that path, so the predicate reaches the provider and fails when the provider compiles it to a physical expression, and an always-false predicate reaches the provider and matches no row. That path was already safe, which is what makes the bug optimizer-dependent; the cases guard it. Assisted-by: Claude Code:claude-opus-5
Co-authored-by: Martin Grigorov <martin-g@users.noreply.github.com>
…unoptimized path fix(core): reject a DELETE or an UPDATE whose WHERE clause cannot reach the provider A DELETE or an UPDATE whose WHERE clause holds an IN or an EXISTS subquery changed every row of the target table, and reported the whole table as affected. The optimizer rewrites the subquery into a semi join, so the condition leaves the `Filter` nodes that `extract_dml_filters()` reads. The provider then received an empty filter list, which is the encoding for "no WHERE clause", and applied the statement to all rows. An always-false WHERE clause reached the provider the same way. The simplifier folds the predicate into an empty relation, so again no filter survived, and `DELETE FROM t WHERE false` emptied the table. Add `classify_dml_input()`, which walks the input plan of a DELETE or an UPDATE before the provider hook runs. An empty relation means that no row matches, so the statement reports a count of 0. A join, a predicate on another table, or any other node that restricts or multiplies rows raises a "not implemented" error. The hook stays untouched in both cases, so a provider that writes to durable storage cannot lose rows. `classify_dml_input()` and `extract_dml_filters()` both need the target table and its aliases, so `collect_dml_target_refs()` collects that set once and each DML arm passes it to both. `classify_dml_input()` also states the LIMIT gap that it leaves open: `DELETE FROM t LIMIT n` deletes every matching row, because a `Limit` carries no predicate and reaches the provider as no filter. `UPDATE ... LIMIT` is already rejected by the SQL planner. The sqllogictest cases run a second time with `datafusion.optimizer.max_passes = 0`. No rule rewrites the subquery on that path, so the predicate reaches the provider and fails when the provider compiles it to a physical expression, and an always-false predicate reaches the provider and matches no row. That path was already safe, which is what makes the bug optimizer-dependent; the cases guard it. Assisted-by: Claude Code:claude-opus-5
michaelsembwever
force-pushed
the
fix/dml-where-subquery-not-supported
branch
from
September 7, 2026 10:15
648f444 to
19a68c0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
#24654
Rationale for this change
See ticket.
What changes are included in this PR?
A
DELETEor anUPDATEwhoseWHEREclause holds anINor anEXISTSsubquery changed every row of the target table, and reported the whole table as affected. The optimizer rewrites the subquery into a semi join, so the condition leaves theFilternodes thatextract_dml_filters()reads. The provider then received an empty filter list, which is the encoding for "no WHERE clause", and applied the statement to all rows.An always-false
WHEREclause reached the provider the same way. The simplifier folds the predicate into an empty relation, so again no filter survived, and aDELETE FROM t WHERE falseemptied the table.Add
classify_dml_input(), which walks the input plan of aDELETEor anUPDATEbefore the provider hook runs:The hook stays untouched in every rejected case, so a provider that writes to durable storage cannot lose rows.
Are these changes tested?
Only with the tests provided in this patch, which are based on the assumptions made in the ticket description.
Are there any user-facing changes?
?