Repository navigation
fix(semantic-layer): block deletion of sources with dependent assets - #45071
mikebridge wants to merge 7 commits into
Conversation
Block semantic-view and layer hard deletes while live charts, dashboards, or schedules still reference their views. Count all dependents in one snapshot, but name only assets visible through their list-access policies. The guard deliberately remains best-effort against concurrent chart creation or repointing, and active schedule references are scanned without a new index.
Parse typed native filter targets after a coarse SQL prefilter so delete guards count filter-only dashboards without confusing legacy table IDs. Normalize alert and report discriminator values and document the 409 contract. This remains a best-effort guard without new locking or indexes.
✅ Deploy Preview for superset-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #45071 +/- ##
==========================================
+ Coverage 82.64% 82.75% +0.10%
==========================================
Files 3007 3013 +6
Lines 187318 191989 +4671
Branches 43399 44509 +1110
==========================================
+ Hits 154811 158878 +4067
- Misses 29742 30076 +334
- Partials 2765 3035 +270
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
@aminghadersohi @rebenitez1802 this is ready for review: deleting a semantic layer or view is now blocked while charts, dashboards (including native filters that target it) or alerts/reports still depend on it, with a message listing the dependents. |
CodeAnt PR Risk: Low Risk
Assessed commit: |
fitzee
left a comment
There was a problem hiding this comment.
Verdict: COMMENT. I found no blocking bugs. The dependency guard does what the PR says on all three delete entry points. One design question is worth settling before merge: archived assets don't block the delete. The rest are follow-ups and nits.
Reviewed head 729053a6. The PR's 153 unit tests pass locally. I also ran _dependent_assets through the real ORM session, not the raw Connection the tests patch in, against SQLite, Postgres 16 and MySQL 8. Both the admin and non-admin visibility paths compile and return the expected (total, dependents, inaccessible_count) on all three. The yield_per streaming and SUM(CASE … IN (subquery)) work on each backend.
What I checked and found correct:
- Charts: matched on
datasource_type == "semantic_view"plus the id. A table chart with the same numeric id does not block. - Dashboards: caught both through chart membership and through typed native-filter / display-control targets.
- Reports and alerts: active ones block, whether they point at a chart or a dashboard.
lower()on theAlert/Reportenum values gives the documented lowercase types. - Layer delete: covers every view in the layer, which the ORM/FK cascade would otherwise remove.
- Bulk delete: rejects the whole batch if any one view has dependents.
- Visibility: names come only from the list-API base filters (
ChartFilter/DashboardAccessFilter/ReportScheduleFilter). Hidden dependents are only counted. - Queries: no N+1. It is one count statement plus one limited fetch.
Should address / discuss
-
Archived charts and dashboards don't block a hard delete, and restoring them later gives broken assets.
superset/commands/semantic_layer/delete.py:123,131,137filterdeleted_at IS NULL. The view delete is a hard delete.BaseRestoreCommand.validate(superset/commands/restore.py:76-104) does not re-check the datasource.- Scenario (with
SOFT_DELETEon): archive chart 7, which uses view 42.DELETE /api/v1/semantic_view/42returns 200.POST /api/v1/chart/<uuid>/restorethen succeeds, and the restored chart has no datasource. - The same applies to an archived dashboard whose native filter targets the view.
- Precedent: deleting a database blocks on soft-deleted datasets and tells the operator to purge them first (
superset/commands/database/delete.py:82-102). That precedent is FK-driven, but users see the same contract: "archived = recoverable". - The limitation is documented (
docs/.../importing-exporting-datasources.mdx:158). Still, it would be better either to count archived dependents (perhaps in a separate field, with a purge hint) or to make restore reject a chart whose semantic view is gone.
- Scenario (with
-
The frontend ignores the structured 409.
- Single view (
superset-frontend/src/pages/DatasetList/index.tsx:823-839) and layer (superset-frontend/src/pages/DatabaseList/index.tsx:784-797): the toast shows only the generic "Semantic source has dependent assets and cannot be deleted." It does not showtotalordependents, so the user can't tell what to fix. - Bulk (
DatasetList/index.tsx:1458-1490): a mixed selection of datasets and semantic views archives the datasets and then gets a 409 on the views. The user sees only "There was an issue deleting the selected datasets", with no hint that the views were refused or why. - Fine as a follow-up. A count in
message(e.g. "used by 3 charts, 1 dashboard") would help with no UI change.
- Single view (
Nits
delete.py:98-113: every view or layer delete scans and JSON-parses each live dashboard whose metadata containssemantic_view. It does this even when the view has no charts. It's acceptable for a delete-only path (the bot's perf flag is real but not blocking). If it ever matters, skip the scan when nothing else matched and no dashboard uses the view type.delete.py:61-68:str.isdecimal()accepts non-ASCII digits ("٤٢"→ 42), and thetry/except ValueErrorcan't be reached after that check. Harmless over-match.raw_id.isascii() and raw_id.isdigit()would be tighter.delete.py:71-95re-implements the target walk insuperset/semantic_layers/import_export.py:44(dashboard_targets), with lenient instead of raising semantics. A shared walker with astrictflag would keep the two from drifting when a third control list is added.delete.py:231:ORDER BY type, idsortsalertbeforechart. A source with 20 or more alerts lists only alerts and never the charts, which are usually what the user needs to act on.superset/semantic_layers/api.py: the 409 OpenAPI block is pasted three times (around lines 487, 580 and 1041), and the handler three times (lines 528, 629 and 1079). A sharedcomponents/responsesentry plus a helper would keep them in sync.- Status code: 409 is reasonable. Note that the closest existing case, a database with datasets attached, returns 422 (
superset/databases/api.py:652). Clients that treat "has dependents" as 422 would need a second case. - Tests:
delete_test.pypatchesdb.session.executewith a rawConnection(e.g. around line 343). That skips ORM execution, so the soft-deletedo_orm_executelistener and ORMyield_perhandling never run in tests. One test that uses thesessionfixture directly would cover the production path. I did this locally and it passes.
The race between check and delete (TOCTOU), the LIKE prefilter evasion and the unindexed schedule lookup are acknowledged in the PR body. I agree they're acceptable for a best-effort guard.
aminghadersohi
left a comment
There was a problem hiding this comment.
Two non-blocking test gaps inline; nothing here blocks.
Address Fitz's review with full dependency counts, actionable mixed-bulk errors, chart-first examples and ORM-path coverage. Preserve archived-asset policy and annotate added Python bindings per the full-branch review.
|
Thanks for spelling out the archived-asset case. We're keeping archived dependents non-blocking for source deletion; dashboard restore is tracked separately in SC-125586. To be precise about charts: #44925 landed after the head you reviewed, and this update merges it. It makes chart version-history restore refuse a version whose datasource no longer exists. Restoring an archived chart from Trash is a different path, and it still succeeds. With #44926, also merged here, the restored chart's datasource-derived permissions are cleared because its source is gone, so it fails closed rather than keeping stale access. It still comes back as a chart without a datasource, though. So this doesn't resolve your example end to end; making Trash restore refuse or warn is a follow-up, not part of this PR. |
aminghadersohi
left a comment
There was a problem hiding this comment.
The _semantic_target_matches guards are now pinned; the Dashboard/ReportSchedule can_read else-arms still are not (replied in that thread), and one new test gap is inline. The red sharded-jest-tests (2) leg is the Chart.test.tsx export-menu flake, unrelated to this PR.
…gle bulk toast Extend the unreadable-dependent case with a dashboard and an active report so each can_read else-arm is pinned, and assert the mixed bulk delete raises exactly one danger toast. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for the follow-up review. Both remaining test gaps are addressed in
This follow-up changes only tests. |
SUMMARY
Reject hard deletion of semantic views or layers while live charts, live dashboards containing those charts or targeting the views in native filters or display controls, or active reports/alerts depend on them. Return 409 with
total, up to 20 dependents visible through corresponding list-access filters, andinaccessible_countfor the rest. Each dependent'sidis the integer key that the existing chart, dashboard and report REST endpoints accept, so a client can act on it directly. Schedule types are lowercase.The dashboard target check uses a SQL
LIKEprefilter followed by exact Python parsing of typed(datasourceType, datasetId)targets in both dashboard control lists; legacy id-only targets remain SQL datasets. Malformed dashboard metadata is ignored with debug logging of the dashboard ID only.Limitations:
datasourceTypetext can evade theLIKEprefilter; normal dashboard writers serialize the literal ASCII value.report_schedulereference lookup is intentionally unindexed, since it only runs on a protected delete; an index can follow if volumes warrant.BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Not applicable; API behavior only.
TESTING INSTRUCTIONS
Create a semantic view with a live chart, a dashboard containing it, a native-filter-only dashboard targeting it, a display-control-only dashboard targeting it, and active alert/report schedules. Verify single-view, bulk-view and layer deletes return 409 with lowercase types. Give a source editor no dashboard read access and verify the response counts but does not name it. Verify same-ID table/legacy targets, malformed metadata, archived dashboards/charts and inactive schedules do not block deletion; a string semantic-view ID does. Run the focused command and API tests and the semantic-layer coverage gate.
ADDITIONAL INFORMATION
SEMANTIC_LAYERS🤖 Generated with Claude Code