feat(datasets): add partition filter mapping model, migration and validation by hughhhh · Pull Request #43757 · apache/superset

hughhhh · 2026-09-01T12:57:28Z

Stack (review bottom-up):

feat(datasets): add partition filter mapping model, migration and validation #43757 — model, migration, serialization, validation
feat(datasets): mirror filters onto the partition column at query time #43758 — query rewrite + evaluator
feat(datasets): add partition mapping preview endpoint and editor plumbing #43759 — preview endpoint + editor plumbing
docs: document partition filter mapping #43760 — docs

SUMMARY

First of a four-PR stack adding partition filter mapping. This one adds the storage and the save-time validation; nothing reads the mapping yet.

Datasets on Hadoop-family engines are often partitioned on a technical column — an epoch integer, a lowercased region key — that no analyst would ever filter on. Unless a query carries a predicate on that column the engine scans every partition, and today the only workaround is hand-writing the predicate as custom SQL in a virtual dataset, which pushes a performance concern onto every chart author and takes the dataset out of the physical/syncable path.

Storage. Four columns, following the always_filter_main_dttm / currency_code_column precedent for "a dataset-level setting that names a column":

Column	Purpose
`tables.partition_column`	the physical partition column
`tables.partition_mapped_column`	explicit override; `NULL` follows `main_dttm_col`
`table_columns.partition_value_transform`	the `:value` expression
`table_columns.partition_transform_is_monotonic`	gates range mirroring (see PR 2)

Effective mapped column is partition_mapped_column or main_dttm_col, which makes "re-pointing the default datetime column moves the mapping with it, unless overridden" fall out of the model rather than needing code.

The monotonic flag is NOT NULL DEFAULT false rather than a nullable tri-state, matching normalize_columns — a nullable boolean invites if x: bugs where None and False need distinguishing and don't get it.

A JSON blob inside tables.extra was considered and rejected: extra is a user-editable free-text box in the dataset editor, buildExtraJsonObject rebuilds column.extra from a hardcoded key list on every save, and there is no validation layer for it today. Every comparable setting shipped in the last year chose a real column.

Validation runs in two tiers, because the PRD wants a mapping to "stay inactive until it parses" while some errors must still hard-block:

Blocks the save: unknown columns, a column mapped onto itself, Jinja in the transform, non-deterministic functions.
Saves, mapping inactive: unparseable transform, transform missing :value, no transform.

Two details worth a look:

The self-mapping check validates the effective mapped column. Checking only the explicit override misses the case an owner actually hits — pointing partition_column at the column that is already main_dttm_col.
SQLStatement.get_niladic_functions is added because the denylist can't be purely name-based: on Hive and Impala unix_timestamp() means "now" while unix_timestamp(x) — the canonical transform for this feature — is pure. Note sqlglot's Hive dialect already resolves the zero-arg form to CURRENT_TIMESTAMP; this is the backstop for dialects that don't normalize.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A — no UI in this PR.

TESTING INSTRUCTIONS

pytest tests/unit_tests/connectors/sqla/partition_mapping_test.py \
       tests/unit_tests/datasets/partition_mapping_serialization_test.py \
       tests/unit_tests/sql/parse_tests.py \
       tests/unit_tests/commands/dataset/update_test.py \
       tests/unit_tests/datasets/commands/export_test.py

Migration, verified on sqlite in both directions:

superset db upgrade     # adds all four columns to tables/table_columns + both _version shadow tables
superset db downgrade 1072de5ed955

The serialization tests assert the mapping survives every layer it passes through (export_fields, the data payload, PUT/import-v1 schemas, the API column lists) — a field missing from any one of them is dropped silently, which is the failure mode they exist to catch.

ADDITIONAL INFORMATION

Has associated issue:
Required feature flags: PARTITION_FILTER_MAPPING (off by default)
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided — four ADD COLUMNs with no backfill and no index; effectively instant on Postgres/MySQL, no downtime expected
Introduces new feature or API
Removes existing feature or API

netlify · 2026-09-01T13:01:41Z

✅ Deploy Preview for superset-docs-preview ready!

Name	Link
🔨 Latest commit	`b4676ae`
🔍 Latest deploy log	https://app.netlify.com/projects/superset-docs-preview/deploys/6abec311aa0b1b0008a2cdaf
😎 Deploy Preview	https://deploy-preview-43757--superset-docs-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

codecov · 2026-09-05T16:15:12Z

Codecov Report

❌ Patch coverage is 90.00000% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.72%. Comparing base (8ab7a85) to head (b4676ae).
⚠️ Report is 11 commits behind head on master.

Files with missing lines	Patch %	Lines
superset/commands/dataset/update.py	65.00%	10 Missing and 4 partials ⚠️
superset/connectors/sqla/partition_mapping.py	94.52%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #43757      +/-   ##
==========================================
+ Coverage   81.71%   81.72%   +0.01%     
==========================================
  Files        2978     2979       +1     
  Lines      181612   181790     +178     
  Branches    41982    42012      +30     
==========================================
+ Hits       148397   148575     +178     
+ Misses      30493    30491       -2     
- Partials     2722     2724       +2

Flag	Coverage Δ
hive	`36.66% <35.00%> (-0.01%)`	⬇️
javascript	`77.13% <ø> (+<0.01%)`	⬆️
mysql	`55.78% <42.77%> (-0.03%)`	⬇️
postgres	`55.79% <42.77%> (?)`
presto	`38.57% <35.00%> (-0.01%)`	⬇️
python	`85.91% <90.00%> (+0.02%)`	⬆️
sqlite	`55.51% <42.77%> (-0.03%)`	⬇️
unit	`78.85% <88.88%> (+0.03%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

bito-code-review · 2026-09-05T23:28:12Z

Code Review Agent Run #e33814

Actionable Suggestions - 0

Additional Suggestions - 4

superset/datasets/schemas.py - 1
- load_default clobbers persisted flag · Line 112-112
  
  `load_default=False` injects `False` into every deserialized column dict when the client omits the field. In `DatasetDAO._upsert_columns`, `setattr(col, key, value)` then overwrites a previously-persisted `True` on any partial column update (e.g. renaming a column), silently disabling range mirroring. Use `allow_none=True` to preserve the persisted value, matching `partition_value_transform`.
superset/commands/dataset/update.py - 1
- Unparseable transform blocks save · Line 452-468
  
  The docstring (lines 402-406) says an unparseable transform is Tier-2 and "deliberately let[s] the save through." But `validate_stored_expression` raises `SupersetSecurityException` for any unparseable non-Jinja expression (models.py 1010-1021), and `parse_skeleton` leaves no Jinja, so an unparseable transform always blocks the save. Guard with `is_parseable(transform, database.backend)` to honor the documented Tier-2 behavior.
superset/dashboards/schemas.py - 1
- CWE-200: Column Metadata Leak · Line 375-377
  
  These new fields are serialized by `DashboardDatasetSchema` but not added to `DASHBOARD_DATASET_INACCESSIBLE_FIELDS` in `superset/dashboards/api.py`. `_serialize_dashboard_dataset` strips that tuple for callers lacking datasource access, so `partition_column`/`partition_mapped_column`/`partition_filter_mapping` now leak column metadata to unauthorized users, unlike the other column fields. Add them to the tuple. ([CWE-200](https://cwe.mitre.org/data/definitions/200.html))
superset/migrations/versions/2026-09-04_00-00_a7f3c2e91d84_add_partition_filter_mapping.py - 1
- Stale migration docstring · Line 31-31
  
  The docstring says `Revises: 1072de5ed955`, but the actual `down_revision` is `7e2c9a4f1b83` (line 42). `1072de5ed955` is revised by a different migration (`39097d124752`), so this docstring is stale and misrepresents the chain. Align it with the real `down_revision` to avoid confusion for readers/tooling.

Review Details

Files reviewed - 18 · Commit Range: f21009f..f21009f
- superset/commands/dataset/update.py
- superset/config.py
- superset/connectors/sqla/models.py
- superset/connectors/sqla/partition_mapping.py
- superset/daos/dataset.py
- superset/dashboards/schemas.py
- superset/datasets/api.py
- superset/datasets/schemas.py
- superset/mcp_service/common/schema_discovery.py
- superset/migrations/versions/2026-09-04_00-00_a7f3c2e91d84_add_partition_filter_mapping.py
- superset/sql/parse.py
- superset/superset_typing.py
- tests/integration_tests/datasets/commands_tests.py
- tests/unit_tests/commands/dataset/update_test.py
- tests/unit_tests/connectors/sqla/partition_mapping_test.py
- tests/unit_tests/datasets/commands/export_test.py
- tests/unit_tests/datasets/partition_mapping_serialization_test.py
- tests/unit_tests/sql/parse_tests.py
Files skipped - 2
- docs/static/feature-flags.json - Reason: Filter setting
- docs/static/resources/openapi.json - Reason: Filter setting
Tools
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

/review - Manually triggers an incremental AI Review.
/review full - Manually triggers a full AI Review.
/pause - Pauses automatic reviews on this pull request.
/resume - Resumes automatic reviews.
/resolve - Marks all Bito-posted review comments as resolved.
/abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by

sadpandajoe · 2026-09-07T06:13:48Z

+        columns_by_name = {column.column_name: column for column in self.columns}
+        mapped_column_name = self.partition_mapped_column or self.main_dttm_col
+        mapped_column = columns_by_name.get(mapped_column_name or "")
+        active = bool(


A nonblank transform that fails validation (for example, one without :value) is deliberately saved as inactive, but this summary marks it active anyway. Should the summary use the same validation result so the Explore indicator does not report a mapping that will never mirror filters?

Yes — fixed in 498d705. The summary now calls the save path's own verdict rather than approximating it:

and is_transform_active( mapped_column.partition_value_transform, self.database.backend )

is_transform_active is just not validate_transform(transform, engine), memoized with @lru_cache(maxsize=LRU_CACHE_MAX_SIZE) on (transform, engine). That answers the hot-path worry the old docstring was defending: the parse happens once per distinct transform per process, and if not self.partition_column: return None short-circuits every unmapped dataset before it.

Blocking issues count as inactive too, not just the Tier-2 ones. CreateDatasetCommand and import run no partition validation, so a Jinja or non-deterministic transform can still be read back — and it would never mirror a filter either.

New tests pin it: the summary reports active: False for unix_timestamp(event_time) (your example), unix_timestamp(:value and {{ current_username() }}, while still naming both columns so the editor can say what needs fixing. There is also test_activity_is_exactly_the_absence_of_issues, asserting is_transform_active(t, e) is (validate_transform(t, e) == []) across the corpus, so the two definitions cannot drift apart again.

Two adjacent gaps I hit while tracing this, which I have left for follow-ups rather than widening this PR:

CreateDatasetCommand and the dataset import command run no partition validation at all, so a mapping can land with a blocking transform and no warning anywhere.

partition_value_transform has no Length validator in datasets/schemas.py, unlike python_date_format (255) and datetime_format (100).

bito-code-review · 2026-09-07T06:14:04Z

The current implementation deliberately separates validation into two tiers to ensure that non-blocking issues (like a missing :value placeholder or an unparseable transform) do not prevent the dataset from being saved. This design choice, as documented in the code, allows users to save a "half-written" transform without losing their other edits, leaving the mapping inactive until it is fully valid.

While the summary could theoretically use the same validation result to reflect the mapping's active status, the current approach prioritizes user experience by allowing partial saves. The Explore indicator's behavior is consistent with this design, as it reports the configuration as it is stored, even if the mapping is currently inactive due to validation issues.

superset/connectors/sqla/partition_mapping.py

if not contains_value_placeholder(transform):
        return [
            MappingValidationIssue(
                field=field,
                message=_(
                    "The value transform must contain the :value placeholder, "
                    "which stands for the filter value being mirrored."
                ),
                blocking=False,
            )
        ]

bito-code-review

Code Review Agent Run #7390a9

Actionable Suggestions - 3

superset/datasets/schemas.py - 1
- POST Skips Mapping Validation · Line 198-199
superset/connectors/sqla/partition_mapping.py - 2
- Unused module logger · Line 58-58
- Unused FEATURE_FLAG constant · Line 60-60

Additional Suggestions - 21

superset/connectors/sqla/partition_mapping.py - 2
- Dead `or ""` operand · Line 97-97
  
  In `contains_value_placeholder` (and identically in `contains_jinja` at line 102), the `transform or ""` operand is unreachable: `bool(transform) and ...` short-circuits, so when `transform` is falsy the `search` call never evaluates. The `or ""` guard suggests a None-handling concern that the conjunction already covers. Same dead operand appears in both helpers; keep the expression honest.
- Unbounded lru_cache keys · Line 330-330
  
  `@lru_cache(maxsize=LRU_CACHE_MAX_SIZE)` (256) memoizes on `(transform, engine)` where `transform` is a user-authored string saved on a dataset. Each distinct transform string is retained as a cache key even after validation, so many datasets or repeated edits grow the cache up to 256 full transform strings per worker - modest but unbounded by any input-length check in this module. If a bound is intended, enforce it here; otherwise consider documenting the retention.
superset/translations/zh/LC_MESSAGES/messages.po - 1
- Untranslated new strings · Line 9095-9099
  
  All 8 new user-facing validation messages from `partition_mapping.py` are added with empty `msgstr ""`, so gettext falls back to the English msgid and Chinese users see English error text. The msgids match the source strings exactly; only the translations are missing. Please fill in the Chinese msgstr values (or route through the backfill process) before merge.
superset/translations/fr/LC_MESSAGES/messages.po - 8
- Empty French translation · Line 8935-8939
  
  New user-facing error message from `partition_mapping.py` is added with an empty `msgstr ""`. French users will see untranslated English text. Per BITO.md rules 12743/13343, fill in the French translation before merging.
- Empty French translation · Line 9632-9634
  
  New user-facing error message from `partition_mapping.py:227` added with empty `msgstr ""`. French users will see English. Fill in the French translation per BITO.md rules 12743/13343.
- Empty French translation · Line 10557-10560
  
  New user-facing error message from `partition_mapping.py:279` added with empty `msgstr ""`. French users will see English. Fill in the French translation per BITO.md rules 12743/13343.
- Empty French translation · Line 11262-11264
  
  New user-facing error message from `partition_mapping.py:215` added with empty `msgstr ""`. French users will see English. Fill in the French translation per BITO.md rules 12743/13343.
- Empty French translation · Line 15938-15942
  
  New user-facing error message from `partition_mapping.py:240` added with empty `msgstr ""`. French users will see English. Fill in the French translation per BITO.md rules 12743/13343.
- Empty French translation · Line 16410-16416
  
  New user-facing error message from `partition_mapping.py:315` added with empty `msgstr ""`. French users will see English. Fill in the French translation per BITO.md rules 12743/13343.
- Empty French translation · Line 16418-16421
  
  New user-facing error message from `partition_mapping.py:291` added with empty `msgstr ""`. French users will see English. Fill in the French translation per BITO.md rules 12743/13343.
- Empty French translation · Line 16423-16426
  
  New user-facing error message from `partition_mapping.py:303` added with empty `msgstr ""`. French users will see English. Fill in the French translation per BITO.md rules 12743/13343.
superset/translations/sk/LC_MESSAGES/messages.po - 1
- Empty msgstr translations · Line 9011-9015
  
  All 8 new Slovak entries ship with empty `msgstr ""`, so Slovak users will see untranslated English for these error messages. BITO.md rules 12743/13343 require complete translations before merge. Fill in the Slovak `msgstr` for each new `msgid` (e.g. lines 9011-9015, 9727-9729, 11365-11367).
superset/translations/pt_BR/LC_MESSAGES/messages.po - 1
- Empty pt_BR translations · Line 9633-9638
  
  All seven newly added entries ship with empty `msgstr ""`, so pt_BR users will see the English fallback for these partition-mapping validation messages (e.g. `Mapped column %(name)s is not a column on this dataset.` from `partition_mapping.py`). Per BITO rules 12743/13343, fill in the Portuguese translations before merging.
superset/translations/zh_TW/LC_MESSAGES/messages.po - 1
- Untranslated new strings · Line 9710-9714
  
  All 8 newly added msgid entries (from `partition_mapping.py` `MappingValidationIssue` messages) have empty `msgstr ""`, so zh_TW users fall back to English for these partition-mapping validation messages. Add zh_TW translations for lines 9710-9714, 10493-10495, 11547-11550, 12281-12283, 17328-17332, 17755-17761, 17763-17766, 17768-17771.
superset/translations/tr/LC_MESSAGES/messages.po - 1
- Empty Turkish translations · Line 9223-9228
  
  All 8 new entries (lines 9223-9228, 9975-9977, 10948-10951, 11645-11647, 16466-16470, 16916-16932) ship with empty `msgstr ""`, so Turkish users see untranslated English on these validation/error paths (e.g. `Partition column %(name)s is not a column on this dataset.`). BITO.md rules 12743/13343 require complete translations before merge. Fill the msgstr values, keeping the `%(name)s`/`%(functions)s` placeholders intact.
superset/translations/uk/LC_MESSAGES/messages.po - 1
- Untranslated strings added · Line 8831-8835
  
  Newly added msgid entries in the Ukrainian catalog have empty `msgstr`, so these strings fall back to English for uk locale. The same applies to the other added blocks at L9531-9533, L10454-10457, L11136-11138, L15734-15738, L16177-16183, L16185-16188, L16190-16193. Fill in the Ukrainian translations.
superset/translations/fi/LC_MESSAGES/messages.po - 1
- Missing Finnish translations · Line 14830-14834
  
  All 8 new user-facing validation error messages (from `partition_mapping.py` `validate_partition_mapping`/`validate_transform`) have empty `msgstr ""`, so Finnish users see untranslated English. Per BITO.md rules 12743/13343, fill each msgstr with a verified Finnish translation, keeping the `%(name)s`/`%(functions)s` placeholders intact.
superset/translations/sl/LC_MESSAGES/messages.po - 1
- Untranslated new strings · Line 8970-8974
  
  New msgids are added with empty `msgstr`, so Slovenian users will see the English source text. Other entries in this file are machine-translated via `backfill_po.py`; these 8 new strings (lines 8970-8974, 9661-9663, 10602-10605, 11281-11283, 15913-15917, 16347-16353, 16355-16358, 16360-16363) were left untranslated.
superset/translations/th/LC_MESSAGES/messages.po - 1
- Untranslated msgstr entries · Line 14619-14623
  
  All 8 newly added entries (14619, 15798, 17406, 18552, 26180, 26822, 26830, 26835) have an empty `msgstr`, so Thai users will see the English `msgid` for these partition-mapping validation messages. Surrounding entries in this file are translated. Fill in the Thai translations to complete the catalog.
tests/unit_tests/commands/dataset/update_test.py - 1
- Untyped mock variables · Line 1385-1385
  
  The new `_mapping_command` helper creates `mapped_column`, `partition_column`, and `mock_dataset` via `mocker.MagicMock()` without explicit type annotations, which the repo's BITO adaptive rule 12787 requires for all mock variables in test files. Add `: MagicMock` annotations to these three bindings to comply.
superset/translations/fa/LC_MESSAGES/messages.po - 1
- Untranslated new strings · Line 9388-9392
  
  All 7 new msgid entries (lines 9388-9392, 10137-10139, 11121-11124, 11824-11826, 16719-16723, 17179-17185, 17187-17195) are added with empty `msgstr`, so fa-locale users will see English fallback for these new partition-mapping validation messages from `partition_mapping.py`. The msgids and `#, python-format` flags are correct; only the translations are missing.

Filtered by Review Rules

Bito filtered these suggestions based on rules created automatically for your feedback. Manage rules.

superset/translations/ta/LC_MESSAGES/messages.po - 1
- Missing Tamil translations · Line 8745-8748
superset/translations/es/LC_MESSAGES/messages.po - 1
- Missing Spanish translations · Line 9243-9243
superset/translations/ca/LC_MESSAGES/messages.po - 8
- Empty Catalan translation · Line 9357-9357
- Empty error translation · Line 10082-10082
- Empty Catalan translation · Line 11050-11050
- Empty error translation · Line 11763-11763
- Empty error translation · Line 16625-16625
- Empty Catalan translation · Line 17097-17097
- Empty Catalan translation · Line 17102-17102
- Empty Catalan translation · Line 17107-17107

Review Details

Files reviewed - 52 · Commit Range: c335d81..f191e0b
- superset-frontend/playwright/tests/mobile/mobile-dashboard.spec.ts
- superset/commands/dataset/update.py
- superset/config.py
- superset/connectors/sqla/models.py
- superset/connectors/sqla/partition_mapping.py
- superset/daos/dataset.py
- superset/dashboards/schemas.py
- superset/datasets/api.py
- superset/datasets/schemas.py
- superset/mcp_service/common/schema_discovery.py
- superset/migrations/versions/2026-09-23_00-01_a7f3c2e91d84_add_partition_filter_mapping.py
- superset/sql/parse.py
- superset/superset_typing.py
- superset/translations/ar/LC_MESSAGES/messages.po
- superset/translations/ca/LC_MESSAGES/messages.po
- superset/translations/cs/LC_MESSAGES/messages.po
- superset/translations/de/LC_MESSAGES/messages.po
- superset/translations/en/LC_MESSAGES/messages.po
- superset/translations/es/LC_MESSAGES/messages.po
- superset/translations/fa/LC_MESSAGES/messages.po
- superset/translations/fi/LC_MESSAGES/messages.po
- superset/translations/fr/LC_MESSAGES/messages.po
- superset/translations/it/LC_MESSAGES/messages.po
- superset/translations/ja/LC_MESSAGES/messages.po
- superset/translations/ko/LC_MESSAGES/messages.po
- superset/translations/lv/LC_MESSAGES/messages.po
- superset/translations/messages.pot
- superset/translations/mi/LC_MESSAGES/messages.po
- superset/translations/nl/LC_MESSAGES/messages.po
- superset/translations/pl/LC_MESSAGES/messages.po
- superset/translations/pt/LC_MESSAGES/messages.po
- superset/translations/pt_BR/LC_MESSAGES/messages.po
- superset/translations/ro/LC_MESSAGES/messages.po
- superset/translations/ru/LC_MESSAGES/messages.po
- superset/translations/sk/LC_MESSAGES/messages.po
- superset/translations/sl/LC_MESSAGES/messages.po
- superset/translations/sr/LC_MESSAGES/messages.po
- superset/translations/sr_Latn/LC_MESSAGES/messages.po
- superset/translations/ta/LC_MESSAGES/messages.po
- superset/translations/th/LC_MESSAGES/messages.po
- superset/translations/tr/LC_MESSAGES/messages.po
- superset/translations/uk/LC_MESSAGES/messages.po
- superset/translations/zh/LC_MESSAGES/messages.po
- superset/translations/zh_TW/LC_MESSAGES/messages.po
- tests/integration_tests/datasets/commands_tests.py
- tests/unit_tests/commands/dataset/update_test.py
- tests/unit_tests/connectors/sqla/partition_mapping_test.py
- tests/unit_tests/dao/dataset_test.py
- tests/unit_tests/dashboards/datasets_test.py
- tests/unit_tests/datasets/commands/export_test.py
- tests/unit_tests/datasets/partition_mapping_serialization_test.py
- tests/unit_tests/sql/parse_tests.py
Files skipped - 2
- docs/static/feature-flags.json - Reason: Filter setting
- docs/static/resources/openapi.json - Reason: Filter setting
Tools
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful
- Eslint (Linter) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

/review - Manually triggers an incremental AI Review.
/review full - Manually triggers a full AI Review.
/pause - Pauses automatic reviews on this pull request.
/resume - Resumes automatic reviews.
/resolve - Marks all Bito-posted review comments as resolved.
/abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by

bito-code-review · 2026-09-29T20:21:29Z

+    partition_column = fields.String(allow_none=True, validate=Length(0, 250))
+    partition_mapped_column = fields.String(allow_none=True, validate=Length(0, 250))


POST Skips Mapping Validation

DatasetPostSchema now accepts partition_column/partition_mapped_column, but CreateDatasetCommand.validate() never calls validate_partition_mapping, unlike the PUT path (UpdateDatasetCommand._validate_partition_mapping, update.py:316). A POST can persist a mapping PUT rejects (self-mapped or nonexistent columns, Jinja transform). Read-time gating keeps it inert, but stored state is inconsistent between the two write paths.

Code Review Run #7390a9

Should Bito avoid suggestions like this for future reviews? (Manage Rules)

Yes, avoid them

bito-code-review · 2026-09-29T20:21:29Z

+from superset.exceptions import SupersetParseError
+from superset.sql.parse import SQLStatement
+
+logger = logging.getLogger(__name__)


Unused module logger

logger is assigned at module scope but never used anywhere in this module: _parse_skeleton swallows SupersetParseError and returns None without logging, and no other function references it. The import logging exists only to feed this dead binding. Either log the swallowed parse failure (useful when owners report a mapping stuck inactive) or drop the logger and the import.

Code Review Run #7390a9

Should Bito avoid suggestions like this for future reviews? (Manage Rules)

Yes, avoid them

bito-code-review · 2026-09-29T20:21:29Z

+
+logger = logging.getLogger(__name__)
+
+FEATURE_FLAG = "PARTITION_FILTER_MAPPING"


Unused FEATURE_FLAG constant

FEATURE_FLAG = "PARTITION_FILTER_MAPPING" is defined but never referenced in this module or elsewhere (grep -rn FEATURE_FLAG finds only this line and test config). The consumer UpdateDatasetCommand._validate_partition_mapping re-hardcodes the string literal "PARTITION_FILTER_MAPPING" in is_feature_enabled(...), so the constant fails its only purpose: keeping the flag name in one place. Either use it at the call site or delete it.

Code Review Run #7390a9

Should Bito avoid suggestions like this for future reviews? (Manage Rules)

Yes, avoid them

…idation Datasets on Hadoop-family engines are often partitioned on a technical column -- an epoch integer, a lowercased region key -- that no analyst would filter on. Unless a query carries a predicate on that column the engine scans every partition, and today the only workaround is hand-writing the predicate as custom SQL in a virtual dataset. This is the first of four PRs making that a dataset setting. It adds the storage and the save-time validation; nothing reads the mapping yet. Four columns, following the `always_filter_main_dttm` / `currency_code_column` precedent for "a dataset-level setting that names a column": tables.partition_column the physical partition column tables.partition_mapped_column override; NULL follows main_dttm_col table_columns.partition_value_transform the `:value` expression table_columns.partition_transform_is_monotonic gates range mirroring The monotonic flag is NOT NULL DEFAULT false rather than a nullable tri-state, matching `normalize_columns` -- a nullable boolean invites `if x:` bugs where None and False need distinguishing and don't get it. Validation runs in two tiers. Structural and safety errors block the save: unknown columns, a column mapped onto itself, Jinja in the transform, and non-deterministic functions. Everything else -- an unparseable transform, a transform missing `:value` -- saves and leaves the mapping inactive, so a half-written transform doesn't cost the owner the rest of their edits. Note the self-mapping check validates the *effective* mapped column. Checking only the explicit override misses the case an owner actually hits: pointing `partition_column` at the column that is already `main_dttm_col`. `SQLStatement.get_niladic_functions` is added because the denylist cannot be purely name-based: on Hive and Impala `unix_timestamp()` means "now" while `unix_timestamp(x)` -- the canonical transform for this feature -- is pure. Gated behind the `PARTITION_FILTER_MAPPING` feature flag, off by default. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The description promised filter mirroring and partition pruning, but this branch adds only the model fields, migration, serialization and save-time validation. superset/models/helpers.py has no partition awareness and no mirroring helper exists, so the sole consumer of partition_mapping is DatasetUpdateCommand calling validate_partition_mapping. An operator who enabled the flag on the strength of the docs would get no pruning. Reword to describe what the flag gates in this state, and note that no query consumer emits the mirrored predicate. Edited in config.py, the source scripts/extract_feature_flags.py generates feature-flags.json from, and regenerated the JSON. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Restacking the branch onto master kept the migration's original parent, 7e2c9a4f1b83, which master has since built on and joined into the merge revisions 60f94cd6cd11 and 95d8a99c822e. That left a7f3c2e91d84 as a second Alembic head, so every job running `superset db upgrade` failed, along with enforce-single-migration-head. Point down_revision at 95d8a99c822e, master's current head, correct the Revises: docstring (which named 1072de5ed955, never the real parent), and redate the file so it sorts after its parent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Restacking the branch folded a four-line note describing the resolution of the parse_tests.py conflict into the feature commit, above the two test groups it had merged. The tests themselves were intact; the note broke ruff's line-length limit and ruff-format's blank-line spacing, failing the pre-commit job. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

… orphaned Three review findings on the partition filter mapping, each a case where the code did not do what its own docstring said: * `DatasetColumnsPutSchema.partition_transform_is_monotonic` had `load_default=False`, and `update_columns` applies the loaded payload field by field onto the stored column -- so a partial column update cleared a monotonic flag it never mentioned, silently stopping range mirroring. Absent now means "unchanged"; `ImportV1ColumnSchema` still defaults it, on purpose. * `_validate_partition_mapping` documents an unparseable transform as a Tier-2 issue (saves, stays inactive) but then ran `validate_stored_expression` on it, which rejects anything it cannot parse -- turning it into a blocking error. Gate it on `is_parseable`. Not a hole for templating: Jinja in a transform is already blocking on its own, and an unparseable transform is never emitted into a query. * `clear_dangling_partition_mapping` only ran on the `override_columns=true` path, but the upsert path -- the one the editor's PUT takes -- also deletes every column the payload omits, which could leave the mapping pointing at a column that no longer exists. Both column writers now return the surviving column names and `update_columns` cleans up after either. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Two committed artifacts had fallen behind the source on this branch: * `superset/translations/messages.pot` — the partition mapping's validation messages are `lazy_gettext`, so they have to reach the template for any of them to be translatable. Regenerated with `scripts/translations/babel_update.sh`, which also picks up the alerts/reports and screenshot strings that came in with the master merge and drops four msgids that no longer exist in source. `.po` catalogs follow from the template; no fuzzy entries were introduced. * `docs/static/resources/openapi.json` — `partition_transform_is_monotonic` no longer carries `load_default=False` on the dataset columns PUT schema, so the published spec goes from `"default": false` to `"nullable": true`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…e validation `partition_filter_mapping_summary` derived `active` from whether the transform was non-blank. The save path is stricter: a transform missing `:value` or one that does not parse is saved deliberately inactive. So a mapping that will never mirror a filter was advertised as active on every Explore and dashboard payload. Reuse the save path's own verdict instead of approximating it. `validate_transform` with the messages discarded is memoized on `(transform, engine)`, so the parse the old docstring was avoiding costs one call per distinct transform per process -- and datasets without a partition column never reach it. Blocking issues count too: Jinja and non-deterministic transforms are rejected on PUT, but create and import do not validate the mapping, so one can still be read back, and it mirrors nothing either. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

`_validate_partition_mapping` ran unconditionally from `_validate_semantics`, so a deployment with `PARTITION_FILTER_MAPPING` off could still have a dataset PUT rejected over a mapping that can never be consumed. Every path that reads a mapping is already gated -- the preview endpoint, `resolve_partition_mapping` on the query path, and the Explore summary -- which left the write path as the only ungated one. The three existing transform tests now declare the flag they were implicitly relying on, and `_mapping_command` gives the partition column a real `None` transform so it can stand in as the effective mapped column. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The "Mobile Dashboard Interaction" tests opened whichever dashboard sorted first in /dashboard/list/, which is ordered by changed_on descending. That made them depend on the contents of a database every other spec in the same CI job mutates, so any dashboard another spec touched - or leaked - sorted ahead of the seeded examples and got opened instead. That is what broke "dashboard loads and shows charts on mobile": the spec opened a chartless leftover titled xref_dash_1_*, left behind by cross-referenced-dashboards.spec.ts, and no chart container ever existed. Open the World Bank's Health sample dashboard by slug instead. load_examples always seeds it with charts, the spec's header docstring already declares it as a fixture assumption, and getMobileFilterButton already navigated straight to it for the same reason. Route both through one helper, using DashboardPage so the navigation picks up the gotoWithRetry wrapper the raw page.goto skipped. The beforeAll card-count guard goes away with it: it only skipped the block when the dashboard list was empty, which no longer has any bearing on what these tests open. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…-migration-api

bito-code-review

Code Review Agent Run #aa7c0e

Actionable Suggestions - 4

superset/connectors/sqla/partition_mapping.py - 1
- Redundant per-name AST walks · Line 147-151
superset/datasets/schemas.py - 1
- CWE-20: Import rejects NULL monotonic flag · Line 373-373
superset/connectors/sqla/models.py - 1
- Duplicated mapping validity checks · Line 1987-1994
superset/daos/dataset.py - 1
- Mapping clear misses fetch_metadata path · Line 649-649

Additional Suggestions - 36

superset-frontend/playwright/tests/mobile/mobile-dashboard.spec.ts - 1
- Removed skip guard · Line 53-57
  
  `openSampleDashboard` removed both prior skip guards (`openFirstDashboard`'s `test.skip(cardCount === 0)` and the `beforeAll` `test.skip()`). If `world_health` isn't seeded, `gotoBySlug` hits a 404 and `waitForLoad` throws after `TIMEOUT.PAGE_LOAD` (10s) — a hard failure where the old code skipped. Consider a short probe + `test.skip()` to preserve graceful degradation.
superset/translations/pl/LC_MESSAGES/messages.po - 6
- Missing Polish translation · Line 9788-9792
  
  New user-facing error message added with empty `msgstr ""`. Polish users will see untranslated English text. Per BITO.md rules 12743/13343, fill in the Polish translation before merging.
- Missing Polish translation · Line 10538-10540
  
  New user-facing error message added with empty `msgstr ""` and `#, python-format`. Polish users will see untranslated English. Fill in the Polish translation per BITO.md rules 12743/13343.
- Missing Polish translation · Line 11561-11564
  
  New user-facing message added with empty `msgstr ""`. Polish users will see untranslated English. Fill in the Polish translation per BITO.md rules 12743/13343.
- Missing Polish translation · Line 12298-12300
  
  New user-facing error message added with empty `msgstr ""` and `#, python-format`. Polish users will see untranslated English. Fill in the Polish translation per BITO.md rules 12743/13343.
- Missing Polish translation · Line 17496-17500
  
  New user-facing error message added with empty `msgstr ""` and `#, python-format`. Polish users will see untranslated English. Fill in the Polish translation per BITO.md rules 12743/13343.
- Missing Polish translation · Line 17971-17987
  
  Three new user-facing messages added with empty `msgstr ""`. Polish users will see untranslated English. Fill in the Polish translations per BITO.md rules 12743/13343.
superset/translations/pt_BR/LC_MESSAGES/messages.po - 8
- Missing pt_BR translation · Line 9659-9659
  
  New msgid added with empty `msgstr ""`. pt_BR users will see the untranslated English string. Fill in the Portuguese translation before merge (BITO.md rules 12743/13343 require complete translations for all user-facing messages).
- Missing pt_BR translation · Line 10438-10438
  
  New msgid added with empty `msgstr ""`. pt_BR users will see the untranslated English string. Fill in the Portuguese translation before merge (BITO.md rules 12743/13343).
- Missing pt_BR translation · Line 11426-11426
  
  New msgid added with empty `msgstr ""`. pt_BR users will see the untranslated English string. Fill in the Portuguese translation before merge (BITO.md rules 12743/13343).
- Missing pt_BR translation · Line 12178-12178
  
  New msgid added with empty `msgstr ""`. pt_BR users will see the untranslated English string. Fill in the Portuguese translation before merge (BITO.md rules 12743/13343).
- Missing pt_BR translation · Line 17257-17257
  
  New msgid added with empty `msgstr ""`. pt_BR users will see the untranslated English string. Fill in the Portuguese translation before merge (BITO.md rules 12743/13343).
- Missing pt_BR translation · Line 17744-17744
  
  New msgid added with empty `msgstr ""`. pt_BR users will see the untranslated English string. Fill in the Portuguese translation before merge (BITO.md rules 12743/13343).
- Missing pt_BR translation · Line 17749-17749
  
  New msgid added with empty `msgstr ""`. pt_BR users will see the untranslated English string. Fill in the Portuguese translation before merge (BITO.md rules 12743/13343).
- Missing pt_BR translation · Line 17754-17754
  
  New msgid added with empty `msgstr ""`. pt_BR users will see the untranslated English string. Fill in the Portuguese translation before merge (BITO.md rules 12743/13343).
superset/translations/ko/LC_MESSAGES/messages.po - 1
- Untranslated new entries · Line 13206-13210
  
  All 8 newly added Korean msgid entries have empty `msgstr`, so Korean users will see the English fallback for these new partition-mapping validation messages from `partition_mapping.py`. Fill in the Korean translations to complete this backfill.
superset/connectors/sqla/partition_mapping.py - 1
- Transform parsed twice · Line 310-310
  
  `validate_transform` parses the transform twice on every call: `is_parseable` (line 286) builds a `SQLStatement`, discards it, then `find_non_deterministic_functions` re-parses the same text (line 310 → `_parse_skeleton`). Parse once via `_parse_skeleton`, return the Tier-2 issue on `None`, and run the non-determinism check on the statement. Related: the per-name walk at lines 147-151.
superset/translations/lv/LC_MESSAGES/messages.po - 1
- Empty msgstr translations · Line 8851-8855
  
  All 8 new entries (8851-8855, 9556-9558, 10482-10485, 11183-11185, 15798-15802, 16248-16254, 16256-16259, 16261-16264) ship with empty `msgstr ""`, so Latvian users will see untranslated English for these partition-mapping validation messages. The msgids match `partition_mapping.py` exactly. Fill each msgstr with a verified Latvian translation, keeping the `%(name)s`/`%(functions)s` placeholders intact.
superset/translations/fi/LC_MESSAGES/messages.po - 1
- Untranslated msgstr entries · Line 14791-14795
  
  All newly added msgid entries (14791-14795, 15983-15985, 17590-17593, 18762-18764, 26459-26463, 27123-27129, 27131-27134, 27136-27139) have empty `msgstr ""`, so Finnish users will see the English source strings. These are untranslated additions to a translation catalog; fill in the msgstr values to complete the Finnish localization.
superset/translations/ru/LC_MESSAGES/messages.po - 1
- Missing Russian translations · Line 8721-8725
  
  All 8 newly added `msgid` entries (Jinja templating, Mapped column, No value transform, Partition column, cannot-be-mapped-onto-itself, transform calls functions, could not be parsed, must contain :value placeholder) have empty `msgstr ""`. These are user-facing validation errors from `partition_mapping.py`; Russian users will see English text. Per BITO.md rule 13343, fill in translations before merging.
superset/translations/ar/LC_MESSAGES/messages.po - 8
- Untranslated error message · Line 9732-9736
  
  Newly added user-facing error message from `partition_mapping.py` has an empty `msgstr`, so Arabic users will see the English text when the Jinja-in-transform validation fires. Per BITO.md rule 13343, fill in the Arabic translation before merge.
- Untranslated error message · Line 10488-10490
  
  Newly added user-facing error message from `partition_mapping.py` has an empty `msgstr`, so Arabic users see English when the mapped-column validation fires. Fill in the Arabic translation per BITO.md rule 13343.
- Untranslated error message · Line 11493-11496
  
  Newly added user-facing error message from `partition_mapping.py` has an empty `msgstr`, so Arabic users see English when the no-transform validation fires. Fill in the Arabic translation per BITO.md rule 13343.
- Untranslated error message · Line 12221-12223
  
  Newly added user-facing error message from `partition_mapping.py` has an empty `msgstr`, so Arabic users see English when the partition-column validation fires. Fill in the Arabic translation per BITO.md rule 13343.
- Untranslated error message · Line 17223-17227
  
  Newly added user-facing error message from `partition_mapping.py` has an empty `msgstr`, so Arabic users see English when the self-mapping validation fires. Fill in the Arabic translation per BITO.md rule 13343.
- Untranslated error message · Line 17696-17702
  
  Newly added user-facing error message from `partition_mapping.py` has an empty `msgstr`, so Arabic users see English when the non-deterministic-function validation fires. Fill in the Arabic translation per BITO.md rule 13343.
- Untranslated error message · Line 17704-17707
  
  Newly added user-facing error message from `partition_mapping.py` has an empty `msgstr`, so Arabic users see English when the parse-failure validation fires. Fill in the Arabic translation per BITO.md rule 13343.
- Untranslated error message · Line 17709-17712
  
  Newly added user-facing error message from `partition_mapping.py` has an empty `msgstr`, so Arabic users see English when the missing-placeholder validation fires. Fill in the Arabic translation per BITO.md rule 13343.
superset/translations/fa/LC_MESSAGES/messages.po - 1
- Untranslated new strings · Line 9414-9418
  
  All 8 newly added msgid entries (lines 9414-9418, 10166-10168, 11151-11154, 11867-11869, 16779-16783, 17247-17253, 17255-17258, 17260-17263) have empty msgstr, so Persian users will see the English source text for these partition-mapping validation messages from `partition_mapping.py`. Add translations (preserving %(name)s/%(functions)s placeholders) or mark them fuzzy for the backfill pass.
superset/translations/zh_TW/LC_MESSAGES/messages.po - 1
- Empty translations added · Line 9739-9743
  
  All 8 newly added entries (e.g. `Mapped column %(name)s is not a column on this dataset.`, `The partition column cannot be mapped onto itself.`) are user-facing validation errors from `partition_mapping.py` but ship with empty `msgstr ""`. zh_TW users will see untranslated English. Fill in the Traditional Chinese translations before merge.
superset/translations/sl/LC_MESSAGES/messages.po - 1
- Missing translations · Line 8997-9001
  
  All 8 newly added entries are user-facing error messages (partition value transform errors) but ship with empty `msgstr ""`, so Slovenian users will see untranslated English. Per BITO.md rules 12743/13343, fill each `msgstr` with a verified translation before merging.
superset/translations/th/LC_MESSAGES/messages.po - 1
- New strings untranslated · Line 14593-14597
  
  The 7 new msgid entries added here (lines 14593, 15775, 17372, 18532, 26164, 26814, 26822, 26827) all ship with an empty `msgstr`, so Thai users will see the English fallback for these new partition-mapping error messages from `partition_mapping.py`. The file already has ~19% empty msgstr, so this matches the existing (incomplete) state, but these are newly added user-facing strings — consider translating them in this PR.
superset/translations/tr/LC_MESSAGES/messages.po - 1
- Untranslated Turkish strings · Line 9244-9248
  
  All 8 new entries (lines 9244-9248, 9999-10001, 10975-10978, 11686-11688, 16526-16530, 16984-16990, 16992-16995, 16997-17000) have empty `msgstr ""`, so Turkish users will see the English source text for these partition-mapping validation messages. The msgids match `partition_mapping.py` exactly and the `#, python-format` flags/`%(name)s`/`%(functions)s` placeholders are correct; only the translations are missing.
superset/translations/ja/LC_MESSAGES/messages.po - 1
- Untranslated new strings · Line 8195-8200
  
  All 8 new msgid entries are added with empty `msgstr ""`, so these validation messages from `partition_mapping.py` will render in English for Japanese users until translated. The repo already carries 666 untranslated strings, but these are new user-facing validation errors (e.g. "Mapped column %(name)s is not a column on this dataset."). Consider filling the translations or confirming they are queued for the translation workflow.
superset/translations/mi/LC_MESSAGES/messages.po - 1
- Untranslated new entries · Line 9304-9308
  
  All 7 new entries (9304, 10032, 10995, 11708, 16579, 17056, 17064, 17069) were added with empty `msgstr`, so the mi locale falls back to English for these partition-mapping validation messages from `partition_mapping.py`. The msgids match the source exactly and the `#, python-format` flags are correct, but the translations are missing.
superset/connectors/sqla/models.py - 1
- Dangling comment reference · Line 1101-1103
  
  The comment directs readers to 'the operator matrix in `superset.connectors.sqla.partition_mapping`', but that module (read in full, 351 lines) contains no operator matrix and no range-operator gating — only `validate_transform`/`is_transform_active`. No backend code reads `partition_transform_is_monotonic` to gate range mirroring. Fix the reference or add the promised matrix.

Filtered by Review Rules

Bito filtered these suggestions based on rules created automatically for your feedback. Manage rules.

superset/translations/fr/LC_MESSAGES/messages.po - 1
- Empty French translations · Line 8972-8972
superset/translations/pt/LC_MESSAGES/messages.po - 1
- Empty pt translations · Line 11478-11478

Review Details

Files reviewed - 52 · Commit Range: a6fd901..ae33f9f
- superset-frontend/playwright/tests/mobile/mobile-dashboard.spec.ts
- superset/commands/dataset/update.py
- superset/config.py
- superset/connectors/sqla/models.py
- superset/connectors/sqla/partition_mapping.py
- superset/daos/dataset.py
- superset/dashboards/schemas.py
- superset/datasets/api.py
- superset/datasets/schemas.py
- superset/mcp_service/common/schema_discovery.py
- superset/migrations/versions/2026-09-23_00-01_a7f3c2e91d84_add_partition_filter_mapping.py
- superset/sql/parse.py
- superset/superset_typing.py
- superset/translations/ar/LC_MESSAGES/messages.po
- superset/translations/ca/LC_MESSAGES/messages.po
- superset/translations/cs/LC_MESSAGES/messages.po
- superset/translations/de/LC_MESSAGES/messages.po
- superset/translations/en/LC_MESSAGES/messages.po
- superset/translations/es/LC_MESSAGES/messages.po
- superset/translations/fa/LC_MESSAGES/messages.po
- superset/translations/fi/LC_MESSAGES/messages.po
- superset/translations/fr/LC_MESSAGES/messages.po
- superset/translations/it/LC_MESSAGES/messages.po
- superset/translations/ja/LC_MESSAGES/messages.po
- superset/translations/ko/LC_MESSAGES/messages.po
- superset/translations/lv/LC_MESSAGES/messages.po
- superset/translations/messages.pot
- superset/translations/mi/LC_MESSAGES/messages.po
- superset/translations/nl/LC_MESSAGES/messages.po
- superset/translations/pl/LC_MESSAGES/messages.po
- superset/translations/pt/LC_MESSAGES/messages.po
- superset/translations/pt_BR/LC_MESSAGES/messages.po
- superset/translations/ro/LC_MESSAGES/messages.po
- superset/translations/ru/LC_MESSAGES/messages.po
- superset/translations/sk/LC_MESSAGES/messages.po
- superset/translations/sl/LC_MESSAGES/messages.po
- superset/translations/sr/LC_MESSAGES/messages.po
- superset/translations/sr_Latn/LC_MESSAGES/messages.po
- superset/translations/ta/LC_MESSAGES/messages.po
- superset/translations/th/LC_MESSAGES/messages.po
- superset/translations/tr/LC_MESSAGES/messages.po
- superset/translations/uk/LC_MESSAGES/messages.po
- superset/translations/zh/LC_MESSAGES/messages.po
- superset/translations/zh_TW/LC_MESSAGES/messages.po
- tests/integration_tests/datasets/commands_tests.py
- tests/unit_tests/commands/dataset/update_test.py
- tests/unit_tests/connectors/sqla/partition_mapping_test.py
- tests/unit_tests/dao/dataset_test.py
- tests/unit_tests/dashboards/datasets_test.py
- tests/unit_tests/datasets/commands/export_test.py
- tests/unit_tests/datasets/partition_mapping_serialization_test.py
- tests/unit_tests/sql/parse_tests.py
Files skipped - 2
- docs/static/feature-flags.json - Reason: Filter setting
- docs/static/resources/openapi.json - Reason: Filter setting
Tools
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful
- Eslint (Linter) - ✔︎ Successful
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

/review - Manually triggers an incremental AI Review.
/review full - Manually triggers a full AI Review.
/pause - Pauses automatic reviews on this pull request.
/resume - Resumes automatic reviews.
/resolve - Marks all Bito-posted review comments as resolved.
/abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by

bito-code-review · 2026-10-01T21:06:55Z

+    found = {
+        name
+        for name in NON_DETERMINISTIC_FUNCTIONS
+        if statement.check_functions_present({name})
+    }


Redundant per-name AST walks

check_functions_present wraps get_disallowed_functions, which walks the whole AST and rebuilds the dialect parser-alias map on every call (superset/sql/parse.py:1798-1841); the comprehension runs it once per name — seven walks for one verdict. One statement.get_disallowed_functions(NON_DETERMINISTIC_FUNCTIONS) call returns the same matched subset in a single walk.

Code Review Run #aa7c0e

Should Bito avoid suggestions like this for future reviews? (Manage Rules)

Yes, avoid them

bito-code-review · 2026-10-01T21:06:55Z

+    partition_value_transform = fields.String(allow_none=True)
+    # Bundles predating the field must not claim their transform preserves
+    # ordering, which would silently enable range mirroring on import.
+    partition_transform_is_monotonic = fields.Boolean(load_default=False)


CWE-20: Import rejects NULL monotonic flag

partition_transform_is_monotonic is nullable in the DB (migration a7f3c2e91d84), and the legacy datasource editor writes NULL for fields its payload omits (see TableColumn.partition_transform_is_monotonic comment in connectors/sqla/models.py). Export round-trips that NULL, and fields.Boolean without allow_none=True rejects it with "Field may not be null", failing the whole dataset import. Siblings is_dttm/is_active set allow_none=True. (CWE-20)

Code Review Run #aa7c0e

Should Bito avoid suggestions like this for future reviews? (Manage Rules)

Yes, avoid them

bito-code-review · 2026-10-01T21:06:55Z

+        active = bool(
+            self.partition_column in columns_by_name
+            and mapped_column is not None
+            and mapped_column_name != self.partition_column
+            and is_transform_active(
+                mapped_column.partition_value_transform, self.database.backend
+            )
+        )


Duplicated mapping validity checks

active re-implements the three tier-1 structural checks of validate_partition_mapping (partition column exists, mapped column exists, not self-mapped) inline, sharing only the transform verdict via is_transform_active. The docstring claims this is 'the save path's own verdict', but the structural half is a parallel copy that can drift. Extract the shared checks into a helper in partition_mapping.py.

Code Review Run #aa7c0e

Should Bito avoid suggestions like this for future reviews? (Manage Rules)

Yes, avoid them

bito-code-review · 2026-10-01T21:06:55Z

-            cls._upsert_columns(model, property_columns)
+            surviving_column_names = cls._upsert_columns(model, property_columns)
+
+        cls.clear_dangling_partition_mapping(model, surviving_column_names)


Mapping clear misses fetch_metadata path

clear_dangling_partition_mapping runs only from update_columns, but SqlaTable.fetch_metadata (RefreshDatasetCommand, the 'Sync columns from source' action, and the MCP update_dataset sync_columns flow) also deletes physical columns that vanished from the source (models.py drops non-calculated leftovers) without touching partition_column/partition_mapped_column. A source-side drop of the partition column still leaves the stored mapping dangling on that path — the exact state this change sets out to prevent.

Code Review Run #aa7c0e

Should Bito avoid suggestions like this for future reviews? (Manage Rules)

Yes, avoid them

github-actions · 2026-10-01T22:09:21Z

⚠️ Possible DB migration conflict

@hughhhh the base branch master picked up a new database migration under superset/migrations/ after this PR was opened, and this PR adds or modifies migration files too. Alembic migrations form a single linear chain, so if both branch off the same parent revision, merging this PR as-is could leave the migration history with two heads instead of one, which makes superset db upgrade fail with Multiple head revisions are present.

To check and, if needed, fix this:

Rebase this branch onto the latest master.
Run superset db heads. If it lists more than one revision, you have a conflict.
In your new migration's script, update down_revision (and the Revises comment) to point at the other head instead, so the chain has a single head again.

See Merging DB Migrations in the contributing guide for a worked example.

Review follow-ups on the model/validation PR, all of them the same shape: a mapping could reach storage through a path that never checked it. `CreateDatasetCommand` validated nothing. The same mapping was accepted on POST and rejected on PUT, and once stored the only thing that reported it was the dataset editor -- which an API-only caller never opens. What create can check is narrower than what update can: its columns are synced by `fetch_metadata` after validation, so "is that a real column?" has no answer yet, and there is no column payload on POST for a transform to arrive in. The self-mapping case is what is left, and it is the one reachable in a single request. Import validated nothing either, and it *can* carry a transform. It now drops one the save path would have rejected, rather than raising: a bundle is imported as a whole, and failing someone's entire dataset over one unusable expression is a worse trade than importing the dataset without it. Same bargain `clear_dangling_partition_mapping` already strikes for a mapping whose column went away. `partition_value_transform` had no length bound, unlike `python_date_format` (255) and `datetime_format` (100). `is_transform_active` parses the stored value on every Explore load, so an unbounded string would make that parse the expensive part of rendering a chart. `MAX_TRANSFORM_LENGTH` is enforced by both the typed column field and the import schema, and the preview request's own bound now names the same constant. Validation also ran too eagerly in one place. A column payload drops every column it omits, and `update_columns` then clears a mapping whose columns went with them -- but that happens later, during `run()`. Validating the pre-cleanup state rejected the very sync the cleanup exists to absorb: an `override_columns` resync whose source table no longer has the mapped column got a blocking 422 before the cleanup ran. Only a *stored* reference is forgiven; asking in the same request to map onto a column that request does not define is still an error. Finally, `add_columns` now lists the two dataset-level fields. `DatasetPostSchema` has always accepted them -- POST deserializes through the schema, not this list -- so this only makes `/_info` and the generated spec agree with what POST does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

hughhhh · 2026-10-02T01:23:24Z

Closing in favour of #43870, which now carries the whole feature against master rather than the five-PR stack this was the model, migration and save-time validation of.

Nothing is lost. The commits are unchanged and still in #43870's history, so this stage is still reviewable on its own, and the 1 unresolved comment from this PR is reproduced in an inventory comment there — each with a link back to the original thread and the commit that addressed it. They were all addressed in the review pass of 2026-10-01; they show as unresolved here only because the threads were never clicked shut.

@sadpandajoe @rusackas — sorry for the churn. If you would rather keep reviewing stage by stage, say so and I will reopen; the branches are all still there.

bito-code-review

Code Review Agent Run #0d43e9

Actionable Suggestions - 3

tests/unit_tests/datasets/commands/importers/v1/import_test.py - 2
- Missing docstring on helper · Line 2811-2811
- Tests bypass import wiring · Line 2838-2857
superset/commands/dataset/importers/v1/utils.py - 1
- CWE-20: Import Skips Expression Security Check · Line 305-309

Additional Suggestions - 4

tests/unit_tests/commands/dataset/test_create.py - 2
- Duplicated test scaffolding · Line 606-627
  
  `_create_command_with_mapping` repeats the `Mock(spec=Database)` construction and four `patch` contexts already in `_validate_virtual_dataset` (lines 46-63), adding only the `DatasetDAO.validate_table_exists` patch. Extracting a shared helper parameterized by properties/patch set would keep the two validation environments from drifting apart.
- Mock missing type annotation · Line 606-606
  
  BITO adaptive rule 12787 requires explicit type annotations on mock variables, and sibling tests in `update_test.py` follow it (e.g. `mock_database: MagicMock = mocker.MagicMock()`). The new helper leaves `mock_database` unannotated; add `mock_database: Mock = Mock(spec=Database)` so the mock type is visible to readers and static checkers.
tests/unit_tests/datasets/partition_mapping_serialization_test.py - 1
- Missing local type annotation · Line 254-254
  
  Per the repo's typing standard for test files (BITO.md rule 13153), locals in test files carry explicit annotations. The sibling test at line 237 does not introduce a local, so this is the only gap; `transform: str = "x" * MAX_TRANSFORM_LENGTH` aligns it.
superset/connectors/sqla/partition_mapping.py - 1
- Comment cites absent enforcement · Line 64-65
  
  The comment states every entry point enforces MAX_TRANSFORM_LENGTH, naming 'the preview request' as the third enforcement site alongside the typed column field (`DatasetColumnsPutSchema`) and the import schema (`ImportV1ColumnSchema`). No preview endpoint or schema validating `partition_value_transform` exists in the repo (repo-wide search of `@expose` routes, schemas, and `mcp_service` found none). Drop or correct the phrase so the documented coverage matches the code.

Review Details

Files reviewed - 10 · Commit Range: ae33f9f..0b4ba99
- superset/commands/dataset/create.py
- superset/commands/dataset/importers/v1/utils.py
- superset/commands/dataset/update.py
- superset/connectors/sqla/partition_mapping.py
- superset/datasets/api.py
- superset/datasets/schemas.py
- tests/unit_tests/commands/dataset/test_create.py
- tests/unit_tests/commands/dataset/update_test.py
- tests/unit_tests/datasets/commands/importers/v1/import_test.py
- tests/unit_tests/datasets/partition_mapping_serialization_test.py
Files skipped - 0
Tools
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

/review - Manually triggers an incremental AI Review.
/review full - Manually triggers a full AI Review.
/pause - Pauses automatic reviews on this pull request.
/resume - Resumes automatic reviews.
/resolve - Marks all Bito-posted review comments as resolved.
/abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by

bito-code-review · 2026-10-02T04:26:58Z

    assert mock_read_bounded.call_args_list[1].args[0] is decompressed
+
+
+def _partition_mapping_config(database_id: int, transform: str) -> dict[str, Any]:


Missing docstring on helper

_partition_mapping_config is the only new unit here without a docstring, while the three tests it feeds all have one. Repo rule (BITO adaptive 12490/12147) requires docstrings on new test helpers. Add a one-liner, e.g. "Build a minimal dataset import config whose dttm column carries transform."

Code Review Run #0d43e9

Should Bito avoid suggestions like this for future reviews? (Manage Rules)

Yes, avoid them

bito-code-review · 2026-10-02T04:26:58Z

+def test_import_drops_a_transform_the_save_path_would_reject(
+    session: Session, transform: str
+) -> None:
+    """
+    Import is the one door into this field that does not go through
+    `UpdateDatasetCommand`, so a bundle can carry a transform that will never
+    mirror a filter. Dropping it leaves a state an owner can see and fix rather
+    than a stored expression that looks configured and is not.
+    """
+    engine = db.session.get_bind()
+    SqlaTable.metadata.create_all(engine)  # pylint: disable=no-member
+    database = Database(database_name="hive_db", sqlalchemy_uri="hive://localhost/db")
+    db.session.add(database)
+    db.session.flush()
+
+    config = _partition_mapping_config(database.id, transform)
+    drop_unusable_partition_transforms(config)
+
+    assert config["columns"][0]["partition_value_transform"] is None
+    assert config["columns"][0]["partition_transform_is_monotonic"] is False


Tests bypass import wiring

These tests are named and docstring'd as import-path behavior ("Import is the one door..."), but all three call drop_unusable_partition_transforms directly and never invoke import_dataset. If the call at utils.py:559 were dropped, these tests still pass while the import door silently stops sanitizing. Exercise import_dataset once, or reframe names/docstrings as helper-level tests.

Code Review Run #0d43e9

Should Bito avoid suggestions like this for future reviews? (Manage Rules)

Yes, avoid them

bito-code-review · 2026-10-02T04:26:59Z

+        if blocking := [
+            issue
+            for issue in validate_transform(transform, database.backend)
+            if issue.blocking
+        ]:


CWE-20: Import Skips Expression Security Check

The docstring promises parity with the save path, but UpdateDatasetCommand._validate_partition_mapping also runs validate_stored_expression on parseable transforms (update.py:524-543), rejecting subqueries and set operations. drop_unusable_partition_transforms applies only validate_transform's blocking tier, so import stores transforms the editor rejects on security grounds. The value is only parsed today (is_transform_active), but the stored state diverges by entry path. (CWE-20)

Code Review Run #0d43e9

Should Bito avoid suggestions like this for future reviews? (Manage Rules)

Yes, avoid them

pull-request-size Bot added the size/XXL label Sep 1, 2026

This was referenced Sep 1, 2026

feat(datasets): mirror filters onto the partition column at query time #43758

Closed

feat(datasets): add partition mapping preview endpoint and editor plumbing #43759

Closed

docs: document partition filter mapping #43760

Closed

github-actions Bot added risk:db-migration PRs that require a DB migration api Related to the REST API doc Namespace | Anything related to documentation labels Sep 1, 2026

sadpandajoe added the review:draft label Sep 2, 2026

hughhhh force-pushed the hughhhh/pfm-1-model-migration-api branch 6 times, most recently from 588f71c to d909e49 Compare September 5, 2026 16:10

hughhhh force-pushed the hughhhh/pfm-1-model-migration-api branch 2 times, most recently from d092638 to f21009f Compare September 5, 2026 18:20

hughhhh marked this pull request as ready for review September 5, 2026 20:23

hughhhh requested review from michael-s-molina, rusackas, sadpandajoe and villebro as code owners September 5, 2026 20:23

sadpandajoe reviewed Sep 7, 2026

View reviewed changes

sadpandajoe reviewed Sep 8, 2026

View reviewed changes

Comment thread docs/static/feature-flags.json Outdated

sadpandajoe removed the review:draft label Sep 8, 2026

github-actions Bot added the requires:rebase Requires rebasing on top of current master label Sep 9, 2026

github-actions Bot removed the requires:rebase Requires rebasing on top of current master label Sep 16, 2026

github-actions Bot added requires:rebase Requires rebasing on top of current master and removed requires:rebase Requires rebasing on top of current master labels Sep 28, 2026

hughhhh force-pushed the hughhhh/pfm-1-model-migration-api branch from 56a68c0 to c10ad3e Compare September 29, 2026 16:31

hughhhh requested review from geido and msyavuz as code owners September 29, 2026 16:31

hughhhh force-pushed the hughhhh/pfm-1-model-migration-api branch from a56a271 to f191e0b Compare September 29, 2026 18:10

bito-code-review Bot reviewed Sep 29, 2026

View reviewed changes

hughhhh mentioned this pull request Sep 30, 2026

feat(datasets): partition filter mapping, stored in tables.extra #44816

Closed

2 of 9 tasks

github-actions Bot added the requires:rebase Requires rebasing on top of current master label Oct 1, 2026

hughhhh and others added 9 commits October 1, 2026 09:12

hughhhh force-pushed the hughhhh/pfm-1-model-migration-api branch from f191e0b to ae33f9f Compare October 1, 2026 17:10

github-actions Bot removed the requires:rebase Requires rebasing on top of current master label Oct 1, 2026

Merge remote-tracking branch 'origin/master' into hughhhh/pfm-1-model…

b4676ae

…-migration-api

bito-code-review Bot reviewed Oct 1, 2026

View reviewed changes

github-actions Bot added the requires:rebase Requires rebasing on top of current master label Oct 2, 2026

hughhhh removed this pull request from stack #43893 October 2, 2026 01:22

hughhhh mentioned this pull request Oct 2, 2026

feat(datasets): partition filter mapping #43870

Open

7 of 9 tasks

hughhhh closed this Oct 2, 2026

bito-code-review Bot reviewed Oct 2, 2026

View reviewed changes

		partition_column = fields.String(allow_none=True, validate=Length(0, 250))
		partition_mapped_column = fields.String(allow_none=True, validate=Length(0, 250))


		logger = logging.getLogger(__name__)

		FEATURE_FLAG = "PARTITION_FILTER_MAPPING"

		assert mock_read_bounded.call_args_list[1].args[0] is decompressed


		def _partition_mapping_config(database_id: int, transform: str) -> dict[str, Any]:

Conversation

hughhhh commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

Uh oh!

netlify Bot commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for superset-docs-preview ready!

Uh oh!

codecov Bot commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

bito-code-review Bot commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Agent Run #e33814

Uh oh!

sadpandajoe Sep 7, 2026

Choose a reason for hiding this comment

Uh oh!

hughhhh Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

bito-code-review Bot commented Sep 7, 2026

Uh oh!

Uh oh!

bito-code-review Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Code Review Agent Run #7390a9

Uh oh!

bito-code-review Bot Sep 29, 2026

Choose a reason for hiding this comment

Uh oh!

bito-code-review Bot Sep 29, 2026

Choose a reason for hiding this comment

Uh oh!

bito-code-review Bot Sep 29, 2026

Choose a reason for hiding this comment

Uh oh!

bito-code-review Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Code Review Agent Run #aa7c0e

Uh oh!

bito-code-review Bot Oct 1, 2026

Choose a reason for hiding this comment

Uh oh!

bito-code-review Bot Oct 1, 2026

Choose a reason for hiding this comment

Uh oh!

bito-code-review Bot Oct 1, 2026

Choose a reason for hiding this comment

Uh oh!

bito-code-review Bot Oct 1, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Oct 1, 2026

⚠️ Possible DB migration conflict

Uh oh!

hughhhh commented Oct 2, 2026

Uh oh!

bito-code-review Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Code Review Agent Run #0d43e9

Uh oh!

bito-code-review Bot Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

bito-code-review Bot Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

bito-code-review Bot Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

hughhhh commented Sep 1, 2026 •

edited

Loading

netlify Bot commented Sep 1, 2026 •

edited

Loading

codecov Bot commented Sep 5, 2026 •

edited

Loading

bito-code-review Bot commented Sep 5, 2026 •

edited

Loading

bito-code-review Bot left a comment •

edited

Loading

bito-code-review Bot left a comment •

edited

Loading

bito-code-review Bot left a comment •

edited

Loading