Skip to content

fix(security): prevent scalar control string iteration in guest anti-tamper check (#43576) - #43697

Open
FrancescoCastaldi wants to merge 25 commits into
apache:masterfrom
FrancescoCastaldi:fix/guest-anti-tamper-and-specs
Open

FrancescoCastaldi wants to merge 25 commits into
apache:masterfrom
FrancescoCastaldi:fix/guest-anti-tamper-and-specs

Conversation

@FrancescoCastaldi

Copy link
Copy Markdown
Contributor

Summary

This PR addresses Issue #43576 (Guest embedded dashboard payload rejection on charts storing single-value scalar controls such as heatmap_v2) and strengthens security manager tamper checks and dialect test coverage across Superset.

Problem Fixed (Issue #43576)

Embedded dashboard charts whose controls store a single scalar string value (e.g. groupby: "division") rather than a list were previously iterated directly as iterables in _columns_metrics_modified and related helpers in superset/security/manager.py. This caused Python to decompose the string character-by-character (e.g. {"d", "i", "v", "s", "o", "n"}), resulting in a false-positive rejection ("Guest user cannot modify chart payload").

Key Changes

  1. Security Manager (superset/security/manager.py):
    • Introduced _ensure_list helper to safely normalize scalar values (strings, numbers, dicts), sequences, and nulls.
    • Guarded _columns_metrics_modified, _stored_param_values, _native_filter_query_modified, _collect_stored_orderby_entries, and _sql_filters_modified to prevent character-level iteration bugs.
  2. Security Unit Tests (tests/unit_tests/security/manager_test.py):
    • Added unit tests validating scalar groupby controls (e.g., heatmap_v2).
    • Added unit tests validating scalar columns, metrics, and query validation.
  3. Database Engine Specs Unit Test Coverage:
    • Added and expanded unit test suites for OceanBase, CockroachDB, Firebolt, MongoDB, Google Sheets, Azure Synapse, MotherDuck, and Amazon DynamoDB specs in tests/unit_tests/db_engine_specs/.

Testing Instructions

  • Run pytest tests/unit_tests/security/manager_test.py
  • Run pytest tests/unit_tests/db_engine_specs/

@dosubot dosubot Bot added the change:backend Requires changing the backend label Aug 30, 2026
@bito-code-review

bito-code-review Bot commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Agent Run #f468fc

Actionable Suggestions - 0
Additional Suggestions - 3
  • tests/unit_tests/db_engine_specs/test_oceanbase.py - 1
    • Test coverage regression · Line 21-21
      The removed parametrized `test_get_column_spec` was the only test exercising `OceanBaseEngineSpec.column_type_mappings`/`get_column_spec` (no other test file references OceanBase type mappings). The new tests only assert static properties/metadata, so the spec's core type-mapping behavior is now untested. Consider re-adding a `get_column_spec` test via `assert_column_spec`.
  • tests/unit_tests/db_engine_specs/test_mongodb.py - 2
    • Reduced convert_dttm coverage · Line 51-54
      This refactor drops the lowercase type cases ("text", "dateTime", "string") and numeric cases ("integer", "number" → None) that the previous test covered. `convert_dttm` relies on `get_sqla_column_type` → `get_column_types`, which matches types via `re.IGNORECASE` regexes; without these cases, a regression in case-insensitive matching or numeric-type handling would go undetected.
    • Reduced grain coverage · Line 66-75
      The refactored `test_time_grain_expressions` only covers None/SECOND/MINUTE/HOUR/DAY/MONTH/YEAR, dropping WEEK, QUARTER, and the WEEK_ENDING/WEEK_STARTING variants that the previous test verified. These grains are still defined in `MongoDBEngineSpec._time_grain_expressions`; without coverage, a regression in their expressions would go undetected.
Filtered by Review Rules

Bito filtered these suggestions based on rules created automatically for your feedback. Manage rules.

  • tests/unit_tests/db_engine_specs/test_gsheets.py - 1
  • tests/unit_tests/db_engine_specs/test_duckdb.py - 1
Review Details
  • Files reviewed - 10 · Commit Range: feb7707..51627fa
    • superset/security/manager.py
    • tests/unit_tests/db_engine_specs/test_crdb.py
    • tests/unit_tests/db_engine_specs/test_duckdb.py
    • tests/unit_tests/db_engine_specs/test_dynamodb.py
    • tests/unit_tests/db_engine_specs/test_firebolt.py
    • tests/unit_tests/db_engine_specs/test_gsheets.py
    • tests/unit_tests/db_engine_specs/test_mongodb.py
    • tests/unit_tests/db_engine_specs/test_mssql.py
    • tests/unit_tests/db_engine_specs/test_oceanbase.py
    • tests/unit_tests/security/manager_test.py
  • Files skipped - 0
  • Tools
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers an incremental AI Review.

  • /review full - Manually triggers a full AI Review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo


metadata = GSheetsEngineSpec.metadata
assert "Google Sheets" in metadata["description"]
assert metadata["logo"] == "google-sheets.png"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: GSheets metadata currently advertises google-sheets.svg, so this assertion fails every time the metadata test runs. [api mismatch]

Assessment: 🟠 Major · 🔁 Occurrence: Often

Use CodeAnt Skill Fix in Cursor Fix in VSCode Claude

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** tests/unit_tests/db_engine_specs/test_gsheets.py
**Line:** 1122:1122
**Comment:**
	*Api Mismatch: GSheets metadata currently advertises `google-sheets.svg`, so this assertion fails every time the metadata test runs.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎

@bito-code-review

Copy link
Copy Markdown
Contributor

The flagged issue in the unit test is correct. The test assertion is failing because the GSheets metadata is returning google-sheets.svg while the test expects a different value. To resolve this, you should update the test expectation in tests/unit_tests/db_engine_specs/test_gsheets.py to match the actual metadata value being returned.

I have checked the available PR comments, and there are no other comments to address. Would you like me to proceed with updating the test file?

@codecov

codecov Bot commented Aug 30, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.37%. Comparing base (ce39d22) to head (7007e94).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff            @@
##           master   #43697    +/-   ##
========================================
  Coverage   82.36%   82.37%            
========================================
  Files        2997     2998     +1     
  Lines      185523   185713   +190     
  Branches    42938    42981    +43     
========================================
+ Hits       152799   152974   +175     
- Misses      29970    29981    +11     
- Partials     2754     2758     +4     
Flag Coverage Δ
hive 36.27% <7.14%> (-0.03%) ⬇️
mysql 55.21% <57.14%> (-0.07%) ⬇️
postgres 55.22% <57.14%> (-0.07%) ⬇️
presto 38.14% <7.14%> (-0.04%) ⬇️
python 86.31% <100.00%> (+0.01%) ⬆️
sqlite 54.95% <57.14%> (-0.07%) ⬇️
unit 79.34% <100.00%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@rusackas
rusackas requested a review from sha174n August 31, 2026 04:50
@netlify

netlify Bot commented Sep 2, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for superset-docs-preview ready!

Name Link
🔨 Latest commit fd77e8d
🔍 Latest deploy log https://app.netlify.com/projects/superset-docs-preview/deploys/6ac4aa4e0afe83000802de65
😎 Deploy Preview https://deploy-preview-43697--superset-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@bito-code-review

bito-code-review Bot commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Agent Run #165dcb

Actionable Suggestions - 0
Review Details
  • Files reviewed - 5 · Commit Range: 51627fa..0f27bb6
    • tests/unit_tests/db_engine_specs/test_duckdb.py
    • tests/unit_tests/db_engine_specs/test_dynamodb.py
    • tests/unit_tests/db_engine_specs/test_gsheets.py
    • tests/unit_tests/db_engine_specs/test_mongodb.py
    • tests/unit_tests/db_engine_specs/test_mssql.py
  • Files skipped - 0
  • Tools
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers an incremental AI Review.

  • /review full - Manually triggers a full AI Review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@FrancescoCastaldi

Copy link
Copy Markdown
Contributor Author

Fixed the missing \TimeGrain\ import in \ ests/unit_tests/db_engine_specs/test_firebolt.py\ and verified Google Sheets metadata assertions in commit 14c596b. CI checks are now running with the fix in place.

@bito-code-review

bito-code-review Bot commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Agent Run #32b24b

Actionable Suggestions - 0
Review Details
  • Files reviewed - 4 · Commit Range: 0f27bb6..c115e1f
    • tests/unit_tests/db_engine_specs/test_firebolt.py
    • superset/security/manager.py
    • tests/unit_tests/db_engine_specs/test_duckdb.py
    • tests/unit_tests/security/manager_test.py
  • Files skipped - 0
  • Tools
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers an incremental AI Review.

  • /review full - Manually triggers a full AI Review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@github-actions github-actions Bot added the requires:rebase Requires rebasing on top of current master label Sep 4, 2026
…mper-and-specs

# Conflicts:
#	tests/unit_tests/security/manager_test.py
@FrancescoCastaldi

Copy link
Copy Markdown
Contributor Author

Merged latest upstream master into branch, resolved merge conflict in \ ests/unit_tests/security/manager_test.py\ by integrating the upstream scalar control tests alongside the scalar columns/metrics test suite, and verified all lint checks pass cleanly. PR is now fully up to date and mergeable.

@github-actions github-actions Bot removed the requires:rebase Requires rebasing on top of current master label Sep 5, 2026
@bito-code-review

bito-code-review Bot commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Agent Run #d45178

Actionable Suggestions - 0
Review Details
  • Files reviewed - 1 · Commit Range: c115e1f..7a854c1
    • superset/security/manager.py
  • Files skipped - 0
  • Tools
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers an incremental AI Review.

  • /review full - Manually triggers a full AI Review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@github-actions github-actions Bot added the requires:rebase Requires rebasing on top of current master label Sep 17, 2026
@FrancescoCastaldi

Copy link
Copy Markdown
Contributor Author

Merged latest \master\ and resolved merge conflicts with recent MSSQL/security tests. Clean and ready for review.

@github-actions github-actions Bot added requires:rebase Requires rebasing on top of current master and removed requires:rebase Requires rebasing on top of current master labels Sep 20, 2026
@codeant-ai-for-open-source

codeant-ai-for-open-source Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

CodeAnt PR Risk: Low Risk

  • The PR appears safe to merge; scalar query values are normalized before security comparisons, with a regression test for columns and metrics.
  • The OceanBase test rewrite removes native-type mapping assertions; consider retaining those checks alongside the new metadata coverage.

Assessed commit: 7007e94cb112

@github-actions github-actions Bot removed the requires:rebase Requires rebasing on top of current master label Oct 6, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

change:backend Requires changing the backend size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant