Skip to content

fix(importers): catch YAMLError instead of narrower ParserError in dataset v0 importer - #42442

Merged
eschutho merged 1 commit into
apache:masterfrom
eschutho:fix-yaml-error-dataset-importer-v0
Jul 27, 2026
Merged

eschutho merged 1 commit into
apache:masterfrom
eschutho:fix-yaml-error-dataset-importer-v0

Conversation

@eschutho

Copy link
Copy Markdown
Member

SUMMARY

ImportDatasetsCommand.validate() in superset/commands/dataset/importers/v0.py (the legacy/unversioned dataset importer) only caught yaml.parser.ParserError, missing sibling exceptions under yaml.error.YAMLError (ScannerError, ComposerError, ReaderError, ConstructorError). A raw error from these classes propagated as an unhandled 500 instead of the intended IncorrectVersionError → (after the dispatcher exhausts all import versions) CommandInvalidError (422) with a clear message. This is the same bug class fixed in #42366, #42401, and #42426 — #42426 fixed the identical narrow-catch pattern in the sibling v1 importer (superset/commands/importers/v1/utils.py's load_yaml()) and explicitly flagged this v0 site as the same pattern but out of scope for that PR. This PR closes that gap.

PROBLEM

ImportDatasetsCommand.validate() parses every file in a dataset import as YAML:

try:
    config = yaml.safe_load(content)
except yaml.parser.ParserError as ex:
    logger.exception("Invalid YAML file")
    raise IncorrectVersionError(
        f"{file_name} is not a valid YAML file"
    ) from ex

yaml.parser.ParserError, yaml.scanner.ScannerError, yaml.composer.ComposerError, etc. are siblings — all subclasses of yaml.error.YAMLError, not of each other (confirmed via yaml.parser.ParserError.__mro__). For example, yaml.safe_load('key: "unterminated string') raises yaml.scanner.ScannerError, which is not caught here.

This is reachable end-to-end: this v0 ImportDatasetsCommand is the last entry in superset/commands/dataset/importers/dispatcher.py's fallback list, reached whenever the v1 importer raises IncorrectVersionError (i.e. exactly the unversioned/legacy export format this v0 path exists to handle). If validate() leaks a raw ScannerError/etc., it propagates through run(), through the dispatcher's own generic except Exception: logger.exception(...); raise (which logs and re-raises, doesn't swallow it), and out through superset/datasets/api.py's import_ endpoint (POST /api/v1/dataset/import/), which has no try/except around command.run() — only Flask-AppBuilder's @safe decorator, turning it into an opaque generic 500. The same path is also reachable via the legacy_import_datasources CLI command. So importing a legacy-format dataset YAML file malformed in a way other than a ParserError currently 500s instead of surfacing a proper validation error.

FIX

Widen the except clause in validate() to except yaml.YAMLError as ex:. No other exception handling in this file was touched (this is the only yaml. try/except in v0.py). No behavior change for the already-working ParserError case; new coverage for ScannerError/ComposerError/ReaderError/ConstructorError.

TESTING INSTRUCTIONS

  • Added two tests to tests/unit_tests/datasets/commands/importers/v0/import_test.py:
    • test_validate_parser_error_raises_incorrect_version_error — confirms the existing ParserError path ("[1, 2") still raises IncorrectVersionError, unchanged.
    • test_validate_scanner_error_raises_incorrect_version_error — confirms a ScannerError-triggering input ('key: "unterminated string') is now also caught and raises IncorrectVersionError, instead of leaking the raw yaml.scanner.ScannerError.
  • Verified the new ScannerError test fails on pre-fix code (raw yaml.scanner.ScannerError propagates uncaught) and passes after the fix.
  • Ran the full test file — all 5 tests pass (3 pre-existing + 2 new), no regressions.
  • Ran ruff check, ruff format --check, and mypy on both changed files — no new issues.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

…taset v0 importer

ImportDatasetsCommand.validate() only caught yaml.parser.ParserError when
loading a legacy/unversioned dataset export, but yaml.safe_load can raise
several sibling exceptions under yaml.YAMLError depending on how the file
is malformed (ScannerError, ComposerError, ReaderError, ConstructorError).
Those siblings were not caught here, so they propagated past this command,
past the dispatcher's generic re-raise, and out through the
POST /api/v1/dataset/import/ endpoint as a raw, opaque 500 instead of the
intended 422 IncorrectVersionError response.

Widen the except clause to yaml.YAMLError, matching the pattern already
fixed in the v1 importer's load_yaml() (apache#42426).
@dosubot dosubot Bot added change:backend Requires changing the backend data:dataset Related to dataset configurations labels Jul 26, 2026
@bito-code-review

bito-code-review Bot commented Jul 26, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Agent Run #b42831

Actionable Suggestions - 0
Review Details
  • Files reviewed - 2 · Commit Range: 7dd7d14..7dd7d14
    • superset/commands/dataset/importers/v0.py
    • tests/unit_tests/datasets/commands/importers/v0/import_test.py
  • Files skipped - 0
  • Tools
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@eschutho
eschutho requested a review from rebenitez1802 July 26, 2026 16:47
@codecov

codecov Bot commented Jul 26, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 65.23%. Comparing base (ba0900d) to head (7dd7d14).

Files with missing lines Patch % Lines
superset/commands/dataset/importers/v0.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #42442      +/-   ##
==========================================
- Coverage   65.23%   65.23%   -0.01%     
==========================================
  Files        2795     2795              
  Lines      157694   157694              
  Branches    36067    36067              
==========================================
- Hits       102871   102868       -3     
- Misses      52847    52849       +2     
- Partials     1976     1977       +1     
Flag Coverage Δ
hive 38.39% <0.00%> (ø)
mysql 57.56% <0.00%> (ø)
postgres 57.60% <0.00%> (-0.01%) ⬇️
presto 40.31% <0.00%> (ø)
python 59.00% <0.00%> (-0.01%) ⬇️
sqlite 57.23% <0.00%> (ø)
unit 100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@eschutho
eschutho merged commit 276b7e2 into apache:master Jul 27, 2026
70 checks passed
eschutho added a commit to preset-io/agor-teammate that referenced this pull request Aug 13, 2026
eschutho added a commit to preset-io/agor-teammate that referenced this pull request Aug 13, 2026
eschutho added a commit to preset-io/agor-teammate that referenced this pull request Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

change:backend Requires changing the backend data:dataset Related to dataset configurations size/M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants