fix(jinja): handle UndefinedError from virtual dataset templates by eschutho · Pull Request #42366 · apache/superset

eschutho · 2026-07-23T23:31:02Z

SUMMARY

Virtual dataset SQL templates that index into filter_values() (e.g. filter_values('col')[0]) can raise a Jinja UndefinedError when no dashboard filter is active for that column. This gives that error a dedicated, more specific validation message instead of the generic template-rendering error message it was previously grouped under.

PROBLEM

When a virtual dataset's SQL template does something like:

WHERE col = {{ "'" + filter_values('col')[0] + "'" }}

and there are no active dashboard filters for col, filter_values('col') returns an empty list, and operating on the resulting undefined value (e.g. concatenating it into a string, a common pattern for quoting filter values in SQL) raises jinja2.exceptions.UndefinedError (e.g. list object has no element 0) during Jinja rendering.

get_rendered_sql() in superset/models/helpers.py already caught this via a broader except (TemplateError, SupersetSyntaxErrorException) clause, since UndefinedError is a subclass of TemplateError — so it was not surfacing as a raw 500. However, it was reported with the same generic message used for unrelated template syntax errors ("Error while rendering virtual dataset query: ..."), which doesn't make it obvious to the dashboard/chart owner that the root cause is a template referencing a value (like an unset filter) that isn't available.

FIX

Added an explicit except UndefinedError branch in get_rendered_sql(), ordered before the existing except (TemplateError, SupersetSyntaxErrorException) branch, that re-raises as QueryObjectValidationError with a dedicated message: "Virtual dataset template error: %(msg)s", using the original exception's message.

Note: other call sites of template_processor.process_template() in superset/models/helpers.py (e.g. _process_sql_expression, get_timestamp_expression, convert_tbl_column_to_sqla_col) are not wrapped in equivalent error handling and could still raise UndefinedError unhandled for adhoc SQL expressions. That's out of scope for this PR, which is narrowly focused on the virtual dataset SQL template path.

TESTING INSTRUCTIONS

Added a regression test in tests/unit_tests/jinja_context_test.py that builds a virtual dataset whose SQL template indexes into filter_values('col')[0] with no active filters, and asserts that get_rendered_sql() raises QueryObjectValidationError with the new "Virtual dataset template error" message. Verified the test fails against the pre-fix code (with the old generic message) and passes with the fix.
Ran pytest tests/unit_tests/jinja_context_test.py tests/unit_tests/models/helpers_test.py locally — all tests pass.
Ran ruff check, ruff format --check, pylint, and mypy on the changed files.

ADDITIONAL INFORMATION

Has associated issue:
Required feature flags:
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

Virtual dataset SQL templates that index into filter_values() (e.g. filter_values('col')[0]) raise a Jinja UndefinedError when no dashboard filter is active for that column, since the call returns an empty list. get_rendered_sql() now catches UndefinedError and re-raises it as a QueryObjectValidationError with a clear message, turning an unhandled 500 into a graceful chart-level error. Co-Authored-By: Claude <noreply@anthropic.com>

bito-code-review · 2026-07-23T23:31:12Z

Code Review Agent Run #46276b

Actionable Suggestions - 0

Review Details

Files reviewed - 2 · Commit Range: cbca3a6..cbca3a6
- superset/models/helpers.py
- tests/unit_tests/jinja_context_test.py
Files skipped - 0
Tools
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

/review - Manually triggers a full AI review.
/pause - Pauses automatic reviews on this pull request.
/resume - Resumes automatic reviews.
/resolve - Marks all Bito-posted review comments as resolved.
/abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by

codecov · 2026-07-23T23:35:39Z

Codecov Report

❌ Patch coverage is 57.14286% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.27%. Comparing base (206fe7a) to head (fe4391a).
⚠️ Report is 189 commits behind head on master.

Files with missing lines	Patch %	Lines
superset/models/helpers.py	57.14%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #42366      +/-   ##
==========================================
+ Coverage   65.25%   65.27%   +0.02%     
==========================================
  Files        2794     2808      +14     
  Lines      157396   158987    +1591     
  Branches    35982    36209     +227     
==========================================
+ Hits       102706   103782    +1076     
- Misses      52713    53162     +449     
- Partials     1977     2043      +66

Flag	Coverage Δ
hive	`38.08% <28.57%> (-0.34%)`	⬇️
mysql	`57.82% <57.14%> (+0.19%)`	⬆️
postgres	`57.86% <57.14%> (+0.19%)`	⬆️
presto	`39.98% <28.57%> (-0.37%)`	⬇️
python	`59.25% <57.14%> (+0.17%)`	⬆️
sqlite	`57.49% <57.14%> (+0.20%)`	⬆️
unit	`100.00% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…on test The previous assertion only matched the exception's message substring, which the pre-existing generic TemplateError handler already produced (UndefinedError is a TemplateError subclass), so the test passed even without the fix. Match on the new "Virtual dataset template error" prefix instead, so the test actually fails without the fix and passes with it. Co-Authored-By: Claude <noreply@anthropic.com>

netlify · 2026-07-31T17:54:33Z

✅ Deploy Preview for superset-docs-preview ready!

Name	Link
🔨 Latest commit	`fe4391a`
🔍 Latest deploy log	https://app.netlify.com/projects/superset-docs-preview/deploys/6a6ce0e2a5f9c8000963aa3c
😎 Deploy Preview	https://deploy-preview-42366--superset-docs-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

codeant-ai-for-open-source · 2026-07-31T17:57:10Z

+            except UndefinedError as ex:
+                # Raised when a template references an undefined value, e.g.
+                # indexing into an empty list returned by `filter_values()`
+                # when no dashboard filter is active for that column.
+                raise QueryObjectValidationError(
+                    _(
+                        "Virtual dataset template error: %(msg)s",
+                        msg=str(ex),
+                    )
+                ) from ex


Suggestion: The new handling only applies when rendering through get_rendered_sql(). Virtual dataset metadata discovery renders the same SQL directly via get_virtual_table_metadata() and catches only SupersetSyntaxErrorException, so an empty filter_values('col')[0] still escapes as an unhandled UndefinedError on metadata-related endpoints. Move this translation into shared template-processing handling or add equivalent handling to the metadata path so both virtual-dataset workflows return the intended validation error. [incomplete implementation]

Severity Level: Major ⚠️

- ❌ Virtual-dataset metadata endpoint can return an unhandled server error. - ⚠️ Dataset column discovery fails for unset template filters. - ⚠️ Metadata refresh also calls `external_metadata()` at `models.py:1978`.

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖

This is a comment left during a code review. **Path:** superset/models/helpers.py **Line:** 2778:2787 **Comment:** *Incomplete Implementation: The new handling only applies when rendering through `get_rendered_sql()`. Virtual dataset metadata discovery renders the same SQL directly via `get_virtual_table_metadata()` and catches only `SupersetSyntaxErrorException`, so an empty `filter_values('col')[0]` still escapes as an unhandled `UndefinedError` on metadata-related endpoints. Move this translation into shared template-processing handling or add equivalent handling to the metadata path so both virtual-dataset workflows return the intended validation error. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix

👍 | 👎

bito-code-review · 2026-07-31T17:57:21Z

The flagged issue is correct. The current implementation only catches UndefinedError within get_rendered_sql(), leaving other paths like get_virtual_table_metadata() vulnerable to unhandled exceptions when template filters are unset.

To resolve this, you should move the template processing logic into a shared helper method that handles both SupersetSyntaxErrorException and UndefinedError. This ensures consistent error handling across all virtual dataset workflows.

Would you like me to fetch all other comments on this PR to validate them and implement a minimal fix for the rest?

superset/models/helpers.py

def process_template_with_error_handling(template_processor, sql):
    try:
        return template_processor.process_template(sql)
    except UndefinedError as ex:
        raise QueryObjectValidationError(
            _("Virtual dataset template error: %(msg)s", msg=str(ex))
        ) from ex
    except SupersetSyntaxErrorException as ex:
        # ... existing error handling logic ...
        raise QueryObjectValidationError(str(ex)) from ex

bito-code-review · 2026-07-31T20:57:09Z

Code Review Agent Run #d90170

Actionable Suggestions - 0

Review Details

Files reviewed - 1 · Commit Range: cbca3a6..fe4391a
- tests/unit_tests/jinja_context_test.py
Files skipped - 0
Tools
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

/review - Manually triggers a full AI review.
/pause - Pauses automatic reviews on this pull request.
/resume - Resumes automatic reviews.
/resolve - Marks all Bito-posted review comments as resolved.
/abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by

pull-request-size Bot added the size/M label Jul 23, 2026

dosubot Bot added the global:jinja Related to Jinja templating label Jul 23, 2026

codeant-ai-for-open-source Bot reviewed Jul 31, 2026

View reviewed changes

eschutho requested a review from rebenitez1802 August 3, 2026 00:13

rebenitez1802 approved these changes Aug 3, 2026

View reviewed changes

eschutho mentioned this pull request Aug 3, 2026

fix(alerts): wrap Jinja rendering errors in AlertCommand._execute_query #42714

Merged

6 tasks

eschutho merged commit 03b3518 into apache:master Aug 4, 2026
62 checks passed

eschutho mentioned this pull request Aug 28, 2026

fix(sql): catch sqlglot ParseError when parsing RLS predicates #43651

Merged

9 tasks

This was referenced Sep 15, 2026

fix(sqla): wrap TypeError from virtual dataset Jinja rendering as a validation error #44312

Merged

fix(datasets): soften render-time UndefinedError in get_virtual_table_metadata #44316

Merged

This was referenced Oct 16, 2026

fix(import): return 422 instead of 500 for corrupt ZIP uploads #45380

Merged

fix(api): return 400 instead of 500 for malformed query_context in legacy /api/v1/query/ #45636

Open

This was referenced Oct 26, 2026

fix(sqla): raise QueryObjectValidationError for template errors in adhoc column metadata lookup #45742

Merged

fix(sqla): raise QueryObjectValidationError for template errors in adhoc metric rendering #45792

Open

Navigation Menu

Repository navigation

fix(jinja): handle UndefinedError from virtual dataset templates - #42366

fix(jinja): handle UndefinedError from virtual dataset templates#42366
eschutho merged 2 commits into
apache:masterfrom
eschutho:fix-jinja-filter-values-index-error

eschutho commented Jul 23, 2026 •

edited

Loading

Uh oh!

bito-code-review Bot commented Jul 23, 2026 •

edited

Loading

Code Review Agent Run #46276b

Uh oh!

codecov Bot commented Jul 23, 2026 •

edited

Loading

Uh oh!

netlify Bot commented Jul 31, 2026

Uh oh!

codeant-ai-for-open-source Bot Jul 31, 2026

Uh oh!

bito-code-review Bot commented Jul 31, 2026

Uh oh!

bito-code-review Bot commented Jul 31, 2026 •

edited

Loading

Code Review Agent Run #d90170

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

eschutho commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

SUMMARY

PROBLEM

FIX

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

Uh oh!

bito-code-review Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Agent Run #46276b

Uh oh!

codecov Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

netlify Bot commented Jul 31, 2026

✅ Deploy Preview for superset-docs-preview ready!

Uh oh!

codeant-ai-for-open-source Bot Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

bito-code-review Bot commented Jul 31, 2026

Uh oh!

bito-code-review Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Agent Run #d90170

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

eschutho commented Jul 23, 2026 •

edited

Loading

bito-code-review Bot commented Jul 23, 2026 •

edited

Loading

codecov Bot commented Jul 23, 2026 •

edited

Loading

bito-code-review Bot commented Jul 31, 2026 •

edited

Loading