Skip to content

perf(datasource): add pagination to datasource editor tables to prevent browser freeze - #37555

Merged
villebro merged 8 commits into
apache:masterfrom
madhushreeag:madhu/feat-add-pagination-datasource-editor
Feb 23, 2026
Merged

villebro merged 8 commits into
apache:masterfrom
madhushreeag:madhu/feat-add-pagination-datasource-editor

Conversation

@madhushreeag

@madhushreeag madhushreeag commented Jan 29, 2026 •

Copy link
Copy Markdown
Contributor

SUMMARY

This update significantly enhances the user experience within the datasource editor, particularly when managing a large number of metrics.

Previously, accessing metrics caused the editor window to become unresponsive for over six minutes when dealing with 100 or more metrics. To resolve this critical performance issue, we've implemented pagination, dramatically reducing response times to under two seconds, even with the same extensive datasets (displaying 100 metrics per page).

Furthermore, this improvement includes enhanced usability features such as sorting, along with resizable windows and expression fields, providing a more fluid and efficient editing environment.

TLDR;

  • Add pagination to the CollectionTable component used in the dataset editor, with configurable page size (default 25, options: 10/25/50/100)
  • Add search/filter inputs for metrics, physical columns, and calculated columns tabs in the dataset editor
  • Make the datasource modal resizable and draggable

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Before -
When opening the metrics tab for datasets with 100+ metrics:

  • Browser freezes for 15+ seconds
  • Network requests queue up (700+ parallel webpack chunk requests)
  • Font files and other assets get stuck in network queue
  • Poor user experience with unresponsive UI
Screen.Recording.-.Pagination.before.mov

After -

  • Tables displays 25 metrics per page by default (configurable)
  • Browser loads smoothly without freezing
  • Network requests reduced from ~700 to ~70 on initial load
  • Fonts and assets load immediately
  • Metrics, columns, and calculated columns tabs have search inputs to filter by name/label
  • Modal is resizable and draggable
Screen.Recording.-.Pagination.after.mov

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@madhushreeag
madhushreeag marked this pull request as ready for review February 17, 2026 21:41
@dosubot dosubot Bot added the change:frontend Requires changing the frontend label Feb 17, 2026

@bito-code-review bito-code-review Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Agent Run #59bf11

Actionable Suggestions - 4
  • superset-frontend/src/components/Datasource/components/DatasourceEditor/components/DatasetUsageTab/index.tsx - 1
  • superset-frontend/src/components/Datasource/components/DatasourceEditor/components/DatasetUsageTab/DatasetUsageTab.test.tsx - 1
  • superset-frontend/src/components/Datasource/components/DatasourceEditor/DatasourceEditor.jsx - 1
  • superset-frontend/packages/superset-ui-chart-controls/src/operators/utils/extractExtraMetrics.ts - 1
    • Logic Error: Incorrect Extra Metrics Extraction · Line 29-44
Additional Suggestions - 2
  • superset-frontend/src/components/Datasource/components/DatasourceEditor/DatasourceEditor.jsx - 1
    • Unsafe null handling in filter · Line 2040-2046
      Replace `col.column_name?.toLowerCase().includes(...)` with `( (col.column_name || '').toLowerCase() ).includes(...)` to safely handle null or undefined column_name.
  • superset-frontend/src/components/Datasource/components/DatasourceEditor/tests/DatasourceEditor.test.tsx - 1
    • Incomplete assertions in case-insensitive test · Line 656-658
      The test checks that 'sum__num' is found with 'SUM' input, but doesn't verify that non-matching metrics like 'avg__num' are hidden. This leaves the test incomplete in asserting the full filtering logic.
Review Details
  • Files reviewed - 10 · Commit Range: 937be9b..937be9b
    • superset-frontend/packages/superset-ui-chart-controls/src/operators/sortOperator.ts
    • superset-frontend/packages/superset-ui-chart-controls/src/operators/utils/extractExtraMetrics.ts
    • superset-frontend/src/components/Datasource/DatasourceModal/index.tsx
    • superset-frontend/src/components/Datasource/components/CollectionTable/index.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/DatasourceEditor.jsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/components/DatasetUsageTab/DatasetUsageTab.test.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/components/DatasetUsageTab/index.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/tests/DatasourceEditor.test.tsx
    • superset-frontend/src/components/Datasource/types.ts
    • superset-frontend/src/explore/components/controls/TextAreaControl.tsx
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • Eslint (Linter) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@villebro
villebro self-requested a review February 18, 2026 20:51
@madhushreeag
madhushreeag force-pushed the madhu/feat-add-pagination-datasource-editor branch from 937be9b to c3a6d49 Compare February 20, 2026 20:05
@github-actions github-actions Bot removed the packages label Feb 20, 2026
Comment thread superset-frontend/src/components/Datasource/components/CollectionTable/index.tsx Outdated
@bito-code-review

bito-code-review Bot commented Feb 20, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Agent Run #611ebf

Actionable Suggestions - 0
Additional Suggestions - 2
  • superset-frontend/src/components/Datasource/components/DatasourceEditor/DatasourceEditor.tsx - 1
    • Runtime Error Risk in Column Filter · Line 2436-2442
      The filter for searching calculated columns can throw a TypeError if any column has a null column_name, since undefined.includes() is invalid. This matches the pattern in the COLUMNS tab and should be fixed consistently.
      Code suggestion
       @@ -2438,5 +2438,5 @@
      -        col.column_name
      -          ?.toLowerCase()
      -          .includes(
      -            this.state.calculatedColumnSearchTerm.toLowerCase(),
      -          ),
      +        col.column_name
      +          ?.toLowerCase()
      +          ?.includes(
      +            this.state.calculatedColumnSearchTerm.toLowerCase(),
      +          ),
  • superset-frontend/src/components/Datasource/components/CollectionTable/index.tsx - 1
    • Inconsistent pagination prop handling · Line 100-100
      The component initializes pageSize from props but hardcodes currentPage to 1. To allow initial current page from props and maintain consistency, initialize currentPage similarly from props.pagination.current.
Review Details
  • Files reviewed - 7 · Commit Range: b09a741..c3a6d49
    • superset-frontend/src/components/Datasource/DatasourceModal/index.tsx
    • superset-frontend/src/components/Datasource/components/CollectionTable/index.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/DatasourceEditor.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/components/DatasetUsageTab/DatasetUsageTab.test.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/components/DatasetUsageTab/index.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/tests/DatasourceEditor.test.tsx
    • superset-frontend/src/components/Datasource/types.ts
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • Eslint (Linter) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@madhushreeag
madhushreeag force-pushed the madhu/feat-add-pagination-datasource-editor branch from c3a6d49 to 2d4e485 Compare February 20, 2026 22:33

@bito-code-review bito-code-review Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Agent Run #1f6ca0

Actionable Suggestions - 1
  • superset-frontend/src/components/Datasource/components/DatasourceEditor/DatasourceEditor.tsx - 1
Additional Suggestions - 2
  • superset-frontend/src/components/Datasource/DatasourceModal/index.tsx - 1
    • Modal behavior change · Line 366-367
      Adding `resizable` and `draggable` props disables the modal mask, allowing background interaction and changing the modal from blocking to non-blocking behavior. This may not be intended for dataset editing, where users should focus on the modal.
  • superset-frontend/src/components/Datasource/components/DatasourceEditor/DatasourceEditor.tsx - 1
    • Inconsistent search filtering logic · Line 2393-2399
      The search for columns and calculated columns only filters on column_name, but metrics search includes both metric_name and verbose_name. Since columns have verbose_name (used in display), users may expect to search by label too. This creates inconsistent behavior where searching by verbose_name works for metrics but not for columns/calculated columns.
      Code suggestion
       @@ -2392,8 +2392,12 @@
      -                        ? this.state.databaseColumns.filter(col =>
      -                            col.column_name
      -                              ?.toLowerCase()
      -                              .includes(
      -                                this.state.columnSearchTerm.toLowerCase(),
      -                              ),
      -                          )
      +                        ? this.state.databaseColumns.filter(col =>
      +                            col.column_name
      +                              ?.toLowerCase()
      +                              .includes(
      +                                this.state.columnSearchTerm.toLowerCase(),
      +                              ) ||
      +                            col.verbose_name
      +                              ?.toLowerCase()
      +                              .includes(
      +                                this.state.columnSearchTerm.toLowerCase(),
      +                              ),
      -                          )
      +                          )
       @@ -2381,1 +2381,1 @@
      -                    placeholder={t('Search columns by name')}
      +                    placeholder={t('Search columns by key or label')}
       @@ -2436,8 +2436,12 @@
      -                        ? this.state.calculatedColumns.filter(col =>
      -                            col.column_name
      -                              ?.toLowerCase()
      -                              .includes(
      -                                this.state.calculatedColumnSearchTerm.toLowerCase(),
      -                              ),
      -                          )
      +                        ? this.state.calculatedColumns.filter(col =>
      +                            col.column_name
      +                              ?.toLowerCase()
      +                              .includes(
      +                                this.state.calculatedColumnSearchTerm.toLowerCase(),
      +                              ) ||
      +                            col.verbose_name
      +                              ?.toLowerCase()
      +                              .includes(
      +                                this.state.calculatedColumnSearchTerm.toLowerCase(),
      +                              ),
      -                          )
      +                          )
       @@ -2423,1 +2423,1 @@
      -                    placeholder={t('Search calculated columns by name')}
      +                    placeholder={t('Search calculated columns by key or label')}
Review Details
  • Files reviewed - 7 · Commit Range: 46250ff..2d4e485
    • superset-frontend/src/components/Datasource/DatasourceModal/index.tsx
    • superset-frontend/src/components/Datasource/components/CollectionTable/index.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/DatasourceEditor.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/components/DatasetUsageTab/DatasetUsageTab.test.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/components/DatasetUsageTab/index.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/tests/DatasourceEditor.test.tsx
    • superset-frontend/src/components/Datasource/types.ts
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • Eslint (Linter) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@madhushreeag madhushreeag changed the title add pagination in datasource modal perf(datasource): add pagination to metrics table to prevent browser freeze Feb 21, 2026
@madhushreeag madhushreeag changed the title perf(datasource): add pagination to metrics table to prevent browser freeze perf(datasource): add pagination to datasource tables to prevent browser freeze Feb 21, 2026
@madhushreeag madhushreeag changed the title perf(datasource): add pagination to datasource tables to prevent browser freeze perf(datasource): add pagination to datasource editor tables to prevent browser freeze Feb 21, 2026
  1. DatasetUsageTab/index.tsx — Search breaks pagination

  Bot: Client-side search filters current page but pagination still fetches unfiltered server data.
  Fix: Disabled pagination entirely when searchTerm is active (pagination={false}). When searching, all matches from the current page are shown without pagination controls.

  ---
  2. DatasetUsageTab/DatasetUsageTab.test.tsx — Inadequate test assertion

  Bot: Test compared 1/1 in both filtered and unfiltered states — identical with only 2 mock charts.
  Fix: Rewrote test to verify pagination is hidden when searching and restored when cleared, and that only matching chart appears during search.

  ---
  3. CollectionTable/index.tsx — currentPage can exceed available pages

  Bot: When collection shrinks, currentPage can exceed maxPage, showing an empty page.
  Fix: Added clamping logic — currentPage = Math.min(this.state.currentPage, maxPage) before passing to the table's pagination config.

  ---
  4. DatasourceEditor.tsx — renderDefaultColumnSettings() accidentally removed

  Bot: The default datetime column and currency code column settings UI was missing from both the Columns and Calculated Columns tabs.
  Fix: Restored {this.renderDefaultColumnSettings()} at the top of both tab content sections.

  ---
  5. DatasourceEditor.tsx + CollectionTable + types.ts — Filtered   5. DatasourceEditor.tsx + CollectionTable + types.ts — Filtered   5.rics/columns to CollectionTable meant edits while searching would drop all non-matching items.
  Fix:
  - Added filterTerm and filterFields props to CRUDCollectionProps and ColumnCollectionTableProps
  - CollectionTable now applies filtering for display only (  - CollectionTable now applies filtering for display only (  - Collection
  - All three tables (metrics, columns, calculated columns) now pass their full data to collection and use filterTerm/filterFields for visual filtering only
@villebro villebro changed the title perf(datasource): add pagination to datasource editor tables to prevent browser freeze perf(datasource): add pagination to datasource editor tables to prevent browser freeze Feb 21, 2026

@bito-code-review bito-code-review Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Agent Run #d6e220

Actionable Suggestions - 1
  • superset-frontend/src/components/Datasource/components/CollectionTable/index.tsx - 1
    • Incorrect pagination clamping with filtering · Line 453-464
Review Details
  • Files reviewed - 7 · Commit Range: 2d4e485..4e630a8
    • superset-frontend/src/components/Datasource/DatasourceModal/index.tsx
    • superset-frontend/src/components/Datasource/components/CollectionTable/index.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/DatasourceEditor.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/components/DatasetUsageTab/DatasetUsageTab.test.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/components/DatasetUsageTab/index.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/tests/DatasourceEditor.test.tsx
    • superset-frontend/src/components/Datasource/types.ts
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • Eslint (Linter) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

Comment thread superset-frontend/src/components/Datasource/components/CollectionTable/index.tsx Outdated
…ted column search is

  case-insensitive) to use getByDisplayValue/queryByDisplayValue instead of getByText/queryByText, since the column_name field in the calculated columns tab is rendered as a TextControl input (not plain
  text) because editableColumnName=true is passed to that table.
@netlify

netlify Bot commented Feb 23, 2026

Copy link
Copy Markdown

✅ Deploy Preview for superset-docs-preview ready!

Name Link
🔨 Latest commit b92e8ac
🔍 Latest deploy log https://app.netlify.com/projects/superset-docs-preview/deploys/699c8fbcf7556a000825760e
😎 Deploy Preview https://deploy-preview-37555--superset-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

…thout an accessible name, so getByRole('list', { name: 'Pagination' }) fails. Replaced with getByTitle('1') to check for the page-1

  listitem (presence = pagination visible) and queryByTitle('Next Page') to check pagination is hidden while searching.
@bito-code-review

bito-code-review Bot commented Feb 23, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Agent Run #ffa242

Actionable Suggestions - 0
Review Details
  • Files reviewed - 7 · Commit Range: 4e630a8..515562c
    • superset-frontend/src/components/Datasource/DatasourceModal/index.tsx
    • superset-frontend/src/components/Datasource/components/CollectionTable/index.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/DatasourceEditor.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/components/DatasetUsageTab/DatasetUsageTab.test.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/components/DatasetUsageTab/index.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/tests/DatasourceEditor.test.tsx
    • superset-frontend/src/components/Datasource/types.ts
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

madhushree agarwal added 2 commits February 23, 2026 13:54
…ayData.length so clamping is based on filtered results

  - Added total: totalItems to paginationConfig so Ant Design Table knows the correct total for the filtered view
@bito-code-review

bito-code-review Bot commented Feb 23, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Agent Run #00d4c9

Actionable Suggestions - 0
Additional Suggestions - 1
  • superset-frontend/src/components/Datasource/components/DatasourceEditor/components/DatasetUsageTab/index.tsx - 1
    • Incorrect pagination reset on search clear · Line 332-337
      When clearing the search, this sets currentPage to 1 but doesn't refetch data, so the table shows the wrong page (e.g., page 3 data labeled as page 1). It should call handleFetchCharts(1) to properly reset and fetch page 1.
      Code suggestion
       @@ -335,1 +335,1 @@
      -             setCurrentPage(1);
      +             handleFetchCharts(1);
Review Details
  • Files reviewed - 7 · Commit Range: 515562c..2757e47
    • superset-frontend/src/components/Datasource/DatasourceModal/index.tsx
    • superset-frontend/src/components/Datasource/components/CollectionTable/index.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/DatasourceEditor.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/components/DatasetUsageTab/DatasetUsageTab.test.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/components/DatasetUsageTab/index.tsx
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/tests/DatasourceEditor.test.tsx
    • superset-frontend/src/components/Datasource/types.ts
  • Files skipped - 0
  • Tools
    • Eslint (Linter) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@villebro villebro left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for this amazing improvement! ❤️

@villebro
villebro merged commit 8f07016 into apache:master Feb 23, 2026
80 checks passed
qfcwell pushed a commit to qfcwell/superset that referenced this pull request May 12, 2026
…nt browser freeze (apache#37555)

Co-authored-by: madhushree agarwal <madhushree_agarwal@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

change:frontend Requires changing the frontend review:draft size/XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants