Skip to content

AutoFilter recalculates only what hidden rows can change (#661) - #3248

Open
hectorviov wants to merge 2 commits into
storytold:mainfrom
hectorviov:perf/filter-recalcs-what-hidden-rows-change
Open

hectorviov wants to merge 2 commits into
storytold:mainfrom
hectorviov:perf/filter-recalcs-what-hidden-rows-change

Conversation

@hectorviov

@hectorviov hectorviov commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #661

What changed

Applying, clearing or reapplying an AutoFilter no longer recalculates the whole workbook. The filter commands used to mark their edit as structural, which sends commit to recalc_all. Hiding or showing rows can only change formulas that skip hidden rows, so they now set a new Ctx::rows_shown_or_hidden flag instead, and commit runs an incremental cells_changed seeded with every formula that calls SUBTOTAL or AGGREGATE. Their dependents follow through the normal dirty closure, and dynamic nodes (volatile functions, names) are always part of an incremental run, as before.

Three-way comparison

Synthetic workbook: 1,500 SUMPRODUCT formulas over another sheet, one SUBTOTAL(9, …) under a 200-row filtered range, release builds, GRIDCRAFT_PROFILE=1.

Step GridCraft main GridCraft with this PR Excel
data.filterBy commit recalc 8.231 ms, every formula evaluated 0.187 ms, only the SUBTOTAL and what reads it instant
data.clearFilter commit recalc 7.56 ms 0.085 ms instant
SUBTOTAL result after filtering 20100 20100 (identical) updates; other formulas untouched

Tests

  • cmd::data::tests::filtering_recalculates_only_what_hidden_rows_change: SUBTOTAL goes 21 → 12 and its dependent follows, AGGREGATE updates, a plain SUM stays, and at most 5 formulas are evaluated. On main it fails (all 303 formulas evaluated).
  • cargo xtask ci passes (all 10 steps).

Anything a reviewer should look at first

  • The seed set is found by formula text (SUBTOTAL( / AGGREGATE(), the same cheap check the issue suggests. A formula that only reaches SUBTOTAL through a defined name is still covered because names are dynamic nodes in incremental runs.
  • The two smaller items in the issue (filter_values per frame and Top 10 sorting per row) are already gone on current main (the Top 10 threshold is computed once per column), so this PR only handles the recalc.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Formulas that calculate totals based on visible rows now update correctly when filters are applied, changed, reapplied, or cleared. Related formulas also reflect the updated results, while unrelated calculations remain unaffected.
    • Filter changes also update these totals and dependent formulas in manual calculation mode.
  • Documentation
    • Updated the compatibility notes to describe recalculation behavior when AutoFilters change, including which formulas are recalculated.

Applying, changing, reapplying or clearing an AutoFilter marked the
commit structural, so every formula in the workbook was recalculated:
on a formula-heavy workbook each filter click took seconds, while Excel
filters it at once. Hiding or showing rows changes no formula result
except SUBTOTAL and AGGREGATE, which leave filtered rows out (and the
volatile formulas, which recalculate on every recalculation).

The filter commands now set Ctx::rows_shown_or_hidden instead. The
commit then recalculates incrementally from the formulas that call
SUBTOTAL or AGGREGATE (found from the formula text), with what depends
on them; volatile formulas and names are recalculated by every
incremental pass already.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Walkthrough

AutoFilter operations now mark row visibility changes instead of structural edits. The commit path recalculates changed cells and formulas that read hidden rows. In manual calculation mode, row-visibility changes trigger a full recalculation.

Changes: AutoFilter recalculation

Layer / File(s) Summary
Row-visibility recalculation support
crates/engine/src/cmd.rs
Ctx tracks row visibility changes. For non-structural edits, commit recalculates changed cells and formulas containing SUBTOTAL( or AGGREGATE(. Structural edits and row-visibility changes in manual calculation mode trigger a full recalculation.
AutoFilter integration and validation
crates/engine/src/cmd/data/filter.rs, crates/engine/src/cmd/data/tests.rs, docs/target-app-parity.md
Applying, clearing, removing, and reapplying filters mark row visibility changes. Tests check formula results, limit evaluations to five, and verify behavior in manual calculation mode. The parity history records the recalculation change and no score change.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: aiongg, prxnvt

Merge Risk: Minimal · up to 79647

Filter changes use targeted recalculation in automatic mode and full recalculation in manual mode. No actionable merge-blocking risk was established.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 14.29% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title states the main change: AutoFilter operations now recalculate only the formulas that hidden or shown rows can affect, instead of the full workbook. The raw summary and objectives both descri…
Linked Issues check Passed Issue #661 requires that applying or clearing an AutoFilter not force a full workbook recalculation. The diff meets this for automatic calculation. In crates/engine/src/cmd/data/filter.rs, toggle_filt…
Out of Scope Changes check Passed All changes in the whole-PR diff (merge base 178f3eb to head 79647c8) relate to issue #661. crates/engine/src/cmd.rs adds the rows_shown_or_hidden field, the hidden_row_readers helper, and the commit …
Full details: Title check

Explanation

The title states the main change: AutoFilter operations now recalculate only the formulas that hidden or shown rows can affect, instead of the full workbook. The raw summary and objectives both describe this exact change (SUBTOTAL/AGGREGATE formulas, their dependents, and volatile formulas). The title is short, specific, and free of file lists or vague terms. The number (#661) refers to the linked issue, which does not affect the assessment.

Full details: Linked Issues check

Explanation

Issue #661 requires that applying or clearing an AutoFilter not force a full workbook recalculation. The diff meets this for automatic calculation. In crates/engine/src/cmd/data/filter.rs, toggle_filter, filter_by, clear_filter and reapply now set cx.rows_shown_or_hidden instead of cx.structural. In crates/engine/src/cmd.rs, commit runs d.calc.recalc_all only for structural edits or for row-visibility edits in manual mode. Otherwise it runs d.calc.cells_changed seeded with the changed cells plus hidden_row_readers(&wb). That function collects every formula whose text contains SUBTOTAL( or AGGREGATE(, which matches the issue's suggested scope. Dependents follow through the existing dirty closure. The issue also names volatile functions as dirty candidates. The diff does not change the volatile handling in cells_changed. The PR states that dynamic nodes stay in incremental runs, but I could not read the calc crate to confirm this, and the new test uses no volatile formulas. The two smaller items in the issue (filter_values per frame, Top 10 per-row sorting) are not in the diff. The PR description says both are already resolved on main; I did not verify this.

Full details: Out of Scope Changes check

Explanation

All changes in the whole-PR diff (merge base 178f3eb to head 79647c8) relate to issue #661. crates/engine/src/cmd.rs adds the rows_shown_or_hidden field, the hidden_row_readers helper, and the commit branch that uses them. crates/engine/src/cmd/data/filter.rs changes the four filter commands from structural to row-visibility edits. crates/engine/src/cmd/data/tests.rs adds tests for this behavior, which are supporting changes. docs/target-app-parity.md records the AutoFilter recalculation change in its revision history; I took this from the change summary and did not read the file diff. No unrelated code changes were found in the inspected hunks.

Full details: Docstring Coverage

Explanation

Docstring coverage is 14.29% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 7 functions across 3 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches
Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.
Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 38ded93 and 79647c8.

Files selected for processing: 2
  • crates/engine/src/cmd.rs
  • crates/engine/src/cmd/data/tests.rs

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 0 remain after this review.

@coderabbitai coderabbitai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Fix comments: Fix CodeRabbit comments on this PR

Comment thread crates/engine/src/cmd.rs Resolved
…workbook

Addresses CodeRabbit review on storytold#3248, "Recalculate dependents after
filtering in manual mode": in manual calculation cells_changed evaluates
only the formulas it is given, so the seeded path left D2 = D1*10 stale
after a filter changed D1 = SUBTOTAL(...). A filter recalculated the
whole workbook before this PR, so in manual mode it does again; in
automatic mode it keeps the seeded recalculation.

Test: filtering_in_manual_calculation_still_updates_what_depends_on_subtotals.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@echelon

echelon commented Nov 3, 2026

Copy link
Copy Markdown
Member

Thanks for this! The analysis looks careful (SUBTOTAL/AGGREGATE seeds plus dirty_closure, dynamic nodes, manual-calc fallback). Because it narrows which cells recalculate after a filter — where a missed dependent would show stale numbers — I've flagged it for a core maintainer's review rather than landing it in this pass.

@eliets2 eliets2 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First-pass review notes (automated; combined from specialist angles: tests, performance). Please treat as suggestions and verify before acting.

What it does: Applying, clearing, or reapplying an AutoFilter no longer marks the edit as structural and triggers a full workbook recalculation. Instead, filter commands set a new Ctx::rows_shown_or_hidden flag, and commit runs an incremental recalculation seeded with formulas calling SUBTOTAL or AGGREGATE, with dependents following through the normal dirty closure and dynamic nodes included as before.

Logical units:

  • Filter commands in filter.rs set rows_shown_or_hidden instead of structural
  • commit in cmd.rs runs incremental cells_changed for that flag
  • hidden_row_readers finds the seed formulas by text search for SUBTOTAL( / AGGREGATE(
  • Tests for filtering recalculation and manual calculation mode

Findings:

  • [tests] crates/engine/src/cmd/data/tests.rs, filtering_recalculates_only_what_hidden_rows_change: the toggle-off assertion is vacuous because data.filterBy with clear:true already restored D1 to 21 before data.filter with {} runs, so a stale toggle path would still pass. Fix: toggle the filter off while rows are still hidden and assert D1/D2 change back.
  • [tests] crates/engine/src/cmd.rs, hidden_row_readers: no test for the claimed defined-name coverage, cross-sheet SUBTOTAL/AGGREGATE, or text variants such as SUBTOTAL ( with a space or _xlfn. prefix, which the contains("SUBTOTAL(") check would miss and leave stale. Fix: add unit tests for hidden_row_readers covering those cases, or normalize the match.
  • [tests] crates/engine/src/cmd/data/tests.rs, same test: no evaluated-count assertion after the clear step, and D3 (AGGREGATE) is not re-checked after clearing, so a regression where only the first filter application is incremental would pass. Fix: assert last_recalc_cells <= 5 and D3 after clear:true.
  • [tests] crates/engine/src/cmd/data/tests.rs / filter.rs: data.reapply also sets rows_shown_or_hidden but has no test. Fix: add a reapply case after changing underlying values or criteria and assert SUBTOTAL/dependent update with a bounded evaluated count.
  • [tests] crates/engine/src/cmd/data/tests.rs, filtering_in_manual_calculation_still_updates_what_depends_on_subtotals: only covers filterBy in manual mode, not clearFilter, toggle-off, or reapply, which take the same recalc_all branch in commit. Fix: extend the manual-mode test to clear/toggle and assert dependents update.
  • [performance] crates/engine/src/cmd.rs, hidden_row_readers: it allocates a full uppercase copy of every formula in the workbook on every filter commit (to_ascii_uppercase() per cell) and scans all sheets/cells. Fix: avoid the per-cell full uppercase allocation on every filter commit.

Overlaps: None listed.

Verdict: Needs changes. The 3 most important points are: (1) the toggle-off test assertion is vacuous and would not catch a stale toggle path, (2) hidden_row_readers text matching may miss variants and lacks tests for defined-name and cross-sheet coverage, leaving stale results possible, and (3) clear and reapply paths lack bounded evaluated-count and result assertions.

@Pavel199760

Copy link
Copy Markdown

I independently implemented the same idea for AutoFilter while digging into a 3.8 M-formula workbook (details in #4462): filter edits stop marking the edit structural, the graph gains a per-node flag for formulas that read row visibility (SUBTOTAL/AGGREGATE), and only their dirty closure is recalculated — branch perf/filter-selective-recalc, PR #4569. Happy to fold my changes into yours or rebase onto your Ctx::rows_shown_or_hidden naming — whichever merges first.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AutoFilter apply/clear triggers full workbook recalc (seconds per filter on formula-heavy books)

4 participants