Skip to content

Roll/Scene baselines: pool by whole frame, outliers keep their own bounds - #1145

Merged
marcinz606 merged 1 commit into
mainfrom
feat/baseline-pooling
Sep 22, 2026
Merged

marcinz606 merged 1 commit into
mainfrom
feat/baseline-pooling

Conversation

@marcinz606

Copy link
Copy Markdown
Owner

Problem

With Use Luma/Color Average on, some frames rendered with a heavy cast (marbella_1 #19: neon cyan) while their own bounds rendered correctly.

  • The baseline IQR-trimmed each channel independently, so each channel could average a different set of frames (PARTY scene: red floor from 016/019/024, G/B floors from 014/016/021).
  • A frame far from the pool was reported, but it still received the baseline.
  • Every frame was measured with the active frame's crosstalk, not its own.

Across three rolls (99 frames), the outliers are the first or last frame of each 6-frame strip. The scans were balanced per frame so that the brightest area reads white: the orange base mid-strip, bare light past the cut at a strip end. So strip-end frames keep the orange mask (0.4–0.6 D denser in G/B). The film-base level alone classifies 95 of 99 frames. The scanner-side fix (an exposure lock in the nkscan backend) is a separate change.

Change

  • pool_frame_bounds (normalization.py): compares frames on luma-free color, and flags whole frames further than POOL_OUTLIER_DISTANCE (0.3 D) from the median. The baseline pools the inliers only: luma is the median, color the mean.
  • Outliers keep their own bounds, with both averages off. Taking the baseline luma with the frame's own color rendered washed out on every real outlier, because the color offset also shifts luma through G.
  • The outlier list is stored with the roll/scene baseline (outliers), so a later Apply keeps it. Old records read as empty.
  • Each frame is measured with its own crosstalk, as it renders. The override_crosstalk_* fields are removed.
  • Status message, confirmation dialog, PIPELINE.md and USER_GUIDE.md are updated.

Validation

  • Tuned on marbella_1. On the held-out rolls 2 and 3, the same rule flags only strip-end frames.
  • Inliers render as before with the new baseline, and outliers render as they do with their own bounds.
  • A fixed threshold was chosen over a spread-scaled one (K × MAD), because the latter flagged nothing on a roll with two balance groups.

Test plan

  • make all: lint, ty and the full pytest suite
  • New tests/test_baseline_pooling.py, plus worker, controller and rolls tests
  • Re-run Roll Analysis / Analyze All Scenes on an existing roll. Old baselines have no outlier list until they are re-run.

…rs keep own bounds

Roll and Scene Analysis IQR-trimmed each channel independently, so every
channel of the baseline could average a different set of frames, and a frame
far from the pool still received the baseline. On marbella_1, frame 019
rendered neon cyan with its scene baseline: its G/B sat 0.5-0.9 D denser
than the scene average.

Across three rolls (99 frames) the outliers were the first or last frame of
each 6-frame strip. The scans were balanced per frame so that the brightest
area reads white: orange film base mid-strip, bare light past the cut at a
strip end, which leaves the orange mask in. The film-base level alone
classifies 95 of 99 frames.

- pool_frame_bounds: compare frames on luma-free color, flag whole frames
  further than 0.3 D from the median, pool luma (median) and color (mean)
  from the inliers only.
- Outliers keep their own bounds (both averages off). Taking the baseline's
  luma alone rendered washed out on every real outlier, since the color
  offset also shifts luma through G.
- The outlier list is stored with the roll/scene baseline, so a later apply
  keeps it.
- Each frame is measured with its own crosstalk, as it renders, instead of
  the active frame's.
@github-actions

Copy link
Copy Markdown

Preview load metrics (github-linux-x64)

format cold load throughput
CR2 0.765 s 29.32 MPx/s
NEF 0.624 s 39.38 MPx/s
ARW 0.219 s 92.17 MPx/s
RAF 2.127 s 7.65 MPx/s
DNG 0.653 s 36.64 MPx/s

fixtures: rawsamples.ch · sha: d7660ce · full JSON

@marcinz606
marcinz606 merged commit b2229d6 into main Sep 22, 2026
2 checks passed
@marcinz606
marcinz606 deleted the feat/baseline-pooling branch September 23, 2026 05:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant