Skip to content

media: apple: isp: temporal filter and mains frequency as macOS sets them (J700, J616s) - #774

Merged
RyanTheTide merged 2 commits into
aurora-wipfrom
camera/isp-ae-flicker-tnr
Nov 1, 2026
Merged

RyanTheTide merged 2 commits into
aurora-wipfrom
camera/isp-ae-flicker-tnr

Conversation

@RyanTheTide

Copy link
Copy Markdown
Contributor

The camera on the Neo and the J616s is much grainier than on macOS. This brings the ISP setup in line with what macOS sends on the H17 interface, using parameter facts from Daybreak's reading of the macOS 27.0 camera stack (opcodes and values only).

What changes

  • Temporal filter per sensor. IMX958 (J616s) gets the 0x18-byte start record (1, 0, 0), high-strength filtering off (0xc120), band 0 modulation off (0xc11c) and band 0 thresholds 120/100 in 1/256 steps (0xc11d), then motion history and enable as before. IMX558 (J700) gets no start at all. macOS disables and stops the filter and the motion history there, so we do the same. Every other sensor keeps the old sequence. The choice sits next to each sensor's setfile entry.
  • Mains frequency. The H17 firmware's auto exposure takes the mains frequency (0x8212, Hz in 16.16). macOS always sends it, 50 Hz unless the country uses 60 Hz. Added the standard V4L2_CID_POWER_LINE_FREQUENCY control on H17 hardware, 50 Hz by default, applied when the stream starts. P010 keeps its minimal sequence.

Everything else macOS sends at stream start already matches (CNR, MBNR 0/1/1, AE stability 32/20, AE rate limits).

Measured on hardware (fleet14 with this series as a test module, NV12 1080p30, read-only captures, 8-bit luma)

Exposure Gain (x/256) Luma temporal noise Chroma U / V
J616s stock, 150 lux 16.7 ms 261 13.2 1.39 / 1.90
J616s this series 10.0 ms 264 13.4 to 13.9 1.63 / 2.23
Neo stock (filter started with 1), 52 lux 16.7 ms 372 8.1 to 8.3 1.02 / 1.11
Neo filter started with 0 16.7 ms 372 7.2 0.83 / 0.91
  • On the J616s the firmware now steps exposure in 10 ms units for 50 Hz lights instead of 8.3 ms units for 60 Hz. That stops beating with 50 Hz lighting. It doesn't change the grain in this light, which is the same within scene variation.
  • On the Neo the old start (DT value 1) leaves more noise than starting with 0. The full macOS sequence (no start) is still to be measured on the Neo. That's booked for tonight, so this stays a draft until then.
  • The bigger gap to macOS isn't in the ISP. Against a macOS capture of the same J616s scene, a flat region measured 13.6 here and 2.1 on macOS (20.7 vs 35.5 dB). macOS also runs a host-side motion-compensated temporal filter on every frame. Our camera-denoise CPU filter on the same Linux frames gives 2.4 (35.7 dB).

Tests

  • KUnit (arm64 QEMU, apple-isp*): 40/40, with three new cases for both temporal filter sequences and the 50/60 Hz encoding.
  • fleet14 config, LLVM, W=1: ISP objects build clean. KCFLAGS=-Werror stops on an existing FORTIFY warning in isp-ipc.c, which this series doesn't touch.
  • checkpatch --strict: clean on both commits.

🤖 Generated with Claude Code

On the H17 interface macOS does not start the ISP's temporal noise filter
the same way for every sensor. For the IMX958 (J616s) it sends a longer
start record, (1, 0, 0), then turns high-strength filtering and band 0
modulation off and sets the band 0 strength thresholds to 120 and 100 in
1/256 steps, before the motion history and the enable. For the IMX558
(J700) it starts nothing: it disables the filter and stops the motion
history and the filter, since the firmware keeps that state between
streams. These values come from Daybreak's reading of the macOS 27.0
camera stack.

Linux sent the short start with the apple,temporal-filter value on every
sensor, 1 on the J700. On the Neo, starting the filter with 0 instead of
1 already leaves about 13% less luma noise and 18% less chroma noise on a
still scene.

Keep the per-sensor choice next to the setfile of each sensor and leave
every other sensor on the original sequence.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
The H17 firmware's auto exposure keeps exposure times in step with the
mains frequency so that lights do not flicker or band in the picture.
macOS tells it the frequency for every camera, 50 Hz unless the country
uses 60 Hz, as Hz in 16.16 fixed point. Linux never sent it.

Add the standard V4L2 power line frequency control on H17 hardware, 50 Hz
by default as on macOS, and send it with the stream setup. It is applied
when the stream starts, like the exposure priority.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
@RyanTheTide

Copy link
Copy Markdown
Contributor Author

Neo measurement of the filter-stopped sequence, fleet14c plus this series as a test module, evening light:

  • Luma noise across stock, start-with-0 and the macOS stop sequence was 7.1 to 8.5 in 8-bit code values. The scene changed more than that between rows (AE moved 32.5 / 16.7 / 25 ms), so the ISP temporal filter setting doesn't change Neo noise by more than about 10% either way.
  • The first stream using the stop sequence right after two streams that had started the filter delivered no frames, with no error logged, until it was stopped. The next two streams with the stop sequence worked. That mixed transition can't happen with this series, since every stream gets the same sequence.
  • Still to do before this leaves draft: this series as shipped, from a fresh boot on the Neo (stop sequence every stream plus 50 Hz), several streams in a row with no stall. Planned tonight.

@RyanTheTide

Copy link
Copy Markdown
Contributor Author

Fresh-boot check on the Neo, 2026-11-01 20:16 (fleet14c, this change as a test module with default settings, boot 2ea81766), very dark room (1.6 lux, gain 4160, 30 ms exposure):

Stream Result
1st after boot (this change) froze after 114 frames (3.8 s), no frames for the next 36 s until the 40 s timeout, no kernel message
2nd to 4th (this change) 150/150 frames at 30.1 fps
stock sequence 150/150 frames at 30.1 fps

This is the second freeze with the filter-stopped sequence on the Neo (the first was at 18:40, 9 minutes with no frames). Stock streams have not frozen in any run today. Keeping this as a draft until the cause is understood. The J616s half (strength controls, mains frequency) is not implicated, so splitting it out is an option.

Logs: reports/camera-quality-20261101/neo-774-freshboot-20261101/ (full dmesg included).

🤖 Generated with Claude Code

@RyanTheTide

Copy link
Copy Markdown
Contributor Author

Fresh-boot A/B on the Neo, 2026-11-02 00:44-00:58 (fleet14d, this change as a signed test module, dark room with gain at maximum). Each boot's first stream was 150 frames at about 23-44 s uptime. Module options switched between the stock sequence and this change:

Boot Sequence Frames
ce18d913 stock 150/150
d22d3923 this change 150/150
6ffb95fc stock 150/150
f9453bf4 this change 150/150
788eeb26 stock 150/150
06e6049b this change 150/150

So the first-stream freeze seen at 20:16 (and the 18:40 stall) didn't reproduce in three fresh boots with this change. One difference from those two runs: then, another camera client (GNOME Camera via camera-denoise) opened the device while the test stream ran, which camera-denoise #6 now handles. I can't fully rule the change out from three boots, so I'm leaving the merge to review with these numbers. Logs: reports/camera-quality-20261101/neo-774-freeze-20261102/ (dmesg of every boot).

🤖 Generated with Claude Code

@RyanTheTide

Copy link
Copy Markdown
Contributor Author

Correction to my last comment: at 20:16 the freeze started at 28.4 s uptime, before GNOME Camera tried to open the device at 45.6 s. So the second client doesn't explain that freeze. The cause is still unknown, and it didn't reproduce in the three fresh boots above.

🤖 Generated with Claude Code

@RyanTheTide

Copy link
Copy Markdown
Contributor Author

Merging as the camera reviewer.

  • Neo, fleet14d: 6/6 fresh-boot first streams complete (3 stock, 3 with this change).
  • J616s: A/B earlier today with the strength controls and 50 Hz showed no regression. Luma noise was within run-to-run variation, and the 50 Hz exposure steps worked.
  • KUnit: 40/40.
  • Not explained: one first-stream freeze on the Neo (20:16, test module on fleet14c) didn't reproduce in the six boots since. If it shows up again in the fleet kernel, the stall is visible as a stream that stops delivering frames with nothing in dmesg, and reverting the Neo half (ISP_TEMPORAL_FILTER_OFF) is the first thing to try.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant