Repository navigation
Run the temporal camera filter on Vulkan compute with CPU fallback - #1
Draft
RyanTheTide wants to merge 7 commits into
Draft
RyanTheTide wants to merge 7 commits into
RyanTheTide wants to merge 7 commits into
Conversation
…lback Reproduce the CPU fixed-point filter and noise estimator in a GPU plugin, prefer it for the temporal stage, and add parity and paced benchmark tooling. Hardware measurements remain pending. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cover single and split plane DMA-BUF offsets, padded strides and invalid buffer ranges without a camera. Keep software Vulkan test workers bounded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cache successful loads while allowing missing or invalid plugins to be retried after installation. Test the optional DMA-BUF companion and legacy ABI v1 plugins without a GPU. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop old camera-pool references after synchronous processing completes, and probe DMA-BUF support again for the next stream. Document CPU fallback warmup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Vulkan loader pkg-config file may be installed without development headers. Auto mode should omit the plugin in that case; explicitly enabled builds should explain the missing dependency at configuration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Export linear allocations on the build GPU and pass them through the plugin companion ABI. Compare every output byte and noise estimate to CPU across offset/stride variations, pool reuse, resets and rejected bounds, without a camera. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 1, 2026
RyanTheTide
marked this pull request as draft
November 1, 2026 05:51
Document exact RADV upload and DMA-BUF parity, strict enabled/disabled builds, and historical CPU reference rows. Keep paired Apple camera latency, CPU, GPU busy and power results pending. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Measured this branch live on the Neo (fleet14b, 1080p30, camera open, 10 s means of the SMC PPMC rail at real governor clocks). Camera node alone 1637 mW. CPU backend 2034 mW (+397 mW, 42% of one core, 14.3 ms per frame). This Vulkan backend 2960 mW (+1323 mW, 3.5% CPU, 6.1 ms per frame). So today it costs about 3.3x the CPU backend in energy. Please keep |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The always-on temporal camera filter spends CPU time on every frame. Add
cdn-backend-gpu.soand prefer Vulkan compute for that stage, with CPU fallback when the plugin/device is unavailable or processing fails.--backend cpuforces the CPU baseline; the low-light backend remains independently triggered.The compute shaders reproduce the CPU noise estimator, fixed-point history, motion tests, edge-preserving luma filter and both chroma passes. An optional companion entry preserves backend ABI v1 while importing linear NV12 DMA-BUF planes with producer-fence waits and foreign ownership transfers. Unsupported imports use one upload. Output copies into the existing PipeWire memory buffer.
Validation on Radeon RX 7900 XTX / RADV NAVI31:
Remains draft for recorded-camera parity and paired latency, filter-process CPU, GPU busy and PMP/SMC power measurements on Neo, J616s, M1 Pro and M2. The validation notes and measurement matrix include earlier CPU references and reproduction commands. Apple GPU performance and power results remain pending the camera-quality lead's on-device run.