Skip to content

media: apple: avd: AV1 picture overlap across the VP engines - #793

Merged
RyanTheTide merged 2 commits into
aurora-wipfrom
avd/av1-overlap-mvp
Nov 1, 2026
Merged

RyanTheTide merged 2 commits into
aurora-wipfrom
avd/av1-overlap-mvp

Conversation

@RyanTheTide

Copy link
Copy Markdown
Contributor

AV1 still decoded one picture at a time on the queued transport, while H.264, HEVC and VP9 overlap and spread over the VP engines since #787. AV1 was the slowest codec relative to macOS.

Fix:

  • media: apple: avd: overlap AV1 pictures with immutable tail inputs. This brings back 2f9bd00, which was held out of media: apple: avd: frame queue, queued open-firmware transport, picture overlap #713 only until it had run on AV1 hardware. The CDF and segmentation inputs a picture reads are copied into consumer-owned storage, so picture N+1's VP stage can start once N's VP DONE has been seen.
  • media: apple: avd: size AV1 scratch from the firmware codec pool and order VP producers. AV1 gets one bank per advertised AV1 VP plus one held by PP, bounded by the FIFO slots. A picture that names a live reference cannot start on another VP before that producer's tagged VP DONE. Banks and buffers are only released by PP retirement or by quiesced error retirement. Old tagged firmware keeps two banks, and the legacy transport keeps one. H.264, HEVC and VP9 are unchanged.

How tested:

  • run-host-tests 158/158, plus the AV1 program, queue and unicorn tests added here. W=1 is clean.
  • J616s (T6040), 7.3.0-aurora-fleet14f with avd-fw 6062e851 and the PMC floor, 3 passes per row, bit-exact against software decode, no DART or AVD kernel lines: AV1 1080p 1612 fps (1054 before), AV1 4K 563 (411). H.264 1080p 1897-1911, HEVC 1080p 2605, VP9 1080p 879: unchanged.
  • Neo (T8140), same kernel and firmware, bit-exact: AV1 1080p 321 fps (264), 4K 90 (80). H.264 and HEVC unchanged.

🤖 Generated with Claude Code

Keep two complete AV1 scratch banks on queued firmware advertising tagged
VP DONE. Look ahead at the pending request's frame control and follow its
actual primary-reference timestamp. Default CDFs and references outside
the in-flight set do not wait for an unrelated picture.

The open firmware's VP event does not establish CPU visibility of AV1
hardware-written CDF, segmentation or motion-field tails. Retain PP
retirement dependencies for those inputs. When frame-end CDF updates or
segmentation-map writes are disabled, copy preparation's immutable input
snapshots into the consumer bank instead of reading live CAPTURE storage.
Only PP retirement releases the saved picture's bank.

Preserve the legacy emitters and tail preparation, add production host
flag/ownership tests and a routed AV1 Unicorn overlap test, and document
the remaining visibility evidence requirement.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Let independent AV1 pictures fill the firmware-advertised VP pool, with one
scratch bank per engine and one retained by PP. Keep the two-bank old-queue
and one-bank legacy fallbacks, and leave VP9's bank limit unchanged.

A different VP no longer supplies the implicit same-engine ordering of the
two-bank path. Track tagged VP completion in each saved AV1 CAPTURE bank
and block a consumer naming that producer until its VP callback. Duplicate
callbacks never release storage. Updated CDF, segmentation and motion-field
inputs still require PP retirement; immutable CDF/map copies stay private
to the consumer bank. Do not infer CAPTURE DMA visibility from a CM3 DMB.

Extend production host and routed firmware tests for pool sizing, all three
AV1 engines, reverse VP completion, reference barriers, outbox pressure,
allocation failure, saved-bank error retirement and reset. Compare legacy
programs and tails with the pre-overlap driver including film grain control
values and show flags at 8/10 bits. Device pixel and throughput qualification
remains required.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
@RyanTheTide
RyanTheTide merged commit f997147 into aurora-wip Nov 1, 2026
mchalunderscore pushed a commit to mchalunderscore/linux that referenced this pull request Nov 4, 2026
…tion/aurora

Bring integration/aurora up to aurora-wip 52f0a6b, the source of
7.3.0-aurora-fleet14h, promoted on all four machines on 2026-11-02.
The range also covers the fleet14g source b35adc7. It adds aurora-silicon#792,
aurora-silicon#750, aurora-silicon#783, aurora-silicon#784, aurora-silicon#793, aurora-silicon#774, aurora-silicon#794, aurora-silicon#796, aurora-silicon#798, aurora-silicon#800, aurora-silicon#802 and aurora-silicon#801.
No conflicts.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant