Skip to content

drm/asahi: retire a G17 render group on its pass record when its stamps stall - #555

Merged
DjDeveloperr merged 1 commit into
integration/aurorafrom
fix/g17-render-stamp-fallback-20261028
Oct 28, 2026
Merged

DjDeveloperr merged 1 commit into
integration/aurorafrom
fix/g17-render-stamp-fallback-20261028

Conversation

@DjDeveloperr

Copy link
Copy Markdown
Contributor

Problem

On the J700, about every other boot under a 6-worker Vulkan CTS load, one client's render fails with ETIMEDOUT about 30 s after it was submitted, and the client sees a lost device:

asahi: GPU work of a client failed (ETIMEDOUT, reported at g17/runtime.rs:1291); the client sees a lost device

It reproduces with the qualified GNOME-era stack too (old perf core + intrec1), so it isn't a regression. It was found on the Omarchy install because CTS sessions now start right after boot, where it shows up more often. A gallium desktop also raises the rate.

Cause

A pair-state dump at the failure (a diagnostic build calling Pair::report_recovery from fail_render_packet) shows the firmware did finish the group:

  • both rings were consumed (done == read == write);
  • the pass record holds its tiler start and fragment end;
  • on one pair, 13 later groups had executed as well.

Both queues' completion stamps, though, stop exactly one kick short. For example, slot 35's front ordinal 406 needs stamps covering kick 407, and both read 406 << 8. No stamp store, completion notification or later event ever moves them.

next_render_observation only observes a pair once the stamps cover its front, so the pair is never observed again. The client then waits through the watchdog and its 14 renewals before failing. This is the same firmware behavior compute already tolerates for its records (STAMP_FALLBACK_NS, 50 ms).

Change

Render gets the same bound:

  • The observation scan, and the scan before a render watchdog attribution, latch the time when a front group is executed and consumed while the stamps lag it.
  • A latched pair stays in the 1 ms rescan without a notification. After 50 ms it is observed and the group retires on its pass record (retirement::poll_stampless).
  • When the watchdog of such a front group itself expires, the group retires on its record instead of failing. Failing would also give up on the stamps.
  • The first and every 64th such retirement are logged with a lifetime count.

KUnit: three new cases in asahi_g17_render_retirement model the stalled pair as read on hardware. The suite runs 161 tests (160 pass, 1 skip).

Hardware

J700 on the Omarchy install, integration/aurora 4460ba6 kernel (asahi there is identical to this base) with this change as a module, and Mesa aurora-wip f3639d31. Each boot starts 6-worker targeted Vulkan CTS (synchronization2 basic/timeline, command buffers, render passes) right after login:

asahi module boots boots with a client ETIMEDOUT
integration/aurora 4460ba6, and diagnostic builds of it 15 9
the qualified GNOME-era module (b3a06ac), on this core and on the older core 2 2
this change 6 0

With the change, every boot logged exactly one retirement on the pass record, 33-47 s after boot during the first heavy load. For example:

asahi: G17: lifetime total 1 render groups retired on their pass record after 50 ms without completion stamps (latest: slot 11 ordinal 1544)

The stall therefore happens on most boots. Without the change, it only becomes a failure when a client waits on the stalled group.

Qualification on the same stack: regression gate PERFORMANCE-REGRESSION-PASS, targeted CTS 4687 pass / 681 not supported / 0 fail, desktop alive, no new kernel errors.

…ps stall

A render pair is observed only once both queues' completion stamps cover
its front group. On the J700, about every other boot under a CTS load,
the stamps of one pair stop one kick short of a group the firmware has
executed: the pass record holds its tiler start and fragment end, both
rings are consumed, and later groups on the pair execute too, but no
stamp store, completion notification or later event moves the stamps.
The pair is never observed again, and after the scheduler watchdog and
its renewals (about 30 s) the client's render fails with ETIMEDOUT and
it sees a lost device.

Give render the bound compute already has for its records:

- When an observation scan finds a front group executed and consumed
  while the stamps lag it, it latches the time. The scan before a render
  watchdog attribution also latches, so a pair whose notification never
  came is found within one watchdog period.
- A latched pair stays in the 1 ms rescan without a notification. After
  STAMP_FALLBACK_NS (50 ms) it is observed and the group retires on its
  pass record.
- When the watchdog of such a front group itself expires, the group
  retires on its record instead of failing; failing would have given up
  on the stamps as well.

The first and every 64th such retirement are logged with a lifetime
count.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant