[DjDeveloperr]

Mirror of aurora-silicon/linux#570, cherry-picked with -x onto integration/aurora; it applied without conflicts. The touched files (g17/context.rs, g17/context/identity.rs, g17/runtime.rs, g17/runtime/render/{direct,submit}.rs) are identical on both trees. The KUnit and hardware results below were measured on the aurora build.

This fixes a host-side deadlock on a G17 context's work-state nodes. It is one of two failures behind gpubench-stress queue.depth (aurora-silicon/gpubench#4) on the Neo. The other is a GPU stall that this PR does not fix (see "Still open").

What broke

A context has 63 work-state nodes, and all engines of one ioctl share its node. The node returns only after every command of the ioctl has retired.

queue.depth keeps 128 compute and 64 render submissions in flight on one queue, and each render submission also carries a compute copy. With that load:

1. Render admission keeps taking nodes for newer render-plus-copy ioctls as their renders retire. Each node then waits for that ioctl's compute copy.
2. The copies sit in the compute queue behind an older compute ioctl, which has no node yet.
3. Once render holds every node, neither queue can admit anything. Nothing in flight can return a node, so the context deadlocks until the client times out.

Evidence

On the stock kernel, debugfs asahi-g17/capacity was sampled every 10 ms alongside every asahi_g17 tracepoint:

• 50 ms after the start, the context holds work_used=63/63, with compute active=0 and render active=0.
• Every published command has completed.
• The compute queue's next command (a compute-only ioctl) and the render queue's next render loop between prepare and wait (schedule phases 5/6, detail=1) for 15 s, until the client gives up.
• No firmware error or stampless retirement occurs, so this face is purely host-side.

Fix

An engine never takes the last free node while it already holds one.

• The other engine can always take a node.
• An engine refused this way waits only for nodes it took itself. Those belong to ioctls ahead of its waiting command, which retire.
• Each engine admits a context's ioctls in submission order, so these waits cannot form a cycle. An ioctl held by render and waiting on compute would have to be both older and newer than one held by compute and waiting on render.

The decision is serialized under the context's work-storage lock, which initialization already took. The lease records which engine took its node, and a release decrements that engine's count.

Qualification

• KUnit: asahi_g17* passes 149/149 under QEMU arm64 with 16K pages. That includes the new asahi_g17_work_states suite, covering the admission rule, the one-node reserve, and release bookkeeping.
• Neo, stock kit (integration/aurora + KVM guest crashes with a "Bad address" error when accessing GPU memory (specifically drm_gem_shmem pfn_mkwrite) on kernel 7.1.5-400.asahi AsahiLinux/linux#563) with this change as its asahi module:
  • Fresh one-shot boot.
  • Regression gate: PERFORMANCE-REGRESSION-PASS.
  • Targeted CTS: synchronization2 basic/timeline, api.command_buffers, render-pass subsets. 4687 Pass, 0 Fail, 681 NotSupported.
• Performance: the rule only refuses a node when one is left. Single-queue workloads never get near that, because renders are bounded by 32 credits and compute by 16. Three alternated rounds of render.light, compute.light, copy fill and render.heavy at d1–d8, render.light d32/d64, compute.light d64, queues d4/d8, alu and sync.signal matched stock within run-to-run spread, with this rule plus a node cap enabled.

Still open: GPU stall under the same load (not fixed here)

With this change, queue.depth still loses the device. After about 0.5–4 s at 128 compute + 64 render on one queue, all completions of the context stop:

• Completion stamps stop for its compute queue and its render pair at the same moment.
• Newly published compute work never runs.
• The 50 ms stampless fallbacks retire what had already executed.
• Then the client hits ETIMEDOUT, and the next client on the same render slot stalls too.

Native macOS runs the same firmware, so until shown otherwise our driver does something native does not; that trigger has not been found. Each change below was tested as a diagnostic knob together with this rule:

Change	Result
Cap live nodes at 56 / 48 / 32	stall at all three; 48 and 32 passed short 6 s runs, then failed 15 s runs about 1 s in
Skip free nodes still reachable from the firmware work list	stall
Reuse a node only when its completion word equals its submitted count	stall
Add an ioctl's planned kicks (render + copy) to its node with its first command, so a node never completes before its last command	stall, with and without this rule
Keep a released node unused for 2 ms	passed 5 of 5, but throughput fell about 15×, so it most likely just avoids the race

• Render-only work at the same depth ran with all 63 nodes live and did not stall.
• Once, the GPU firmware crashed 54 ms after the boot's first stampless render retire.
• Separately, one CTS synchronization pass (64871 cases, 6 workers) retires about 9.1k compute commands on the 50 ms stampless fallback on the stock kernel, with either Mesa: lifetime counter 1 → 9216 with the baseline Mesa, 9216 → 18304 with mesa#98.
• Next step: run the same mix natively on the Neo under macOS, and measure how many command buffers per queue native keeps in flight and how it orders render and compute.
• drm/asahi: retire a G17 render group on its pass record when its stamps stall (stacked on #244) #264 and t603x atcphy (review/do not merge) AsahiLinux/linux#555 stay on hold until this is resolved or explicitly split out.

[scottjones]

We read this at fbb8ca1 against integration/aurora, and ran it on a Neo with #278 and #276.

The rule does what the body says. may_take (g17/context/identity.rs:26) is decided under the work lock (g17/context.rs:421), and the lease returns the node to the engine that took it. The no-cycle argument rests on each engine admitting in submission order, and that holds: each queue has its own Context (g17/runtime.rs:757), and compute_direct steps aside once a context has seen a render (g17/runtime.rs:1790). An exhaustive search over small cases (2 and 3 nodes, R, C and R+C ioctls) found 0 deadlocks with the rule and 16 and 20 without it. A refused compute command isn't lost: its wait on progress ends on the next event pass (g17/runtime.rs:1854, 2340). Lock order is fine.

One small fix: g17/context.rs:429 calls release_state() with work still held, so notify() signals fences under that lock, and g17/availability.rs:98 says not to. It's only reachable when initialize() fails and we don't see a deadlock. #276 keeps the same call, so it's easiest to fix there.

The mirror is faithful: the patch is byte-identical to 3b5847f. Scope is g17 only. The body's line that #264 and AsahiLinux#555 stay on hold is out of date.

What we ran, all through a private Vulkan driver with omacom/mesa#20 (18ff4cdc9a), gpubench db3fec6, queue.depth 5 times and sparse.churn 3 times at 60 s each:

• 2dd4aab + #278, #275, #276: queue.depth 5/5 (838k-876k deep completions per run, worst latency 84 ms), sparse.churn 3/3, no flagged kernel line.
• 1358ab0 + those three + #264, #277: the same, 5/5 and 3/3.
• The 7.2.11-10-1 release: device lost in the first run at 6.4 s, ETIMEDOUT at g17/runtime.rs:1291, then no GPU for any client until reboot. That is one failure, since the second run started on the same GPU right after.

So we didn't see the stall in those runs. Our failure on the release kernel isn't the 15 s client timeout your body calls stock, and we haven't separated those two. The limits: one job per kernel, one Neo, our clang 22 build rather than the release toolchain, no T6040, and no CTS on the three-PR build (synchronization passed 0 fail only on the five-PR build, on the pinned Mesa). The base moved between those builds for display audio only; drivers/gpu/drm/asahi is the same. Isolating this PR rests on your numbers.

Your "Still open" section says that alone this turns a client timeout into a GPU stall, so land this with #276, with no release between them.