[DjDeveloperr]

One CTS synchronization2 pass retires about 9.8k G17 compute commands on the 50 ms stampless fallback ("lifetime total N compute commands retired on their completion record after 50 ms without a completion stamp"). That happens on stock and with #570/#571. A synchronization pass retires about 9.1k more. These stamps aren't missing. The firmware stores them later than the fallback assumed.

Cause

The firmware stores the completion stamps of an ioctl's commands only after every kick counted on the ioctl's work-state node has completed. A command whose record (compute) or pass record (render) is already present waits for its stamp for as long as the rest of its ioctl runs.

The 50 ms bound was measured from the record, so it retired those members on their records while the firmware was still running their node. It released their fences, VM pins and render leases before the post-maintenance stamp that the retirement path otherwise requires. This is the visibility window chriskearney asked about on omacom/linux-aurora#264. In these cases it wasn't a stalled stamp: the node was still running.

Evidence from a diagnostic build (this PR's base plus report tracepoints). It covered one full 6-worker synchronization2 pass (36516 pass, 0 non-pass), with the work-state node read at every fallback:

compute fallback retirements	9792
... whose node still had kicks running (completion word below the host count)	9792 (100%)
... whose stamp covered them later	9430 of 9430 tracked; 4.3 / 27.6 / 86 / 299 ms after the fallback (p10 / p50 / p90 / max)
render pass-record fallbacks in the same pass	6, all with their node still running

• Node layout as read: completion word at 0xc, which is the sum of per-engine counts at 0x14/0x18/0x1c. Those counts are cleared when the node drains.
• The most common shapes were:
  • one compute and one render in one ioctl, where the render hadn't finished (node of 3 kicks);
  • signal_order's single command buffer of twelve read operations, with the compute copies between them (12 renders + 13 computes = 37 kicks), where each render waits on the other queue.
• Host traces of the stock kernel agree: every fallback retirement happened while another command of the same context was still executing. In more than half of the runs of consecutive fallbacks on a queue, the queue's stamp moved for the rest of the run within 0.2 ms of the last fallback.

The fallback added no latency here. The ioctl's out-fences wait for all its members, so they signalled at node completion either way. A synchronization2 pass takes 3 min 47 s with or without this change.

Fix

Count the bound from the drain of the command's node instead of from its record:

• Once the 50 ms have passed, a compute command or render group whose stamps lag asks its node whether every counted kick has completed (WorkStateLease::drained(): node completion word ≥ host count).
• If not, its stamps aren't late, and the wait starts again from now. The render pair stays latched for the 1 ms rescan, as before.
• A drained node whose stamps still lag after the bound keeps the existing fallback.
• A node that can't be read keeps the old bound.

The node is read only when the bound has already passed, so the common path is unchanged.

What a stampless retirement releases, and when that can still happen

Retirement (Deferred::Retired → Completion::complete() / release_after_retirement()) does the following:

• signals the command's member fence (the ioctl's out-fence once all members are done);
• publishes its timestamps (Output::publish());
• releases its input sources, its VM job pin (vm_job) and, for a render, its render-slot lease;
• for compute, frees its slot in the queue's 16-entry FIFO;
• for a render, completes its free-list/QoS accounting and retires its slot memory.

With this change none of that happens on the fallback while the command's node still runs. A stampless retirement now needs one of two things:

• the node has drained and the stamps still lag for 50 ms; or
• the scheduler watchdog has expired an executed, consumed render group (expire_stamp_wait(), which sets the latch to 0). That one is unchanged and doesn't consult the node.

The #555 stall is still covered

The original render stall (#555) had the stamps stop one kick short while the group and later groups had executed, and no later event moved them:

• If that node has drained, the stamps lag past the bound exactly as before, and the group retires on its pass record after 50 ms.
• If that node never drains, the group no longer retires at 50 ms. The watchdog's last resort still retires it on its pass record at expiry instead of failing the client. The KUnit cases cover both paths.

No hardware repro of a drained-node stall exists today. In this PR's qualification boot, nothing retired stampless: no render or compute fallback line at all, including the first heavy CTS load after boot, where #555's stall used to show up once per boot.

Qualification

• KUnit: asahi_g17* 155/155 under QEMU arm64, 16K pages (aurora-asahi-1028.kunitconfig). Two new cases:

  • asahi_g17_compute_stamp_wait::a_stamp_waits_for_the_rest_of_its_ioctl;
  • asahi_g17_render_retirement::stamps_wait_for_the_rest_of_the_ioctl, which also checks that the watchdog's expiry doesn't wait for the node.

  On the omacom tree (mirror, stacked on drm/asahi: retire a G17 render group on its pass record when its stamps stall (stacked on #244) omacom/linux-aurora#264) it is 130/130.

• Neo, stock kit clpcd2 with this change as its asahi module (drm/asahi: g17: keep a work-state node for the other engine #570 + drm/asahi: g17: admit a context's ioctls to work states in submission order #571 base). One fresh one-shot boot:

  • Regression gate: PERFORMANCE-REGRESSION-PASS.
  • Targeted CTS: 4687 Pass, 0 Fail, 681 NotSupported (same as drm/asahi: g17: admit a context's ioctls to work states in submission order #571).
  • Full synchronization2: 36516 Pass, 0 non-pass, 3 min 47 s.
  • Full synchronization: 28647 Pass, 0 non-pass, 3 min 14 s.
  • Stampless retirements over the whole boot: 0. No compute or render fallback line was printed (the first one always prints). The same groups on stock print ~9.7k for synchronization2 and ~9.1k for synchronization.
  • No asahi/G17 error, warning, timeout, fault or recovery line.

• Diagnostic A/B with the node gate as a knob (same boot, synchronization2): 9792 fallbacks with the knob off, 17 with it on. Those 17 retired on the record the moment their node drained and their stamp landed 0.0–0.3 ms later. The production change restarts the bound at that point instead, so they retire on their stamps.

Performance

Same-boot A/B on the Neo. A diagnostic twin of this change has a knob that restores the bound from the record (the old behaviour). Arms were alternated with rotated order, 6 rounds, quietdesk on. Medians, with the change vs the old bound (positive = faster with the change):

Rows	Delta
render.light tri d1/d2/d4/d8	−0.2 / −0.2 / −0.3 / −0.4%
render.light tri d32/d48/d64/d128	−0.8 / −1.2 / −0.8 / −1.1%
compute.light d1/d2/d4/d8/d64	−0.3 / −1.1 / +0.3 / −1.3 / +1.0%
render.heavy fill 1080p d1–d8, copy.bw fill d1–d8, compute.heavy alu d16	−0.4…+0.1%
render.light queues d4/d8, sync.signal timeline, concurrent.cr interleave/queues	−0.1…+0.3%

All of this is noise. The two arms run the same instructions on these rows: the node is read only once a stamp wait has passed 50 ms, and no stampless line appeared during the A/B (the harness stops at the first one). Every median with the change lies within the old arm's own spread over its rounds (render.light d32: old 26.76–27.85 µs, with the change 27.44; compute.light d8: old 15.11–15.53 µs, with the change 15.37). The cross-module regression gate above passed as well.