Refresh backend README (#21385)

[JCNTH]

Summary:

Document the current WebGPU backend and measured browser results

The existing README describes an early five-operator prototype. This refresh aligns it with the landed backend and validated browser artifacts.

Key changes:

• README.md — document the current architecture and representative 100+ operator registry
• README.md — report accepted Llama browser medians and conservative Qwen validation status
• README.md — replace stale setup and test guidance with current paths

Documentation only; runtime behavior is unchanged.

Co-authored-with: Claude Code.

Reviewed By: psiddh

Differential Revision: D113580415

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21385

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-codesync]

@JCNTH has exported this pull request. If you are a Meta employee, you can view the originating Diff in D113580415.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[Copilot]

Pull request overview

Refreshes the WebGPU backend README to reflect the current backend architecture, operator surface, and measured browser performance/validation claims.

Changes:

• Updates the README’s backend overview, architecture diagram, and operator support summary.
• Adds a performance/results section and an optimization-techniques summary.
• Rewrites the quick-start/testing guidance and updates the directory structure sketch.

Comments suppressed due to low confidence (3)

backends/webgpu/README.md:121

• "This scripted path" is ambiguous (no script is referenced here). Since test_build_webgpu.sh is the scripted entrypoint and it sources .ci/scripts/setup-webgpu-linux-deps.sh to install Dawn/SwiftShader, call that out explicitly so readers know what to run and where the deps come from.

This scripted path installs a Dawn and SwiftShader prebuilt for Linux. On
macOS, provide a configured Dawn installation instead of using this script.

backends/webgpu/README.md:154

• The quick-start exports model.pte, but the follow-up test_build_webgpu.sh script doesn’t run that artifact; it exports its own reference models under /tmp and runs native tests. Clarifying this avoids users thinking their exported .pte is being exercised.

The script exports a `.pte`, builds the native runtime, and validates GPU
output.

backends/webgpu/README.md:176

• The directory tree under scripts/ is incomplete: the repo currently also has backends/webgpu/scripts/test_webgpu_native_ci.sh. Keeping this listing accurate helps readers discover the right entrypoints.

├── scripts/
│   └── gen_wgsl_headers.py          # Generate embedded WGSL headers

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[JCNTH]

@claude Review for staleness, hard coding, and support for scaling webgpu in future

[claude]

Claude finished @JCNTH's task in 2m 11s —— View job

---

Review: backends/webgpu/README.md — staleness, hard-coding, and future scaling

I verified every claim in the refreshed README against the current tree (backends/webgpu/). Findings below are organized by the three axes you asked about.

---

🔴 Hard-coded values that will drift

1. 89 Python operator-test modules (line 109)
This is stale-prone and, depending on how you count, already slightly off:

• 87 test_*.py at the top level of test/ops/
• 89 if you count test_*.py recursively (two more live in test/ops/index/ and test/ops/dynamic_shape/)

So "89" is defensible today, but the count changes every time an op test is added. Copilot flagged the same thing. Recommend dropping the number:

Operator tests: Python operator-test modules under test/ops/ exercise export and delegation.

Fix this →

2. more than 100 operator symbols (lines 10, 52) — this one is good ✅
There are currently 133 WEBGPU_REGISTER_OP registrations. The "more than 100" phrasing is a floor rather than an exact count, so it ages gracefully and stays true as ops are added. Keep this style.

3. Performance table (lines 20–26)
2,176 tok/s prefill / 151 tok/s decode on a specific artifact will drift as kernels improve. The surrounding caveats ("accepted hash-pinned artifact, not the performance of every model or hardware configuration") are exactly the right hedge, so this is acceptable — but consider a "measured YYYY-MM" datestamp so a future reader can tell whether the number is current.

---

🟡 Staleness / accuracy against the tree

4. Directory structure omits test_webgpu_native_ci.sh (line 175)
scripts/ currently contains two files, but the README lists only gen_wgsl_headers.py:

backends/webgpu/scripts/
├── gen_wgsl_headers.py
└── test_webgpu_native_ci.sh   # <-- missing from the README tree

Copilot flagged this too. Worth adding so readers can find the CI entrypoint.

5. Quick Start: exported model.pte is never exercised by the script (lines 143–154)
Section 1 walks the user through exporting model.pte, then Section 2 runs test_build_webgpu.sh. But that script does not consume model.pte — it exports its own reference models to /tmp (/tmp/dispatch_order, /tmp/webgpu_update_cache_test.pte, SDPA sweeps) and runs native tests against those. A reader will reasonably assume their .pte is being validated. Either note that the export snippet is illustrative-only, or have Section 2 actually run the artifact from Section 1.

6. "This scripted path installs a Dawn and SwiftShader prebuilt" (line 119)
The dependency install is real but happens indirectly — test_build_webgpu.sh sources .ci/scripts/setup-webgpu-linux-deps.sh (confirmed at line 78 of the script). Since this sentence sits above the section that first names the script, "this scripted path" has no antecedent yet. Naming the script and the sourced deps file would remove the ambiguity Copilot noted.

---

🟢 Architecture claims — verified correct

• VH00 header + VK00 FlatBuffer identifier: confirmed (WebGPUDelegateHeader.cpp:31 kExpectedMagic = {'V','H','0','0'}; WebGPUBackend.cpp:78 expects VK00). The "Reuses Vulkan serialization" note is accurate.
• "No Python AOT layer" / consumes .pte exported with VulkanPartitioner: confirmed — tester.py lowers via VulkanPartitioner and tests assert a VulkanBackend delegate id.
• Embedded WGSL headers with build-time drift check: confirmed (gen_wgsl_headers.py --check in the build script).

---

📈 Support for scaling WebGPU in the future

The doc is mostly well-positioned for growth, with two structural risks:

• Prefer floors and generated lists over fixed counts. The "more than 100 operators" pattern (Re-sync with internal repository #2) is the right model; the "89 test modules" count (Add support for quantized LeakyReLU #1) is the anti-pattern. As the backend scales, every exact count becomes a maintenance liability. If you want an accurate op/test inventory in the README long-term, generate it (e.g. a small doc-gen step or a --check that fails when the README count drifts, mirroring the WGSL header drift check you already have). Otherwise stay qualitative.

• The "Vulkan-derived" framing needs a long-term stance. The README leans on Vulkan serialization/partitioner/VK00 today. That's honest about the current implementation, but as WebGPU scales it will help readers to state intent: is reusing the Vulkan FlatBuffer a permanent design choice or a bootstrapping shortcut? A one-line note on the intended trajectory (e.g. "WebGPU will continue to consume the Vulkan serialization" vs. "a dedicated WebGPU serialization is planned") prevents the doc from reading as accidentally coupled.

• Per-model validation table (lines 41–48) scales fine — adding rows as models are validated is low-friction, and the "registry coverage alone does not guarantee end-to-end" caveat is exactly right for setting expectations as the op surface grows.

---

Summary: Documentation-only refresh is a solid improvement over the stale prototype README, and the architecture claims all check out. The one change I'd treat as blocking is the hard-coded 89 (#1); #4 and #5 are worth fixing while you're here since they can mislead. The "more than 100 operators" phrasing is the pattern to standardize on for future scaling.

---

• Read the changed README
• Verify claims against the codebase
• Assess hard-coding
• Assess scaling implications
• Post review
  · branch export-D113580415

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

[JCNTH]

@claude Review for staleness, hard coding, and support for scaling webgpu in future

[claude]

Claude finished @JCNTH's task in 1m 13s —— View job

---

Re-review: backends/webgpu/README.md — staleness, hard-coding, future scaling

I re-verified the refreshed README against the current tree. Every finding from my earlier review has been addressed, and the remaining claims all check out. Details below.

---

✅ Previously-flagged issues — all resolved

#	Earlier finding	Status in current README
1	Hard-coded 89 Python operator-test modules	Fixed — line 110 now reads "Python operator-test modules exercise export and delegation" with no count.
4	scripts/ tree omitted test_webgpu_native_ci.sh	Fixed — line 183 now lists it with a # CI entry point for native tests comment.
5	Quick Start implied model.pte was validated by the script	Fixed — lines 150–151 explicitly state "This snippet demonstrates the export path. The validation script below exports and runs its own native reference models; it does not consume model.pte."
6	"This scripted path" had no antecedent	Fixed — lines 120–123 now name test_build_webgpu.sh and the sourced .ci/scripts/setup-webgpu-linux-deps.sh, and clarify the macOS path.

The macOS caveat also resolves Copilot's second-round comment about setup-webgpu-linux-deps.sh being Linux/apt-specific — the README now says macOS users should "provide a configured Dawn installation instead of using this script."

---

✅ Hard-coding posture — now consistent

• more than 100 operator symbols (lines 10, 53): verified — there are currently 219 WEBGPU_REGISTER_OP call sites under runtime/ops/. "More than 100" is a floor that ages gracefully. This is the right pattern and it's now used everywhere; the one exact count (89) that would drift has been removed. 👍
• Performance table (lines 16–24): now carries a "Measured in July 2026" datestamp plus the hash-pinned-artifact caveat — exactly the hedge I asked for. A future reader can tell whether the numbers are current.

---

✅ Architecture & tree claims — verified against source

• VH00 header + VK00 FlatBuffer identifier: confirmed — WebGPUDelegateHeader.cpp:31 (kExpectedMagic = {'V','H','0','0'}) and WebGPUBackend.cpp:78 (expects VK00).
• "No Python AOT layer" / consumes .pte exported with VulkanPartitioner: confirmed — test/tester.py lowers via VulkanPartitioner (line 89 comment: "WebGPU consumes the Vulkan VK00 serialization").
• WGSL drift check: confirmed — test_build_webgpu.sh:19 runs gen_wgsl_headers.py --check and fails on drift.
• scripts/ tree matches the two files actually present (gen_wgsl_headers.py, test_webgpu_native_ci.sh).

---

📈 Scaling posture — good, one optional suggestion

The doc is well-positioned for growth: floors-over-counts is now applied consistently, the per-model validation table (lines 42–46) adds rows cheaply, and the "registry coverage alone does not guarantee end-to-end" caveat sets the right expectation as the op surface grows.

One optional item carried over from last round (not blocking):

• State the long-term stance on the Vulkan-derived serialization. The README leans on the Vulkan FlatBuffer / partitioner / VK00 today and is honest about it, but doesn't say whether that reuse is a permanent design choice or a bootstrapping shortcut. A one-line note on intent would keep the doc from reading as accidentally coupled as WebGPU scales. Purely a nicety — nothing here is inaccurate.

---

Verdict: All blocking and should-fix items from the prior review are resolved and every architecture claim verifies against the tree. This is ready from a staleness/hard-coding standpoint. The remaining Vulkan-intent note is optional polish.

• Read the current README (post-updates)
• Verify operator registry / test counts against the tree
• Verify directory structure & script references
• Verify architecture claims (VH00/VK00, partitioner)
• Assess hard-coding & scaling posture
• Post findings
  · branch export-D113580415

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

backends/webgpu/README.md:123

• The “Linux Native Quick Start” section mentions macOS, but doesn’t describe a runnable macOS path and could be read as implying test_build_webgpu.sh works on macOS. Since the script unconditionally sources .ci/scripts/setup-webgpu-linux-deps.sh (Linux-specific), it’d be clearer to state that this dependency install flow is Linux-only and that macOS users must provide Dawn_DIR themselves.

The `test_build_webgpu.sh` flow sources
`.ci/scripts/setup-webgpu-linux-deps.sh` to install Dawn and SwiftShader
prebuilts on Linux. On macOS, provide a configured Dawn installation instead
of using this script.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

backends/webgpu/README.md:134

• The README’s sample export code imports VulkanPartitioner from an internal module path. The public import (executorch.backends.vulkan) already re-exports VulkanPartitioner, and this is the import style used by WebGPU op tests (e.g. backends/webgpu/test/ops/test_sdpa.py). Using the public import keeps the documentation aligned with the stable API surface and avoids coupling to internal module layout.

from executorch.backends.vulkan.partitioner.vulkan_partitioner import (
    VulkanPartitioner,
)

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

backends/webgpu/README.md:145

• The snippet imports VulkanPartitioner from the deep module path, even though executorch.backends.vulkan re-exports VulkanPartitioner as a public entrypoint. Using the public import keeps the README example shorter and avoids coupling docs to internal module layout.

from executorch.backends.vulkan.partitioner.vulkan_partitioner import (
    VulkanPartitioner,
)

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.