[reeselevine]

Overview

• Fixes issue here: WebGPU: Qwen2 models produce garbled output (repeated @ token) #21602, by doing the following:
  • updates register tiling matmul to use f32 for accumulation, for increased precision and removing NaN on some older Qwen models
  • Fixes weird issue with q4/q5_k quants that only affects Chrome. May be issue with Chrome's WebGPU compiler which we are planning on reporting soon
• Updates shader batching to 64, which should help with performance on iOS while maintaining stability
• Removes unnecessary callback_mode() accidentally introduced in ggml-webgpu: address quantization precision and backend lifecycle managment #21521.
• Also fixes issue compiling with WebGPU GPU profiling turned on.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: no

[abhijitramesh]

The f32 accumulator fix seems to be helping us. I hooked this branch with wllama and ran the Qwen models; it seems to be working as expected on Firefox and Safari now. Chrome still seems to be broken, but it's no longer printing @@@.

Chrome output:

[abhijitramesh]

Now it works on Chrome as well!