Skip to content

opencl: default the FA c8 cluster width to 16 on X1E (performance tuning) #26433

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lhez merged 1 commit into from
Aug 12, 2026
Merged

opencl: default the FA c8 cluster width to 16 on X1E (performance tuning) #26433

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion ggml/src/ggml-opencl/ggml-opencl.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4911,8 +4911,13 @@ static bool ggml_opencl_ensure_fa_variant(ggml_backend_opencl_context * backend_
const int x = (e && e[0]) ? atoi(e) : 0;
return (x == 8 || x == 16 || x == 32) ? x : 0; // 0 = per-gen default
}();
// X2E needs 16 to keep per-lane o_acc at 128B (the compiler spills the
// kernel-default width); X1E does not spill, but C=16 is still a measured
// +28-30% DK128-GQA4 decode win there (X1-85, kv 4096/8192), neutral on
// DK64 / GQA1 / quant-KV.
const int fa_cl_c_gqa4 = fa_cl_c_env ? fa_cl_c_env
: (backend_ctx->adreno_gen == ADRENO_GPU_GEN::X2E ? 16 : 0);
: (backend_ctx->adreno_gen == ADRENO_GPU_GEN::X2E ||
backend_ctx->adreno_gen == ADRENO_GPU_GEN::X1E ? 16 : 0);
const std::string opts_cl_c_gqa4 = fa_cl_c_gqa4
? " -D FA_CL_C=" + std::to_string(fa_cl_c_gqa4) : std::string();
const std::string fa_cl_c_g8_val = std::to_string(fa_cl_c_gqa4 ? fa_cl_c_gqa4 * 2 : 16);
Expand Down
Loading