Skip to content

ggml-tsavorite: don't report OPU Target for ops with no kernel invocation #160

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
akapoor3518 wants to merge 1 commit into from
Closed

ggml-tsavorite: don't report OPU Target for ops with no kernel invocation #160

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion ggml/src/ggml-tsavorite/ggml-tsavorite.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6261,6 +6261,11 @@ std::lock_guard<std::mutex> _lk(g_tsavorite_compute_mutex);
src1 = node->src[1];
min_num_of_elem = 0;
max_num_of_elem = 0;
// Some op types with no kernel/blob (GET_ROWS, SOFT_MAX, etc.) are handled
// by calling ggml's own CPU compute function directly further below,
// rather than launching a Tsavorite kernel. Snapshot the kernel-run count
// here so the perf Target label below can tell the two cases apart.
const int64_t tsi_kernel_runs_before_node = node->tsi_kernel_runs;
if(node->type == GGML_TYPE_F32 && src0->type == GGML_TYPE_F32 && (!src1 || src1->type == GGML_TYPE_F32))
kernel_sub_type = DATA_TYPE_F32_INDEX;
/*
Expand Down Expand Up @@ -6871,7 +6876,15 @@ std::lock_guard<std::mutex> _lk(g_tsavorite_compute_mutex);
#if defined(GGML_PERF) || defined(GGML_PERF_RELEASE) || defined(GGML_PERF_DETAIL)
int64_t t_end = ggml_time_us();
node->perf_runs++;
node->ggml_compute_backend = GGML_COMPUTE_BACKEND_TSAVORITE;
// Only report this node as having run on OPU if a kernel was actually
// launched for it this call. Op types with no kernel (GET_ROWS, SOFT_MAX,
// etc.) reach this point too, having been computed via a direct call to
// ggml's own CPU function further up -- without this check they were
// unconditionally marked TSAVORITE/OPU despite zero kernel runs, which is
// exactly what the Target/TSI_KERNEL-RUN columns are meant to make visible.
node->ggml_compute_backend = (node->tsi_kernel_runs > tsi_kernel_runs_before_node)
? GGML_COMPUTE_BACKEND_TSAVORITE
: GGML_COMPUTE_BACKEND_CPU;

@atrivedi-tsavoritesi atrivedi-tsavoritesi Aug 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the way we used to display was everything as OPU but the txe_kernel run being counted only when txe executes it otherwise it is empty. Is it the case even now ?

if (t_end >= t_start) {
node->perf_time_us += (t_end - t_start);
} else {
Expand Down