Skip to content

cuda : add GGML_CUDA_GRAPH_DEBUG and read env vars once - #30327

Merged
ggerganov merged 6 commits into
masterfrom
gg/cuda-gate-debug-logs
Oct 13, 2026
Merged

cuda : add GGML_CUDA_GRAPH_DEBUG and read env vars once#30327
ggerganov merged 6 commits into
masterfrom
gg/cuda-gate-debug-logs

Conversation

@ggerganov

@ggerganov ggerganov commented Oct 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Overview

  • The CUDA graph debug logs (warmup complete/reset, graph id reused, graph update failure, graph disabling) are now gated behind the new GGML_CUDA_GRAPH_DEBUG env var; they only print when it is set to a non-zero value, and are silent by default even at debug log level.
  • Env vars in ggml-cuda.cu are now read exactly once per process: duplicate getenv calls are consolidated into ggml_cuda_env_flag / ggml_cuda_env_int / ggml_cuda_env_set helpers, and per-call reads (device malloc, host malloc, host buffer register/unregister, device memory/props, P2P, RPC min batch) now go through lazily-initialized static accessors.
  • No behavior change except the graph log gating; the presence-based semantics of GGML_CUDA_P2P (set = on, even with value 0) are preserved.

Additional information

  • The new env var GGML_CUDA_GRAPH_DEBUG controls the per-call CUDA graph debug logs: set it to a non-zero value (e.g. GGML_CUDA_GRAPH_DEBUG=1) to print them; when unset they stay silent even at debug log level.

Requirements

@github-actions github-actions Bot added ggml changes relating to the ggml tensor library for machine learning CUDA Related to the CUDA backend labels Oct 11, 2026
@ggerganov
ggml-cuda : drop env helper comments
06342d0 
Assisted-by: pi:llama.cpp/Qwen3.8-27B
@ggerganov
ggerganov marked this pull request as ready for review October 11, 2026 11:27
@ggerganov
ggerganov requested a review from a team as a code owner October 11, 2026 11:27
@ggerganov ggerganov changed the title ggml-cuda : gate CUDA graph debug logs behind GGML_CUDA_GRAPH_DEBUG and read env vars once ggml-cuda : add GGML_CUDA_GRAPH_DEBUG and read env vars once Oct 11, 2026
@ggerganov ggerganov changed the title ggml-cuda : add GGML_CUDA_GRAPH_DEBUG and read env vars once cuda : add GGML_CUDA_GRAPH_DEBUG and read env vars once Oct 11, 2026
@ggerganov

ggerganov commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

Bump - should be a simple review.

@ggerganov
ggerganov merged commit f851e24 into master Oct 13, 2026
17 checks passed
@ggerganov
ggerganov deleted the gg/cuda-gate-debug-logs branch October 13, 2026 12:24
@ggerganov ggerganov mentioned this pull request Oct 15, 2026
Merged
@BrewTestBot BrewTestBot mentioned this pull request Oct 15, 2026
Merged
1 task done
mclrcha added a commit to mclrcha/llama.cpp that referenced this pull request Oct 16, 2026
@mclrcha
Merge ggml-org/llama.cpp master (7182ef7) into r9700-rocm-performance
3c23e34 
153 upstream commits (merge-base 1623d8c). Conflicts (6 files) and semantic fixes:
- gated_delta_net.cu: fork kernels (column_lanes, indexed state, RDNA4 chunked WY prefill) kept; upstream GDN MMA
  path (ggml-org#29353, HIP only on RDNA3.5) runs after ours, skipped when the state gather was deferred to the op.
- ggml-cuda.cu: n_stable reset kept, warmup reset log behind GGML_CUDA_GRAPH_DEBUG (ggml-org#30327).
- llama-batch: upstream state embeddings for MTP hooks (ggml-org#30257) next to the fork embd alias; R57 extended to them:
  the allocator points at the batch state rows and a ubatch of consecutive rows reads them in place (no copies).
- llama-context: upstream sched_new (realloc callback, ggml-org#30329) also used for the second MTP scheduler.
- llama-graph: draft LM head and logits top-k kept next to build_hadamard_input / build_logits; the draft head
  applies the Hadamard input transform too.
- llama-model: load_tensors_hadamard, then build_mtp_draft_head.
- common: R59 full-ubatch warmup also sets zero state embeddings.
edwardyoon pushed a commit to edwardyoon/focus-llama that referenced this pull request Oct 24, 2026
@ggerganov @edwardyoon
cuda : add GGML_CUDA_GRAPH_DEBUG and read env vars once (ggml-org#30327)
508a49a 
* ggml-cuda: gate CUDA graph debug logs behind GGML_CUDA_GRAPH_DEBUG

* ggml-cuda: cache env var lookups in static const instead of re-reading per call

* ggml-cuda: read env vars exactly once via ggml_cuda_env_flag/ggml_cuda_env_int

* ggml-cuda: consolidate env flag reads into single static accessors

* ggml-cuda : drop env helper comments

Assisted-by: pi:llama.cpp/Qwen3.8-27B

* cont : remove NDEBUG gates for CUDA graph logs
auRose94 pushed a commit to auRose94/llama.cpp that referenced this pull request Oct 28, 2026
@ggerganov
cuda : add GGML_CUDA_GRAPH_DEBUG and read env vars once (ggml-org#30327)
82490d6 
* ggml-cuda: gate CUDA graph debug logs behind GGML_CUDA_GRAPH_DEBUG

* ggml-cuda: cache env var lookups in static const instead of re-reading per call

* ggml-cuda: read env vars exactly once via ggml_cuda_env_flag/ggml_cuda_env_int

* ggml-cuda: consolidate env flag reads into single static accessors

* ggml-cuda : drop env helper comments

Assisted-by: pi:llama.cpp/Qwen3.8-27B

* cont : remove NDEBUG gates for CUDA graph logs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@am17an am17an am17an approved these changes

Assignees

No one assigned

Labels

CUDA Related to the CUDA backend ggml changes relating to the ggml tensor library for machine learning

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ggerganov @am17an