Repository navigation
server: add exceed_context_size_error type - #15780
Merged
Merged
Conversation
ggerganov
approved these changes
Sep 4, 2025
allozaur
approved these changes
Sep 4, 2025
gabe-l-hart
added a commit
to gabe-l-hart/llama.cpp
that referenced
this pull request
Sep 4, 2025
…upport * origin/master: (72 commits) metal : Add template specialization for mul_mm_id w/ ne20 == 10 (ggml-org#15799) llama : set n_outputs to 1 to avoid 0 outputs mean-pooling (ggml-org#15791) CANN: Refactor ND to NZ workspace to be per-device (ggml-org#15763) server: add exceed_context_size_error type (ggml-org#15780) Document the new max GPU layers default in help (ggml-org#15771) ggml: add ops for WAN video model (cuda && cpu) (ggml-org#15669) CANN: Fix precision issue on 310I DUO multi-devices (ggml-org#15784) opencl: add hs=40 to FA (ggml-org#15758) CANN: fix acl_rstd allocation size in ggml_cann_rms_norm (ggml-org#15760) vulkan: fix mmv subgroup16 selection (ggml-org#15775) vulkan: don't use std::string in load_shaders, to improve compile time (ggml-org#15724) vulkan : update ggml_vk_instance_validation_ext_available (ggml-org#15666) ggml vulkan: add hardsigmoid and hardswish operations (ggml-org#15762) CUDA: Optimize `rms_norm_f32` kernel and its fused variants, giving 1-6% perf E2E (ggml-org#15715) model-conversion : fix pyright errors (ggml-org#15770) sampling : optimize dist sampler (ggml-org#15704) llama : fix incorrect model type for Gemma 270M (ggml-org#15764) model-conversion : remove hardcoded /bin/bash shebangs [no ci] (ggml-org#15765) CANN: Add RoPE contiguous check for 310I DUP device (ggml-org#15735) ggml-cpu : optimize RVV kernels (ggml-org#15720) ...
gabe-l-hart
added a commit
to gabe-l-hart/llama.cpp
that referenced
this pull request
Sep 5, 2025
…g-model-disabled-agent-prefill * origin/master: (84 commits) CUDA: fastdiv, launch bounds for mmvq + q8_1 quant (ggml-org#15802) tests : add --list-ops and --show-coverage options (ggml-org#15745) gguf: gguf_writer refactor (ggml-org#15691) kv-cache : fix SWA checks + disable cacheless iSWA (ggml-org#15811) model-conversion : add --embeddings flag to modelcard.template [no ci] (ggml-org#15801) chat : fixed crash when Hermes 2 <tool_call> had a newline before it (ggml-org#15639) chat : nemotron thinking & toolcalling support (ggml-org#15676) scripts : add Jinja tester PySide6 simple app (ggml-org#15756) llama : add support for EmbeddingGemma 300m (ggml-org#15798) metal : Add template specialization for mul_mm_id w/ ne20 == 10 (ggml-org#15799) llama : set n_outputs to 1 to avoid 0 outputs mean-pooling (ggml-org#15791) CANN: Refactor ND to NZ workspace to be per-device (ggml-org#15763) server: add exceed_context_size_error type (ggml-org#15780) Document the new max GPU layers default in help (ggml-org#15771) ggml: add ops for WAN video model (cuda && cpu) (ggml-org#15669) CANN: Fix precision issue on 310I DUO multi-devices (ggml-org#15784) opencl: add hs=40 to FA (ggml-org#15758) CANN: fix acl_rstd allocation size in ggml_cann_rms_norm (ggml-org#15760) vulkan: fix mmv subgroup16 selection (ggml-org#15775) vulkan: don't use std::string in load_shaders, to improve compile time (ggml-org#15724) ...
walidbr
pushed a commit
to walidbr/llama.cpp
that referenced
this pull request
Sep 7, 2025
* server: add exceed_context_size_error type * change error code to 400
blime4
referenced
this pull request
in blime4/llama.cpp
Feb 5, 2026
* server: add exceed_context_size_error type * change error code to 400
Seunghhon
pushed a commit
to Seunghhon/llama.cpp
that referenced
this pull request
Apr 26, 2026
* server: add exceed_context_size_error type * change error code to 400
my-other-github-account
pushed a commit
to my-other-github-account/llama.cpp
that referenced
this pull request
May 15, 2026
* server: add exceed_context_size_error type * change error code to 400
fukuro-kun
pushed a commit
to fukuro-kun/fukuro-llama-cpp-turboquant
that referenced
this pull request
Jul 5, 2026
* server: add exceed_context_size_error type * change error code to 400
MrLordCat
referenced
this pull request
in MrLordCat/llama.cpp-rdna-lab
Jul 16, 2026
* server: add exceed_context_size_error type * change error code to 400
AMD-melliott
pushed a commit
to AMD-melliott/llama.cpp
that referenced
this pull request
Aug 4, 2026
* server: add exceed_context_size_error type * change error code to 400
zommiommy
pushed a commit
to zommiommy/llama.cpp
that referenced
this pull request
Aug 18, 2026
* server: add exceed_context_size_error type * change error code to 400
Aspis0
added a commit
to Aspis0/kalsa
that referenced
this pull request
Oct 4, 2026
… dead-ends past n_ctx — the window reaches the send, and an engine overflow is halved and retried the way the Room does it. P1 (the owner accepted LFM2.5-VL-3B at 32_768 on slow PCs, conditional on this). Fix 1 — the window is known before the send: the `/props` read `useServerFacts` already makes now also parses and stores `n_ctx` (chat.ts `fetchSamplingDefaultsWithStatus` gains `nctx`, through the ONE parser, `parseContextSize`; the hook stores it via `rememberContextSize`, tagged with THIS read's own endpoint — never a later render's), so a text-only chat with no attach and no panel open prunes its oldest turns at send time instead of sending an unknown-size history. A model restart changes `brain_state.kind`, which the facts read already depends on: `/props` is re-asked and the new answer overwrites the cache entry, so a restart that changes `n_ctx` heals without a clear (the settings-save clear at useChat.ts:1005 stays as it was; it never fired on a restart — the facts re-read is what covers that). Fix 2 — engine overflow recovery, like the Room's (crates/kalsa-door/src/room/turn.rs:364): `streamRound` reads the body of a 400 only to classify llama.cpp's overflow answer (structured type `exceed_context_size_error` — upstream ggml-org/llama.cpp#15780 — and the sentence measured against this repo's own engine, `request (4225 tokens) exceeds the available context size (4096 tokens), try increasing it`, docs/MULTI-DEVICE-SHAPE.md:129; the engine is not vendored here, so both markers are matched) and throws kind `oversize`. The tool loop catches it, sheds the older half of the history before this turn's last user message, and asks again, down to the newest message alone; a cut through a tool exchange backs up so no orphan `tool` message goes on the wire. Only when nothing is left to shed does the failure land — and Thread's existing oversize sentence shows (the i18n key is reused; no new strings), never the generic "Wait a moment, then try again". The body's words are never logged, shown, or carried in the error. Fix 3 — the tool loop's own fit: before each follow-up round the working conversation is re-measured with the same estimator (attachments.ts `wireSize`, injected as `StreamOptions.contextFit` with budget = `n_ctx` - `CONTEXT_RESERVE_TOKENS`) and older history is pruned until it fits. Chosen and stated: the round's own tool results are NOT truncated — a page the model reads in a silently partial form is worse than a smaller window — and the engine stays the authority; if the estimate is still wrong, the 400 recovery above takes over. The estimator is injected rather than imported so toolLoop keeps the attachment/pdfjs code out of the esbuild bundles the harnesses compile (the import broke `context-size.mjs` before this shape). Fix 4 — the chars/4 ratio is unchanged and declared unmeasured: the repo has no `/tokenize` fixture (the mock has none; only dev/*.py scripts against a live engine), and the door estimates 4 bytes/token too (crates/kalsa-door/src/room/turn.rs:572), so the undercount guard is the recovery, not a larger divisor. Tests — chat/scripts/verify.mjs appended only (215 added lines, 0 removed), chat/scripts/mock-server.mjs extended: (a) `textprune` — a text-only chat on /tight, no chip and the panel never opened, prunes Q00–Q04 and keeps Q05–Q19 at send time; RED before fix 1 (mutation: `rememberContextSize` disabled → "textprune: the oldest turns fell" FAIL). (b) `overflowrecover` — the mock's new `overcontext-demo` refuses any prompt over 400 true tokens while /props says 1024, with the engine's own 400 body; the first body carries Q00, the automatic retry has shed it and includes Q12 and the current turn, the answer arrives, and the thread shows neither the generic retry text nor the oversize text; RED before fix 2 (mutation: `askRound`'s catch removed → 1 request, wait timed out, generic retry text). (c) `overflowfloor` — `overhard-demo` refuses everything: 4 requests ending at exactly [system, user], then "too much for Kalsa at once", never the retry text. (d) `toolfit` — a 300-char search result pushes the follow-up round over the tight budget and the second body has shed the older turns while the tool's words still ride (mutation: the pre-round fit check disabled → FAIL). The appended cases answer `brain_capability`, whose absence from the shared `seed` leaves the home blank — the pre-existing harness gap declared in efc338d, untouched here. Runs: cd chat && npx tsc --noEmit EXIT=0; node scripts/verify.mjs textprune overflowrecover overflowfloor toolfit EXIT=0 (17 checks); node scripts/context-size.mjs EXIT=0; node scripts/system-prompt.mjs EXIT=0; node scripts/prefill-liveness.mjs EXIT=0; node scripts/{tool-calls,thinking,slot-gate,slot-hold,tier-switch,brain-progress,brain-poll-coalescing,ui-log}.mjs EXIT=0 each; node scripts/{vision-offer,turn-failures,pdf-attach,markdown-stream,image-attach}.mjs EXIT=0 each; npm run build EXIT=0 (2079.31 kB / 619.67 kB gzip; the chunk-size warning is pre-existing). Not verified: a real engine's overflow round trip (the sentence is the one measured in MULTI-DEVICE-SHAPE and the type is upstream's, quoted — no engine runs in this session; the mock pins the contract); an Italian paragraph's real chars/token (left unmeasured, divisor left at 4); and the pre-existing verify.mjs cases prunekeep/oversizesend, which cannot run at HEAD — the shared seed answers no `brain_capability` so the home renders blank and both time out on selectors, and oversizesend additionally waits for the "exceeds the context" copy removed by 372f867. No push, no tag, no new branch.
frostyautumnleaf
pushed a commit
to frostyautumnleaf/llama.cpp
that referenced
this pull request
Oct 5, 2026
* server: add exceed_context_size_error type * change error code to 400
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prepare for #14839
Added a new error type
exceed_context_size_errorWhen a request exceed slot context size, error response will now include the token count:
{ "error": { "code": 500, "message": "the request exceeds the available context size. try increasing the context size or enable context shift", "type": "exceed_context_size_error", "n_prompt_tokens": 1407, "n_ctx": 256 } }