Error: 400: {"type":"invalid_request_error","message":"prompt (38963 tokens) + max tokens (226281) exceeds the context (262144); requests are never truncated"} #545

Activity

1. Niko1221

   added a commit that references this issue on Oct 3, 2026

   serve: the "exceeds the context" 400 says how to get past it (#545)

   ...

   dca932f
2. Niko1221 commented on Oct 3, 2026

   

   Niko1221

   on Oct 3, 2026

   Owner

   More actions

   Thanks. Pi asked for 226,281 output tokens on a 38,963-token prompt. Together that is about 3K over the 262,144-token context, and the server refuses such a request instead of cutting anything. Pi bases the output size on its context setting, so a larger context doesn't help. Two ways around it:

   • add "fit_max_tokens": true to the model's strata-<model>.json in the Strata folder and restart. The server then shortens max_tokens to the room that's left;
   • or give the model a smaller max output in Pi's model settings (e.g. 32768).

   From 0.1.38 the error message itself says this and gives the largest max_tokens that fits.
3. Niko1221

   closed this as completedon Oct 3, 2026
4. avarakin commented on Oct 3, 2026

   

   avarakin

   on Oct 3, 2026

   ContributorAuthor

   More actions

   I added the parameter but still not able to open problematic session. I am able to open it with another qwen model and llama.cpp
5. avarakin commented on Oct 3, 2026

   

   avarakin

   on Oct 3, 2026

   ContributorAuthor

   More actions

   one more thing: looks like it happens after compaction. I manually requested pi to compact, compact was successful, but after that i started getting this error and was not able to re-open session
6. avarakin commented on Oct 3, 2026

   

   avarakin

   on Oct 3, 2026

   ContributorAuthor

   More actions

   Never mind, it worked. I applied the change incorrectly.
7. Niko1221

   added a commit that references this issue on Oct 6, 2026

   serve: the "exceeds the context" 400 says how to get past it (#545)

   ...

   2233e81
8. anon761

   added 2 commits that reference this issue on Oct 6, 2026

   serve: the "exceeds the context" 400 says how to get past it (Niko122…

   ...

   64e275d

   serve: the "exceeds the context" 400 says how to get past it (Niko122…

   ...

   2876d1a