You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
jinja input marking is never consumed by llama-server #26532
#18462 added jinja::string with an is_input flag so special tokens coming from user content can be told apart from ones the template emitted, and listed "implement input marking on llama-server" as a follow-up. That follow-up doesn't seem to have landed.
At 3581ba0 the flag is produced but nothing reads it:
tools/server/ has zero references to is_input or mark_input
all four tokenize_input_prompts calls in server-context.cpp pass parse_special = true
So a two-message conversation with a ChatML payload in the user content tokenizes to the same ids as a real system turn (151644, 8948 on qwen2.5). Nothing downstream can distinguish them.
I know special token injection is a known class and this is a documented TODO rather than a surprise. What made me want to bring it up, transformers has no working equivalent. apply_chat_template(tokenize=True) forges the turns, split_special_tokens isn't a parameter it accepts, and on the two-step path that flag strips the template's own delimiters too, leaving no role structure at all. Keeping the template's delimiters while dropping the content's needs origin tracking, and is_input looks like the only implementation of it anywhere.
Is the follow-up still on the roadmap, or was it dropped deliberately? Happy to help if it's just unowned.
Activity
CISC commentedon Aug 4, 2026
I think it's still on @ngxson TODO, but it's not trivial (if at all 100% feasible) so not high priority.