You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
/v1/tokenize returns a flat list when the text array holds a single string #4664
The shape of the /v1/tokenize response follows the number of strings in the request instead of the
form of the request. src/tokenize/tokenize_parser.cpp nests the tokens only when the tokenizer
output has more than one row (outputShape[0] > 1), and a one-element array produces one row, the
same as a plain string:
A client that sends a list and reads tokens[i] for each input gets a list of token lists for
two or more inputs, but a flat list of integers for exactly one. Batches that happen to contain one
string need special handling on the client.
None of the tests send a one-element array: the list cases in tokenize_parser_test.cpp and tokenize_endpoint_test.cpp all use two or more strings. The docs show only the single-string
response.
Reproduced with the parser and a real Qwen3 tokenizer, following the same sequence as src/llm/servable.cpp: parse the request, encode, build the response.
Expected behavior
An array of strings always gives a list of token lists, including an array with one string.
Note that this changes the response for clients that currently send a one-element array and rely on
the flat list.
Activity
Return a list of token lists for a text array with one string (#4665)