Repository navigation
perf(openai): stop copying the whole input array when stripping namespaces - #8870
Merged
Wei-Shaw merged 2 commits intoNov 20, 2026
Merged
Conversation
…paces stripOpenAIResponsesInputNamespaces runs on every OpenAI OAuth / API-key HTTP forward attempt. Its entry check only proves that the *body* contains a namespace literal, and Codex clients advertise the image_gen namespace tool on ordinary requests, so the check passes for most traffic and the function runs even when there is nothing to strip. Three avoidable copies showed up in that path: 1. gjson.GetBytes(body, "input") materialises the input array. On a 1MiB body that is a 1MiB allocation on every request, including the no-op case. Use the zero-copy parse view (parseRawJSONView) instead. 2. The rewrite built the new input array in a buffer and then handed it to sjson.SetRawBytes, which allocates a second full body copy. Assemble the output directly instead: prefix + rewritten array + suffix in one buffer sized to the body. 3. sjson.DeleteBytes([]byte(item.Raw), ...) converted each changed item to a byte slice first. Use the string form. hasOpenAIResponsesNamespaceToolDeclaration reads input the same way and gets the same treatment; it runs for API-key accounts on every request. Benchmark (Apple M5 Pro, -benchmem), legacy = the previous implementation: 1MiB tools_only 545us / 1056768 B / 1 alloc -> 495us / 0 B / 0 alloc 1MiB strip_first 882us / 3170624 B / 8 allocs -> 548us / 1056936 B / 4 1MiB strip_last 841us / 3170624 B / 8 allocs -> 556us / 1056928 B / 4 8MiB strip_first 5.95ms / 25190720 B / 8 -> 4.30ms / 8396960 B / 4 So the no-op path stops allocating entirely and the rewriting path copies one body instead of three. Behaviour is unchanged for valid JSON: equivalence tests keep the previous implementation as legacy* and compare output bytes, errors and whether the input slice was returned untouched, across 36 bodies (duplicate keys, escaped keys, nested namespace fields, non-object items, keep-tool-call mode, missing or non-array input) plus 3000 randomized combinations and a fuzz target. One deliberate difference, pinned by a test: on input that is not valid JSON, gjson.GetBytes and gjson.Parse return different spans for the same key, and the new implementation refuses to rewrite an unparseable fragment instead of truncating it. The only caller forwards a body that the handler has already validated, so this path is not reachable in production.
|
补充一组生产环境数据,供评估这类拷贝在超大 Codex 请求体上的影响。 场景
内存实测
v0.2.22 的零拷贝改动对这条 OpenAI Responses 路径只改善了约 5%。 后果
与本 PR 的关系 我们没有保存请求体,无法从日志确认这些请求是否命中改写分支( |
This was referenced Nov 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
stripOpenAIResponsesInputNamespaces在 OpenAI OAuth / API Key 的每次 HTTP 转发尝试上都会跑(openai_gateway_forward.go:132)。它的入口预检查只有bytes.Contains(body, "\"namespace\""),而 Codex 客户端在普通请求的tools里就带image_gennamespace 声明——预检查对主流流量必然放行,即使input里没有需要清理的字段。修改
三处可以省掉的整块拷贝:
gjson.GetBytes(body, "input")→parseRawJSONView(body).Get("input")。前者会把input数组物化成一份拷贝(1 MiB body 上每次请求 1 MiB 分配),后者是零拷贝视图。sjson.SetRawBytes→ 直接组装输出。原来先把新 input 拼进bytes.Buffer,再交给SetRawBytes分配第二份完整 body;现在按prefix + 改写后的 input + suffix一次写进按 body 大小预留的缓冲,总共一份。sjson.DeleteBytes([]byte(item.Raw), ...)→sjson.Delete(item.Raw, ...),省掉每个待改 item 的[]byte转换。hasOpenAIResponsesNamespaceToolDeclaration用同样的方式读input(API Key 账号每请求都会走到),一并改掉。收益
Apple M5 Pro,
-benchmem,legacy = 改动前的实现(benchmark 里两个变体同一次运行对比):tools_onlystrip_firststrip_laststrip_firstno-op 路径(Codex 常态)不再分配;改写路径从三份完整副本降到一份。
行为保证
legacy*,比较输出字节、错误、以及是否原样返回了传入的 slice,覆盖 36 组语料(重复键、\u转义键、嵌套 namespace 字段、非对象 item、keepToolCallNamespaces两种取值、input 缺失/非数组/重复、非法 JSON),外加 3000 组确定性随机组合与一个 fuzz 目标(约 146 万次执行通过)。一处有意保留的差异
在非法 JSON 上,
gjson.GetBytes与gjson.Parse对同一个键返回的区间不同:前者一路扫到结尾,后者遇到非法结构就停。典型例子:新实现选择不改写无法解析的片段。该函数只有一个调用点,且拿到的 body 已被 handler 校验过,这条路径在生产上不可达;行为已用单独用例固定。
Fixes #8869