Repository navigation
feat(messages): add the Anthropic Message Batches API - #546
Merged
Merged
Conversation
Implements /v1/messages/batches as an Anthropic-dialect ingress over the
existing native-batch pipeline (same edge-translation pattern as
/v1/messages, ADR-0007):
- POST/GET/list/cancel/results routes plus DELETE for ended batches.
Batch IDs are dialect views of one resource: msgbatch_<uuid> here,
batch_<uuid> on /v1/batches.
- anthropicapi translators: create body ({requests:[{custom_id,params}]})
decodes each params as a Messages request and translates to canonical
chat items, so a batch routes to any provider with native batch
support; egress renders message_batch objects (processing_status,
request_counts, expires_at, results_url) and the results JSONL with
successful items converted back to the Anthropic Messages shape.
- /v1/messages/batches* classified as an anthropic-dialect batches
operation, so error envelopes, audit, usage, budgets, and rate limits
apply exactly like /v1/batches.
- Delete support through the stack: batchstore.Store.Delete (memory,
sqlite, postgres, mongo), orchestrator Delete (refresh, ended-only
guard, store removal), optional NativeBatchDeleteProvider with an
anthropic implementation; providers without native delete fall back to
gateway-local removal.
- OpenAI-compatible providers now materialize inline batch requests into
an uploaded JSONL input file, fixing inline submissions for the
file-based batch API on both dialects.
Verified live with the Anthropic Python SDK: full lifecycle including an
OpenAI gpt-4o-mini batch created, polled to ended, results streamed as
Anthropic messages, and deleted; plus anthropic-native create, retrieve,
list, and cancel.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (5)
📝 WalkthroughWalkthroughAdds full Anthropic Message Batches support, translating requests and results through the canonical batch pipeline, exposing lifecycle endpoints, supporting deletion across stores and providers, handling OpenAI-compatible inline inputs, and documenting the API. ChangesAnthropic Message Batches
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/anthropicapi/batch.go`:
- Around line 118-127: Update the customID validation in the batch request
processing flow to use trimming only for blank-value detection, while retaining
the original item.CustomID for deduplication, storage, and returned results.
Ensure IDs differing only by surrounding whitespace remain distinct and preserve
the caller’s exact correlation key.
- Around line 201-205: Update the ended-batch handling in the response
normalization flow around firstUnix and out.ProcessingStatus so it does not
derive ended_at from time.Now() on each retrieval. Use the provider’s persisted
transition timestamp when available, or a stable canonical fallback that remains
identical across responses, while preserving existing timestamp conversion
behavior.
In `@internal/providers/openai/compatible_provider_test.go`:
- Around line 239-308: Expand
TestCompatibleProvider_CreateBatch_UploadsInlineRequestsAsInputFile into a
table-driven test with separate cases for successful creation, /files failure,
and /batches failure. Configure the mock server per case, assert that
file-upload errors prevent any /batches request, and verify batch-creation
errors are returned after a successful upload while preserving the existing
success assertions.
In `@internal/providers/openai/compatible_provider.go`:
- Around line 438-446: Update the batch preparation and creation flow around
uploadInlineBatchInput to track the gateway-created temporary file in batch
metadata, then perform best-effort deletion whenever a subsequent /batches
operation fails, including the additional failure path around lines 473-484.
Preserve the original error while attempting cleanup, and do not delete
caller-provided InputFileID files.
In `@internal/server/messages_batch_handler_test.go`:
- Around line 50-53: Replace the map[string]any response decoding and subsequent
type assertions in the affected tests with Anthropic response structs or
dedicated typed test DTOs. Update the assertions to access typed fields
directly, including the response handling around the create, batch, and related
test cases, while preserving the existing validation behavior.
In `@internal/server/messages_batch_service.go`:
- Around line 139-143: Update the batch-results response flow to avoid calling
EncodeBatchResults or buffering the complete payload. Iterate through
result.Response and encode each result directly to the response writer as JSONL,
setting the application/x-jsonl content type and preserving appropriate error
handling for encoding or write failures.
In `@internal/server/messages_handler.go`:
- Around line 135-146: Update the `@Produce` annotation in MessagesBatchResults to
advertise application/x-jsonl instead of application/octet-stream, matching the
endpoint’s JSONL response while leaving the handler behavior unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: ce144f5d-60f5-4ce0-ac34-d51c49fff8f2
📒 Files selected for processing (26)
docs/advanced/anthropic-messages-api.mdxinternal/anthropicapi/batch.gointernal/anthropicapi/batch_test.gointernal/batch/store.gointernal/batch/store_memory.gointernal/batch/store_memory_test.gointernal/batch/store_mongodb.gointernal/batch/store_postgresql.gointernal/batch/store_sqlite.gointernal/batch/store_sqlite_test.gointernal/core/batch.gointernal/core/endpoints.gointernal/core/endpoints_test.gointernal/core/interfaces.gointernal/core/semantic.gointernal/core/semantic_test.gointernal/gateway/batch_orchestrator.gointernal/providers/anthropic/batch.gointernal/providers/openai/compatible_provider.gointernal/providers/openai/compatible_provider_test.gointernal/providers/router.gointernal/server/handlers_test.gointernal/server/http.gointernal/server/messages_batch_handler_test.gointernal/server/messages_batch_service.gointernal/server/messages_handler.go
Comment on lines
+50
to
+53
| var created map[string]any | ||
| if err := json.Unmarshal(rec.Body.Bytes(), &created); err != nil { | ||
| t.Fatalf("decode create response: %v", err) | ||
| } |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
Decode API responses into typed structs.
These map[string]any assertions weaken schema checks and introduce panic-prone type assertions. Use the Anthropic response structs or dedicated typed test DTOs.
Based on learnings, API request and response payloads under internal/ should prefer strongly typed structs over map[string]any.
Also applies to: 63-75, 96-101, 111-119, 215-220, 267-272
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@internal/server/messages_batch_handler_test.go` around lines 50 - 53, Replace
the map[string]any response decoding and subsequent type assertions in the
affected tests with Anthropic response structs or dedicated typed test DTOs.
Update the assertions to access typed fields directly, including the response
handling around the create, batch, and related test cases, while preserving the
existing validation behavior.
Source: Learnings
Contributor
Author
There was a problem hiding this comment.
Skipping: map-based decoding is the established pattern across this package's handler tests (see handlers_test.go); switching one file would leave the suite inconsistent without changing coverage.
Contributor
There was a problem hiding this comment.
@SantiagoDePolonia, agreed. Keeping the established map-based decoding pattern in internal/server handler tests is reasonable here; a one-file typed-DTO conversion would be inconsistent without improving coverage. I’ll avoid raising this as a standalone concern in similar handler tests.
🐇
✏️ Learnings added
Learnt from: SantiagoDePolonia
Repo: ENTERPILOT/GoModel PR: 546
File: internal/server/messages_batch_handler_test.go:50-53
Timestamp: 2026-07-17T16:01:32.557Z
Learning: In the Go `internal/server` package, handler tests conventionally decode HTTP JSON responses into `map[string]any` and assert selected fields dynamically (for example, in `internal/server/handlers_test.go`). Do not request a one-off conversion to typed response DTOs in individual handler test files unless the package-wide convention is being changed or dynamic decoding causes a concrete test defect.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
🧠 Learnings used
Learnt from: SantiagoDePolonia
Repo: ENTERPILOT/GOModel PR: 33
File: internal/auditlog/auditlog_test.go:272-279
Timestamp: 2026-01-15T23:07:26.646Z
Learning: Go code guideline: Avoid using interface{} and map[string]interface{} for API request/response payload types. This should be restricted to public API surfaces and data payloads. It is acceptable to use interface{} in internal infrastructure packages (e.g., storage abstractions) only to break import cycles, and only when type assertions occur at initialization (not on every request) to minimize runtime type checks and improve readability. Apply this rule broadly to files under internal/ that deal with payload-like types, but not to internal components that are pure infrastructure or have runtime-per-request type assertions.
Learnt from: SantiagoDePolonia
Repo: ENTERPILOT/GOModel PR: 33
File: internal/auditlog/factory.go:112-143
Timestamp: 2026-01-15T23:07:37.652Z
Learning: Guideline: Do not use interface{} or map[string]interface{} for API request/response payload types. Prefer strongly-typed structs for API payload definitions to improve type safety, serialization, and documentation. Allow interface{} only in internal infrastructure code paths where pragmatic flexibility is necessary (e.g., to avoid import cycles or to handle highly dynamic internal contracts). In internal/auditlog/factory.go and similar non-API implementation files, applying this restriction is optional and should be evaluated on a case-by-case basis based on whether the type remains internal and does not define API boundary shapes.
- Preserve custom_id verbatim (trim only for blank detection) so results match the caller's correlation key byte-for-byte. - Never fabricate ended_at at render time; it stays null unless the provider reported a transition timestamp. - Best-effort delete the gateway-uploaded inline batch input file when the subsequent provider batch creation fails. - Table-drive the inline-batch provider test with upload-failure and create-failure (cleanup) cases. - Align the batch results swagger media type with application/x-jsonl. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
Contributor
|
Caution Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted. Error details |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/providers/openai/compatible_provider.go`:
- Around line 461-465: Update the cleanup path around DeleteFile in the
batch-create failure handling to use a detached context with a short timeout
instead of the already-canceled ctx, ensuring uploadedInputFileID is still
deleted after cancellation or timeout. Preserve the existing warning logging and
add a regression case covering cleanup when the request context is canceled.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: ab1fc192-dcbb-4e7c-a479-2e8af7ce9c81
📒 Files selected for processing (5)
internal/anthropicapi/batch.gointernal/anthropicapi/batch_test.gointernal/providers/openai/compatible_provider.gointernal/providers/openai/compatible_provider_test.gointernal/server/messages_handler.go
Comment on lines
+461
to
+465
| if uploadedInputFileID != "" { | ||
| if _, cleanupErr := p.DeleteFile(ctx, uploadedInputFileID); cleanupErr != nil { | ||
| slog.Warn("failed to clean up inline batch input file after batch create failure", | ||
| "provider", p.providerName, "file_id", uploadedInputFileID, "error", cleanupErr) | ||
| } |
Contributor
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Detach file cleanup from the failed request context.
When /batches fails due to cancellation or timeout, ctx is already canceled, so DeleteFile immediately fails and leaves the uploaded request content orphaned. Use a detached context with a short cleanup timeout, and add a canceled-context regression case.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@internal/providers/openai/compatible_provider.go` around lines 461 - 465,
Update the cleanup path around DeleteFile in the batch-create failure handling
to use a detached context with a short timeout instead of the already-canceled
ctx, ensuring uploadedInputFileID is still deleted after cancellation or
timeout. Preserve the existing warning logging and add a regression case
covering cleanup when the request context is canceled.
Source: Coding guidelines
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the Anthropic Message Batches API (
/v1/messages/batches) soclient.messages.batches.*in the Anthropic SDKs works against GoModel — closing the last gap from the drop-in compatibility work in #543. The dialect is an edge translation over the existing native-batch pipeline (same pattern as/v1/messages, ADR-0007): decode at the edge, reuse the orchestrator/store/budget/usage/audit machinery, render back in the Anthropic shape.User-visible impact
limit/after_id), cancel, results (JSONL), and DELETE for ended batches.paramsis a Messages request translated to the canonical chat shape, so a Message Batch routes to any provider with native batch support. Verified live: anopenai/gpt-4o-minibatch created through the Anthropic SDK, polled toended, results streamed back as Anthropic-shaped messages, deleted./v1/messages/batchesand/v1/batchesare views of the same gateway batch; IDs interconvert (msgbatch_<uuid>↔batch_<uuid>).results_urlis set once a batch ends and the SDK'sresults()follows it; successful items are converted to the Anthropic Messages shape regardless of serving provider; provider-native canceled/expired item outcomes map to their dedicated result types.Store.Deleteacross all four persistence backends, an ended-only guard in the orchestrator, and native upstream deletion via a new optionalNativeBatchDeleteProvider(implemented for anthropic). Providers without a native delete (OpenAI has none) fall back to gateway-local removal.Provider-specific behavior
POST /batches(the OpenAI batch API is file-only). This fixes inline submissions on the OpenAI dialect'srequests[]extension too — previously they were forwarded verbatim and rejected upstream withMissing required parameter: 'input_file_id'.request_countsmaps the canonical total/completed/failed aggregates: unfinished requests report asprocessingwhile running; after the batch ends, the remainder is attributed by outcome (canceled/expired/errored).Testing
Delete(memory + sqlite), handler lifecycle (create/get/list/results/cancel/delete incl. the ended-only guard and Anthropic error envelopes), and the inline→file upload in the OpenAI provider.go test ./internal/...green; gofmt/vet clean.Docs
docs/advanced/anthropic-messages-api.mdx: endpoints table extended, new Message Batches section with cross-provider notes and an SDK example.🤖 Generated with Claude Code
Summary by CodeRabbit