Skip to content

test(billing): give inflight estimate fixture a user group rate resolver - #7823

Closed
Ogannesson wants to merge 1 commit into
Wei-Shaw:mainfrom
Ogannesson:fix/inflight-estimate-fixture-resolver
Closed

Ogannesson wants to merge 1 commit into
Wei-Shaw:mainfrom
Ogannesson:fix/inflight-estimate-fixture-resolver

Conversation

@Ogannesson

Copy link
Copy Markdown
Contributor

Summary

TestInflightEstimate_AccountMappingNoDBAndBoundedMemory can fail on a loaded CI runner even though nothing grows per model name:

billing_inflight_reservation_test.go:464:
    Error: "8664872" is not less than "8388608"

We hit this twice in a row (8,664,872 and 8,675,904 bytes) when running go test -tags=unit ./... on GitHub-hosted runners. On an idle machine, the same measurement ranges from 1 to 6 MB.

This PR adds the missing userGroupRateResolver to the test fixture. Only the test fixture changes. Production code is not touched.

Root cause

newInflightEstimateGateway builds a bare GatewayService{...} without userGroupRateResolver. So every EstimateInflightReservation call reaches the fallback in getUserGroupRateMultiplier:

resolver := s.userGroupRateResolver
if resolver == nil {
    resolver = newUserGroupRateResolver(...) // cache == nil -> gocache.New(ttl, time.Minute)
}

Each call creates a new go-cache with its own janitor goroutine and ticker. They are only freed after their finalizers run. The test runs 20,000 iterations, so the HeapAlloc delta measures how far finalization has progressed, not per-model cache growth.

A heap profile diff (pprof -base) of the measured window shows that 100% of the growth comes from getUserGroupRateMultiplier → newUserGroupRateResolver → gocache.New (janitor, time.NewTicker, newCache). Goroutine dumps taken right after the window show hundreds of go-cache.(*janitor).Run goroutines.

The production constructors (NewGatewayService, NewOpenAIGatewayService) always set userGroupRateResolver, so this fallback is not reached in production.

Verification

We ran the full internal/service package on Linux, with a probe test that repeats the same 20,000-iteration window 6 times right after the original test:

per-window HeapAlloc delta
before 0.9 – 5.8 MB, varies from round to round
after 320 – 1,456 bytes, stable

go test -tags=unit ./internal/service/ passes.

newInflightEstimateGateway built a bare GatewayService without
userGroupRateResolver, so every EstimateInflightReservation call fell
back to newUserGroupRateResolver and allocated a fresh go-cache with its
own janitor goroutine and ticker. Those are only released after their
finalizers run, so TestInflightEstimate_AccountMappingNoDBAndBoundedMemory
measured finalizer progress rather than per-model cache growth, and
exceeded its 8 MiB bound on loaded CI runners (8.66 MB, twice in a row).
With the resolver set as the production constructors do, the measured
delta drops to under 2 KB.
Copilot AI balanced review requested due to automatic review settings October 2, 2026 16:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Ogannesson
Ogannesson requested a balanced review from Copilot October 2, 2026 16:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Ogannesson

Copy link
Copy Markdown
Contributor Author

Superseded by #8598, which fixes the same fixture (the missing userGroupRateResolver in newInflightEstimateGateway) and has been merged. Closing this one. Thanks!

@Ogannesson Ogannesson closed this Nov 18, 2026
@Ogannesson
Ogannesson deleted the fix/inflight-estimate-fixture-resolver branch November 18, 2026 04:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants