Skip to content

perf(scheduler): omit expanded groups from account cache - #8845

Merged
Wei-Shaw merged 1 commit into
Wei-Shaw:mainfrom
r266-tech:codex/scheduler-compact-account-groups
Nov 20, 2026
Merged

Wei-Shaw merged 1 commit into
Wei-Shaw:mainfrom
r266-tech:codex/scheduler-compact-account-groups

Conversation

@r266-tech

@r266-tech r266-tech commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Refs #8842.

When an account belongs to many groups, sched:acc:<id> serializes each expanded group both in Groups and in AccountGroups[].Group. Neither graph is used by scheduler hydration: group admission uses the membership IDs, while admin/ops group displays read the account repository.

Before marshaling the value copy, omit Groups and reuse the existing membership projection for AccountGroups. This retains account/group IDs, priorities and creation times while dropping the nested Account/Group pointers. Credentials, extra settings, proxy configuration and account scheduling/billing fields remain in the full record. Both SetAccount and snapshot rebuilds use this serializer; the repository's original object stays intact.

Synthetic account round trips through both write paths produce:

Groups Expanded bytes Cached bytes
1 12,533 1,345
10 114,240 2,329
100 1,131,877 12,412
241 2,727,151 28,627

These fixtures include expanded group descriptions and account edges; the numbers describe the fixtures, not the reporter's production data. The new regression fails on the original main baseline 530c5e3a for all eight write-path/size combinations and passes with this change. It checks the complete hydrated account, all memberships and the untouched source object. A legacy-payload control remains readable, and the Redis integration fixture checks compact memberships alongside full credentials.

Local validation of 18c42a29d, rebased onto main 15871f904, on Go 1.27.4 / macOS arm64:

  • Unmodified full unit suite (go test -p 1 -tags=unit ./... -count=1): 28,229 test/subtest entries passed, 41 skipped.
  • Unmodified full integration suite (go test -p 1 -tags=integration ./... -count=1, real Docker fixtures): 16,733 passed, 16 skipped.
  • CI-pinned golangci-lint v2.14.0: passed, 0 issues.
  • govulncheck ./...: exit 0, no reachable vulnerabilities and no affected imported packages reported; six module-level advisories remain outside the imported package/call results.

The rebase incorporates the already merged Go 1.27.4 upgrade from #8767, including the CI and Docker toolchain pins. The previous Go 1.27.3 GO-2026-6859 failure no longer appears in the local scan. git range-diff confirms this PR's patch is unchanged. Both full test suites above use the branch as written, with no test or product-code overlays. Hosted CI for the new head is still pending.

This addresses payload size within #8842. Repeated identical SETs, cross-bucket/instance coordination and the reported Sentinel replication behavior remain for separate investigation. #8833 handles read-side loading and fallback coalescing; this serializer change is independent of it. No live provider requests or production changes were made.

AI-assisted implementation and local verification.

@r266-tech
r266-tech force-pushed the codex/scheduler-compact-account-groups branch from 0d1380d to 18c42a2 Compare November 17, 2026 09:12
@Wei-Shaw
Wei-Shaw merged commit 84a2930 into Wei-Shaw:main Nov 20, 2026
9 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 20, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants