Describe the bug / perf issue
FoundryAgentSessionStore (the default agent-session SessionStore for hosted MAF agents) rebuilds its backing FoundryStateStore on every get/set/delete. Each _get_store() call goes through FoundryStateStore.get_or_create("agent_sessions", user_isolation=True), which:
- constructs a fresh credential (empty token cache → a new managed-identity token fetch), and
- issues an
agent_sessions metadata round-trip (GET/POST state_stores) before the actual item operation.
Because the hosting infra calls set() in the finally of every Responses request, this redundant credential + metadata work lands on the critical path of every request.
Component
python / foundry_hosting — FoundryAgentSessionStore (_state_store.py).
Expected behavior
The agent-session scope ("agent_sessions", user_isolation=True) is identical for every request, so the backing store should be resolved once and reused for the process lifetime. Only the real item GET/PUT/DELETE should remain on the hot path — not a per-request credential build + metadata round-trip.
Measured impact
500 cold + 500 warm streaming requests per agent, concurrency 50, private/VNet Foundry project, gpt-4o-mini, 0 errors / 0 throttled. Same-conditions A/B, baseline vs cached Responses agent (ms, p50):
| Metric |
Baseline |
Cached |
Δ |
| WARM TTLB p50 |
7,736 |
7,464 |
−272 ms (−3.5%) |
| WARM TTFB p50 |
5,680 |
5,683 |
~flat |
The win is the elimination of the per-request credential + metadata round-trip; the remaining warm latency is the model first-token floor (~5.7 s TTFB), unaffected by this change.
Proposed fix
Cache one per-event-loop, per-scope FoundryStateStore for the agent-session scope (guarded by a per-loop lock with double-checked init; keyed by the running loop via a WeakKeyDictionary so a closed loop's store is GC'd, and keyed by scope so subclasses overriding DEFAULT_ROOT_SCOPE are isolated). Item get/set/delete semantics stay byte-for-byte identical; checkpoint / function-approval stores are left untouched (not on the per-request hot path).
Fix implemented in PR #8178 (closed pending this tracking issue).
Describe the bug / perf issue
FoundryAgentSessionStore(the default agent-sessionSessionStorefor hosted MAF agents) rebuilds its backingFoundryStateStoreon everyget/set/delete. Each_get_store()call goes throughFoundryStateStore.get_or_create("agent_sessions", user_isolation=True), which:agent_sessionsmetadata round-trip (GET/POST state_stores) before the actual item operation.Because the hosting infra calls
set()in thefinallyof every Responses request, this redundant credential + metadata work lands on the critical path of every request.Component
python/foundry_hosting—FoundryAgentSessionStore(_state_store.py).Expected behavior
The agent-session scope (
"agent_sessions",user_isolation=True) is identical for every request, so the backing store should be resolved once and reused for the process lifetime. Only the real itemGET/PUT/DELETEshould remain on the hot path — not a per-request credential build + metadata round-trip.Measured impact
500 cold + 500 warm streaming requests per agent, concurrency 50, private/VNet Foundry project,
gpt-4o-mini, 0 errors / 0 throttled. Same-conditions A/B, baseline vs cached Responses agent (ms, p50):The win is the elimination of the per-request credential + metadata round-trip; the remaining warm latency is the model first-token floor (~5.7 s TTFB), unaffected by this change.
Proposed fix
Cache one per-event-loop, per-scope
FoundryStateStorefor the agent-session scope (guarded by a per-loop lock with double-checked init; keyed by the running loop via aWeakKeyDictionaryso a closed loop's store is GC'd, and keyed by scope so subclasses overridingDEFAULT_ROOT_SCOPEare isolated). Itemget/set/deletesemantics stay byte-for-byte identical; checkpoint / function-approval stores are left untouched (not on the per-request hot path).Fix implemented in PR #8178 (closed pending this tracking issue).