Repository navigation
Description
If you haven't already, check out our contributing guidelines.
Version Number: Multiple production versions observed (e.g. 9.3.54–9.3.64)
Reproducible in staging?: Unknown
Reproducible in production?: Yes
Email or phone of affected tester (no customers): N/A — reproduced via production telemetry, not a single test account
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL: https://github.com/Expensify/Expensify/issues/644858
Issue reported by: Internal engineering investigation
What performance issue do we need to solve?
The JS thread is blocked for 10+ seconds (dead clicks / fully unresponsive UI) when large OpenApp, ReconnectApp, and GetMissingOnyxMessages responses arrive in rapid succession. Sentry traces show this drives a storm of Onyx updates and deep equality work across subscribers.
Symptom tracking (aggregate): https://expensify.sentry.io/issues/7323151341/ (~14k events / 90 days)
What is the impact on end-users?
- App appears frozen after navigation, reconnect, or opening reports/search/settings
- Affects iOS, Android, Windows, and Mac (not platform-specific)
- Both new and established accounts
Benchmarks / evidence (from production traces)
Payload sizes observed: ~0.5 MB – 4 MB per response on sessions that should be lightweight.
Pattern A — Same-endpoint stampede
One session fires the same command many times in minutes:
- 40×
ReconnectAppin ~10 min (iOS 9.3.54-7, scan flow, every 8–30s) — https://expensify.sentry.io/explore/traces/trace/6c175d912b7d4bf5b238d131fdcd10a7 - 21×
ReconnectAppin ~5 min (iOS, route change) — https://expensify.sentry.io/explore/traces/trace/688bf8372e4744f3b9fc57afb30eda21 - 13×
GetMissingOnyxMessagesin ~28s (iOS, Report page) — https://expensify.sentry.io/explore/traces/trace/61d5b4e4d45f4fea979548f86352c912 - 20×
ReconnectAppin ~10s (Windows, ~146 KB each, ~2 calls/sec) — https://expensify.sentry.io/explore/traces/trace/298128ff3d274b70860b2060c9fc3581 - 27×
ReconnectAppin ~9s (Windows,/settings/profile/contact-methods) — https://expensify.sentry.io/explore/traces/trace/e3d86d00ed3148bf85d85162c4218e57
Pattern B — Byte-identical duplicate responses (≤3s apart)
Server returns the same payload twice; client processes duplicate Onyx data:
- 2×
ReconnectApp, 516,004 / 516,004 bytes identical, 1s apart (Windows) — https://expensify.sentry.io/explore/traces/trace/61bc0c35ab24458fac3e76592dbcc28b - 2×
OpenApp, ~1 MB each, ~90s apart (iOS) — https://expensify.sentry.io/explore/traces/trace/44c8c38f52a941f7bf6cd4549fb1bbfd - 4× byte-identical ~224 KB
ReconnectAppin 2s on/search(Windows) — https://expensify.sentry.io/explore/traces/trace/9d20e02d47d047aeaad3f0afcc82fa1d
Pattern C — Cross-command overlap (≤10s)
GetMissingOnyxMessages475 KB →ReconnectApp475 KB, 1s apart (Mac, Report) — https://expensify.sentry.io/explore/traces/trace/1d72808fd7c8414e8b86bdc38d349ed1ReconnectApp→GetMissingOnyxMessages, byte-identical 250,893 B, 1s apart (Windows, Home) — https://expensify.sentry.io/explore/traces/trace/da50a56c2ec4443685eada0f86e486e2
Example hang timeline (iOS HybridApp 9.3.64-31)
OpenAppon Home — https://expensify.sentry.io/explore/traces/trace/940b5545caec4d5ba1ac8ce1c0b8738bReconnectAppon Onboarding_Purpose ~32s later — https://expensify.sentry.io/explore/traces/trace/0df88e7f01bc4a1ea048f491b87f021d- APP HANG ~52s after open — https://expensify.sentry.io/issues/7323151341/events/8677a76e805d4948bd37c2412b90e3fd/
Proposed investigation / solution direction
Open questions (from internal triage):
- Why are payloads so large? — Possible whole-collection responses; correlate server
output.sizein logs forOpenApp/ReconnectApp. - Why are commands fired repeatedly? — Network flapping vs uncoordinated client callers (e.g.
subscribeToFullReconnect.tsand other call sites). - Sentry ↔ server correlation — Confirm
requestIDstill appears in HTTP breadcrumbs for recent sessions. - Delegate sessions — Check overlap with known
GetMissingOnyxMessagesover-fetch in delegate mode (PR in review internally). - Mitigation — Client in-flight dedup by
updateIDFrom/updateIDTomay help, but root cause (why duplicate/large calls happen) is priority.
Sentry traces above are shareable; customer emails and FullStory remain on the internal issue only.
Platforms
- iOS: App
- Android: App (inferred from cross-platform scope)
- Windows: Chrome
- MacOS: Chrome / Safari
Issue Owner
Current Issue Owner: @mallenexpensifyMetadata
Metadata
Labels
Type
Projects
- StatusShow more project fieldsDone
Activity
MelvinBot commentedon Jun 3, 2026
Issue Analysis
Root cause (client side): The hang is amplified by two missing dedup guards in App — but the trigger (why duplicate/large responses are produced at all) is server side and needs an internal backend investigation.
Stampede amplifier (Pattern A) — no in-flight dedup.
ReconnectApp/OpenApp/GetMissingOnyxMessageshave ~6 uncoordinated trigger paths (network reachability, app-foreground, Pusher multi-event, auth-screen mount,subscribeToFullReconnect, delegate/priority flows) with no throttle, debounce, or in-flight guard. The only dedup —writeWithNoDuplicatesConflictAction→RequestConflictUtils.ts:57-73— matches only against the queued requests returned byPersistedRequests.getAll(), which excludes the in-flightongoingRequest. So while oneReconnectAppis mid-flight, every new trigger pushes a fresh copy that fires the instant the in-flight one returns → N calls per stampede.Duplicate-apply amplifier (Patterns B & C) — large payloads reapplied in full.
OnyxUpdates.apply()has anisUpdateOldguard that normally skips reapplyingonyxDatawhenlastUpdateID <= lastUpdateIDAppliedToClient— but it is explicitly bypassed forOpenAppand fullReconnectAppatOnyxUpdates.ts:133-136(these commands also skip gap-detection via the bypass set inSaveResponseInOnyx.ts:12-18). Result: a byte-identical 0.5–4 MB duplicateOpenApp/full-ReconnectApppayload is written to Onyx again in full, firing subscriber callbacks + deep-equality work across every subscriber — the 10 s+ JS-thread block.Recommended fix (App-side mitigation): These two guards are exactly the "client in-flight dedup" the issue calls out. They're implementable in App and will collapse most of the storm, but they are a mitigation, not the root cause — they reduce the cost of duplicate/large responses without stopping them from being produced.
ongoingRequest(or add an in-flight no-op guard inreconnectApp/openApp) so a second identical command attaches to the in-flight one instead of queueing a fresh copy.OpenApp/full-ReconnectApp, skip reapplyingonyxDatawhenlastUpdateIDhasn't advanced pastlastUpdateIDAppliedToClient(revisit the bypass atOnyxUpdates.ts:133-134) so byte-identical duplicates don't re-storm Onyx.Needs internal backend investigation (outside the App repo): The priority questions from your triage — (1) why payloads are 0.5–4 MB (whole-collection responses?) and (2) why the server returns byte-identical duplicates ≤3 s apart — live in
Auth/Web-Expensifyand can't be diagnosed or fixed from App. These should be escalated to an internal engineer with backend access; correlating serveroutput.sizeandrequestIDagainst the Sentry traces is the next step there.Investigation details & evidence
In-flight dedup gap (Pattern A):
reconnectApp()andopenApp()have no in-flight flag/Promise guard; only conditionals arehasLoadedApp(init) andisUsingImportedState.getMissingOnyxUpdates()usesmakeRequestWithSideEffectswith no conflict checker at all.Reconnect.ts(network-restored and app-foreground),User.ts(Pusher multi-event),AuthScreensInitHandler.tsx(per-mount),subscribeToFullReconnect.ts.RECONNECT_STAMPEDE_JITTER_MSjitter inNetworkState.ts:127-147, whose comment even asserts "Duplicate reconnectApp() calls are safe — SQ deduplicates them" — which is false for the in-flight case because of thegetAll()/ongoingRequestgap above.processNextRequestmoves it toongoingRequest, leavingpersistedRequestsempty → next call sees empty queue → pushes another fresh copy → repeats.Duplicate-apply gap (Patterns B & C):
isUpdateOldguard (OnyxUpdates.ts:132-159) stripsonyxDatafor ordinary duplicates but exemptsOpenAppand fullReconnectApp(noupdateIDFrom). When not exempted-but-old, onlysuccessData/failureDataare reapplied; when exempted, the fullonyxDatais handed toOnyx.update(...).lastUpdateID, and it's intentionally disabled for the two biggest-payload commands.GetMissingOnyxMessages/ partialReconnectAppduplicates can be caught by the guard, but only if the asynclastUpdateIDAppliedToClientcache (OnyxUpdates.ts:17-23) has propagated before the duplicate is processed — a race when two land ≤3 s apart.Alternative explanations considered: Could the hang be pure render cost from one large legitimate payload (no duplication)? Partly — a single 4 MB
OpenAppis expensive regardless. But that doesn't explain the repeated traces showing byte-identical duplicates and 20–40× same-command bursts, which the two gaps above directly account for. The deep-equality "storm across subscribers" in the Sentry traces is consistent with full-payload reapplication, not a one-shot apply.Confidence: High that both client gaps exist and amplify the hang (verified at file:line). Medium on attribution — whether these client amplifiers vs. the server over-fetch/duplication are the dominant driver needs the log correlation below.
Logs that would confirm dominance (internal): count, per affected session,
[OnyxUpdateManager] Applying update type: https with lastUpdateID: <N>(OnyxUpdates.ts:130) for the samelastUpdateIDwithin 3 s broken out by command; and[SequentialQueue] Conflict action determinedwithcommand: ReconnectAppandconflictType: pushvsreplace. Correlate against serverReconnectApp/OpenAppoutput.sizeandrequestID.Next Steps: Reply with
@MelvinBot implement thisto draft a PR for the two App-side mitigations (in-flight dedup + duplicate-payload guard), or@MelvinBot <feedback>to refine. The backend over-fetch/duplication questions (priority root cause) need an internal engineer withAuth/Web-Expensifyaccess.melvin-bot commentedon Jun 8, 2026
@adhorodyski Eep! 4 days overdue now. Issues have feelings too...
adhorodyski commentedon Jun 10, 2026
First, I wrote down this PRD which sums up the problem I saw in more detail, it was AI-assisted and some context might be off for the model but issues are real. This includes proposed solutions.
Reconnect Storm Prevention — PRD
Problem Statement
Users experience "dead clicks" and app hangs — taps that do nothing, a UI frozen for seconds to minutes. Over the last 90 days there are ~14k such reports. The hangs trace to the JS thread being saturated by a flood of large and/or rapid HTTP calls in the reconnect family:
ReconnectApp,OpenApp, andGetMissingOnyxMessages.Production telemetry (Sentry, prod host, 14 days) shows the scale of the storm:
ReconnectApp: 816,044 calls. p50 328ms, p95 2,223ms, max 544,963ms (9 minutes). One trace fired 31ReconnectAppcalls in 9 seconds (~3.4/s), each ~285ms, all succeeding fast.GetMissingOnyxMessages: 486,535 calls. p50 185ms, p95 1,744ms, max 81,676ms (82s). One trace fired 28 calls in 9 seconds with zeroReconnectApp.OpenApp: 213,265 calls. p50 329ms, p95 1,325ms, max 382,956ms (6.4min). Bursts cap around 9.http.response_content_length.Total: ~1.5M reconnect-family calls per 14 days. The leadership ask (tgolen, ionatan) is explicit: find and fix the root cause of why so many fire, not add another dedup band-aid.
The investigation isolated three independent failure modes plus one amplifier:
ReconnectApploop — a self-re-arming full-reconnect trigger.GetMissingOnyxMessagesloop — the OnyxUpdateManager gap-detection path re-firing, bypassing the SequentialQueue entirely so nothing dedups it.Solution
From the user's perspective: the app stops freezing. Taps register. Coming back online, returning from background, or receiving a server "please reconnect" signal triggers at most one reconnect cycle, not a storm. A slow reconnect on a bad network degrades gracefully instead of locking the UI.
We achieve this by closing the re-arm hazards at each loop's source and closing the dedup gap that lets duplicates escape:
ReconnectApp/OpenApp/GetMissingOnyxMessagescollapse against it.The unifying principle: every reconnect entry point converges on a single in-flight cycle. A trigger that fires while a cycle is active is a no-op (or a coalesced replace), never a new request.
Root cause detail (code-verified)
Mode 1 — the
subscribeToFullReconnectre-arm hazard.subscribeToFullReconnect.tssubscribes to two Onyx keys —LAST_FULL_RECONNECT_TIMEandNVP_RECONNECT_APP_IF_FULL_RECONNECT_BEFORE— and on any change callsdoFullReconnectIfNecessary(), which firesreconnectApp()wheneverlastFullReconnectTime < reconnectAppIfFullReconnectBefore. There is no in-flight guard and no debounce.The loop: every
OpenApp/ full-ReconnectAppresponse re-deliversNVP_RECONNECT_APP_IF_FULL_RECONNECT_BEFOREinresponse.onyxData.applyHTTPSOnyxUpdatesappliesresponse.onyxDatabeforesuccessData. The only writer ofLAST_FULL_RECONNECT_TIME = now(the loop-breaker) issuccessData(App.ts:381-387, only onisOpenApp || isFullReconnect). So when the NVP callback runs, the timestamp is still stale → comparison says "reconnect needed" → fires anotherreconnectApp()→ whose response re-delivers the NVP → repeat. The burst trace7d45b533…(31 bareReconnectAppin 9s, kicked off by Ping/PusherPing, zeroGetMissingOnyxMessages) is this loop.The team already patched the clear-Onyx / delegate / supportal paths by pre-seeding the timestamp (
clearOnyxAndSeedFullReconnect.ts). The steady-state path (network reconnect, PusherRECONNECT_APP) was never seeded or guarded.Mode 2 — the OnyxUpdateManager gap re-detect.
getMissingOnyxUpdates()callsAPI.makeRequestWithSideEffects, which bypasses the SequentialQueue, so there is no dedup at all.OnyxUpdateManagerhas partial guards (getMissingOnyxUpdatesQueryPromise()in the!lastUpdateIDFromClientbranch;areDeferredUpdatesQueued || isFetchingForPendingUpdatesin the gap branch) but the gap branch's guard is defeated under concurrent Pusher updates, producing 28-in-9s stampedes (traceff14342c…).Amplifier — SQ in-flight dedup gap. The no-duplicates resolver (
resolveDuplicationConflictAction, inRequestConflictUtils.ts:57) runsfindIndexoverpersistedRequestsonly — the waiting queue.processNextRequest()removes the in-flight request from that array (slice(1)) before sending. ThedeepEqualongoing-guard (PersistedRequests.ts:150) misses it because the duplicate enqueued mid-flight has a differentrequestID. So a reconnect enqueued while another is in flight is never collapsed.Mode 3 — slow giants. Trace
a389b884…: a loneReconnectAppof 544,963ms (9 min) running in parallel with anAuthenticatePusherof 544,926ms from the same instant — socket open, no progress. This is a network/device stall returning a 7–22MB full payload, matching tgolen's flaky-net reports and vit's thermal/low-mem reports. Distinct from the burst loops.User Stories
ReconnectAppcalls.GetMissingOnyxMessagesfetch to cover them, so that I don't fire one request per gap detection.GetMissingOnyxMessages, so that Mode 2 stays fixed.Implementation Decisions
Module 1 — Full-reconnect decision (Mode 1)
Extract the decision out of the Onyx-subscription side effects in
subscribeToFullReconnectinto a pure, deep module with a tiny interface:The subscription file becomes a thin shell that owns the mutable state (the two timestamps + an
isReconnectInFlightflag), calls the pure decision, and on atrueresult:LAST_FULL_RECONNECT_TIME = nowoptimistically before firing, so the NVP re-delivered in the response can't re-arm the loop in the steady-state path (mirrorsclearOnyxAndSeedFullReconnect).isReconnectInFlight = true, firesreconnectApp(), and clears the flag when the cycle settles (success or failure).Decision: prefer the in-flight flag + optimistic seed over a time-based debounce. A debounce papers over the loop with a delay; the flag + seed removes the re-arm cause.
reconnectApp()must return/expose a settle signal so the flag can be cleared (today it returnsvoidon thewriteWithNoDuplicatesConflictActionpath — we thread a promise through, or resolve the flag off thesuccessData/finallyDatawrite).Module 2 — Missing-updates single-flight guard (Mode 2)
getMissingOnyxUpdatesbypasses the SequentialQueue, so it needs its own guard. Introduce an explicit single-flight wrapper keyed on "a missing-updates fetch is outstanding":The OnyxUpdateManager gap branch routes through this wrapper instead of calling
getMissingOnyxUpdatesdirectly. This makes theareDeferredUpdatesQueued || isFetchingForPendingUpdatesguard authoritative even under concurrent Pusher updates — the inferred guard-defeat path is closed structurally rather than by reasoning about ordering.Note (honest gap): the exact guard-defeat sequence inside OnyxUpdateManager is inferred, not line-traced. The single-flight wrapper is defensive — it holds regardless of which intermediate guard leaks.
Module 3 — SequentialQueue ongoing-request dedup (Amplifier)
Close the gap so the no-duplicates resolver considers the in-flight request, not just the waiting queue. Two candidate shapes:
resolveDuplicationConflictActioncan returnnoActionwhen the new request matches the ongoing one. RequireswriteWithNoDuplicatesConflictActionto surfacegetOngoingRequest()to the matcher.PersistedRequests.save(), after the waiting-queue dedup, also compare the incoming request againstongoingRequestby command + matcher (notrequestID, notdeepEqual— the in-flight copy has a differentrequestID).Decision: Option B, scoped to the reconnect-family commands (
RECONNECT_APP,OPEN_APP) that use the no-duplicates conflict action, to avoid changing dedup semantics for unrelated commands. The matcher must compare on command identity, since the whole point is thatrequestID/deepEqualdiffer.Module 4 — Slow-giant resilience (Mode 3, client side)
Cross-cutting decisions
NetworkStateis per-tab); onlySequentialQueue.flush()is. Mode 2's path already leader-gates inside OnyxUpdateManager (isClientTheLeader()). For Mode 1, evaluate adding leader-gating todoFullReconnectIfNecessaryso N tabs don't each seed+fire. The NVP and timestamp are shared Onyx keys, so the optimistic seed already dampens cross-tab fan-out, but explicit leader-gating is the clean fix. (Multi-tab is a contributing factor to the exact 31-count, which remains not fully explained.)clearOnyxAndSeedFullReconnect.ts(response.onyxData applies before successData). Keep the comment lineage so the why is preserved.Testing Decisions
A good test here asserts external behavior — "how many requests fired" / "was the duplicate collapsed" — not internal flag values. Tests drive the public entry points (Onyx key changes, network transitions, enqueue calls) and assert on the resulting API calls via the existing API/network mocks. Prior art: the existing SequentialQueue and PersistedRequests test suites, and OnyxUpdateManager tests, already mock
APIand assert on emitted commands — follow that style.Tests to write (all chosen modules):
Full-reconnect decision module (
shouldTriggerFullReconnect, pure):truewhen stale timestamp < NVP and no reconnect in flight.falsewhen a reconnect is in flight (the re-arm guard).falsewhen timestamp ≥ NVP.OpenAppresponse re-delivering the NVP with a stale timestamp → assert exactly onereconnectApp()fires (reproduces the 31-in-9s burst, asserts it collapses to 1). This is the headline regression test for the leadership ask.LAST_FULL_RECONNECT_TIMEis written before the request, so a re-delivered NVP in the same response does not re-trigger.Missing-updates single-flight guard:
GetMissingOnyxMessagesrequest.SQ ongoing-request dedup:
RECONNECT_APP, mark it in flight (processNextRequest), enqueue an identicalRECONNECT_APP→ assert the second is collapsed (noAction) and not sent twice.RECONNECT_APPis not collapsed (no over-dedup regression).Slow-giant / observability:
Verification & Success Metrics
Principle: every fix is provable against production numbers before and after. A fix is not "done" because the code looks right — it is done when a specific Sentry query that showed the storm before deploy shows it gone after. We cannot claim "the API attack is over" from aggregate volume drift alone (volume moves for many reasons); each fix therefore emits its own suppression counter so we can attribute the drop directly to the guard that caused it.
Measurement protocol
Per-fix metric contract
7d45b533…span.description:*ReconnectApp*aggregated count per trace_id; assert the distribution's tail collapses. PlusFull reconnect suppressed (in-flight)log count > 0 in the storm population.ff14342c…span.description:*GetMissingOnyxMessages*count per trace_id. PlusMissing-updates fetch coalesced (single-flight)log count.Collapsed duplicate against ongoing requestlog count, broken down by command.a389b884…had reconnect ‖ AuthenticatePusher hanging from same instanthttp.response_content_lengthdistribution;Slow reconnect exceeded thresholdlog count.North-star metrics (end-to-end)
Instrumentation requirements (implementation must add these)
lastFullReconnectTime,reconnectAppIfFullReconnectBefore,isReconnectInFlight). Reuse the existingLog.info('Full reconnect triggered', …)line and add a pairedFull reconnect suppressed (in-flight)line.count-aggregated in Sentry by command and grouped per trace.adhorodyski commentedon Jun 10, 2026
I'd just treat the above as context (it's too content heavy for a human I) and the implementation might differ, but even proposed solutions tackle the real problem, we're missing a single source of truth for when/when to not reconnect, it's not a single module that owns this.
adhorodyski commentedon Jun 10, 2026
Also, here's a verification session I ran for this later:
Reconnect Storm — Verification State
Every failure mode in the PRD is confirmed on two independent axes: live code (GitHub permalinks below) and live production telemetry (Sentry, 14d,
environment:production, orgexpensify/ projectapp).Production telemetry (Sentry, 14d, prod)
Total reconnect-family ≈ 1.53M calls / 14d.
ReconnectApp response bodies reach 22,843,956 bytes (22.8 MB) via
http.response_content_length(p95 of measured bodies = 137 KB; 342k of 820k spans carry the attribute).Dashboard: https://expensify.sentry.io/explore/traces/?query=span.description%3A*ReconnectApp*+environment%3Aproduction&statsPeriod=14d
Query form: use plain
span.description:*ReconnectApp*— free-text gets coerced to exact-match and misses.Signature traces
7d45b533b0a5490481b09871a2c886c5: 31 ReconnectApp spans, top of the per-trace distribution.ff14342c65c54ff48df07eb4a8b23df9: 28 GetMissingOnyxMessages spans, zero ReconnectApp.a389b884c52f4c11a46c6a6024052255(2026-05-28 12:34:27): a 544,963ms ReconnectApp running parallel to a 544,926ms AuthenticatePusher from the same instant — socket open, no progress. The same trace holds a 545,538ms "Route Change" / "Navigation … cancelled or screen mounted" and ~13 ActivityIndicator spinners each ~544,992ms — the multi-minute frozen UI behind a dead-click report.Code
Mode 1 —
subscribeToFullReconnectre-arm hazardsubscribeToFullReconnect.tsL6–35 — subscribes toLAST_FULL_RECONNECT_TIMEandNVP_RECONNECT_APP_IF_FULL_RECONNECT_BEFORE;doFullReconnectIfNecessary()firesreconnectApp()onlastFullReconnectTime < reconnectAppIfFullReconnectBeforewith no in-flight guard, no debounce.App.tsL381–387 —successDatais the only writer ofLAST_FULL_RECONNECT_TIME=now, gated onisOpenApp || isFullReconnect(the loop-breaker).App.tsL511 —const isFullReconnect = !updateIDFrom(arg-less ⇒ full dump).clearOnyxAndSeedFullReconnect.tsL6–21 — the comment statesapplyHTTPSOnyxUpdatesappliesresponse.onyxDatabeforesuccessData, and the path pre-seeds the timestamp; the steady-state path is left unseeded.User.tsL989–991 — PusherRECONNECT_APP→ barereconnectApp().Mode 2 — OnyxUpdateManager gap re-detect, bypasses SequentialQueue
OnyxUpdateManager/index.tsL152–191 —!lastUpdateIDFromClientdiverts to full ReconnectApp; the gap branch is guarded only byareDeferredUpdatesQueued || isFetchingForPendingUpdates, then callsgetMissingOnyxUpdates.App.tsL547–560 —getMissingOnyxUpdates→API.makeRequestWithSideEffects(bypasses SequentialQueue, no in-flight guard).App.tsL528–540 —finalReconnectAppAfterActivatingReliableUpdates(declared L528, call at L538) also usesmakeRequestWithSideEffects, bypassing the SequentialQueue.Amplifier — SequentialQueue in-flight dedup gap
RequestConflictUtils.tsL57–73 —resolveDuplicationConflictActiondoesfindIndexover the waiting persisted requests only; wired into the queue viaAPI.writeWithNoDuplicatesConflictAction→checkAndFixConflictingRequest(API/index.tsL193).PersistedRequests.tsL384 —processNextRequestremoves the in-flight request viaslice(1).PersistedRequests.tsL150 —deepEqual(nextRequestToProcess, ongoingRequest)guard misses the mid-flight duplicate (differentrequestID).SequentialQueue.tsL336–342 —flush()is leader-gated viaisClientTheLeader(); reconnect triggers are not.State
All four failure modes are real and live in production. The code paths exist exactly as the PRD describes them, and the production telemetry — the call volumes, the byte-level payloads, and the three signature traces (
7d45b533,ff14342c,a389b884) — confirms each mode is firing in the field.72 remaining items
melvin-bot commentedon Aug 17, 2026
@mkhutornyi Uh oh! This issue is overdue by 2 days. Don't forget to update your issues!
[-][Performance] App hangs when OpenApp / ReconnectApp / GetMissingOnyxMessages fire in rapid stampedes[/-][+][Due for payment 2026-08-18] [Performance] App hangs when OpenApp / ReconnectApp / GetMissingOnyxMessages fire in rapid stampedes[/+]melvin-bot commentedon Aug 18, 2026
Triggered auto assignment to @mallenexpensify (
Awaiting Payment)mallenexpensify commentedon Aug 18, 2026
@mkhutornyi can you propose what amount you're due and provide reasoning? Thx
mkhutornyi commentedon Aug 18, 2026
They are independent PRs and required testing for each. There were no regressions. So $1000 total
mallenexpensify commentedon Aug 19, 2026
Payment Summary
Contributor+: @mkhutornyi due $1000 via NewDot
cc @blimpich , @mountiny , in case you disagree with the four PRs above all being $250 each.
trjExpensify commentedon Sep 1, 2026
approved $1,000 for @mkhutornyi