You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[$175] Web - Queued ApproveMoneyRequest never reaches the server after a SequentialQueue pause, but the Approve button disappears #105121
A Copilot approved an expense report on web. The app applied the optimistic approval and hid the Approve button. The ApproveMoneyRequest write never reached the server, and no error came back. The report stayed in Processing on the server. Later, a different Copilot opened the same report, saw Approve, and approved it successfully.
What the client logs show
The SequentialQueue logged "Queue already paused" repeatedly for about 2 minutes. Then "Pause watchdog fired — queue stuck paused with no progress, recovering" fired and flushed 2 persisted requests.
About 6 minutes later, the user opened the report (OpenReport succeeded) and clicked Approve.
The app pushed ApproveMoneyRequest to the SequentialQueue with currentQueueLength: 2, while isOffline was not set. It persisted the request (queue length 3) and then logged "Queue is not running. Flushing the queue."
No "Finished API request" line ever appeared for that request. The server has no record of it.
Other writes in the same session succeeded, including many ApproveMoneyRequest calls on other reports. The network was up.
Expected result
The app sends the queued approval. If the request fails, the app rolls back the optimistic data and shows an error so the user can retry.
Actual result
The request is silently stuck or dropped. The UI shows the report as approved, so the Approve button is gone, but the server never approved it. The user had to hold the report and ask someone else to approve it.
Related
#97331 added the pause watchdog. Here the watchdog fired, but a request queued after it still never went out.
Most likely, the tab that approved was not the "leader" tab. Only the leader tab sends queued writes. On any other tab, flush() quietly returns early, and the request waits for the leader to send it. In this case the leader was a tab that was dead or frozen, so the request never went out.
flush() returns early with "Unable to flush. Client is not the leader." when this tab is not the leader (SequentialQueue.ts:662-671).
The leader is simply the last client ID in the shared ACTIVE_CLIENTS list (ActiveClientManager/index.ts:63-77). IDs are removed only on beforeunload (index.ts:99-106). That event does not fire when Chrome kills, discards, or freezes a tab, so a dead tab can stay leader.
push() logs "Queue is not running. Flushing the queue." without checking whether this tab is the leader or whether the queue is paused (SequentialQueue.ts:941-944). The log makes it look like the request was sent when it wasn't.
"Queue already paused" for 2 minutes: a follower tab still pauses on Onyx update gaps. But handleMissingOnyxUpdates returns early on followers, so nothing fetches the missing updates or unpauses. Each Pusher update calls pause() again.
Watchdog fired, "flushed 2 requests": the watchdog's escalation is skipped on followers. unpause() then calls flush(false), which returns at the leader check. The "2" is numberOfPersistedRequests on the "Unpausing the queue" log, not a count of requests sent.
OpenReport worked, Approve didn't: reads go through MainQueue, which any tab can use. Writes go through SequentialQueue, which only the leader processes.
Earlier approvals worked: the tab was leader earlier. Leadership moves whenever another tab mounts or becomes visible (Expensify.tsx:196-221). Nothing calls flush() when a tab becomes leader again.
No rollback, no error: the request never failed. It is still sitting in PERSISTED_REQUESTS, so the optimistic data stays.
Other causes considered
Queue paused again after the watchdog. Less likely. Every new pause() re-arms the watchdog, so a 6-minute pause should log more than one "Pause watchdog fired" alert.
First queued request hung inside the network middleware. Unlikely. That would leave isSequentialQueueRunning true, but push() logged that the queue was not running.
To confirm from the session logs, find the line logged right after "Queue is not running. Flushing the queue." for ApproveMoneyRequest:
"Unable to flush. Client is not the leader." confirms this root cause.
"Unable to flush. Queue is paused." points to the re-pause cause instead.
Also look for "[OnyxUpdateManager] Pause watchdog escalation skipped — not the leader client" near the watchdog alert.
What changes do you think we should make in order to solve the problem?
Make leadership exclusive and self-healing. Use the Web Locks API (navigator.locks.request) for the sequential-queue leader on web. The browser releases a lock when its tab dies. Only one tab can hold it, which avoids the duplicate-send problem that got the earlier fix reverted. Other platforms keep the current single-client behavior.
Flush when a tab becomes leader. When isClientTheLeader() changes from false to true, call SequentialQueue.flush(). This drains requests that other tabs left behind.
Stop hiding stuck requests. Add a check that fires when persisted requests exist and the queue is online, not paused, and not running for more than N seconds. It should raise a Log.alert, re-run ActiveClientManager.init(), and flush. Also log the real reason in push() instead of always logging "Flushing the queue."
What alternative solutions did you explore? (Optional)
Use a heartbeat with expiry: each tab writes a timestamp to ACTIVE_CLIENTS, and stale entries lose leadership. This works on every platform, but it can briefly produce two leaders. That risk is the reason the earlier fix was reverted.
Confidence: medium. The code fully explains the log sequence, but I could not see this session's logs to confirm the "Client is not the leader" line.
Next Steps for Contributor+ team: To accept:@MelvinBot implement [this](https://github.com/Expensify/App/issues/105121) to create a draft PR. To refine:@MelvinBot <your feedback> To reject: Explain why you are rejecting Melvin's proposal.
changed the title [-]Web - Queued ApproveMoneyRequest never reaches the server after a SequentialQueue pause, but the Approve button disappears[/-][+][$175] Web - Queued ApproveMoneyRequest never reaches the server after a SequentialQueue pause, but the Approve button disappears[/+]on Oct 27, 2026
Activity
MelvinBot commentedon Oct 27, 2026
Proposal
What is the root cause of that problem?
Most likely, the tab that approved was not the "leader" tab. Only the leader tab sends queued writes. On any other tab,
flush()quietly returns early, and the request waits for the leader to send it. In this case the leader was a tab that was dead or frozen, so the request never went out.flush()returns early with "Unable to flush. Client is not the leader." when this tab is not the leader (SequentialQueue.ts:662-671).ACTIVE_CLIENTSlist (ActiveClientManager/index.ts:63-77). IDs are removed only onbeforeunload(index.ts:99-106). That event does not fire when Chrome kills, discards, or freezes a tab, so a dead tab can stay leader.push()logs "Queue is not running. Flushing the queue." without checking whether this tab is the leader or whether the queue is paused (SequentialQueue.ts:941-944). The log makes it look like the request was sent when it wasn't.flush()(SequentialQueue.ts:211-246). On a follower tab, that flush also returns early. This explains why the 2 requests the watchdog "recovered" were still queued 6 minutes later (currentQueueLength: 2).How this fits each log line
handleMissingOnyxUpdatesreturns early on followers, so nothing fetches the missing updates or unpauses. Each Pusher update callspause()again.unpause()then callsflush(false), which returns at the leader check. The "2" isnumberOfPersistedRequestson the "Unpausing the queue" log, not a count of requests sent.MainQueue, which any tab can use. Writes go throughSequentialQueue, which only the leader processes.Expensify.tsx:196-221). Nothing callsflush()when a tab becomes leader again.PERSISTED_REQUESTS, so the optimistic data stays.Other causes considered
pause()re-arms the watchdog, so a 6-minute pause should log more than one "Pause watchdog fired" alert.isSequentialQueueRunningtrue, butpush()logged that the queue was not running.Known history: [No QA] Fix leader-election deadlock in ActiveClientManager (SequentialQueue unable to flush) targeted this same deadlock. It was reverted in [CP Staging] Revert "[No QA] Fix leader-election deadlock in ActiveClientManager (SequentialQueue unable to flush)" because several tabs believed they were leader and sent the same requests more than once.
To confirm from the session logs, find the line logged right after "Queue is not running. Flushing the queue." for
ApproveMoneyRequest:What changes do you think we should make in order to solve the problem?
navigator.locks.request) for the sequential-queue leader on web. The browser releases a lock when its tab dies. Only one tab can hold it, which avoids the duplicate-send problem that got the earlier fix reverted. Other platforms keep the current single-client behavior.isClientTheLeader()changes from false to true, callSequentialQueue.flush(). This drains requests that other tabs left behind.Log.alert, re-runActiveClientManager.init(), and flush. Also log the real reason inpush()instead of always logging "Flushing the queue."What alternative solutions did you explore? (Optional)
Use a heartbeat with expiry: each tab writes a timestamp to
ACTIVE_CLIENTS, and stale entries lose leadership. This works on every platform, but it can briefly produce two leaders. That risk is the reason the earlier fix was reverted.Confidence: medium. The code fully explains the log sequence, but I could not see this session's logs to confirm the "Client is not the leader" line.
Next Steps for Contributor+ team:
To accept:
@MelvinBot implement [this](https://github.com/Expensify/App/issues/105121)to create a draft PR.To refine:
@MelvinBot <your feedback>To reject: Explain why you are rejecting Melvin's proposal.
view run
melvin-bot commentedon Oct 27, 2026
Triggered auto assignment to Contributor-plus team member for initial proposal review - @situchan (
External)[-]Web - Queued ApproveMoneyRequest never reaches the server after a SequentialQueue pause, but the Approve button disappears[/-][+][$175] Web - Queued ApproveMoneyRequest never reaches the server after a SequentialQueue pause, but the Approve button disappears[/+]melvin-bot commentedon Oct 27, 2026
Job added to Upwork: https://www.upwork.com/jobs/~022115045530174165300