Congestion control holds back updates when the round trip time varies, as on mobile networks #381

Describe the bug
On a link whose round trip time varies, as on 4G mobile networks, the congestion control in Congestion.cxx keeps its window far below what the link can carry, so writeFramebufferUpdate() sends fewer frames than the screen produces. We emulated a 4G link we had measured, first without packet loss: 40 Mbit/s down and a round trip of about 215 ms that rises to 300–500 ms for a second or two every 10–20 seconds. With a 30 fps animation on the remote screen (about 8 Mbit/s), Chrome drew 15.5 frames per second and 29% of the time fell in stalls longer than 100 ms. With a stable round trip (held at its lowest value), it drew 20.5 frames per second with 12% of the time in stalls. Over the same link, a plain TCP stream of frames of similar size arrived at 29 frames per second with 3% of the time in stalls, whether the round trip varied or not. Xvnc's own estimate of the bandwidth (the congestion window divided by the base round trip, from msgTypeNetworkStats) was 6.5 Mbit/s with the varying round trip, below the 8 Mbit/s that the animation takes, and 11.2 Mbit/s with the stable one.

With 1% packet loss added, most stalls come from TCP itself: a lost segment holds back everything after it until it is resent, so the plain TCP stream also spent 66–73% of the time in stalls. Chrome then drew only 6.5–9 frames per second from Xvnc, and Xvnc's estimate fell to 2.9–4.5 Mbit/s.

System Description

PRETTY_NAME="Debian GNU/Linux 13 (trixie)"
Linux 6.12.113+deb13-amd64 x86_64 (4 vCPU, no GPU)

KasmVNC Details
Package kasmvncserver_debian_trixie_1.5.3_master_faded1_amd64.deb (master at faded1d).

Xvnc KasmVNC 1.5.1.faded1d95d659cd69d0821ac8adc3ebbd54de316 - built Oct 25 2026 10:22:50

Xvnc runs with -FrameRate 60 and software H.264. The client uses the Custom preset with framerate 60, video_stream_quality 28, gop 60 and the Canvas2D renderer. The browser reaches Xvnc through a proxy that keeps its buffers small (TCP_NOTSENT_LOWAT 32 KiB), like our TLS proxy in #377 but without TLS. With 256 KiB and 1 MiB instead, on the link with a varying round trip and loss, the results were the same or worse. The server uses BBR and net.ipv4.tcp_notsent_lowat = 131072.

To Reproduce

1. Play a 30 fps animation in a 1120×630 area of a 1600×900 remote screen. We used a textured scene that pans and slowly zooms, with a few small areas redrawn every frame; at the settings above it takes about 8 Mbit/s.
2. Shape only the browser's connection with netem on the server: 40 Mbit/s down, 10 Mbit/s up, and in each direction a delay of half the round trip of a ping series we recorded over a 4G phone hotspot, plus 7.5 ms. The series has one sample every 200 ms and is replayed in a loop (lowest 196 ms, median 203 ms, 90th percentile 302 ms, 98th percentile 420 ms, maximum 495 ms). For the runs with a stable round trip, the delay stays at the value for the lowest sample. For the runs with loss, add 1% loss downstream and 0.5% upstream.
3. Connect from Chrome and record when each video frame is drawn. We also drew a clock code into the remote screen and read it from each decoded frame, which gives the delay from the screen change to the draw, and requested msgTypeNetworkStats and the bottleneck stats once a second.
4. For comparison, send frames over plain TCP through the same shaping: 30 per second, 30 KB each, with a 200 KB keyframe every 60 frames. The sender uses BBR and TCP_NOTSENT_LOWAT 32 KiB, like the proxy. A frame is skipped while the previous one is still waiting to be written to the socket, as isCongested() does while the output buffer is not empty.

Each run with shaping lasted 60 seconds. Without shaping, the browser drew 24.2 frames per second (software decoding in headless Chrome on the same server), at 7.9 Mbit/s and a median delay of 71 ms.

Xvnc:

Plain TCP over the same link:

The congestion window is the bandwidth estimate times the base round trip that Xvnc reported (218–237 ms). The delay for Xvnc runs from the screen change to the draw and includes capture, encoding and decoding; for plain TCP it runs from the send to the arrival of the whole frame.

Expected behavior
When the link has spare bandwidth, a varying round trip alone does not hold back updates: the frame rate stays close to the one with a stable round trip, as with the plain TCP stream.

Browser

• Headless Chrome 156 on the server, with software H.264 decoding, for the runs above.
• The link parameters come from a 4G phone hotspot, where video on the remote desktop was much less smooth than on a fixed line (Chrome 156 on Windows 11, hardware decoding).

Additional context
After each update with data, writeRTTPing() sends a fence with fenceFlagBlockBefore (VNCSConnectionST.cxx:1387), and Congestion::gotPong() takes the time until its reply as a round trip sample. Congestion::updateCongestion() (Congestion.cxx:394) runs after three samples and compares their minimum, minRTT, with baseRTT, the lowest sample since the last idle reset:

• more than max(100 ms, baseRTT/2) above it, the window is scaled by baseRTT/minRTT and slow start ends (line 408);
• in slow start, more than 25 ms above it, the same (line 422);
• otherwise, more than 50 ms above it, the window shrinks by 4 KiB (line 440);
• the window grows only when the minimum of the samples taken while congested is less than 25 ms above baseRTT: it doubles in slow start (line 435), and otherwise grows by 8 KiB when that minimum is less than 5 ms above baseRTT, or by 4 KiB (lines 449–454).

In our 4G trace, which is 30 seconds long, the round trip stays 50–300 ms above its lowest value for a second or more three times, and these rises cut the window. It grows back slowly: each adjustment needs three samples from pings sent after the previous adjustment (line 185), so adjustments are at least a round trip apart, and after slow start each one adds at most 8 KiB, which is at most about 36 KiB per second at a 220 ms round trip. The window in the table was 177 KiB with the varying round trip, against 298 KiB with a stable one. A lost segment has a similar effect, because the replies to fences sent after it wait for the retransmission, about one more round trip. The network score of the bottleneck stats is lower the more often isCongested() finds the congestion window full and expects the wait to be longer than a frame interval (VNCSConnectionST.cxx:1414); data still in the output buffer does not count. It fell from 7.8 on the stable link to 4.4 with a varying round trip and to 0–1.1 with loss.

The check is there to keep data from queuing ahead of a slow link. On Linux the kernel reports that directly: TCP_INFO has tcpi_notsent_bytes, tcpi_unacked and tcpi_snd_cwnd, and SIOCOUTQ has the bytes not yet acknowledged; Congestion::debugTrace() already reads both when built with CONGESTION_TRACE. Two possible directions:

1. On Linux, treat the connection as congested while the kernel still holds more than a small amount of unsent data, and leave pacing, loss and the round trip to the kernel's congestion control. Behind a proxy, Xvnc's socket only sees the hop to the proxy, so this works only if the proxy keeps its own buffers small, as ours does.
2. Keep the delay-based window, but compare the samples with a threshold that follows the measured variation (gotPong() already computes rttvar), and skip samples during which TCP retransmitted (tcpi_total_retrans changed between the ping and its reply).

Neither removes the stalls that loss causes in TCP itself; those would need a transport without head-of-line blocking.

#377 is about the stalls that keyframes cause on a stable link; this one is separate. We can test a build with either change on this setup and post the same measurements.