tls: coalesce buffered records into one read

[ronag]

TLSWrap::ClearOut() currently emits one JavaScript read per TLS record. Peers that send small records cause allocation, callback and downstream processing costs once per record, even when several records arrived together.

This change decrypts records that are already buffered into a 64 KiB output buffer before emitting a read. It does not wait for additional network input. The cap increases from 16 KiB to 64 KiB, allowing several full-sized records or a burst of small records to share a read. This also increases the temporary stack buffer by 48 KiB and changes allocation sizes and stream chunk boundaries.

SSL errors and close notifications are inspected before entering JavaScript and reported after preceding data. Accumulated data is flushed before visible control records and before handshake, session and key-log callbacks, including TLS 1.3 encrypted control records. Reentrant writes and shutdown retain the existing SSL-library guards. Destruction requested from a read callback while OpenSSL is on the stack is deferred; subsequent output and callbacks stop immediately, and a strong reference keeps the wrapper alive until cleanup.

Tests cover coalescing, the 64 KiB chunk limit, complete delivery through a small onread buffer, clean EOF, corrupted records, TLS 1.2 renegotiation, TLS 1.3 KeyUpdate ordering, reentrant writes and synchronous SSL destruction. test-tls-max-send-fragment now checks record sizes on the wire, since readable chunks can contain multiple records.

The existing benchmark/tls/throughput-s2c.js benchmark gains TLS-version and record-size selectors: 1 KiB, 4 KiB and 16 KiB records under TLS 1.2 and TLS 1.3. Application write size and receive-buffer configuration remain independently selectable.

Validation

• Base: c883ccf3b249033e417368366a6f6c16f185bc5a; candidate: f09b719c804aac271d8ee82f0332316ad647055f.
• Both built from source with plain ./configure and GCC 14.2.0 on Linux x64; Node v27.0.0-pre, bundled OpenSSL 3.5.9.
• C++ lint, import checks, ESLint, C++ diff formatting and git diff --check pass.
• The TLS 1.2 and TLS 1.3 callback-ordering regressions were reproduced before their fixes. The KeyUpdate regression passes on the base and candidate, including _destroySSL() from the preceding data callback.
• The final candidate passes all 597 focused TLS, HTTPS, HTTP/2 and net.onread cases.
• The full make -j16 test PARALLEL_ARGS="-j16 --timeout=120 -p tap" target passes: 7,518 JavaScript/native-addon TAP cases, with expected platform skips, plus 275 C++ tests and the tool tests. The TLS benchmark smoke test also passes. The container runs tests as a non-root user with a subreaper and standard privileged-port permissions; fixture ownership was corrected after reproducing environment-only failures on the base.

Before/after benchmark

AMD EPYC 9355P, Debian 13, Docker limited to 16 CPUs and 32 GiB. Both benchmark processes and their children were pinned to the same CPU, with a 256 MiB JavaScript heap. Eight runs per binary and configuration, three seconds each, using 16 MiB Buffer writes and the default read allocation path. Values are mean ± sample standard deviation in MiB/s.

TLS	Record size	Base	Candidate	Change
1.2	1 KiB	652.1 ± 3.5	928.3 ± 14.6	+42.37%
1.3	1 KiB	599.9 ± 13.5	834.4 ± 12.7	+39.09%
1.2	4 KiB	1368.4 ± 59.4	1801.1 ± 16.7	+31.63%
1.3	4 KiB	1424.3 ± 42.7	1743.5 ± 27.3	+22.41%
1.2	16 KiB	2204.6 ± 38.3	2256.0 ± 51.7	+2.33%
1.3	16 KiB	2321.1 ± 39.9	2220.9 ± 56.9	−4.32%

Node's comparison analyzer reports all six differences significant at 5% after Holm–Bonferroni correction. The TLS 1.3 full-record regression is a tradeoff of the candidate. These measurements compare the complete patch against the base; they do not isolate the effect of the 64 KiB cap from coalescing and callback handling.

taskset -c 57 "$CANDIDATE_NODE" --max-old-space-size=256 benchmark/compare.js \
  --old "$BASE_NODE" --new "$CANDIDATE_NODE" --runs 8 \
  --filter throughput-s2c --set dur=3 --set type=buf \
  --set sendchunklen=16777216 --set recvbuflen=0 --set recvbufgenfn=false \
  --no-progress --analyze --csv throughput.csv tls

Separate --trace-gc runs used the same workload and limits, three repetitions per case. With 1 KiB records, peak process RSS rose from 116–130 MiB to 175–186 MiB. Major collections rose from zero to 12–20 per three-second run, with 13–26 ms of summed major-GC pause time. With 4 KiB records, peak RSS rose from 133–157 MiB to 170–182 MiB, while major collection counts generally fell. With 16 KiB records, peak RSS remained roughly 171–179 MiB and collection counts varied. These whole-process measurements include both client and server, and the candidate transfers more data in several cases; they are not a per-socket allocation measurement or a claim that GC pressure falls.

/usr/bin/time -f '%U %S %M' taskset -c 57 "$NODE_BINARY" \
  --max-old-space-size=256 --trace-gc benchmark/tls/throughput-s2c.js \
  dur=3 type=buf sendchunklen=16777216 recvbuflen=0 recvbufgenfn=false \
  fragment=1024 tlsVersion=TLSv1.3

[nodejs-github-bot]

Review requested:

• @nodejs/crypto

[Copilot]

🟡 Changes recommended

Additional SSL reads can let renegotiation callbacks overtake preceding application data.

1 open finding

• TLS renegotiation callback can overtake buffered application data · New

What changed in this PR

Coalesces buffered TLS records into 16 KiB reads to reduce per-record overhead.

Changes:

• Batches decrypted records while preserving EOF and TLS error handling.
• Adds coalescing, small-buffer, EOF, and corruption tests.
• Updates fragment-size validation to inspect wire records.

File	Description
src/​crypto/​crypto_tls.cc	Implements record coalescing and deferred error emission.
src/​crypto/​crypto_tls.h	Declares extracted TLS error helpers.
test/​parallel/​test-tls-coalesce-records.js	Adds coalescing regressions.
test/​parallel/​test-tls-max-send-fragment.js	Validates fragment sizes on the wire.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

TLS 1.3 protocol records can trigger callbacks before preceding application data is emitted.

1 open finding

• TLS 1.3 control records can overtake buffered application data · New

1 resolved since last review

• TLS renegotiation callback can overtake buffered application data

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

The implementation raises the read cap to 64 KiB despite the stated requirement to preserve 16 KiB behavior.

1 open finding

• Restore the 16 KiB chunk limit or justify the 64 KiB increase · New

2 resolved since last review

• TLS 1.3 control records can overtake buffered application data
• TLS renegotiation callback can overtake buffered application data

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

A data callback can synchronously destroy the SSL object while an OpenSSL callback is still using it.

2 open findings

• Defer SSL destruction during callbacks to prevent use-after-free · New
• Restore the 16 KiB chunk limit or justify the 64 KiB increase

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

Native TLS callback and lifetime changes require human review and completion of the stated runtime validation.

1 open finding

• Restore the 16 KiB chunk limit or justify the 64 KiB increase

1 resolved since last review

• Defer SSL destruction during callbacks to prevent use-after-free

🧠 Review effort: Balanced

[codecov]

Codecov Report

❌ Patch coverage is 79.43925% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.46%. Comparing base (c883ccf) to head (f09b719).
⚠️ Report is 6 commits behind head on main.

Files with missing lines	Patch %	Lines
src/crypto/crypto_tls.cc	79.43%	6 Missing and 16 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #66698      +/-   ##
==========================================
- Coverage   92.82%   90.46%   -2.37%     
==========================================
  Files         422      791     +369     
  Lines      194189   277293   +83104     
  Branches    30039    53326   +23287     
==========================================
+ Hits       180257   250844   +70587     
- Misses      13607    16835    +3228     
- Partials      325     9614    +9289     

Files with missing lines	Coverage Δ
src/crypto/crypto_tls.h	90.00% <ø> (ø)
src/crypto/crypto_tls.cc	78.59% <79.43%> (ø)

... and 502 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.