Conversation
apache#4677's "re-profile Host startup" acceptance item lacks a current baseline because the startup anchors (composition open, five-phase domain recovery, first catalog query, orderly close) had no repeatable local measurement. This adds a self-contained runtime-host script that seeds a usage-shaped fixture (Sessions x Turns x artifacts), reopens the composition for the configured number of rounds, and reports per-anchor medians per restart. Medians measured on this head's ancestor fffc19f (Windows 11, Node 26.10.0, one machine for all rounds): | fixture | open | recover | firstQuery | close | restart total | | --------------------------------------------- | ---: | ------: | ---------: | ----: | ------------: | | 100 sessions x 3 turns x 2 artifacts | 16ms | 159ms | 1.1ms | 10ms | 185ms | | 300 sessions x 900 turns, 12,000 artifacts | 14ms | 397ms | 1.2ms | 23ms | 435ms | The 12k-artifact shape is the apache#4027 original scale. Against the historical ~130s cold-start profile (~90s in the artifact phase), that residual no longer exists on current main, and recovery scales sub-linearly (30x the data, 2.5x the recover time). These are single-machine synthetic-fixture measurements, not production observations; the script is the deliverable so the baseline stays reproducible on any machine. Refs apache#4677. Generated-by: GLM-5.3-Flash (ZCode)
FakeBackend paces non-steering responses as 9-char chunks with a 45ms sleep per chunk, so the 283-char seed text spent ~1.4s per turn in that typing simulation - 32 chunks of scripted pacing, not anything the startup anchors measure. A ~25-char seed turn streams a few deltas over the same durable path. Seeding one session with two turns drops from 4.5s to 1.9s (same machine, same shape otherwise), which compounds across fixture builds: the 300-session fixture seeds in roughly 15 minutes instead of 40. Generated-by: GLM-5.3-Flash (ZCode)
|
Pushed |
hqhq1025
left a comment
hqhq1025
left a comment
There was a problem hiding this comment.
Reviewed commit ca2c639. This adds a Runtime Host startup benchmark that seeds sessions, completed turns and artifacts, then times composition open, recovery, first catalog query and close over repeated restarts (packages/runtime-host/scripts/startup-profile.mjs:45-49,69-194,231-296). It does not change production startup behavior.
One P3 measurement issue is noted inline. A Node 24 smoke run with 1 session, 1 turn, 2 artifacts and 1 round completed and reported all four anchors. With PROFILE_DEBUG=1, 3 artifacts yielded total=0.04s, artifacts=0.06s, other=-0.02s, confirming the per-session stage breakdown is wrong; the main restart-anchor table is unaffected. build:test passed locally; current-head hosted checks are green. Fresh main 71bc045 merges cleanly and git diff --check passes. I did not run the default-size fixture, multiple restart rounds, or a real Desktop process restart. This is a COMMENTED review, not merge approval.
Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.
| offset = accepted.nextOffset; | ||
| } | ||
| await ingest({ kind: 'commit', sessionId, uploadId }); | ||
| artifactsMs += performance.now() - artifactStart; |
There was a problem hiding this comment.
[P3] The artifact timer starts once before the loop (line 140), but this line adds the entire elapsed time after every artifact. With N artifacts it sums cumulative durations rather than the artifact phase once; PROFILE_DEBUG=1 reports impossible totals (3 artifacts: total 0.04s, artifacts 0.06s, other -0.02s). Move the start inside the loop or assign the phase elapsed time after the loop.
The seed pass started the artifact-phase timer once before the upload loop but added the cumulative elapsed time after every artifact, so with N artifacts `PROFILE_DEBUG=1` reported N times the phase duration (3 artifacts: artifacts 0.06s against a 0.04s session total, driving `other` negative). Start the timer inside the loop per artifact, matching the existing turnsMs pattern, so the phase is summed exactly once. Generated-by: GLM-5.3-Flash (ZCode)
|
Fixed in c044a0d: the artifact timer now starts inside the upload loop, per artifact, matching the existing Verified with Segments now add up and (review: 5340780590) |
There was a problem hiding this comment.
Reviewed commit c044a0d. The only delta from the previous reviewed head moves artifactStart inside the artifact loop (packages/runtime-host/scripts/startup-profile.mjs:140-141), so each artifact contributes only its own elapsed time. I reran the Node 24 debug probe with 1 session and 3 artifacts: total 0.04s, artifacts 0.02s, other 0.02s; the previous impossible negative other value is gone. The benchmark's four restart anchors are unchanged. No substantiated P0-P3 remains in this increment. Current-head hosted checks are green; fresh main 2f32205 merges cleanly and git diff --check passes. I did not repeat the full default-size fixture or a real Desktop restart. This is not merge approval.
Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.
This branch has not been deployed
2 participants
Summary
#4677's "Re-profile Host startup" acceptance item lacks a current baseline because the startup anchors had no repeatable local measurement. This adds a self-contained
packages/runtime-host/scripts/startup-profile.mjs(registered asbenchmark:startup-profile, alongside the two existing runtime-host benchmarks) that:FakeBackend/SessionManagerpath;open(composition construction + store open),recover(the five-phase domain-module recovery),firstQuery(firstsession.catalog.queryafter recovery),close(orderly shutdown);Why now
The historical profile (#4027/#4037/#4038: ~130s cold start, ~90s in the artifact phase) is flagged in #4677 as "not the current baseline", and the re-profiling acceptance item is open pending fresh numbers. Fresh medians on
fffc19fb5(Windows 11, Node 26.10.0, one machine, medians across restart rounds):The 12k-artifact shape is the #4027 original scale. That historical ~130s/90s residual does not exist on current main, and recovery scales sub-linearly (30x the data, 2.5x the recover time). Startup therefore looks like maintenance-level acceptance rather than an optimization frontier. Whether the script should land was raised on #4677; this PR is the concrete proposal, so the baseline stays reproducible on any machine.
Verification
biome checkon changed filescheck:asf-headersprotocol-epoch-check --stagedgit diff --checkNo runtime behavior is changed, so no test coverage is added; the script's own run is the acceptance. Local note: the husky pre-commit spawns
biome.cmdviaspawnSync, which current Node rejects withEINVALon Windows (CVE-2024-27980 mitigation), so the four staged checks were executed manually with the same scripts and the commit carries--no-verify.Does this PR entail a change of behavior?
AI use
Select exactly one:
Tool(s) and scope: GLM-5.3-Flash (ZCode) authored the script, fixture, and measurements under direction.
Checklist