Skip to content

perf: Measure RSS and multi-monitor cost on macOS - #590

Merged
omesser merged 1 commit into
mainfrom
perf/424-rss-multimonitor
Sep 10, 2026
Merged

omesser merged 1 commit into
mainfrom
perf/424-rss-multimonitor

Conversation

@omesser

@omesser omesser commented Sep 9, 2026

Copy link
Copy Markdown
Owner

Why

#424 asks for RSS and multi-monitor scaling numbers, and #423 cannot rank
optimization targets until its children come back with baselines. Nothing in
the repository measures memory, and the ticket's hypothesis — "RSS scales with
display count, each webview ~100–200 MB" — was unchecked.

This machine is macOS with two displays attached, which is the multi-monitor
axis the ticket asks about, and RSS is readable from the shell. So the macOS arm
is answerable here. Linux and Windows are not, and this pull request does not
guess at them.

What changed

Two files, plus a pointer.

  • scripts/bench-rss.sh — launches target/debug/ai-buddy, settles, samples
    every process's RSS to a TSV, and reads each one's peak physical footprint
    before stopping the app. Scenarios are chosen by environment
    (AI_BUDDY_INSTANCES, AI_BUDDY_CHARACTERS), so the script has no knowledge
    of them.
  • docs/research/memory-rss-and-multi-monitor.md — the four runs, the
    method, and what could not be measured. docs/agents/docs.md puts
    investigation in docs/research/.
  • docs/DEVELOPMENT.md — one line under Automated Scripts.

Three decisions a reviewer would otherwise have to reverse-engineer:

  • The script measures five processes, not one. WKWebView content runs
    out-of-process and those helpers are children of launchd, so ps on the app
    pid reports about a third of the total. The script takes the set of WebKit
    helpers before and after launch and calls the difference the app's — and warns
    when the count is not displays + 2, because another application starting a
    helper in that window would land in the set. That happened once during this
    work and the run was discarded.
  • Comparisons are made on peak physical footprint, not on RSS. RSS is what
    macOS has let a process keep. On a machine shared with other agents building
    and running, the four-Instance run read 33 MB lighter than the one-Instance
    run purely because it was sampled when free memory was 25–29 % rather than
    33–43 %. Peak footprint only rises, so it survives that. Both are reported.
  • Every run settles 300 s first. A launch peaks near 400 MB, drops to 176 MB
    by 150 s, and comes back to ~225 MB by 300 s. A number taken in the first
    minute is the loader's.

What it does not do: no heap profile. #424 asks for Instruments Allocations or
heaptrack, and Instruments needs a GUI session and a human. The document says
so and names the 47.6 MB finding below as the first thing to point one at.

How to verify

cd src-tauri && cargo build --bin ai-buddy && cd ..
HOME=/tmp/bench-home \
AI_BUDDY_DIRECTOR=0 AI_BUDDY_DIRECTOR_API_KEY=x AI_BUDDY_CAPTURABLE=1 \
AI_BUDDY_TRACE_ENGINE=1 AI_BUDDY_CHARACTERS="$PWD/characters" \
AI_BUDDY_INSTANCES="bmo:One" \
scripts/bench-rss.sh --settle 300 --seconds 300 --out /tmp/one.tsv

Ran: four 10-minute runs on 2026-09-09 (MacBook Pro Mac15,7, 36 GB,
macOS 26.6.2; DELL P2414H 1920×1080 @1× and the built-in 3456×2234 @2×), a
15-minute soak for the settling curve, and a 50-second smoke run of the final
script (exit 0, app stopped, no contamination warning). pre-commit run --files
passes on all three files; no Rust changed.

Peak physical footprint, MB:

Run ai-buddy GPU Net 1080p overlay Retina overlay Total
1 Instance, 8 Characters installed 82.9 146.3 8.0 132.7 213.2 583
4 Instances of one Character 82.9 147.6 7.3 162.0 239.5 639
4 Instances, 4 Characters 85.1 150.2 7.9 170.9 280.5 695
1 Instance, 1 Character installed 35.3 139.8 8.4 122.6 197.7 504

What the numbers say:

  • The hypothesis is half right. One webview per display, yes — but the cost
    is pixels, not displays. The Retina overlay is 75–110 MB heavier than the
    1080p one in every run, and the ratio is the ratio of backing stores. Plan on
    120–170 MB for a 1080p display and 200–280 MB for a Retina one.
  • Instances cost webview, not engine. Four Instances of one Character is
    +56 MB, all of it in the two WebContent processes; the Rust process's peak
    was 82.9 MB in both runs, to the tenth of a MB.
  • A Character switch cannot leak. load_all_characters loads every installed
    package at launch, so the art switched to is already resident. Read off the
    code path, not a driven switch — the tray menu needs a human.
  • Idle perched and walking cost the same. Matching each sample to the
    animation live at that instant: idle 260 MB, walk 245, sit 260, talk 260,
    climb 240. The spread between animations is smaller than the spread within any
    one of them.
  • Seven unused installed Characters cost 79 MB of peak, 47.6 MB of it in the
    Rust process — ten times the 4.5 MB of base64 they amount to. Unexplained, and
    flagged as the target for the heap profile this PR does not have.

Notes

  • Debug build. perf: Benchmark memory usage (RSS) and multi-monitor scaling (all platforms) #424 asked to run target/debug; a user runs release. The
    Rust process's share is an upper bound. The WebKit helpers are release code
    regardless.
  • A single-display comparison was not run. Both displays are permanently
    attached and an agent cannot unplug one. The per-display cost is read off the
    process split instead, which answers the same question but is an inference
    from the architecture rather than a two-configuration measurement.
  • Linux and Windows keep perf: Benchmark memory usage (RSS) and multi-monitor scaling (all platforms) #424 open. Neither compiles here.
  • The machine was shared with other agents throughout. One run was killed by
    another agent's ai-buddy and one was contaminated by another application's
    WebKit helper; both were discarded and repeated. The free-memory column in the
    document is that load.

Closes #424

🤖 Generated with Claude Code

— Claude Code, on @omesser's behalf.

Adds scripts/bench-rss.sh and the numbers it produced for #424.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@omesser omesser added performance CPU/GPU/memory/idle cost and benchmarking research Defining or executing a research into an aspect: tech, direction, decision macos macos / OSX specific labels Sep 10, 2026
@omesser
omesser merged commit dc2786d into main Sep 10, 2026
9 checks passed
cursor Bot pushed a commit that referenced this pull request Sep 11, 2026
…mode

Change all three RSS bench scripts (macOS, Linux, Windows) from long research
defaults (settle 300s, sample 300s) to brief smoke-test defaults (settle ~3s,
sample ~10s). Add --research (bash) / -Research (PowerShell) flag to restore
the long soak for measurement studies.

Verification/benches must be brief so tens/hundreds of scenarios can
stage/test/verify quickly. The 300s+300s (~10 min) default was cargo-cult from
macOS #590 research soak; inappropriate for everyday verification.

Changes:
- scripts/bench-rss.sh: settle 3s, seconds 10s, interval 2s by default
- scripts/bench-rss-linux.sh: same brief defaults
- scripts/bench-rss.ps1: same brief defaults
- All three scripts: add optional research mode flag for 300s+300s soak
- docs: Updated to document both modes, default is brief smoke test
- DEVELOPMENT.md: Note brief defaults, research mode available

Short defaults enable fast scenario matrices; optional research mode preserves
bathtub curve analysis capability when needed.

Co-authored-by: Oded Messer <odedmesser@gmail.com>
cursor Bot pushed a commit that referenced this pull request Sep 11, 2026
…mode

Change all three RSS bench scripts (macOS, Linux, Windows) from long research
defaults (settle 300s, sample 300s) to brief smoke-test defaults (settle ~3s,
sample ~10s). Add --research (bash) / -Research (PowerShell) flag to restore
the long soak for measurement studies.

Verification/benches must be brief so tens/hundreds of scenarios can
stage/test/verify quickly. The 300s+300s (~10 min) default was cargo-cult from
macOS #590 research soak; inappropriate for everyday verification.

Changes:
- scripts/bench-rss.sh: settle 3s, seconds 10s, interval 2s by default
- scripts/bench-rss-linux.sh: same brief defaults
- scripts/bench-rss.ps1: same brief defaults
- All three scripts: add optional research mode flag for 300s+300s soak
- docs: Updated to document both modes, default is brief smoke test
- DEVELOPMENT.md: Note brief defaults, research mode available

Short defaults enable fast scenario matrices; optional research mode preserves
bathtub curve analysis capability when needed.

Co-authored-by: Oded Messer <odedmesser@gmail.com>
omesser added a commit that referenced this pull request Sep 11, 2026
…mode

Change all three RSS bench scripts (macOS, Linux, Windows) from long research
defaults (settle 300s, sample 300s) to brief smoke-test defaults (settle ~3s,
sample ~10s). Add --research (bash) / -Research (PowerShell) flag to restore
the long soak for measurement studies.

Verification/benches must be brief so tens/hundreds of scenarios can
stage/test/verify quickly. The 300s+300s (~10 min) default was cargo-cult from
macOS #590 research soak; inappropriate for everyday verification.

Changes:
- scripts/bench-rss.sh: settle 3s, seconds 10s, interval 2s by default
- scripts/bench-rss-linux.sh: same brief defaults
- scripts/bench-rss.ps1: same brief defaults
- All three scripts: add optional research mode flag for 300s+300s soak
- docs: Updated to document both modes, default is brief smoke test
- DEVELOPMENT.md: Note brief defaults, research mode available

Short defaults enable fast scenario matrices; optional research mode preserves
bathtub curve analysis capability when needed.

Co-authored-by: Oded Messer <odedmesser@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

macos macos / OSX specific performance CPU/GPU/memory/idle cost and benchmarking research Defining or executing a research into an aspect: tech, direction, decision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: Benchmark memory usage (RSS) and multi-monitor scaling (all platforms)

1 participant