Skip to content

emrg: global cross-project session index (rant 2026-08-13T16:42:22) - #756

Merged
argszero merged 1 commit into
masterfrom
feature/sessions-index
Aug 13, 2026
Merged

emrg: global cross-project session index (rant 2026-08-13T16:42:22)#756
argszero merged 1 commit into
masterfrom
feature/sessions-index

Conversation

@argszero

Copy link
Copy Markdown
Owner

Summary

Implements the host's cross-project session global index design: a single ~/.emrg/sessions_index.json mapping session_id → absolute session directory, so any session can locate and read another project's conversation records.

Design (host-finalized, minimal index)

The index stores only the id→path map; everything else (title, message_count, updated_at, history, memory) is read on demand from the target session's meta.json / history.jsonl / memory/MEMORY.md.

Changes

  • emrg/sessions_index.py (new): upsert_session_index / remove_session_index / rebuild_sessions_index with atomic write (tmp + os.replace), corrupt-file tolerance (never raises), idempotent upsert (skips rewrite when the path is unchanged). The startup scan walks ~/.emrg recursively (pruning install/.git/node_modules/.venv/etc.) plus each registered project path.
  • emrg/session.py: Session._save_meta_with_title upserts (create/append/compact/rename/clear all funnel through it); Session.delete removes the entry.
  • emrg/server/daemon.py: _build_system_prompt injects config_dir; serve() calls _rebuild_sessions_index() at startup (best-effort, never crashes).
  • emrg/server/prompts/system.j2: documents the discovery flow (read index → meta.json → history.jsonl → memory).
  • tests/conftest.py: autouse fixture redirects the index to a per-test tmp file (hermeticity — prevents the whole session suite from polluting the host's real index, same class as the projects.yml leak guard).

Verification

  • +20 tests (tests/test_sessions_index.py): 779→799; full suite green.
  • import + CLI checks green.
  • system.j2 renders the new block correctly.

Acceptance

  • New session → index entry appears; compact/rewind/clear/rename → path unchanged (idempotent).
  • Delete → entry removed; daemon restart → startup scan backfills pre-existing + unregistered-project sessions.
  • Any session can read the index to find another project's session.

Add ~/.emrg/sessions_index.json mapping session_id → session directory so any
session can locate and read another project's conversation. Minimal index
(only the id→path map; everything else read on demand from meta.json /
history.jsonl / memory/MEMORY.md).

- emrg/sessions_index.py: upsert/remove/rebuild with atomic write (tmp +
  os.replace), corrupt-file tolerance, never raises; startup scan backfills
  sessions from registered projects + unregistered ones under ~/.emrg
  (prunes install/.git/node_modules/etc. to keep the walk fast).
- Session._save_meta_with_title upserts (create/append/compact/rename/clear
  all funnel through it); Session.delete removes. Idempotent (skips rewrite
  when path unchanged).
- daemon _build_system_prompt injects config_dir; serve() rebuilds the index
  at startup; system.j2 documents the discovery flow.
- conftest redirects the index to a per-test tmp file (hermeticity).
- +20 tests (779→799).

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle (1/3)

Full review of the cross-project session index (rant 2026-08-13T16:42:22), head 42520c7, 7 files +443/-1:

  • emrg/sessions_index.py (new, 181 lines): clean minimal design — index stores only session_id → absolute dir; atomic writes (mkstemp + os.replace), corrupt/missing/non-dict loads → empty dict, idempotent upsert (skips rewrite when path unchanged — mtime-bump test pins this), prune-dirs set keeps the recursive ~/.emrg scan fast (install/, node_modules/, .venv, .git, logs...).
  • Write/delete hooks: Session._save_meta_with_title (create/append/compact/rename/clear all funnel through it) upserts; Session.delete removes. Correct single-funnel placement.
  • Daemon startup backfill _rebuild_sessions_index(): covers registered projects (projects.yml paths) + unregistered nested under config_root; best-effort (try/except → debug log, never crashes startup); _projects_log confirmed present (daemon.py:175).
  • Prompt: config_dir injected into system.j2 context (import confirmed, daemon.py:31) + Cross-Session Discovery section — clean, tool-based (read index → meta.json → history.jsonl → memory).
  • Test isolation: autouse conftest fixture redirects the index to tmp_path (same class as the projects.yml leak guard) — prevents the suite from polluting the real ~/.emrg index.
  • Verification: 20/20 new tests pass; full suite 799 passed (matches Agent.md doc count 779→799); client import + emrg --help OK.

No issues found. Needs 2 more independent-cycle ✅ to merge.

@pm25coder

Copy link
Copy Markdown
Contributor

Tested end-to-end on branch head (checked out locally, full suite):

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle (2/3).

Fresh independent re-verification of head 42520c7 (post-#755 merge, no file overlap):

  • Import graph: sessions_index.py imports only emrg.config (config_dir) — no cycle (config imports nothing from session/sessions_index). session.py adds from emrg.sessions_index import ... alongside its existing emrg.memory import; full suite + from emrg.client.app import run_client already green.
  • Single-funnel upsert: _save_meta_with_title (the one path create/append/compact/rename/clear all funnel through) calls upsert_session_index exactly once at the end; delete calls remove_session_index after shutil.rmtree. Verified via grep on the actual head.
  • Idempotency: upsert short-circuits when the path is unchanged (mtime-bump test pins it), so repeated meta saves don't rewrite the index file.
  • Startup backfill wired at daemon.py:298 (_rebuild_sessions_index before the scheduler starts), best-effort try/except → debug log.
  • Prompt: ctx["config_dir"] injected at daemon.py:1082, consumed by the new Cross-Session Discovery section in system.j2.
  • Hermeticity: autouse conftest fixture (line 57) redirects sessions_index_path to tmp — prevents the session suite from polluting the real ~/.emrg/sessions_index.json.

CI green (test + test-windows). No conflict with the just-merged #755 (that PR touched only i18n.js; this PR touches no GUI files). Needs 1 more independent-cycle ✅.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle (3/3)

Third independent-cycle approval. Re-confirmed on head 42520c7:

  • No overlap with just-merged #755 (that touched only i18n.js; this PR touches no GUI files) — mergeable with master d89811b.
  • Imports: sessions_index.py → only emrg.config (no cycle); session.py imports it alongside emrg.memory.
  • Write hooks single-funnel through _save_meta_with_title (create/append/compact/rename/clear); delete removes from index after rmtree.
  • Startup backfill wired at daemon.py:298 (before scheduler), best-effort.
  • Prompt: config_dir → system.j2 Cross-Session Discovery.
  • Test hermeticity: autouse fixture redirects index to tmp.
  • CI test + test-windows PASS (31683984369).

3 consecutive ✅ from 3 cycles — merging.

@argszero
argszero merged commit 6da1592 into master Aug 13, 2026
2 checks passed
argszero added a commit that referenced this pull request Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants