Skip to content

feat(autoresearch): keep training after the first burst, on a research interval - #116686

Merged
trunk-io[bot] merged 5 commits into
masterfrom
posthog-self-driving/featautoresearch-keep-training-after-5fc90e
Oct 16, 2026
Merged

trunk-io[bot] merged 5 commits into
masterfrom
posthog-self-driving/featautoresearch-keep-training-after-5fc90e

Conversation

@posthog

@posthog posthog Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem

  • An autoresearch model stops scheduled training after about 5 days. The champion then keeps scoring for months on what it learned in week one.
  • activity_kickoff_training took a hardcoded min(10, remaining) from the lifetime iteration_budget_remaining. With the default budget of 50, every sweep after day 5 returned budget_exhausted. A rescue run for an unscorable champion was the only exception.
  • The later runs, which are the first to see realized outcomes in realized_context, never happened. Users could not change this: the Setup tab showed only a read-only "Budget remaining".

Closes #116673

Origin

  • Issues: #116673
  • First signal: 2026-10-13
  • Inbox report: open
  • Likely cause: f4b9fbf
  • Task started by: auto-start, after the report was rated P2 and ready to fix

Changes

  • Kickoff follows a research schedule, not a budget. training/schedule.py derives the phase from the pipeline's training runs. Nothing stores it.
Phase When a run starts Ends when
Explore each sweep (at most one run per UTC day) a completed run does not promote, or 5 research runs (completed or failed, rescue runs excluded) have run
Maintain research_interval_days calendar days after the latest run of any kind, so a manual run resets the clock never
  • New skip reasons: not_due, and manual_only for a null interval. budget_exhausted is gone. The coordinator body does not change, so replay is safe.
  • Rescue is a fallback. A rescue run starts only when no explore or scheduled run is due. It still works with a null interval and keeps the rescue_throttled window. An explore or scheduled run already gives the agent the unscorable-champion brief.
  • New fields (migration 0006, additive, with db_default):
    • iterations_per_run: 1 to 50, default 5. Each run takes this many iterations.
    • research_interval_days: 7, 14, 30, 60 or 90, or null for "only when I ask". Default 7.
    • AutoresearchTrainingRun.kickoff_reason: explore, scheduled, rescue or manual. Older rows stay null.
  • Manual starts (Train now, /train, MCP, open_training_run) default to iterations_per_run and record manual. They work in every mode and keep the explicit override.
  • API and MCP: both settings can be set on create and update. The response adds read-only training_phase and next_research_at. iteration_budget and iteration_budget_remaining are marked deprecated in help text. The columns stay. tools.yaml and the generated types are regenerated.
  • UI: on the Setup tab, "Budget remaining" becomes two controls, "Look for a better model" and "Experiments per run", with a save button that guards against double submission. A status line shows the phase or the next research date. Each training-run row in Agent research shows a short kickoff label. A new usage event, autoresearch training schedule changed, fires on save.
Setup tab Training run rows
Interval select, experiments input and status line ("Next research run on Nov 12.") replace the budget card Run meta line ends with "Exploring", "Scheduled", "Rescue" or "Manual"

Warning

On deploy, existing pipelines that spent their old budget already have 5 runs, so they land in maintain. Most passed their 7-day mark long ago, so expect each live pipeline to start one research run in the first sweep after deploy.

  • Kickoff still runs only on a scoring cadence day. A pipeline with cadence_days > 1 explores on scoring days, not every day. The copy says "each scoring day" for this reason.
  • The stub training path still debits the old budget column. It is a dev-only path and nothing reads that column now.

How did you test this code?

  • hogli test products/autoresearch/backend passes locally. This covers the rewritten kickoff tests, the API tests and the agent-recorded run tests.
  • pnpm --filter=@posthog/frontend typescript:check passes.
  • The Setup tab and Agent research tab were rendered in Storybook at wide and narrow widths. Editing "Experiments per run" kept the saved interval and turned on the save button. "Only when I ask" can be selected.
  • Not run: repo-wide mypy, frontend Jest, hogli lint:tach. CI runs these.

Test rationale:

  • test_kickoff_follows_the_research_schedule replaces the budget cases with one parameterized test. It covers explore, scheduled, not_due, manual_only, a rescue with a null interval, rescue throttling, and the burst that ends on a non-promoting run or after 5 runs. It also covers a manual run that resets the clock and a rescue run that does not count toward the burst. It catches a schedule regression that would stop or over-run paid training.
  • test_start_training and test_open_training_run now assert the iterations_per_run default and kickoff_reason="manual".
  • test_metadata_editable_after_training asserts that PATCH stores both settings, including a null interval.
  • The serializer range test adds bounds cases for both new fields.

Release status

  • This change is behind a feature flag and is not available to users

Docs update

The product AGENTS.md, backend/temporal/AGENTS.md, the frontend AGENTS.md and the working-with-autoresearch skill now describe the schedule.

🤖 Agent context

Autonomy: Fully autonomous

Agent: Claude Code (PostHog Desktop), claude-opus-5-5

  • Built from the spec in feat(autoresearch): keep training after the first burst, on a research interval #116673. A search for an existing open PR found none.
  • Skills invoked: /django-migrations, /writing-tests.
  • Design choice: schedule first, then rescue as the fallback. This means an unscorable champion during explore gets a full explore run, not a 3-iteration rescue run.
  • The run hit its budget before /simplify and the full lint pass could run.

Created with PostHog Desktop from this inbox report, addressing #116673.

🤖 Generated with Claude Code

…h interval

Replace the lifetime iteration budget with a derived explore and maintain schedule.
Explore starts a run on each scoring day until a run does not promote or 5 research runs have run.
Maintain starts a run every research_interval_days after the latest run.
Add iterations_per_run, research_interval_days and kickoff_reason, expose them in the API, MCP and the Setup tab.

Closes #116673

Generated-By: PostHog Desktop
Task-Id: f27d5f67-f89d-4b4f-97bc-c7dd72eb8810
@posthog posthog Bot added the self-driving label Oct 14, 2026
@posthog
posthog Bot marked this pull request as ready for review October 14, 2026 00:02
@trunk-io

trunk-io Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

😎 Merged successfully - details.

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approved yet — waiting on the conditions below.

The Migration risk check has not finished for this commit, so stamphog cannot tell a safe migration from a risky one yet. The review runs again on the next push, or you can re-request it once the check reports.

Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✗ matches: migrations
size ✓ 707L, 16F substantive, 1221L/30F incl. docs/generated/snapshots — within ceiling
tier ✗ classified as T2-never: T2-never (1221L, 30F, two-areas, feat)
stamphog 2.5.0 .stamphog/policy.yml @ unknown · reviewed head d2029db

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CI report

⚠️ Trunk lane — backend Python lane

This PR is assigned to the backend Python lane. It runs backend Python tests and may merge in parallel with PRs in other lanes.

⚠️ Complexity (TypeScript) — 1 function above the limit (max 32)

Cyclomatic complexity above the limit in changed typescript files (10 for production files, 15 for test files). Warn only: worth simplifying when you next touch these functions.

Function Location Complexity Limit
TrainingRunRow products/autoresearch/frontend/pipeline/TrainingRunRow.tsx:73 32 10
✅ Duplication (Python) — clean

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Duplication (TypeScript) — clean

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

⚠️ Comment density — 3% of added code lines are comments (19 of 605)

This section warns when comments are more than 3% of the code lines a PR adds, and alerts above 6%. Before agent-assisted PRs, the typical share was about 2%. Only full-line comments count. Docstrings, generated files, snapshots, migrations, and workflow files are left out.

Comments that restate the code, record how the change came about, or narrate the next line add noise for the next reader. Keep the comments that explain a reason the code cannot show, and remove the rest. See .agents/skills/writing-code-comments/SKILL.md for the house rules.

Files with the most added comment lines:

File Comment lines Added lines
products/autoresearch/backend/temporal/workflows.py 9 46
products/autoresearch/backend/training/schedule.py 5 73
products/autoresearch/backend/models.py 2 30
products/autoresearch/frontend/trainingScheduleLogic.ts 2 160
products/autoresearch/backend/temporal/test_workflows.py 1 68

This check does not block merging. It updates on every push and clears when the share drops.

⚠️ Bundle size — 🔺 +4.1 KiB (+0.0%)

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 76.60 MiB · 🔺 +4.1 KiB (+0.0%)

File Size Δ vs base
posthog-app/_parent/products/autoresearch/frontend/AutoresearchPipelineScene.js 114.3 KiB 🔺 +4.1 KiB (+3.7%)

Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report

✅ Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root Eager (shipped) Δ vs base Budget
entry (logged-out pages, app bootstrap)
src/index.tsx
1006.4 KiB · 23 files no change █████████░ 85.9% of 1.14 MiB
logged-out boot: index + App + bootApp (preloaded by every page, including /login)
src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
2.87 MiB · 606 files no change █████████░ 94.6% of 3.03 MiB
authenticated shell (every logged-in page)
src/scenes/AuthenticatedShell.tsx
7.50 MiB · 2,563 files no change ██████████ 98.1% of 7.64 MiB
dashboard scene
src/scenes/dashboard/Dashboard.tsx
9.65 MiB · 3,620 files no change █████████░ 93.7% of 10.30 MiB
today home path
src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
7.52 MiB · 2,573 files no change ██████████ 95.4% of 7.89 MiB
events scene
src/scenes/activity/explore/EventsScene.tsx
9.23 MiB · 3,447 files no change █████████░ 93.5% of 9.88 MiB
replay detail scene
src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
11.96 MiB · 4,291 files no change ████████░░ 79.4% of 15.05 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 node_modules/@posthog/brand/dist/generated/hoggies/svg/ stays out of src/index.tsx
🟢 node_modules/@posthog/brand/dist/generated/hoggies/components/ stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/layout/navigation-3000/navigationLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/scenes/dashboard/dashboardLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/lemon-ui/LemonMarkdown/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/RichContentEditor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/CodeSnippet/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/taxonomy/core-filter-definitions-by-group.json stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 node_modules/zod/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/scenes/billing/billingLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/ProductEmptyState/productSetupPreloadLogic.ts stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 products/dashboards/frontend/widgets/previews/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/session-recordings/player/sessionRecordingPlayerLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 zod/v4/locales/de.js stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/@posthog/brand/dist/generated/hoggies/svg/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/@posthog/brand/dist/generated/hoggies/components/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/session-recordings/playlist/SessionRecordingsPlaylist.tsx stays out of src/scenes/dashboard/Dashboard.tsx
🟢 src/scenes/web-analytics/tiles/WebAnalyticsTile.tsx stays out of src/scenes/dashboard/Dashboard.tsx
🟢 src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
🟢 src/scenes/project-homepage/today/TodayReportPage.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
🟢 src/queries/Query/Query.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
🟢 src/scenes/session-recordings/playlist/SessionRecordingsPlaylist.tsx stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 src/scenes/web-analytics/tiles/WebAnalyticsTile.tsx stays out of src/scenes/activity/explore/EventsScene.tsx

Largest files eagerly shipped from src/index.tsx
Size File
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
24.6 KiB ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
6.3 KiB ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js
4.5 KiB ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js
3.9 KiB ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js
1.4 KiB ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js
1.3 KiB src/RootErrorBoundary.tsx
1.3 KiB src/index.tsx
912 B ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js
839 B src/scenes/ChunkLoadErrorBoundary.tsx
Largest files eagerly shipped from src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
Size File
319.8 KiB ../node_modules/.pnpm/posthog-js@1.438.8_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
220.2 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
105.9 KiB src/products.tsx
99.6 KiB src/lib/api.ts
69.0 KiB src/lib/lemon-ui/icons/icons.tsx
38.7 KiB ../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js
33.9 KiB ../node_modules/.pnpm/kea@4.0.0-pre.6_patch_hash=139b8d1f1304f9d9da452a9a1244c94ea679dbcb85687d8999563146879fb6f5_react@18.3.1/node_modules/kea/lib/index.cjs.js
28.9 KiB src/scenes/scenes.ts
27.8 KiB src/lib/constants.tsx
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
Size File
319.8 KiB ../node_modules/.pnpm/posthog-js@1.438.8_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
281.1 KiB src/taxonomy/core-filter-definitions-by-group.json
220.2 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
185.5 KiB ../packages/quill/packages/quill/dist/index.js
153.7 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
139.9 KiB ../packages/quill/packages/primitives/dist/index.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
105.9 KiB src/products.tsx
99.6 KiB src/lib/api.ts
93.3 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
Largest files eagerly shipped from src/scenes/dashboard/Dashboard.tsx
Size File
319.8 KiB ../node_modules/.pnpm/posthog-js@1.438.8_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
281.1 KiB src/taxonomy/core-filter-definitions-by-group.json
220.2 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
185.5 KiB ../packages/quill/packages/quill/dist/index.js
181.9 KiB src/queries/validators.js
153.7 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
139.9 KiB ../packages/quill/packages/primitives/dist/index.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
105.9 KiB src/products.tsx
99.6 KiB src/lib/api.ts
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
Size File
319.8 KiB ../node_modules/.pnpm/posthog-js@1.438.8_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
281.1 KiB src/taxonomy/core-filter-definitions-by-group.json
220.2 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
185.5 KiB ../packages/quill/packages/quill/dist/index.js
153.7 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
139.9 KiB ../packages/quill/packages/primitives/dist/index.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
105.9 KiB src/products.tsx
99.6 KiB src/lib/api.ts
93.3 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
Largest files eagerly shipped from src/scenes/activity/explore/EventsScene.tsx
Size File
319.8 KiB ../node_modules/.pnpm/posthog-js@1.438.8_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
281.1 KiB src/taxonomy/core-filter-definitions-by-group.json
220.2 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
185.5 KiB ../packages/quill/packages/quill/dist/index.js
181.9 KiB src/queries/validators.js
153.7 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
139.9 KiB ../packages/quill/packages/primitives/dist/index.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
105.9 KiB src/products.tsx
99.6 KiB src/lib/api.ts
Largest files eagerly shipped from src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
Size File
319.8 KiB ../node_modules/.pnpm/posthog-js@1.438.8_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
316.0 KiB ../node_modules/.pnpm/posthog-js@1.438.8_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/rrweb.js
281.1 KiB src/taxonomy/core-filter-definitions-by-group.json
220.2 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
185.5 KiB ../packages/quill/packages/quill/dist/index.js
181.9 KiB src/queries/validators.js
153.7 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
105.9 KiB src/products.tsx
99.6 KiB src/lib/api.ts

Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479

✅ Toolbar bundle — eager 2.23 MiB within budget

What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.

Metric Size Δ vs base Budget
Eager (shipped)
entry + static imports
2.23 MiB · 19 files no change ████░░░░░░ 39.1% of 5.72 MiB
Deferred (lazy) 2.10 MiB · 40 files no change n/a — loads on demand
Loader dist/toolbar.js 1.2 KiB no change █░░░░░░░░░ 6.0% of 19.5 KiB
Largest eagerly-shipped chunks
Size File
875.4 KiB dist/toolbar/toolbar-app-J6A7FF3M.css
664.1 KiB dist/toolbar/chunk-chunk-XX36DFAD.js
257.8 KiB dist/toolbar/chunk-chunk-OZR3ITD3.js
131.8 KiB dist/toolbar/chunk-chunk-FDH2IBXT.js
129.6 KiB dist/toolbar/chunk-chunk-TXT4JZUE.js
74.7 KiB dist/toolbar/toolbar-app-2CXOFYXU.js
69.0 KiB dist/toolbar/chunk-chunk-TSAL54PB.js
35.6 KiB dist/toolbar/chunk-chunk-E6C7YD4O.js
21.7 KiB dist/toolbar/chunk-chunk-C3JJZZTP.js
6.8 KiB dist/toolbar/chunk-chunk-DV7IWQNF.js

Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile

✅ Dist folder size — 🔺 +46.3 KiB (+0.0%)

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 1015.29 MiB · 🔺 +46.3 KiB (+0.0%)

ℹ️ MCP UI apps size — 31 app(s), 25418.0 KB JS

Built size of each MCP UI app (main.js + styles.css).

App JS CSS
debug 836.4 KB 206.4 KB
action 760.5 KB 206.4 KB
action-list 827.4 KB 206.4 KB
cohort 759.5 KB 206.4 KB
cohort-list 826.4 KB 206.4 KB
email-template 759.3 KB 206.4 KB
error-details 764.3 KB 206.4 KB
error-issue 760.8 KB 206.4 KB
error-issue-list 828.0 KB 206.4 KB
experiment 824.5 KB 206.4 KB
experiment-list 828.2 KB 206.4 KB
experiment-results 827.0 KB 206.4 KB
feature-flag 826.8 KB 206.4 KB
feature-flag-list 830.5 KB 206.4 KB
feature-flag-testing 762.7 KB 206.4 KB
inline-scan 759.9 KB 206.4 KB
insight-actors 825.9 KB 206.4 KB
llm-costs 823.2 KB 206.4 KB
session-recording 761.7 KB 206.4 KB
survey 761.0 KB 206.4 KB
survey-global-stats 823.2 KB 206.4 KB
survey-list 828.1 KB 206.4 KB
survey-stats 823.2 KB 206.4 KB
trace-span 759.8 KB 206.4 KB
trace-span-list 827.3 KB 206.4 KB
vision-observation-list 826.5 KB 206.4 KB
workflow 759.8 KB 206.4 KB
workflow-list 826.7 KB 206.4 KB
query-results 1076.9 KB 206.4 KB
render-ui 1148.1 KB 206.4 KB
visual-review-snapshots 764.3 KB 206.4 KB
ℹ️ MCP agent API — agent-facing tool changes

What this PR changes for agents, from the tool schema snapshots and tool definitions.

Tools changed (3):

Tool Params Scopes Annotations Schema chars
autoresearch-create description changed
autoresearch-partial-update description changed
autoresearch-train-create description changed
⚠️ Playwright — 1 flaky

🎭 Playwright report · View test results →

⚠️ 1 flaky test:

  • Creating a SQL insight with a variable and overriding it on a dashboard (chromium)

These issues are not necessarily caused by your changes.
Annoyed by this section? Help fix flakies and failures and it will go green!

⚠️ Backend coverage — 91.0% of changed backend lines covered — 12 uncovered

🧪 Backend test coverage

Patch coverage — changed backend lines (products + core): ██████████████████░░ 91.0% (124 / 136)

File Patch Uncovered changed lines
products/autoresearch/backend/facade/contracts.py 28.6% 134–137, 238
products/autoresearch/backend/training/schedule.py 85.7% 28–31, 36, 39
products/autoresearch/backend/facade/api.py 88.9% 190

🤖 Agents: add a test only if an uncovered line exposes a realistic regression that existing tests miss. Otherwise explain why no new test is needed under "How did you test this code?". Gap list: the patch-coverage artifact on this run (gh run download 135379417337325 -n patch-coverage), or the coverage-data block at the end of this comment.

Per-product line coverage (touched products)
Product Coverage Lines
autoresearch ██████████████████░░ 91.2% 10,921 / 11,976

Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.

⚠️ Django migration SQL — 1 new migration to review

We've detected new migrations on this PR. Review the SQL output for each migration:

products/autoresearch/backend/migrations/0007_research_schedule.py

-- sqlmigrate failed for this migration (see the Validate migrations job log)

Last updated: 2026-10-15 11:12 UTC (e294bd9)

✅ Django migration risk — migration analysis complete

We've analyzed your migrations for potential risks.

Summary: 1 Safe | 0 Needs Review | 0 Blocked

✅ Safe

Brief or no lock, backwards compatible

autoresearch.0007_research_schedule
  └─ #1 ✅ AddField
     Adding NOT NULL field with constant default (safe in PG11+)
     model: autoresearchpipeline, field: iterations_per_run
  └─ #2 ✅ AddField
     Adding nullable field requires brief lock
     model: autoresearchpipeline, field: research_interval_days
  └─ #3 ✅ AddField
     Adding nullable field requires brief lock
     model: autoresearchtrainingrun, field: kickoff_reason

📚 How to Deploy These Changes Safely

AddField:

This operation acquires a brief lock but doesn't rewrite the table.

Deployment uses lock timeouts with automatic retries, so lock contention will cause retries rather than connection pile-up.

Last updated: 2026-10-15 11:12 UTC (e294bd9)

@parameterai

parameterai Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Risk: No findings

This increment adds an agent-writable headline field on iterations and training runs (validated by HeadlineField, truncated to 60 chars, never reaching a markdown/dangerous sink in the frontend), a best-so-far holdout-AUC trend on live runs and champion histories (team-scoped reads only), and regenerates types/MSDN tool descriptions. No new security surface is opened: both new write fields go through request.validated_data in the viewset actions, all new reads stay inside for_team(team_id) scopes, and the pipeline-write sandbox guard is unchanged and intact.

Sentinel reviewed e294bd9 · Review settings

@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team October 14, 2026 00:02

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed d2029db: 2 Should fix, 1 Consider.

if not allow_action_target and pipeline.target_definition.get("type") == "action":
raise InvalidTarget("An action target needs the action:read scope.")
budget = iteration_budget or pipeline.iteration_budget
budget = iteration_budget or pipeline.iterations_per_run

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 · Wire the create form to the new per-run setting

When a user changes “Experiment budget” during creation, autoresearchNewLogic still sends iteration_budget and calls /train without an override. This new default ignores that control and runs five experiments even when the user selected one, increasing spend beyond their chosen limit. Update the create form to submit and validate iterations_per_run.

Comment on lines +148 to +149
actions.loadPipelineSuccess(await autoresearchPartialUpdate(String(currentTeamId), props.id, changes))
actions.resetDraft()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 · Preserve schedule edits made while saving

When a user changes either schedule input while the PATCH is pending, the controls remain editable and update draft. On success, resetDraft() discards those newer edits, although the request saved only the earlier changes, and the success toast hides that loss. Disable the inputs during saving or clear only draft values that match the submitted changes.

Comment on lines +79 to +82
rows = (
AutoresearchTrainingRun.objects.for_team(team_id)
.filter(pipeline_id__in=list(pipeline_ids))
.values_list("pipeline_id", "kickoff_reason", "status", "summary__champion_promoted", "created_at")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 · Aggregate schedule state instead of loading the full run history

For long-lived pipelines or frequent manual training, schedule_runs_for sorts and materializes every historical run on each pipeline read and daily kickoff. Request memory and query work grow with lifetime history, and kickoff holds the pipeline lock throughout. Compute per-pipeline counts, non-promotion existence and latest timestamps in SQL instead.

Comment thread products/autoresearch/backend/presentation/views/serializers.py

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approved yet — waiting on the conditions below.

The Migration risk check has not finished for this commit, so stamphog cannot tell a safe migration from a risky one yet. The review runs again on the next push, or you can re-request it once the check reports.

Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✗ matches: migrations
size ✓ 707L, 16F substantive, 1221L/30F incl. docs/generated/snapshots — within ceiling
tier ✗ classified as T2-never: T2-never (1221L, 30F, two-areas, feat)
stamphog 2.5.0 .stamphog/policy.yml @ unknown · reviewed head 59dfe93

@trunk-io

trunk-io Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Static Badge   Static Badge   Static Badge

Failed Test Failure Summary Logs
Scenes-App/Project Homepage ProjectHomepage smoke-test The test timed out while waiting for loading indicators or spinners to disappear. Logs ↗︎
Scenes-App/Experiments ExperimentWithHealthFindings play-test The Storybook UI test could not find the text “Why?” on the experiment page. Logs ↗︎
Dashboards › Creating a SQL insight with a variable and overriding it on a dashboard The test failed because the page URL did not match the expected URL. Logs ↗︎

View Full Report ↗︎ ⋅ Docs

@posthog

posthog Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor Author

✅ Visual changes approved by @andrewm4894 — baseline updated in 7089794.

View this run in PostHog

4 changed.

Install the Visual Review Chrome extension to see visual review results at the top of your pull requests.

4 updated
Run: 37337c20-664f-4001-b38d-2fe9d9b2e0c6

Co-authored-by: andrewm4894 <2178292+andrewm4894@users.noreply.github.com>

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approved yet — waiting on the conditions below.

The Migration risk check has not finished for this commit, so stamphog cannot tell a safe migration from a risky one yet. The review runs again on the next push, or you can re-request it once the check reports.

Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✗ matches: migrations
size ✓ 715L, 17F substantive, 1229L/31F incl. docs/generated/snapshots — within ceiling
tier ✗ classified as T2-never: T2-never (1229L, 31F, cross-cutting, feat)
stamphog 2.5.0 .stamphog/policy.yml @ unknown · reviewed head 7089794

@andrewm4894

Copy link
Copy Markdown
Member

/trunk merge

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approved yet — waiting on the conditions below.

The Migration risk check has not finished for this commit, so stamphog cannot tell a safe migration from a risky one yet. The review runs again on the next push, or you can re-request it once the check reports.

Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✗ matches: migrations
size ✓ 716L, 16F substantive, 1230L/30F incl. docs/generated/snapshots — within ceiling
tier ✗ classified as T2-never: T2-never (1230L, 30F, two-areas, feat)
stamphog 2.5.1 .stamphog/policy.yml @ unknown · reviewed head e294bd9

@andrewm4894

Copy link
Copy Markdown
Member

/trunk merge

1 similar comment
@andrewm4894

Copy link
Copy Markdown
Member

/trunk merge

@trunk-io
trunk-io Bot merged commit ef7af98 into master Oct 16, 2026
264 checks passed
@trunk-io
trunk-io Bot deleted the posthog-self-driving/featautoresearch-keep-training-after-5fc90e branch October 16, 2026 21:48

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed e294bd9: 1 Should fix.

if (!next.isAfter(now)) {
return 'Next research run on the next scoring day.'
}
return `Next research run on ${next.format('MMM D')}.`

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 · Account for scoring cadence when displaying the next research date

For a maintained pipeline last trained and scored on Nov 1 with cadence_days=30 and research_interval_days=7, this displays Nov 8, but the coordinator cannot launch training until Dec 1. next_research_at represents eligibility, while kickoff also requires score_due. Compute the next scoring day or describe this date as the earliest eligible date.

@deployment-status-posthog

deployment-status-posthog Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-10-16 22:18 UTC Run
prod-us ✅ Deployed 2026-10-16 22:31 UTC Run
prod-eu ✅ Deployed 2026-10-16 22:32 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(autoresearch): keep training after the first burst, on a research interval

1 participant