Skip to content

fix(signals): merge survey answers per question in the surveys scout recipe - #116683

Merged
trunk-io[bot] merged 1 commit into
masterfrom
posthog-self-driving/fixsurveys-scout-preserve-earlier-60edff
Oct 14, 2026
Merged

trunk-io[bot] merged 1 commit into
masterfrom
posthog-self-driving/fixsurveys-scout-preserve-earlier-60edff

Conversation

@posthog

@posthog posthog Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Problem

  • The surveys scout can miss survey feedback. When a later event in a submission omits an answer, the scout reads that answer as empty.
  • Before this change, response-querying.md kept only argMax(uuid, timestamp) for each submission and read the answers from that one event.
  • Partial-response events and manual captures can put each answer on a different event. The scout then dropped the earlier answers from its rating trends and theme reads.
  • This is a defect in the scout's query recipe only. Stored responses and SDK events are correct.

Origin

  • Scout: a custom scout
  • First signal: 2026-10-13
  • Inbox report: open
  • Task started by: auto-start, after the report was rated P3 and ready to fix

Changes

  • The recipe now groups survey sent events by submission and keeps the latest nonempty value of each question (argMaxIf(answer, timestamp, answer != '')). This is the same merge that fetch_response_rows() in the surveys responses API uses.
  • The recipe keeps the existing rules: the modern and legacy key coalesce, the $survey_id filter, the uuid fallback when $survey_submission_id is missing, and $survey_iteration as a grouping boundary.
  • The recipe reads completion separately: a submission is complete when one or more of its events is not marked $survey_completed: false.
  • The recipe gives a rule for deleted answers. An empty value and an omitted key are the same, so an earlier answer stays. A cleared answer can stay, but a given answer never gets lost.
  • The rating-trend and open-text queries now read from one per-submission base subquery.
Case Old recipe (latest event) New recipe
Answers split across events keeps the last event's answers only keeps every answer
Cumulative events correct correct
Changed answer later value later value
Later event has an empty answer empty earlier answer
No submission ID one row per event one row per event

How did you test this code?

  • Ran a synthetic HogQL fixture through execute-sql. It had complementary partial events, cumulative events, a changed answer, a later empty answer, and legacy events without an ID. The merge gave the expected value in each case. The old latest-event read returned empty for the later-empty case.
  • Ran the exact rating-trend SQL against events with a placeholder survey ID to confirm that it compiles.
  • On one survey with multi-event submissions, the merged row count matched the count of distinct submission keys, and the completion flag separated complete from partial submissions. Only aggregate counts were read.
  • posthog/test/repo_invariants/test_signals_scout_tool_citations.py passes, and Prettier is clean.

Test rationale: This change edits only a Markdown prompt reference. No test reads the content of a scout reference, so the fixture check above stands in for a unit test.

Release status

  • No feature flag controls this change

Docs update

None. The edited file is the scout's own reference. lazy_seed mirrors it to teams that have not changed their copy.

🤖 Agent context

Autonomy: Fully autonomous

Agent: PostHog Desktop (Claude Code), claude-opus-5-5

  • Started from an inbox report about the surveys scout recipe. Checked scout notes and the scratchpad first, and found nothing that blocks this area.
  • Ported the merge from products/surveys/backend/responses/fetch_rows.py and did not design a new one. This keeps the scout and the product in agreement.
  • Left products/surveys/backend/util.py (get_unique_survey_event_uuids_sql_subquery) unchanged, because the report scopes the fix to the scout reference.
  • Duplicate search: gh pr list --state open --search "survey scout response" found no PR for this defect.
  • All fixture data is invented. Nothing from the session is in this PR.

Created with PostHog Desktop from this inbox report.

🤖 Generated with Claude Code

…recipe

The surveys scout reference kept only the latest event of each submission and read answers from it. A later event that omits an earlier answer made the recipe drop that answer. The recipe now keeps the latest nonempty value of each question per submission, the same merge the responses API uses, and reads completion separately.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Generated-By: PostHog Desktop
Task-Id: dab5f15d-dd9c-401a-a205-7850ca5ea961
@posthog posthog Bot added the self-driving label Oct 13, 2026
@posthog
posthog Bot marked this pull request as ready for review October 13, 2026 23:53
@trunk-io

trunk-io Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

😎 Merged successfully - details.

@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team October 13, 2026 23:53
@github-actions

Copy link
Copy Markdown
Contributor

🤖 CI report

⚠️ Trunk lane — backend Python lane

This PR is assigned to the backend Python lane. It runs backend Python tests and may merge in parallel with PRs in other lanes.

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved.

Contained scout-reference update with no open review concerns or identified showstoppers. The platform-attested Inbox provenance exempts the machine author from the bot-author refusal rule.

  • PostHog Review (Standard) reviewed the current head: no findings.
Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✓ no deny categories matched
size ✓ 0L, 0F substantive, 110L/1F incl. docs/generated/snapshots — within ceiling
tier ✓ T0 auto-approve: T0-deterministic (110L, 1F, single-area, fix)
stamphog 2.5.0 .stamphog/policy.yml @ a08353f · reviewed head a08353f

@andrewm4894

Copy link
Copy Markdown
Member

/trunk merge

@trunk-io
trunk-io Bot merged commit 1e62b13 into master Oct 14, 2026
298 of 375 checks passed
@trunk-io
trunk-io Bot deleted the posthog-self-driving/fixsurveys-scout-preserve-earlier-60edff branch October 14, 2026 06:01
@deployment-status-posthog

deployment-status-posthog Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-10-14 06:20 UTC Run
prod-us ⏳ Pending
prod-eu ✅ Deployed 2026-10-14 06:34 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant