Skip to content

Node state-transition model: execution_id stamping + derived statuses - #263

Merged
Pino de Candia (pinodeca) merged 1 commit into
mainfrom
node-transition-model
Jun 29, 2026
Merged

Node state-transition model: execution_id stamping + derived statuses#263
Pino de Candia (pinodeca) merged 1 commit into
mainfrom
node-transition-model

Conversation

@pinodeca

@pinodeca Pino de Candia (pinodeca) commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Summary

Implements read-time node-status inference for pg_durable so that
df.instance_nodes() and df.explain() report structure-aware statuses for each
node, including branches that were never taken and nodes superseded by later loop
iterations.

The implementation keeps the physical node statuses as pending, running,
completed, and failed. Derived statuses such as skipped, and loop
re-entry-as-pending, are computed from graph structure plus each node's
status_details.execution_id lineage.

What changed

Schema

  • Adds a nullable df.nodes.status_details JSONB column holding the worker-written
    {"execution_id": "<instance::generation[::...]>"} stamp. The column is added
    for fresh installs in src/lib.rs and for existing installs through the
    0.2.3 -> 0.2.4 upgrade script.

Write path

  • Stamps each running and terminal node transition with deterministic execution
    lineage.
  • Applies a monotonic generation fence so stale updates from superseded
    older-generation executions cannot overwrite newer node state.
  • Keeps the new .so compatible with older schemas that do not yet have
    df.nodes.status_details by probing for the column at runtime.

Read path

  • df.instance_nodes(instance_id) returns one row per node with stored physical
    status plus status_details, inferred_status, and
    inferred_status_from_ancestor_id.
  • inferred_status is derived by walking the graph from the root:
    • a non-terminal node under a terminal ancestor reads as skipped;
    • a node from a prior loop iteration reads as pending because it will re-run;
    • a node that physically ran in the current generation keeps its stored status.
  • df.explain(instance_id) uses the same inference helper so the tree view agrees
    with df.instance_nodes().
  • df.instance_nodes(text, integer) remains callable for old schemas and upgraded
    callers as a shape-compatible adapter. It no longer queries execution history,
    ignores last_n_executions, and returns one row per node with execution_id = 1.

Tests & docs

  • Adds tests/e2e/sql/53_inferred_status.sql covering untaken df.if() arms,
    failed df.then() downstream nodes, clean sequences, and df.explain() output.
  • Updates USER_GUIDE.md, docs/api-reference.md, and the pg-durable-sql skill
    for the inferred-status model and the new primary df.instance_nodes() shape.

Related issues

Upgrade & migration

  • The 0.2.3 -> 0.2.4 script adds df.nodes.status_details and recreates
    df.instance_nodes() so upgraded schemas match fresh installs.
  • The new binary remains compatible with previous 0.2.x schemas before
    ALTER EXTENSION UPDATE: write/read paths avoid referencing status_details
    when the column is absent, and the old two-argument df.instance_nodes() symbol
    remains callable.
  • Drain in-flight instances before upgrading. The worker now records execution
    stamps in activity inputs, and duroxide compares recorded activity inputs by
    exact equality during replay.

@pinodeca Pino de Candia (pinodeca) changed the title Implement node state-transition model (#171, #240) Node state-transition model Jun 23, 2026
Pino de Candia (pinodeca) added a commit that referenced this pull request Jun 23, 2026
The proposal previously framed the #240 skipped-node work as part of
'this PR'. That work landed earlier in PR #249; this PR (#263) defines
the consolidated state model and adds the remaining implementation
(cancelled, status_reason, race-loser reconciliation, loop reset).
Pino de Candia (pinodeca) added a commit that referenced this pull request Jun 23, 2026
Adds docs/node-state-model.md, a design proposal consolidating the
node-status work for issues #240 (skipped downstream nodes) and #171
(race-loser nodes left running/pending). Defines instance and node
lifecycle states, legal transitions, a coarse status set plus nullable
status_reason, loop iteration-scoping/reset semantics, and prior-art
alignment with Airflow/Temporal/BPMN/Step Functions.

The #240 skipped-node work shipped earlier in PR #249; this PR (#263)
defines the consolidated model and will add the remaining implementation
(cancelled, status_reason, race-loser reconciliation, loop reset) after
review.

Design only; implementation to follow on this PR.
@pinodeca
Pino de Candia (pinodeca) changed the base branch from main to copilot/fix-loop-restart-issue June 27, 2026 13:30
@pinodeca
Pino de Candia (pinodeca) force-pushed the copilot/fix-loop-restart-issue branch from 3a4255f to b26327a Compare June 27, 2026 13:51
@pinodeca
Pino de Candia (pinodeca) changed the base branch from copilot/fix-loop-restart-issue to main June 28, 2026 15:18
@pinodeca
Pino de Candia (pinodeca) force-pushed the node-transition-model branch 2 times, most recently from 36adea3 to f508143 Compare June 28, 2026 21:18
@pinodeca Pino de Candia (pinodeca) changed the title Node state-transition model Node state-transition model: execution_id stamping + derived statuses Jun 28, 2026
@pinodeca
Pino de Candia (pinodeca) force-pushed the node-transition-model branch 2 times, most recently from bfe179b to e88ad3e Compare June 28, 2026 21:42
@pinodeca
Pino de Candia (pinodeca) force-pushed the node-transition-model branch 3 times, most recently from 0f21033 to bb71981 Compare June 29, 2026 20:27
Add status_details metadata to df.nodes and stamp each node transition
with deterministic execution lineage. Use the lineage to fence stale
loop-generation writes while preserving older-schema write compatibility.

Move df.instance_nodes() and df.explain() to a shared read-time
inference model. It reports skipped branches and superseded loop-body
nodes without adding new physical node statuses.

Keep df.instance_nodes(text, integer) callable for old schemas and
upgraded callers as a shape-compatible adapter. It no longer queries
execution history and returns one row per node with execution_id = 1.

Update the 0.2.3 to 0.2.4 migration, user/API docs, SQL skill guidance,
and E2E coverage for the inferred-status model.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No "skipped" indication for downstream steps after a step fails df.instance_nodes leaves race-loser nodes running or pending after race completion

1 participant