Skip to content

feat(workflow): DAG conditions and composable sub-workflows - #1129

Merged
jeff-r2026 merged 17 commits into
Tencent:mainfrom
SaulMoro:feat/workflow-dag-1078
Oct 15, 2026
Merged

jeff-r2026 merged 17 commits into
Tencent:mainfrom
SaulMoro:feat/workflow-dag-1078

Conversation

@SaulMoro

@SaulMoro SaulMoro commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #1078. Independent of #1126 (run records): whichever merges second adapts run records to graph runs (step lookup by path, a skipped count, end). If #1126 lands first without that, I follow up with the adaptation. One commit per slice; the plan is in #1078 (comment).

Summary

Workflow steps form a graph: needs, if, named ends, inputs, and sub-workflows (from a file or inline).

 leaseNext
-  first pending step; any earlier non-succeeded step fails the run
+  settleRun: skip steps whose needs stopped (unless their `if` reads them) or were all skipped,
+            or whose `if` is false; end the run at a reached `end` or when nothing is left
+  lease the first ready step in declaration order (child steps by path: check.classify)
 reportAttempt
-  a failed report fails the run on the spot
+  record the outcome; the next `next` keeps settling the graph
inputs:
  title: string
ends:
  shipped: succeeded
steps:
  - id: check
    workflow: check                 # file sub-workflow, snapshotted at start
    with:
      title: "{{ inputs.title }}"
  - id: fix
    needs: [check]
    if: steps.check.outputs.verdict == 'fix'
    command: node fix.js
    env:
      TITLE: "{{ inputs.title }}"   # inputs reach commands only through env
  - id: drop
    needs: [check]
    if: steps.check.outputs.verdict == 'drop'
    command: node drop.js
  - id: verify
    needs: [fix]
    workflow:                       # inline sub-workflow
      ends:
        green: succeeded
      steps:
        - id: test
          command: npm test
          end: green
  - id: shipped
    needs: [verify]
    if: steps.verify.end == 'green'
    end: shipped

Pull checks each file, then checks a step that runs another workflow against that workflow's runnable version: missing child, recursion, with keys against the child's inputs, and reads of outputs or ends the child (file or inline) does not declare. Pull and the engine take the templated fields (with, prompt, env) and the step actions from one list, so they cannot disagree on what is a template. docs/dev/workflow-engine.md lists the engine rules that span more than one file, for the next change to src/workflow/.

Pull settles every file's version before any check reads it, so no file is checked against a version that ends up rejected:

start: each file at its revision if it passed its own checks, else at its kept copy (last valid version) unless a code dropped it
repeat until no version fails, checking each against the versions the others are at now (child exists, with/inputs, outputs, ends, cycle):
  a failing version falls back (revision -> kept copy -> nothing) once nothing it runs, directly or not, still fails
  if every failing version waits on another, they run in a cycle: the new revisions on it fall back first
cache = exactly the settled versions: an unreadable file keeps its copy; a missing or symlinked file has none

Decisions worth a look:

  • needs is required after the first step; needs: [] declares another root. Order alone never implies a dependency.
  • skipped is a step status, not a report outcome: drivers still report only success | failed | released.
  • Templates never expand inside command; ${{ … }} stays literal elsewhere (GitHub Actions text), but is flagged inside command.
  • A read of an output or end the child does not declare drops the parent's cached copy even when the parent file is unchanged, so a renamed child output cannot leave a parent that always skips.
  • A kept copy is kept only if it fits what the workflows it runs settle to; the cache stores the source for that check. New revisions that run each other in a cycle are all rejected, since picking one of them by file name would be arbitrary.
  • A run stored by a version without needs makes next return run_unreadable instead of running later steps as roots.

Evidence

  • Before (main): needs, if, end, inputs, env, with and workflow are not step keys (STEP_KEYS = id, command, prompt, prompt_file, agent, output), so the file below fails pull and a failed step fails the whole run.
    After, real CLI on 62d004f (npm run build, sandboxed HOME, teamai init --self, the two files above with check.yaml classifying the title):

    $ teamai pull
    ✔ [workspace] Synced 2 workflows
    $ teamai workflow start ship --driver local --inputs '{"title":"crash on save; rm -rf /"}'
    {"data":{"done":{"run":"r611f1d1e","outcome":"succeeded","end":"shipped"}}}
    $ steps in the run file
    check            succeeded  success
    check.classify   succeeded  success
    fix              succeeded  success
    drop             skipped
    verify           succeeded  success green
    verify.test      succeeded  success
    shipped          succeeded
    

    And a typo in an inline workflow read is caught at pull:

    $ teamai pull
    ⚠ [workflows] workflows/inline-typo.yaml:19: field "steps[1].if": step "use" if reads steps.grp.outputs.verdcit, which is not an output the inline workflow of step "grp" declares
    ⚠ [workflows] workflows/inline-typo.yaml has no runnable version
    
  • Kept copies, real CLI on cc799fb (same setup). A parent whose new revision is invalid no longer keeps a copy that reads an output its child renamed:

    $ teamai pull
    ⚠ [workflows] workflows/ship.yaml:3: field "description": description must be a non-empty string
    ⚠ [workflows] Dropped the last valid version of ship: step "fix" if reads steps.check.outputs.verdict, which is not an output workflow "check" declares
    ⚠ [workflows] workflows/ship.yaml has no runnable version
    $ teamai workflow start ship --driver local
    {"error":{"code":"workflow_not_found","message":"Workflow \"ship\" has no runnable version. Check workflows/ship.yaml and pull again."}}
    
  • Kept copies and cycles, real CLI on f251447 (same setup). Cache ship -> check with a command-only check, then make ship.yaml invalid and check.yaml run ship.
    Before (cc799fb): the cycle check followed check's rejected revision and dropped ship:

    $ teamai pull
    ⚠ [workflows] Dropped the last valid version of ship: step "check" runs workflow "check" in a cycle: ship -> check -> ship
    $ teamai workflow start ship --driver local
    {"error":{"code":"workflow_not_found","message":"Workflow \"ship\" has no runnable version. Check workflows/ship.yaml and pull again."}}
    

    After (f251447): check's new revision is rejected for the cycle, and both keep their last valid version:

    $ teamai pull
    ⚠ [workflows] workflows/check.yaml:9: field "steps[1].workflow": step "again" runs workflow "ship" in a cycle: check -> ship -> check
    ⚠ [workflows] Kept the last valid version of check; this revision was not cached
    ⚠ [workflows] workflows/ship.yaml:3: field "description": description must be a non-empty string
    ⚠ [workflows] Kept the last valid version of ship; this revision was not cached
    $ teamai workflow start ship --driver local
    {"data":{"done":{"run":"r806c5436","outcome":"succeeded"}}}
    
  • Tests on f251447, on main 3865561: npx tsc --noEmit, npm run lint, npx vitest run (9968 passed; gitlab-clone-realspawn times out here, and also on main), full npm run test:e2e (1246 passed; the 6 workflow files pass; dashboard-unified times out here and on main, submodule-checkout-1041 passes when run alone, source-freshness-recall is the macOS /private/var path also failing on main). A table test runs a parent and its child through accept, keep, delete, unreadable and symlink revisions, a renamed output and a cycle, in both name orders. It starts every workflow each sync caches or keeps and checks the cache holds nothing else. The conformance suite gains scenarios for skipped steps, rendered env, sub-workflow paths and recovery after a failure.

  • Not checked locally: providers other than git and agents other than Claude; left to CI.

Merge Danger

Door: two-way

Workflows are new in the 0.28 beta; no stable release carries them. An older teamai reading a cache written by this version runs the graph as a linear list (it fails open), so a downgrade during the beta should be followed by teamai pull.

Blast Radius: workflows

A step after the first now needs needs, so existing beta workflow files without it fail pull until updated. A workflow file that is a symlink, or a symlinked workflows/ directory, now loses its cached version at the next pull, like a deleted file. Nothing outside src/workflow/, workflow start --inputs, and the workflow guide changes.

Follow-ups noted on the issues they affect: lazy child creation and inline readiness need revisiting for parallel leases (#1079); env keys may shadow declared secrets (#1082).

@jeff-r2026

Copy link
Copy Markdown
Collaborator

@SaulMoro Thanks for all your recent contributions to TeamAI. We really appreciate the consistent, thoughtful work you've been doing. I've upgraded your repository access to write—well deserved!

@SaulMoro

Copy link
Copy Markdown
Collaborator Author

@jeff-r2026 Thank you, I appreciate the trust. I use TeamAI both with my team and on my own, and it's very useful in both cases. In my personal projects, it gives me a single source of truth for skills, rules and hooks, and every AI tool I use pulls from it, so I dogfood it a lot. The new workflows are very powerful. I've enjoyed contributing, and I have more ideas to improve it, so I'll keep sending them as issues and PRs.

@SaulMoro SaulMoro changed the title [After #1126] feat(workflow): DAG conditions and composable sub-workflows feat(workflow): DAG conditions and composable sub-workflows Oct 14, 2026
@SaulMoro
SaulMoro marked this pull request as ready for review October 14, 2026 13:58
@SaulMoro SaulMoro added the enhancement New feature or request label Oct 14, 2026
@jeff-r2026 jeff-r2026 self-assigned this Oct 14, 2026
@github-actions

Copy link
Copy Markdown
  • [P1 blocking] Revalidate retained parents against updated child contracts — src/workflow/cache.ts:190. Cache a parent whose branch reads steps.check.outputs.verdict, then invalidate its description while renaming the child’s export to decision. The parent bypasses reference checking and retains its cached definition. Its condition now reads null, silently skips the branch, and can report success. Check retained definitions against the final child versions before keeping them.

  • [P2 non-blocking] Check cycles against effective fallback definitions — src/workflow/cache.ts:247. Start with cached a → b and command-only b; change a → missing and b → a. Cycle detection follows the invalid new a, but pull subsequently retains old a → b and caches b → a. Both starts then fail with workflow_recursion despite b being reported synced. Include fallback definitions in cycle validation.

Testing: The description provides sufficient build and representative real-CLI evidence. Deferred provider/agent coverage is acceptable. No PR code was run, built, or installed.

A read of steps.<inline>.outputs.<name> or a comparison of steps.<inline>.end with a name the inline workflow does not declare passed pull and then always read null. Both now report undeclared_output / undeclared_end, as they already do for a workflow file.
Bundle the parse helpers' shared arguments into one context, walk referenced workflows through workflowsRunBy at start as at pull, name the cache-drop rule and the settled-workflow result once, share one isRecord, and type the outcome an expression reads. No behaviour change.
Pull and the engine now read the templated texts of a step (with, prompt, env) from one templatedTexts() in template.ts, and the step actions from one STEP_ACTIONS map with hasAction(). A new templated field or action is added in one place.

Pull now also checks `with` templates under keys YAML reads as numbers or booleans, which the engine already rendered.
Pull kept the last valid version of a file without checking it again. A kept
parent could then read an output its child had renamed and silently skip the
branch, or close a cycle with a child cached in the same sync, so both starts
failed with workflow_recursion.

A kept copy that runs a workflow file is now checked again, from the source
the cache now stores, against what each child settles to in the same sync.
A copy that no longer fits is dropped, and the file has no runnable version.
Settling follows the versions that would actually run, so a cycle through a
kept copy is caught at pull.
@SaulMoro
SaulMoro force-pushed the feat/workflow-dag-1078 branch from 4975af4 to cc799fb Compare October 14, 2026 16:42
@github-actions

Copy link
Copy Markdown
  • [P1 blocking] Check cycles against settled child versions — src/workflow/cache.ts:264. Cache ship → check with command-only check, then invalidate ship’s description and change check → ship. Pull rejects the new check and retains its command-only copy, but asked() still follows the rejected candidate when checking retained ship. It falsely reports a cycle and deletes ship, so workflow start ship returns workflow_not_found despite a valid, acyclic fallback pair. Cycle checks must account for the versions actually retained.

Earlier findings: Both previously reported failure scenarios are resolved in this diff.

Testing: The description supplies sufficient build and representative real-CLI evidence. Deferred provider/agent coverage is acceptable. No PR code was run, built, or installed.

Pull checked a kept copy for cycles against a child's new revision even
when that revision was then rejected, so a valid parent lost its last
valid version. Every version is now checked only against the versions
the other files are at, and a failing one falls back until none fails;
a cycle falls back on the new revisions in it first. A file that cannot
be read keeps its last valid version under the same checks, and the
cache keeps only what the sync settled.
@github-actions

Copy link
Copy Markdown

No new findings in the specified diff.

Earlier findings: All three previously reported failure scenarios are resolved. Retained parents are revalidated against settled child contracts, and cycle checks use the effective versions, rejecting cyclic revisions before removing valid fallback copies.

Testing: The PR description supplies sufficient build and representative real-CLI evidence, including verification on f251447c. Deferred provider/agent coverage is acceptable.

No PR code was run, built, or installed.

@jeff-r2026
jeff-r2026 merged commit 474dfc0 into Tencent:main Oct 15, 2026
13 checks passed
kittimzhe added a commit to kittimzhe/teamai-cli that referenced this pull request Oct 15, 2026
Tencent#1129 merged before Tencent#1126, so by the agreement in
Tencent#1126 (comment)
this branch carries the adaptation, prepared by @SaulMoro
(records-on-graph-runs fd5b709) and re-verified here:

- the end record publishes from leaseNext's settleRun (the one settlement
  point), still after the run lock is released; the reportAttempt
  publication path is gone;
- one timestamp per operation, so a settled run's end record and its run
  file agree;
- definitions looked up by path via the exported locateStep; stepCounts
  gains skipped; an end-only step that succeeded gets outcome 'success',
  a skipped step none;
- the e2e fixture gains needs (required after the first step now) and
  passes the driver via startRun's options, not inputs.

Three sub-workflow tests from the pre-squash branch layout were stale
against main's final engine (they predated the saveCachedCopy/restore
mechanics); this restores main's versions of those tests verbatim.
kittimzhe added a commit to kittimzhe/teamai-cli that referenced this pull request Oct 16, 2026
… e2e tests

Per review: the two Tencent#1129 e2e tests compare the done envelope with
toEqual, so the new record field broke the strict match. Both fixtures
end with the end record still pending, so pin record: 'pending' to keep
the match strict and document the field.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feat] Add DAG conditions and composable sub-workflows

2 participants