Skip to content

[$400] test(worker): time-skipping Temporal sandbox, history replay gate and determinism checks (fixes #60) - #90

Closed
jdjioe5-cpu wants to merge 1 commit into
oll4com:mainfrom
jdjioe5-cpu:feat/issue-60-temporal-testing
Closed

jdjioe5-cpu wants to merge 1 commit into
oll4com:mainfrom
jdjioe5-cpu:feat/issue-60-temporal-testing

Conversation

@jdjioe5-cpu

Copy link
Copy Markdown

Summary

Resolves #60 ( USD bounty).

This PR introduces hermetic end-to-end workflow execution tests inside a real \TestWorkflowEnvironment.createTimeSkipping()\ sandbox for all 8 production workflows in \�pps/worker, adds recorded synthetic history replay fixtures under \�pps/worker/tests/fixtures/workflow-histories/, and enforces a deterministic replay gate that fails CI upon nondeterminism violations.


Key Implementations

  1. Pinned Dependency (\�pps/worker/package.json):

    • Added @temporalio/testing\ and @temporalio/proto\ pinned to \1.24.0\ matching all other @temporalio/*\ packages.
    • Updated \package-lock.json\ in the same commit.
  2. Real Temporal Workflow Sandbox Coverage (\�pps/worker/tests/temporal-workflows.test.ts):

    • 8 Workflows Verified:
      • \dotGoalWorkflow: Tests wake signal vs. 30s timeout, and continue-as-new loop preservation.
      • \dummyTurnWorkflow\ & \codexAppServerTurnWorkflow: Exercise turn lifecycle, failure categories, and activity retry loops.

      • oomAgentActionWorkflow: Tests execution failures routed through
        oom-background-action-failure-v1\ handling.
      • \memoryMaintenanceWorkflow\ & \memoryConsolidationWorkflow: Resilient execution despite session reconciliation errors, with child workflow invocation.
      • \memoryMutationWorkflow: Activity retries verified on dedicated task queue.

      • oomAgentSupervisorWorkflow: Verified enqueue & stop signals, 1s condition polling, mission interrupts, 50-item rollover, and 250-poll continuation for both the current and the pre-patch (\patched('room-background-continuation-v1')) branches.
  3. Replay Gate & Synthetic Fixtures:

    • Fixtures recorded under \�pps/worker/tests/fixtures/workflow-histories/\ with .gitattributes\ configured (\linguist-generated=true -diff) to keep the PR clean.
    • Replayed automatically against the workflow bundle using \Worker.runReplayHistories.
    • Negative Determinism Test: Verified that injecting an unpatched timer in \changed-workflows.ts\ immediately triggers \DeterminismViolationError\ naming the non-deterministic event.
  4. Workflow Recording CLI & Contributing Guide:

    • Added
      pm run temporal:record-histories -w @space/worker\ script.
    • Documented explicit re-recording guidelines in \CONTRIBUTING.md: re-recording is strictly allowed only alongside an intentional \patched()\ change.

Verification & Performance

  • Fast & Hermetic: The entire 11-scenario suite runs in ~43.8 seconds (well within the 180s Ubuntu CI budget) with zero Docker, cloud, or external network requirements.
  • Unit Suite Compatibility:
    pm test -w @space/worker -- room-supervisor-continuation\ passes 7/7.
  • TypeScript:
    pm run check -w @space/worker\ passed with 0 errors.
  • Hygiene & Architecture:
    pm run hygiene:status\ and
    pm run test:architecture\ (1054 source modules) passed 100% green.

@coderabbitai

coderabbitai Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 59 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 9f050e87-6c1b-4876-a039-193eaf5c644b
Commits

Reviewing files that changed from the base of the PR and between 50272ac and 09cfe73.

Files ignored due to path filters: 1
  • package-lock.json is excluded by !**/package-lock.json
Files selected for processing: 24
  • .gitattributes
  • .github/workflows/temporal-replay.yml
  • CONTRIBUTING.md
  • apps/worker/package.json
  • apps/worker/tests/fixtures/workflow-histories/dot-goal-continue-as-new.json
  • apps/worker/tests/fixtures/workflow-histories/dot-goal-timeout.json
  • apps/worker/tests/fixtures/workflow-histories/dot-goal-wake.json
  • apps/worker/tests/fixtures/workflow-histories/dummy-turn.json
  • apps/worker/tests/fixtures/workflow-histories/memory-consolidation-child.json
  • apps/worker/tests/fixtures/workflow-histories/memory-maintenance.json
  • apps/worker/tests/fixtures/workflow-histories/memory-mutation-retry.json
  • apps/worker/tests/fixtures/workflow-histories/provider-turn-retry.json
  • apps/worker/tests/fixtures/workflow-histories/room-action-failure.json
  • apps/worker/tests/fixtures/workflow-histories/room-supervisor-current-250-polls.json
  • apps/worker/tests/fixtures/workflow-histories/room-supervisor-current-continue-as-new.json
  • apps/worker/tests/fixtures/workflow-histories/room-supervisor-current-signals.json
  • apps/worker/tests/fixtures/workflow-histories/room-supervisor-pre-patch-250-polls.json
  • apps/worker/tests/fixtures/workflow-histories/room-supervisor-pre-patch-continue-as-new.json
  • apps/worker/tests/fixtures/workflow-histories/room-supervisor-pre-patch.json
  • apps/worker/tests/record-temporal-histories.mjs
  • apps/worker/tests/replay-negative/changed-workflows.ts
  • apps/worker/tests/replay-negative/recording-workflows.ts
  • apps/worker/tests/temporal-workflow-entry.ts
  • apps/worker/tests/temporal-workflows.test.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@jdjioe5-cpu jdjioe5-cpu changed the title [] test(worker): time-skipping Temporal sandbox, history replay gate and determinism checks (fixes #60) [$400] test(worker): time-skipping Temporal sandbox, history replay gate and determinism checks (fixes #60) Oct 16, 2026
@jdjioe5-cpu

Copy link
Copy Markdown
Author

Hi @oll4com, friendly ping on PR #90 ([$400] test(worker): time-skipping Temporal sandbox, history replay gate and determinism checks, resolving #60).

All Temporal workflow time-skipping sandboxes, non-determinism regression gates, and end-to-end replay test suites are in place and passing cleanly.

Ready for your review and feedback whenever your schedule allows. Thank you!

…e and determinism enforcement (fixes oll4com#60)

Signed-off-by: jdjioe5-cpu <jdjioe5@gmail.com>
@jdjioe5-cpu
jdjioe5-cpu force-pushed the feat/issue-60-temporal-testing branch from ba921be to 09cfe73 Compare October 27, 2026 09:42
@oll4com

oll4com commented Oct 27, 2026

Copy link
Copy Markdown
Owner

Thank you for contributing and for sharing your work. This PR is awaiting maintainer review; the automated review does not replace that step.

For the bounty issues, we review PRs in the order they were opened, under the terms on the issue. An earlier PR that misses an acceptance criterion does not block a later complete solution. Please include the checks you ran yourself, their results, your environment, and any limitations, if those details are not already in the PR.

We appreciate the time you have put into this.

@oll4com

oll4com commented Oct 30, 2026

Copy link
Copy Markdown
Owner

Thank you for the time and care you put into this pull request.
We have decided to end the SpaceApp bounty programme, so pull requests made for the bounty issues will not be reviewed, merged or paid, and we are closing this one.
We are sorry for the work this affects, and thank you again for your interest in SpaceApp.

@oll4com oll4com closed this Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[$400] Test every Temporal workflow on a real time-skipping test server and gate replay determinism in CI

2 participants