Skip to content

embedder/streams: a lifted wrapper's HostActivity arm outlives the hand-back to the guest — one identity round-trip suppresses deadlock verdicts forever #162

Description

@lannbot

Found while scoping deadlock-detection blind spots after #160 (thread: #156's investigation → #160's "external pending" caveat). Record-only for now; the fix design below is worked out but not scheduled.

The bug

A host import that receives a stream and immediately hands it back — the minimal shape is identity: async func(s: stream<u8>) -> stream<u8> — leaves the store permanently unable to trap genuine deadlocks.

Mechanics (all in runtime/src/exec/host_streams.ts + runtime/src/embedder/streams.ts):

  1. Lifting the guest's stream argument into the host builds a wrapper via hostStreamFor, which mints a HostActivity and arms it immediately (bindOnLoweractivity.bind(boundStore), the lift-time branch). The arm is a promise parked in store.pendingHostCalls meaning "the embedder may still act".
  2. The handler returns the same Stream; lowerStreamSource takes the same-copy fast path → Stream.takeValue() → marks the handle #consumed and lowers the shared object back into the guest's table. Nothing on this path touches the arm.
  3. HostActivity.close() fires only on DROPPED or end-of-pump, and is terminal (#closed latch). There is no non-terminal disarm. The arm lives forever.

The arm class is correctly excluded from hasRealHostCall/storeQuiescent (the pumps are unaffected), but both deadlock verdicts in driveAsync read raw pendingHostCalls.size — the probe precondition (exec/boundary.ts :882) and the bottom trap (:1001). So after one identity round-trip:

  • both verdicts are suppressed for the store's remaining lifetime;
  • the driver's final race parks on a promise only notify() — embedder activity on a wrapper the embedder already gave back — would fire;
  • every later genuine guest deadlock presents as the documented "embedder may act" hang, when the embedder provably cannot act: it retains no end.

The dropForTeardown asymmetry already flagged in review (embedder/streams.ts ~:309, "the arm can outlive the stream… misreporting a later genuine deadlock as the documented hang") is the same class, but scoped to already-faulted stores; the identity round-trip does it to a healthy store.

Fix design: arm liveness = host retention

The wrapper can account retention exactly, so the arm should be live iff the host retains a way to act on the shared object (a retained end, a parked op of ours, or an unfinished producer pump):

  • Lifted wrapper (hostStreamFor): the host holds exactly the end the guest passed. takeValue transfers it back → retained ends hit zero → disarm.
  • Host-created wrapper (hostStream()Stream.create(), lowerStreamSource producer adaptation): the host retains the writer after lowering → arm stays; end-of-pump/DROPPED already close it (R-fix advisory 2). So the change introduces no false-trap risk: wherever "embedder may act" is true, the arm survives.
  • Disarm must be non-terminal (a new HostActivity.disarm() beside terminal close()): the A5 wrapper cache returns the same wrapper if the guest passes the stream out again, and that re-lift must re-arm. This is a real new edge, not a flag flip — today bind happens only at wrapper construction, so a cache-hit lift needs an explicit re-arm hook (and onLowered fires once per shared object, per bindOnLower's single-hook assert).
  • Futures: identical shape (Future.takeValue, hostFutureFor), identical fix. This exhausts the inventory — resources/subtasks/waitables never arm activity, and the import-call promise itself settles when the handler returns.
  • Adjacent decision forced by the fix: post-transfer read()/write() through a consumed wrapper. Ownership went back to the guest, but Stream.#require() only checks for a bound host end, so acting through the wrapper after takeValue operates a phantom duplicate of an end the guest now owns. Refusing loudly is the coherent companion to disarming, but it is a (minor) embedder-visible behavior change to adjudicate alongside.
  • The same non-terminal disarm primitive closes the dropForTeardown asymmetry.

Scope notes

Suggested tests (when scheduled)

  1. Identity round-trip unit test: after takeValue, no arm remains in pendingHostCalls; a manufactured guest deadlock afterward TRAPS (pre-fix: hangs).
  2. Retention case: Stream.create() lowered into a guest with the writer retained — arm stays; end-of-pump still closes it.
  3. Re-lift after round-trip: guest passes the same stream out again — cache-hit wrapper re-arms.
  4. Futures: (1)–(3) mirrored.
  5. dropForTeardown on a store with nothing parked — arm released.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingp1Correctness bugs likely to impact consumers; high-priority missing features

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions