Skip to content

Support Sync and Async on non-blocking fibers via Fiber.blocking.#460

Open
samuel-williams-shopify wants to merge 4 commits into
mainfrom
sync-on-non-blocking-fiber
Open

Support Sync and Async on non-blocking fibers via Fiber.blocking.#460
samuel-williams-shopify wants to merge 4 commits into
mainfrom
sync-on-non-blocking-fiber

Conversation

@samuel-williams-shopify

@samuel-williams-shopify samuel-williams-shopify commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Sync and Async could not be invoked from a non-blocking fiber that has no scheduler — e.g. inside an Enumerator block or a bare Fiber.new { ... }.resume. Doing so raised:

RuntimeError: Running scheduler on non-blocking fiber! (lib/async/scheduler.rb)

Minimal reproduction:

e = Enumerator.new do |yielder|
  yielder << Sync { fetch_something }
end
e.next   # => RuntimeError: Running scheduler on non-blocking fiber!

This shows up whenever Sync/Async is reached beneath a bare non-blocking fiber (job runners, Enumerator, other libraries), and previously required an awkward caller-side workaround such as Fiber.new(blocking: true) { Sync { ... } }.resume.

Change

Run the reactor within Fiber.blocking (kernel/sync.rb, kernel/async.rb). When there is no current task and no scheduler, the reactor is started inside Fiber.blocking { ... }, so the event loop always runs on a blocking fiber — on the current fiber, preserving fiber identity and fiber-locals (no extra fiber is spawned). Task fibers still terminate normally, so control returns to the reactor loop via the resume chain as before.

Tests

  • test/kernel/sync.rb — a new with "a non-blocking fiber" context covering: running the scheduler on a non-blocking fiber, return values, asynchronous work with blocking operations (barrier + semaphore), exception propagation, fiber-storage preservation, nested Sync, use within an Enumerator (value yielded after Sync returns), and the clean-failure case when trying to yield to the enumerator from inside the block.
  • test/kernel/async.rb — running Async on a non-blocking fiber, running work to completion, and passing options through.

Notes

This handles the common case, where the reactor fiber is entered via resume (which covers Enumerator, bare Fiber.new { ... }.resume, and typical job runners). The case where the reactor fiber is entered via Fiber#transfer and is off the resume chain (e.g. owned by another transfer-based scheduler) is not addressed here — it requires the Ruby-level fix tracked in https://bugs.ruby-lang.org/issues/20081, since fiber termination needs to return control to the transferring fiber.

Previously, calling Sync or Async from a non-blocking fiber with no
scheduler (e.g. inside an Enumerator or a bare Fiber.new) raised
'RuntimeError: Running scheduler on non-blocking fiber!', because the
reactor's event loop must run on a blocking fiber.

The reactor is now run within Fiber.blocking, so the scheduler always
runs on a blocking fiber regardless of the calling fiber.

In addition, when a task finishes it now explicitly transfers control
back to the scheduler via Fiber.scheduler&.transfer. This ensures the
event loop regains control even when it is running on a fiber that is
not the thread's root fiber (e.g. a fiber owned by another
transfer-based scheduler), instead of relying on fiber termination
returning control to the expected fiber. The transfer is placed after
the task body so that a critical exception propagating out of the task
is not swallowed.

Assisted-By: devx/5f4cfab6-1d74-4100-8755-82d7c62c70ab
The unconditional Fiber.scheduler&.transfer at task completion left task
fibers parked (alive but finished) instead of terminating, which breaks
Fiber#alive? assumptions in io-event (timers/waiters transfer into
finished fibers) and caused downstream servers to hang.

The Fiber.blocking change alone fixes the reported cases (Enumerator,
bare Fiber.new) since those enter the reactor fiber via resume, so task
termination returns control correctly. The transfer-based-scheduler case
requires the Ruby-level fix in https://bugs.ruby-lang.org/issues/20081
and is not addressed here.

Also fix RuboCop block delimiter spacing in tests.

Assisted-By: devx/5f4cfab6-1d74-4100-8755-82d7c62c70ab
@samuel-williams-shopify samuel-williams-shopify changed the title Support Sync and Async on non-blocking fibers. Support Sync and Async on non-blocking fibers via Fiber.blocking. Jul 1, 2026
- Do not bump version.rb in the PR.
- Move the release note under ## Unreleased.
- Trim the non-blocking fiber tests to a focused set.

Assisted-By: devx/5f4cfab6-1d74-4100-8755-82d7c62c70ab
It exercised a workaround the fix does not use, and was a slow (2s)
deadlock-via-timeout test.

Assisted-By: devx/5f4cfab6-1d74-4100-8755-82d7c62c70ab
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant