Skip to content

Fix debug_launcher on Windows - #4360

Open
CJstate wants to merge 1 commit into
huggingface:mainfrom
CJstate:debug-launcher-windows
Open

CJstate wants to merge 1 commit into
huggingface:mainfrom
CJstate:debug-launcher-windows

Conversation

@CJstate

@CJstate CJstate commented Oct 2, 2026

Copy link
Copy Markdown

What does this PR do?

debug_launcher is meant to let a training script be debugged on CPU, but on Windows it cannot get as far as a
first step. Three things in it only hold on Unix-like platforms:

  • fork — start_processes(..., start_method="fork") raises ValueError: cannot find context for 'fork' on
    Windows, before a single worker starts.
  • the rendezvous file — it is created with tempfile.NamedTemporaryFile(), which keeps the file open in the
    parent process for the whole launch. Workers are started with spawn, and a spawned worker cannot open a file
    that is still open elsewhere, so FileStore fails with PermissionError.
  • gloo_socket_ifname="lo" — lo is the Unix name of the loopback interface. On a platform that names it
    differently, gloo fails with RuntimeError: Unable to find address for: lo.

This PR:

  • picks the first start method torch.multiprocessing actually reports. That is fork wherever it exists, so
    behaviour on Unix-like platforms is unchanged, and spawn on Windows;
  • gives the workers a path inside a TemporaryDirectory instead of an open file handle, letting FileStore
    create the file in the workers themselves;
  • pins lo only when an interface with that name is present, and otherwise leaves gloo's own interface selection
    alone, exactly as it does today whenever the variable is unset.

These three problems were reported together in #4285, which was closed unmerged; this PR fixes the same three
points and adds regression tests.

Tests

Three tests are added to tests/test_launch.py. They replace torch.multiprocessing.start_processes and
socket.if_nameindex, so they assert the start method, the rendezvous path and the interface pinning that
accelerate hands to its workers, and they run on any platform.

I reproduced all three failures on Windows (Python 3.13, torch 2.9.1): the ValueError comes out of
start_processes itself, and the PermissionError out of a worker opening the rendezvous path. The tests fail
on the current main and pass with this change.

A note on what is not covered: the new tests replace the process spawn, so they exercise the contract accelerate
controls rather than a full worker run. After these fixes the launch on my machine stops later inside gloo, which
refuses every interface it is offered on the Windows torch build I have — a limitation of that build rather than
of accelerate. The three failures fixed here all happen before, or independently of, gloo device selection.

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed.

debug_launcher is a core part of the library: @BenjaminBossan @SunMarc

`debug_launcher` relies on three assumptions that only hold on Unix-like
platforms, each of which stops it on Windows:

* it starts the workers with `fork`, which Windows does not have
  (`ValueError: cannot find context for 'fork'`),
* it hands the workers a `NamedTemporaryFile` path, but that file stays
  open in the parent process, so a spawned worker cannot open it
  (`PermissionError`),
* it pins gloo to the `lo` interface, a name that only Unix-like
  platforms use (`RuntimeError: Unable to find address for: lo`).

Pick the first start method torch actually offers, hand the workers a
path inside a temporary directory instead of an open file handle, and
only pin `lo` where that interface exists. Platforms that have both
`fork` and `lo` behave exactly as before.

The three tests added to `tests/test_launch.py` replace the process
spawn and the interface listing, so they exercise the arguments and
environment accelerate passes to its workers on every platform.

Signed-off-by: CJstate <142857225+CJstate@users.noreply.github.com>
CJstate added a commit to CJstate/CJstate that referenced this pull request Oct 2, 2026
@CJstate

CJstate commented Oct 3, 2026

Copy link
Copy Markdown
Author

Small process note on CI: the five pull_request workflows for this branch (ef5b3a75) all completed as action_required — they are queued for a maintainer to approve the run for a first-time contributor, so no check has actually reported yet.

Could someone click "Approve and run workflows" when convenient? Until then the checks stay empty and the PR cannot be evaluated by the bot.

Meanwhile the description has the local before/after evidence for the Windows debug_launcher failure (ValueError: cannot find context for 'fork' and the NamedTemporaryFile PermissionError), and I am happy to run anything else you want on a Windows box.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant