iOS testbed fails when the dir ~/Library/Developer/XCTestDevices is empty

Bug report

Bug description:

iOS testbed run fails when the dir ~/Library/Developer/XCTestDevices is empty.

The dir ~/Library/Developer/XCTestDevices is empty on machines that are clean, e.g. CI images.

The traceback is:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/private/var/folders/x8/34szswsn0zn1ktnx8k1v16qr0000gn/T/cibw-run-pztsyqge/cp313-ios_arm64_iphonesimulator/testbed/__main__.py", line 548, in <module>
    main()
    ~~~~^^
  File "/private/var/folders/x8/34szswsn0zn1ktnx8k1v16qr0000gn/T/cibw-run-pztsyqge/cp313-ios_arm64_iphonesimulator/testbed/__main__.py", line 530, in main
    asyncio.run(
    ~~~~~~~~~~~^
        run_testbed(
        ^^^^^^^^^^^^
    ...<3 lines>...
        )
        ^
    )
    ^
  File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ~~~~~~~~~~^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 719, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/private/var/folders/x8/34szswsn0zn1ktnx8k1v16qr0000gn/T/cibw-run-pztsyqge/cp313-ios_arm64_iphonesimulator/testbed/__main__.py", line 411, in run_testbed
    simulator = await select_simulator_device()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/var/folders/x8/34szswsn0zn1ktnx8k1v16qr0000gn/T/cibw-run-pztsyqge/cp313-ios_arm64_iphonesimulator/testbed/__main__.py", line 129, in select_simulator_device
    raw_json = await async_check_output(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
        "xcrun", "simctl", "--set", "testing", "list", "-j"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/private/var/folders/x8/34szswsn0zn1ktnx8k1v16qr0000gn/T/cibw-run-pztsyqge/cp313-ios_arm64_iphonesimulator/testbed/__main__.py", line 118, in async_check_output
    raise subprocess.CalledProcessError(
    ...<4 lines>...
    )
subprocess.CalledProcessError: Command '('xcrun', 'simctl', '--set', 'testing', 'list', '-j')' returned non-zero exit status 1.

On a machine with empty ~/Library/Developer/XCTestDevices the above command returns:

$ xcrun simctl --set testing list -j
  
Using Parallel Testing Device Clones Device Set: '/Users/runner/Library/Developer/XCTestDevices'
Provided set path does not exist: /Users/runner/Library/Developer/XCTestDevices

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

Thanks for the report. This is one of those bugs that is infuriatingly difficult to reproduce, because as soon as you've run the simulator successfully, you never see the problem again. Thankfully it looks like this one might be reproducible on GitHub Actions.

I've previously fixed an issue related to this - #130388 - however, when I added support for autodetecting the default simulator in #133132, it looks like I've missed the corresponding error handling. Your proposed solution of removing the use of the testing set for device discovery makes sense; I'll check a couple of other edge cases to be sure.

gh-135101: When choosing the default simulator device, don't use `sim…

pythongh-135101: When choosing the default simulator device, don't us…

[3.13] gh-135101: When choosing the default simulator device, don't u…

[3.14] gh-135101: When choosing the default simulator device, don't u…

[3.14] pythongh-135101: When choosing the default simulator device, d…

[3.13] pythongh-135101: When choosing the default simulator device, d…

Thanks for merging!

This is one of those bugs that is infuriatingly difficult to reproduce, because as soon as you've run the simulator successfully, you never see the problem again.

I found that I could reproduce it by doing-

mv ~/Library/Developer/XCTestDevices ~/Library/Developer/XCTestDevices_bkup

Then I'd play around with it, then delete the dir and restore the bkup once done.

I found that I could reproduce it by doing-...

Good to know; my concern is whether that's actually "clean" from the perspective of Xcode - is the directory the only artefact that Xcode is relying on? It may well be... but it's impossible to know for sure.

pythongh-135101: When choosing the default simulator device, don't us…

[3.10] pythongh-135101: When choosing the default simulator device, d…

[3.11] pythongh-135101: When choosing the default simulator device, d…

[3.12] pythongh-135101: When choosing the default simulator device, d…

[3.11] pythongh-135101: When choosing the default simulator device, d…

[3.12] pythongh-135101: When choosing the default simulator device, d…

[3.11] pythongh-135101: When choosing the default simulator device, d…

[3.12] pythongh-135101: When choosing the default simulator device, d…

Navigation Menu

Repository navigation

iOS testbed fails when the dir ~/Library/Developer/XCTestDevices is empty #135101

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

iOS testbed fails when the dir ~/Library/Developer/XCTestDevices is empty #135101

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

freakboy3742 commented on Jun 3, 2025

joerick commented on Jun 4, 2025

freakboy3742 commented on Jun 4, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions