Repository navigation
With asyncio subprocess, send_signal() and the child process watcher will both call waitpid() #87744
Description
Activity
lincheney commented
on Mar 21, 2021 lincheneymannequinMannequinAuthorUnder unix, when creating a asyncio subprocess, the child process watcher will call waitpid() to reap the child, but if you call send_signal() (or terminate() or kill() ) on the asyncio subprocess, this will also call waitpid(), causing exactly one of these to fail, as you cannot call waitpid() on a PID more than once.
If the send_signal() fails, this doesn't seem much of an issue.
If the child process watcher fails however, it sets the returncode to 255 and also returns 255 when running wait() and also emits a warning.I've seen this behaviour with the ThreadedChildWatcher, but possibly other Unix child watchers that use waitpid() suffer from the same problem.
The behaviour is racey (it depends on which one completes the waitpid() first), but if you try it enough it will appear:
import asyncio import signal async def main(): while True: proc = await asyncio.create_subprocess_exec('sleep', '0.1') await asyncio.sleep(0.1) try: proc.send_signal(signal.SIGUSR1) except ProcessLookupError: pass assert (await proc.wait() != 255) asyncio.run(main())The output looks like:
Unknown child process pid 1394331, will report returncode 255 Traceback (most recent call last): File "/tmp/bob.py", line 14, in <module> asyncio.run(main()) File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete return future.result() File "/tmp/bob.py", line 12, in main assert (await proc.wait() != 255) AssertionErrorThis would be expected behaviour if I were explicitly calling waitpid() myself (ie I'm shooting my own foot, so I'd deserve the bad behaviour), but that's not the case here nor any other exotic code.
Did you find a workaround for this? I tried the different watcher classes
asyncioprovides using your example script, but all of them show this error after a whileThis one is new to me. I’m not sure how to approach it yet.
The only workaround I can think of without digging in the code would be to ignore the error using try/except, but that’s a last resort.@graingert Any idea here?
Oops, I see the problem isn't an exception so much as a lost exit status. I'm guessing we need to improve the communication between asyncio.subprocess.Process.wait() and the child watchers.
On my TODO list but low priority, I can review a PR fixing this though.
11 remaining items
- added a commit that references this issue
on Jul 1, 2024 This is a major issue for us, I am surprised that this issue is not getting more attention. It can be reproduced with the test case above on Python 3.9 and 3.10; please add corresponding labels. If I understand it correctly, the problem was introduced with https://bugs.python.org/issue38630 . Is it correct that the only workaround is to use os.kill instead of proc.terminate/proc.kill?
This is a major issue for us, I am surprised that this issue is not getting more attention. It can be reproduced with the test case above on Python 3.9 and 3.10; please add corresponding labels. If I understand it correctly, the problem was introduced with https://bugs.python.org/issue38630 . Is it correct that the only workaround is to use os.kill instead of proc.terminate/proc.kill?
Temper your rhetoric, please. Just upgrade to Python 3.13.
This is a major issue for us, I am surprised that this issue is not getting more attention. It can be reproduced with the test case above on Python 3.9 and 3.10; please add corresponding labels. If I understand it correctly, the problem was introduced with https://bugs.python.org/issue38630 . Is it correct that the only workaround is to use os.kill instead of proc.terminate/proc.kill?
Temper your rhetoric, please. Just upgrade to Python 3.13.
Easier said than done. In our case the situation is the same as for most users, I believe: Python is provided by the platform. In our case, Yocto Kirkstone, which includes Python 3.10. In theory we could upgrade to the next Yocto LTS release which is Scarthgap, but it only includes Python 3.12; not 3.13.
The workaround with os.kill seems to work, though, at least with SafeChildWatcher.
- added a commit that references this issue
on Jun 18, 2026 - added a commit that references this issue
on Sep 12, 2026
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
Linked PRs