Skip to content

Task SDK: importable entry point for callback subprocesses so they can use fork+exec #73143

Description

@gang-zh

Body

#72164 lets the task process opt into fork+exec on Linux via [core] execute_tasks_new_python_interpreter, which closes the inherited-OpenSSL-lock hang in #71707 for tasks. Callbacks are still bare-forked: CallbackSubprocess.start passes a closure target, which WatchedSubprocess.start rejects under use_exec, so an on_failure_callback (or any other task callback) that makes a TLS call keeps the fork-inherited lock exposure after an operator flips the option.

Proposal: give callback subprocesses a named, importable target (the way tasks use _subprocess_main) so the same option covers them, and drop the "callbacks keep bare fork" caveat from the config text and the #72164 newsfragment once it lands.

Raised by @kaxil in #72164 (comment).

Are you willing to submit a PR?

Yes

Code of Conduct

  • I agree to follow this project's Code of Conduct
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions