Skip to content

Go SDK: Shut down Edge Workers gracefully on SIGTERM - #69923

Closed
viiccwen wants to merge 1 commit into
apache:mainfrom
viiccwen:fix-go-sdk-sigterm-shutdown
Closed

viiccwen wants to merge 1 commit into
apache:mainfrom
viiccwen:fix-go-sdk-sigterm-shutdown

Conversation

@viiccwen

@viiccwen viiccwen commented Jul 15, 2026 •

Copy link
Copy Markdown
Contributor

Handle SIGTERM through the same graceful drain path as the first interrupt received by a Go SDK Edge Worker.

The worker registered SIGTERM with signal.Notify, which replaced the signal's default termination behavior, but its signal switch did not handle SIGTERM. Container runtimes and service managers could therefore leave the worker running until escalating to a forced kill, bypassing task draining and normal lifecycle cleanup.

Separate OS signal registration from the core worker loop so tests can inject a signal channel without signaling the test process. Stop signal delivery when the wrapper exits, and route both SIGINT and SIGTERM through the existing drain behavior.

Add a regression test that injects SIGTERM and uses a test Edge API server to verify the worker reports an idle heartbeat followed by offline as it drains. The test also verifies the path repeatedly without installing process-wide signal handlers.


Was generative AI tooling used to co-author this PR?
  • Yes - GPT-5.6 Sol

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@viiccwen
viiccwen force-pushed the fix-go-sdk-sigterm-shutdown branch 2 times, most recently from 7de54c1 to c64e58a Compare July 16, 2026 11:25
Container runtimes and service managers use SIGTERM for normal shutdown. Ignoring it or waiting for an Edge API heartbeat can exhaust the shutdown grace period, bypass task draining, and prevent a second termination signal from taking effect.

Signed-off-by: viiccwen <vicwen@apache.org>
@viiccwen
viiccwen force-pushed the fix-go-sdk-sigterm-shutdown branch from c64e58a to 0bff3ec Compare July 16, 2026 11:35

@jason810496 jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix.

Comment thread go-sdk/edge/worker.go
@viiccwen

viiccwen commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

@jason810496 politely tag for further review.

@jason810496 jason810496 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, as #71874 was merged. We don't need this PR any more. Thanks for your contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants