soroban-rpc: Add systemd sd_notify support for debian package

[paulbellamy]

What problem does your feature solve?

sd_notify is a "new" system allowing systemd to inspect a service's status, and for the service to notify systemd when it is: ready, stopping, unhealthy, etc... And includes a watchdog system which can restart a service if it is unhealthy.

github.com/coreos/go-systemd/v22/daemon explains the protocol simply

sd_notify(3) man page

What would you like to see?

If sd_notify is enabled, soroban-rpc should emit READY=1 when ready, STOPPING=1 when stopping, and any STATUS=... which seem relevant.

If the systemd watchdog is enabled, it should emit WATCHDOG=1 at the right frequency.

Once this is all enabled, we should update the soroban-rpc systemd service in the debian package to use Type=notify, to enable these, and configure the watchdog frequency.

We should ensure the startup timeouts are long enough.

Question: Should we return READY=1 when soroban-rpc is running, or when it is first healthy?

What alternatives are there?

• Don't do it, and keep systemd integration more rudimentary. Encourage infra to deploy in k8s/docker, and do healthchecks there.
• Use a different service for watchdog healthchecks (e.g. monit).

[sreuland]

wanted to mention that the debian package is only published for amd64 platform arch, so it's usage would be limited to only clients that are on hosts with that platform and want to do this type of bare-metal install.