Keep forked processes from resending the parent's log lines, and add Logger#flush

[PetrHeinz]

After a fork, the HTTP device keeps the queues it inherited and only restarts its dead threads (write → ensure_flush_threads_are_started), so every child sends the lines the parent buffered before forking, and the parent sends them as well. The red-team saw each pre-fork line arrive 3 times with Puma preload_app! and 2 workers, and 5 times with 4; logger.reopen doesn't help. A child that ends with exit!, as Resque's job processes do, skips the at_exit hooks and loses its lines, and there was no Logger#flush to call first. An idle worker that exits with the parent's lines in its queue waited 20 seconds for an outlet thread it never started. HTTP#flush waited those 20 seconds with flush_continuously: false too, and delivered nothing.

• The device remembers the pid that owns its queues and threads. When another process writes, flushes or closes it, it starts over in that process: empty queues, no threads, and the threads start when the child first logs. It's a plain Process.pid check, so it works on Ruby 2.5 and needs no Process._fork hook. A lock keeps two threads of a new child from resetting it twice.
• Logtail::Logger#flush flushes the logger's devices, extra loggers included. On the HTTP device it waits until the buffered lines are delivered, about 5 seconds at most (it was 20).
• When no outlet thread runs, with flush_continuously: false or in a child that hasn't logged yet, flush delivers in the calling thread, with 5-second connect and read timeouts.

Behaviour and compatibility:

• Rails calls Rails.logger.flush after every request (ActiveSupport::LogSubscriber.flush_all!, in every Rails from 5.0 to 8.1). Waiting for delivery there would hold up every request, and on Rails 7.0 and older it would delay the response itself, so Logger#flush returns right away when flush_all! calls it. That is what the call did before, when the logger had no flush (ActiveSupport::TaggedLogging#flush only calls super when there is one). The background thread delivers those lines as usual. Checked with ActiveSupport 5.0, 7.1 and 8.1, through TaggedLogging and BroadcastLogger.
• flush and close wait about 5 seconds for the outlet thread instead of 20, so a backlog that takes longer to deliver at exit is cut off sooner.
• A synchronous delivery counts any HTTP response as delivered, like the outlet does today. Its errors only go to the Logtail::Config debug log, also those outside StandardError (WebMock refuses connections with one, so a test suite with WebMock and a real HTTP device would otherwise fail at exit); signals such as Interrupt are raised as usual.
• The process tests start Ruby subprocesses with RbConfig.ruby -Ilib against a local TCP server on an ephemeral port. The fork tests skip where fork isn't available, TruffleRuby included.

Locally, with the red-team's fork.rb (50 lines before forking 3 children that log 100 lines each, then 100 more in the parent) against a local fake ingest: main delivered 600 rows, the 50 pre-fork lines 4 times each, and this branch 450 rows, each once, on Ruby 2.7, 3.0 and 3.4, with and without reopen. An idle child took 20.1 s to exit on main and no measurable time here.

#58 (fast, quiet shutdown) also changes http.rb. Both PRs add the same deliver_synchronously(requests) helper, SYNCHRONOUS_DELIVERY_TIMEOUT and spec/support/processes.rb, and otherwise touch different lines, so they merge cleanly in either order; a local merge of both branches passes the suite on Ruby 2.7, 3.0, 3.2 and 3.4.

Follow-up after the E2E run of the release candidate (two more commits, the first one only a test, red on the Ruby 2.5 to 2.7 jobs): when Net::HTTP can't start the thread that times out connecting, as while Ruby shuts down, deliver_synchronously reads @last_resp before anything set it, and Ruby 2.7 and older warn about that with -w. It's set in initialize now, on the same line as in #58, so the two still merge cleanly.

Follow-up (e9b9c7a): spec/logtail/logger_flush_all_spec.rb runs ActiveSupport's real LogSubscriber.flush_all! in a subprocess. It covers a Logtail::Logger on its own, in TaggedLogging and, from ActiveSupport 7.1, in a BroadcastLogger, against a local host that holds its answers. flush_all! has to return within a second without delivering, where waiting would take 5 seconds, and a direct flush of the same logger has to deliver. An ActiveSupport that moves or renames the call now fails CI instead of holding up every request; with the shortcut disabled, every example fails. CI resolves ActiveSupport 6.1 on Ruby 2.5 and 2.6, where the BroadcastLogger examples are pending, 7.1 on 2.7 and 3.0, 7.2 on 3.1 and 8.1 on the others, TruffleRuby included. The Gemfile gets the same activesupport lines as #53, so the two merge cleanly.

Follow-up (3b995ad test, red on every job, then 970f2c3): synchronous deliveries now wait up to 5 seconds to connect and 5 seconds for the response (SYNCHRONOUS_DELIVERY_TIMEOUT, previously 2), Petr's choice, on the same line as in #58. The test has flush deliver two requests in the calling thread to a host that answers after 3 seconds; with 2 seconds the first one timed out and the second was dropped.

Targets the logtail 0.1.21 patch release. No dependencies on the other open PRs.

The first commit only adds the tests and is expected to fail on CI. The next one fixes how one of those tests reads WebMock requests, then come the fix, a commit that passes the requests to the helper shared with #58, and a test for the idle child, which fails against main (20.07 s). The next two add a failing test and the fix for an error outside StandardError escaping a synchronous delivery, and the follow-ups above come last.

🤖 Generated with Claude Code