Release the binary log file handle when a logger fails during shutdown - #14658
Draft
JanProvaznik wants to merge 1 commit into
Draft
JanProvaznik wants to merge 1 commit into
JanProvaznik wants to merge 1 commit into
Conversation
The binary log file handle is released by ProjectCollection.Dispose, but only once BuildManager.EndBuild has shut its logging service down: that shutdown is what detaches the build-time event source from the ReusableLogger wrapping each logger. ReusableLogger.Shutdown infers which of its two lifetimes is ending from its own state, so if the build-time detach never happens, the single remaining Shutdown call takes the build-time branch and silently never shuts down the wrapped logger. In EndBuild, ShutdownLoggingService was the last statement of an unguarded block that first raises BuildFinished and then gathers telemetry. A logger throwing from BuildFinished - which the engine otherwise tolerates everywhere - skipped it, so the BinaryLogger was never shut down and kept its file handle for the lifetime of the process. In msbuild.exe that was harmless because the process exits. It is not harmless in an MSBuild Server node, which is long-lived, reused across builds, and (since -mt implies the server) now used by ordinary command lines. The leaked handle wedges the log file permanently: every subsequent build the node serves fails with MSB4104 "Failed to write to log file ... because it is being used by another process", the file cannot be deleted, and the leaked binlog is left truncated. Nothing recovers it short of killing the server, which users do not know exists. Run ShutdownLoggingService from a finally so it cannot be skipped, and close the BinaryLogger's stream from a finally as well so the handle is released even if the import-archive or additional-copy work throws. The regression test deliberately does not dispose the BuildManager, mirroring MSBuild Server: XMake skips BuildManager.DefaultBuildManager.Dispose for server nodes, and disposing it shuts the logging service down as a side effect, which hides the leak. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9ce22e5a-3ad9-490d-943b-cd5c7442ab31
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
A user-visible symptom in MSBuild Server: after certain builds, the node keeps the
.binlogfile open forever. Every subsequent build served by that node fails withthe file cannot be deleted, and the leaked binlog is left truncated (gzip header only, unreadable). Nothing recovers it short of killing the server — which users don't know exists.
This matters much more than it used to:
-mtnow implicitly enables MSBuild Server (ShouldUseMSBuildServer), so ordinary command lines land in a long-lived, reused node.Root cause
The binlog handle is not build-scoped. It is released by
ProjectCollection.Dispose(), and only ifBuildManager.EndBuild()ran first.ProjectCollectionwraps every logger inReusableLoggerso oneILoggerinstance can serve both the design-time (evaluation) and build-time lifetimes. BecauseILogger.Shutdown()carries no identity,ReusableLogger.Shutdown()infers which lifetime is ending from its own state:Balanced, ordered calls → correct. One missing call → silently wrong, permanently, with no way to detect it.
In
EndBuild,ShutdownLoggingService— the call that performs the build-time detach — was the last statement of an unguarded block that first raisesBuildFinishedand then gathers telemetry:A logger throwing from
BuildFinished— something the engine tolerates everywhere else — skips it. TheBinaryLoggeris then never shut down and keeps its file handle for the lifetime of the process.In
msbuild.exethis was harmless: the process exits. In a resident server node it is not.Fix
Minimal and targeted:
BuildManager.EndBuild— runShutdownLoggingServicefrom afinallyso it cannot be skipped. This restores the invariant the rest of the file already relies on; the same "on failure, shut the logging service down" compensation already exists in three other places (InitializeLoggingService's innercatch, the_threadExceptioncheck, andCreateLoggingService).BinaryLogger.Shutdown— close the stream from afinally, so the handle is released even if the import-archive or additional-copy work throws. Same fail-open shape, contained.No public API change; the diff in
BuildManager.csis mostly re-indentation.Verification
Regression test
BinaryLoggerFileHandleIsReleasedWhenAnotherLoggerThrowsDuringShutdown:main—The log file '...binlog' is still held open after the build that created it completedThe test deliberately does not dispose the
BuildManager, mirroring MSBuild Server: XMake skipsBuildManager.DefaultBuildManager.Dispose()for server nodes, and disposing it shuts the logging service down as a side effect, which hides the leak. It also usesUseSynchronousLogging = trueto match the command line, which is what makes the failing logger surface on the thread runningEndBuild.End-to-end against a real server node with a locally built MSBuild:
Unable to read beyond the end of the stream-blpathMSB4104foreverAlso ran the
BinaryLogger,BuildManager_Tests,ProjectCollection, and logging suites (437 tests). Remaining failures are pre-existing and reproduce unchanged on a clean tree — 30sExecMSBuildprocess-spawn timeouts in this environment, plus twoGraphBuild*ArgumentNullExceptionfailures.Not addressed here (deliberately)
The deeper issue is the ownership inversion: for a build invocation, logger lifetime should be
BeginBuild..EndBuild, but the binlog handle is owned by theProjectCollection. For a regular CLI build that ownership is incidental —BuildParameters(ProjectCollection)doesn't copy loggers, and XMake round-trips them viaparameters.Loggers = projectCollection.Loggersmainly to obtain theReusableLoggerwrappers. Removing that inversion (or givingReusableLoggerexplicit owner/borrower semantics instead of inference) is a behavior-visible refactor and is out of scope for this fix.Two adjacent issues also found while investigating, not fixed here:
MSB4104doesn't fail the build — the binlog failure above still exits 0.OutOfProcServerNodereturns fromRun()onLinkStatus.Failedwithout waiting for the in-flight build.