warnaserror not failing the build with exit 1 if only errors present were warnings

[tf-jorge]

When there are no errors in a build, but only warnings that have been promoted to error with WarnAsError then the build will not fail with an exit of 1.

$ dotnet build --no-incremental /WarnAsError

...

Build FAILED.

....cs(28,1): error SA1507: Code must not contain multiple blank lines in a row [....csproj]
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:01.55
Exit code =>  0

Given that there are errors in the build, would expect an exit code of 1

Note that if there are genuine errors in the build, the build will fail correctly with the correct exit code

OS: Linux Ubuntu 18.04
dotnet version: 2.2.3-preview

[rainersigwald]

This appears to work in the trivial case:

<Project>
 <Target Name="Build">
  <Warning Text="F" />
 </Target>
</Project>

$ dotnet build /warnaserror     
Microsoft (R) Build Engine version 16.1.54-preview+gd004974104 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

S:\repro\dotnet\core-sdk\issues\1708\test.proj(3,3): error : F

Build FAILED.

S:\repro\dotnet\core-sdk\issues\1708\test.proj(3,3): error : F
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:00.35
$ $LASTEXITCODE
1

Can you tell us more about your repro? What kinds of projects do you have, and how many are in the solution? What target is generating the error?

[juddgaddie]

Thanks for your response. I will test your trivial case and look at the target and see if we experience the same problem. One obvious difference is we are running dotnet on Linux.

[juddgaddie]

4 projects in the solution

I just discovered that if I run the exact same build without making any other changes I get an intermittnet 1 exit code, here is my exit codes for running the same command and make no other changes. The output is always displayed correctly.
$ /dotnet build --no-incremental /WarnAsError
0
$ /dotnet build --no-incremental /WarnAsError
1
$ /dotnet build --no-incremental /WarnAsError
0
$ /dotnet build --no-incremental /WarnAsError
0
$ /dotnet build --no-incremental /WarnAsError
0
$ /dotnet build --no-incremental /WarnAsError
1
$ /dotnet build --no-incremental /WarnAsError
1
$ /dotnet build --no-incremental /WarnAsError
1
$ /dotnet build --no-incremental /WarnAsError
0

$ /dotnet build --no-incremental /WarnAsError

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <TargetFramework>netstandard2.0</TargetFramework>
        <CodeAnalysisRuleSet>Myproject.ruleset</CodeAnalysisRuleSet>
        <AssemblyName>My.Project</AssemblyName>
        <RootNamespace>My.Project</RootNamespace>
    </PropertyGroup>

    <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
      <DocumentationFile>docgen\docfile.xml</DocumentationFile>
      <TreatWarningsAsErrors>false</TreatWarningsAsErrors>
      <NoWarn>1701;1702;1591</NoWarn>
    </PropertyGroup>

    <ItemGroup>
        <PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.9.2">
            <PrivateAssets>all</PrivateAssets>
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
        </PackageReference>
        <PackageReference Include="StyleCop.Analyzers" Version="1.0.2">
          <PrivateAssets>all</PrivateAssets>
          <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
        </PackageReference>
        <AdditionalFiles Include="stylecop.json" />
    </ItemGroup>

    <ItemGroup>
      <ProjectReference Include="..\My.Project\My.Project.csproj" />
    </ItemGroup>

</Project>

[BTSmolders]

@juddgaddie @rainersigwald any news on this issue?
I'm experiencing the same issues (since last month) where the exitcode is not consistent when using dotnet build /WarnAsError and alternates between 0 and 1 on different runs without code changes.

This used to work in the past in a consistent way. Is there a workaround maybe?

[BTSmolders]

I researched various dotnet sdks en think the issue was introduced in the 2.2.2* series/ since msbuild 16.0.450+ga8dc7f1d34:
2.2.101 works; msbuild: 15.9.20+g88f5fadfbe
2.2.104 works; msbuild: 15.9.20+g88f5fadfbe
2.2.202 flaky; msbuild: 16.0.450+ga8dc7f1d34
2.2.203 flaky; msbuild: 16.0.450+ga8dc7f1d34
2.2.300 flaky; msbuild: 16.1.76+g14b0a930a7
2.2.301 flaky; msbuild: 16.1.76+g14b0a930a7
3.0-preview untested

Hopes this helps in tracking down the issue.

[juddgaddie]

Good to hear its been reproduced independently. I unfortunately haven't made any progress from above.

[BTSmolders]

In the meantime I tested 3.0-preview7 and it has the same flaky behaviour as previous versions.

@richlander I read your blog posts on 3.0-preview and was wondering if you (or dotnet/msbuild team) know about this issue, since I cannot find it on any backlog/roadmap.
Maybe you can mention someone that can look into this issue?

[TysonMN]

I too have a solution with only warnings. I am building with -warnAsError and using .NET Core SDK 3.0.100 on Azure DevOps.

• If I build with "Enable system diagnostics" checked, then the build fails, the exit code is 1, and the pipeline step fails.
• If I build without "Enable system diagnostics" checked, then the build fails, the exit code is not given in the logs, and the pipeline step passes.

I expected the pipeline step to fail with or without "Enable system diagnostics" checked.

Is this essentially the same problem that this issue is about or should I create a new issue?

[TysonMN]

Oh, maybe this is just the flaky behavior. I will retest later.

[TysonMN]

Indeed. Retesting has confirmed that I was just experiencing the previously described flaky behavior.

[TysonMN]

I researched various dotnet sdks en think the issue was introduced in the 2.2.2* series/ since msbuild 16.0.450+ga8dc7f1d34:
2.2.101 works; msbuild: 15.9.20+g88f5fadfbe
2.2.104 works; msbuild: 15.9.20+g88f5fadfbe
2.2.202 flaky; msbuild: 16.0.450+ga8dc7f1d34
2.2.203 flaky; msbuild: 16.0.450+ga8dc7f1d34
2.2.300 flaky; msbuild: 16.1.76+g14b0a930a7
2.2.301 flaky; msbuild: 16.1.76+g14b0a930a7
3.0-preview untested

Hopes this helps in tracking down the issue.

@BTSmolders identified 2.2.202 as the earliest SDK he tested that exhibited the bug, but there were more SDKs that could have been tested. I think I have tested all the relevant ones, and I would like to share what I learned.

• 2.1.602 exhibited the bug for me
• 2.1.506 worked for me
• 2.1.600-preview-009426 worked for me

The git history of tag 2.1.602 has no parallel branches at tag v2.1.500. The released SDKs between those two tags form two parallel branches: the "2.1.600-preview-009426 branch" and the "2.1.50X branch". The last 2.1.50X release before the release of 2.1.602 was 2.1.506.

In summary, I think the bug was introduced somewhere before 2.1.602 and after 2.1.600-preview-009426 and 2.1.506.

I tested all these SDKs on Azure DevOps using the Use .NET Core task.

[pleonex]

We have this issue all the time with .NET Core 3.1.200 building in Azure DevOps, which makes harder to review PR (you can't rely in the build result but actually go and check the build task). Should this issue be moved to the dotnet/sdk (before dotnet/cli) repository? Maybe there it can be tracked better and it doesn't look a problem of installation.

[rainersigwald]

@pleonex Can you share a repro of the problem?

Some of the behaviors described sound like #3046, but I'm not sure that describes everything.

[pleonex]

I was able to reproduce it here:

• Build pipeline
• Code

I am not sure if it happened by accident, but this repo started to show the behavior when the only warning started with CSC. In my company projects it's reproduced also when there are warnings without the CSC start (but it's in a on-premise installation of Azure DevOps).

[adhoc-am]

Hello,
I've the same issue with Azure DevOps and task dotnet build with -warnaserror arguments.
First build exit with code 1 and dotnet build task considered as failed, but if further re-run build without changes, inside task logs I see Build Failed, but task considered as success.
In my case with Azure DevOps seems work below solution:

• I separated dotnet restore and dotnet build tasks.

• Run dotnet build task with -warnaserror --no-restore arguments

after that dotnet build task considered as failed for first build attempt and all subsequent attempts.
But need more investigation and also not clear why it works like this.

also be sure that don't used sinarqube prepare task before build
https://docs.sonarqube.org/latest/analysis/scan/sonarscanner-for-msbuild/

The "begin" step will modify your build like this:

the active CodeAnalysisRuleSet will be updated to match the SonarQube quality profile
WarningsAsErrors will be turned off
If your build process cannot tolerate these changes we recommend creating a second build job for SonarQube analysis.