Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions .github/workflows/Harnesses.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: Harnesses

# Generated inputs run against the library on every change to it. Each harness exits non-zero
# on a defect -- a value Simplify changed at a boundary, a root missing from a solution set, a
# wrong answer to a known problem -- and its report is uploaded either way. Sources/Tests/Harnesses/README.md
# says what each asks. https://github.com/asc-community/AngouriMath/issues/1256
#
# Path filters are globs relative to the repository root, with no leading "./": CSharpTest.yml
# says what a leading "./" once cost.
on:
push:
branches:
- master
paths:
- 'Sources/AngouriMath/**'
- 'Sources/Tests/Harnesses/**'
- '.github/workflows/Harnesses.yml'
pull_request:
paths:
- 'Sources/AngouriMath/**'
- 'Sources/Tests/Harnesses/**'
- '.github/workflows/Harnesses.yml'
workflow_dispatch:

jobs:
Harnesses:
runs-on: ubuntu-latest
env:
HARNESS_REPORTS: ${{ github.workspace }}/harness-reports

steps:
- uses: actions/checkout@v6

- name: Setup .NET 10
uses: actions/setup-dotnet@v5
with:
dotnet-version: '10.x'
dotnet-quality: 'preview'

- name: Build
run: |
dotnet build -c Release Sources/Tests/Harnesses/BoundCheck
dotnet build -c Release Sources/Tests/Harnesses/RootCheck
dotnet build -c Release Sources/Tests/Harnesses/CasBench

# One step each, every one run whatever the one before did, so a red run names the harness.
- name: BoundCheck
run: dotnet Sources/Tests/Harnesses/BoundCheck/bin/Release/net10.0/boundcheck.dll

- name: RootCheck
if: success() || failure()
run: dotnet Sources/Tests/Harnesses/RootCheck/bin/Release/net10.0/rootcheck.dll

- name: CasBench
if: success() || failure()
run: dotnet Sources/Tests/Harnesses/CasBench/bin/Release/net10.0/casbench.dll

- name: Reports
if: always()
uses: actions/upload-artifact@v6
with:
name: harness-reports
path: harness-reports
12 changes: 7 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,14 @@ nothing else is read:
[`Contributing/SimplificationContract.md`](Sources/AngouriMath/Docs/Contributing/SimplificationContract.md).
A rule states the assumptions under which it holds, or it is asserting there are none.

The measurement harnesses live outside this repository, in the analysis workspace one directory up
(`work/`): a self-verifying solver corpus, a property checker, root-completeness and
simplification sweeps, a boundary checker, a crash harness that survives a stack overflow, and a
checker for the documentation's code samples. Run them before claiming anything is fixed.
Three measurement harnesses live in `Sources/Tests/Harnesses` and run in CI on every change to the
library: the boundary checker, root completeness and the self-verifying solver corpus. Each fails on
a defect; its README says which. The rest are still in the analysis workspace one directory up
(`work/`): a property checker, simplification sweeps, a crash harness that survives a stack
overflow, and a checker for the documentation's code samples. Run them before claiming anything is
fixed.

Inside the repository there is one measurement, and it is a *gate* rather than a harness:
There is also a *gate*, which is not a harness:
`Sources/Tests/UnitTests/Corpus` runs forty problems on every commit and reports **solved / unsolved
/ wrong / error / timeout**. It fails on a wrong answer, and on any case that stops matching its
recorded verdict — including one that improves, which is a prompt to record the improvement rather
Expand Down
4 changes: 4 additions & 0 deletions Sources/Tests/Harnesses/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# The reports the harnesses write when run by hand. CI uploads them instead.
boundcheck.md
rootcheck.md
coverage.md
28 changes: 28 additions & 0 deletions Sources/Tests/Harnesses/BoundCheck/BoundCheck.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
<Project Sdk="Microsoft.NET.Sdk">

<!--
Does Simplify keep the value at the boundary: across a branch cut, off the real line,
outside a principal interval, at a pole.
A harness, not a test project: it is run by .github/workflows/Harnesses.yml and by hand,
and `dotnet test` does not pick it up. See ../README.md.
-->

<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0</TargetFramework>
<Nullable>disable</Nullable>
<LangVersion>preview</LangVersion>
<AssemblyName>boundcheck</AssemblyName>
<RootNamespace>BoundCheck</RootNamespace>
<InvariantGlobalization>true</InvariantGlobalization>
<IsPackable>false</IsPackable>
</PropertyGroup>

<ItemGroup>
<Compile Include="../Shared/*.cs" />
</ItemGroup>

<ItemGroup>
<ProjectReference Include="../../../AngouriMath/AngouriMath.csproj" />
</ItemGroup>
</Project>
Loading
Loading