Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/workflows/Harnesses.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,13 @@ on:
- master
paths:
- 'Sources/AngouriMath/**'
- 'Sources/Wrappers/AngouriMath.FSharp/**'
- 'Sources/Tests/Harnesses/**'
- '.github/workflows/Harnesses.yml'
pull_request:
paths:
- 'Sources/AngouriMath/**'
- 'Sources/Wrappers/AngouriMath.FSharp/**'
- 'Sources/Tests/Harnesses/**'
- '.github/workflows/Harnesses.yml'
workflow_dispatch:
Expand Down Expand Up @@ -116,3 +118,38 @@ jobs:
with:
name: crashcheck-report
path: harness-reports

# A job of its own: it checks the website out beside the library, clones the wiki, and builds a
# project per language from their samples.
DocSamples:
runs-on: ubuntu-latest
env:
HARNESS_REPORTS: ${{ github.workspace }}/harness-reports

steps:
- uses: actions/checkout@v6

- name: Check out the website
uses: actions/checkout@v6
with:
repository: asc-community/AngouriMathSite
path: site

- name: Setup .NET 10
uses: actions/setup-dotnet@v5
with:
dotnet-version: '10.x'
dotnet-quality: 'preview'

- name: Build
run: dotnet build -c Release Sources/Tests/Harnesses/DocSamples

- name: DocSamples
run: dotnet Sources/Tests/Harnesses/DocSamples/bin/Release/net10.0/docsamples.dll --site=site

- name: Report
if: always()
uses: actions/upload-artifact@v6
with:
name: docsamples-report
path: harness-reports
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -77,5 +77,10 @@ docsamples.md
sympyparity.md
egraph.md

# DocSamples clones the wiki beside itself and generates a project per language there.
Sources/Tests/Harnesses/DocSamples/wiki/
Sources/Tests/Harnesses/DocSamples/generated/
Sources/Tests/Harnesses/DocSamples/generated-fsharp/

# Agent worktrees and local tool state.
.claude/
10 changes: 5 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@ nothing else is read:
[`Contributing/SimplificationContract.md`](Sources/AngouriMath/Docs/Contributing/SimplificationContract.md).
A rule states the assumptions under which it holds, or it is asserting there are none.

Nine measurement harnesses live in `Sources/Tests/Harnesses` and run in CI on every change to the
Ten measurement harnesses live in `Sources/Tests/Harnesses` and run in CI on every change to the
library: the boundary checker, root completeness, the self-verifying solver corpus, the property
checker, the simplification sweep, the rule-set checker, a crash harness that survives a stack
overflow, the canonical-form checker and the arm-order checker. Each fails on a defect, or the last
two on a change to the findings they list; the README says which. The rest are still in the analysis workspace
one directory up (`work/`), among them a checker for the documentation's code samples. Run them before claiming anything is
fixed.
overflow, the canonical-form checker, the arm-order checker, and a checker for the code samples in
the wiki and on the website. Each fails on a defect, or the canonical-form and arm-order checkers on
a change to the findings they list; the README says which. The rest are still in the analysis
workspace one directory up (`work/`). Run them before claiming anything is fixed.

There is also a *gate*, which is not a harness:
`Sources/Tests/UnitTests/Corpus` runs forty problems on every commit and reports **solved / unsolved
Expand Down
33 changes: 33 additions & 0 deletions Sources/Tests/Harnesses/DocSamples/DocSamples.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
<Project Sdk="Microsoft.NET.Sdk">

<!--
Does every code sample in the wiki and on the website compile, run, and print what its page
says. A harness, not a test project: it is run by .github/workflows/Harnesses.yml and by hand,
and `dotnet test` does not pick it up. See ../README.md.

It references no library: it generates a project per language beside it, under generated/ and
generated-fsharp/, which reference Sources/AngouriMath and the F# wrapper, and builds those.
Default compile items are off so that neither those nor the wiki clone are compiled into it.
-->

<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0</TargetFramework>
<Nullable>disable</Nullable>
<LangVersion>preview</LangVersion>
<AssemblyName>docsamples</AssemblyName>
<RootNamespace>DocSamples</RootNamespace>
<InvariantGlobalization>true</InvariantGlobalization>
<IsPackable>false</IsPackable>
<EnableDefaultCompileItems>false</EnableDefaultCompileItems>
</PropertyGroup>

<ItemGroup>
<Compile Include="Program.cs" />
<Compile Include="Snippets.cs" />
<Compile Include="Generator.cs" />
<Compile Include="FSharpGenerator.cs" />
<Compile Include="Report.cs" />
<Compile Include="../Shared/*.cs" />
</ItemGroup>
</Project>
113 changes: 113 additions & 0 deletions Sources/Tests/Harnesses/DocSamples/FSharpGenerator.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
//
// Copyright (c) 2019-2026 Angouri.
// AngouriMath is licensed under MIT.
// Details: https://github.com/asc-community/AngouriMath/blob/master/LICENSE.md.
// Website: https://am.angouri.org.
//

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;

namespace DocSamples;

/// <summary>
/// The F# side of the same check. `AngouriMath.FSharp` is a published package, and the wiki
/// page for it is the only documentation of the wrapper's names, so a wrong name there --
/// `dy/dx` for `d/dx` -- has nothing else to catch it.
///
/// One file, one module per sample, so each sample keeps its own `open` directives. F#
/// wants `open` before anything else in a module, so they are hoisted out of the body.
/// </summary>
static class FSharpGenerator
{
public static Dictionary<string, int> Write(string dir, string wrapperFsproj,
IEnumerable<Snippet> snippets, ISet<string> exclude)
{
var wanted = snippets.Where(s => s.Mode != Mode.Skip && !exclude.Contains(s.Id)).ToList();
Directory.CreateDirectory(dir);

var sb = new StringBuilder();
var offsets = new Dictionary<string, int>();
sb.AppendLine("module DocSamples.Generated.FSharpSamples");
sb.AppendLine();

foreach (var s in wanted)
{
var opens = s.Body.Where(l => l.TrimStart().StartsWith("open ")).ToList();
sb.AppendLine($"module {s.Id} =");
foreach (var o in opens) sb.AppendLine(" " + o.Trim());
sb.AppendLine(" let run () =");
offsets[s.Id] = sb.ToString().Count(c => c == '\n') + 1;
var wrote = false;
foreach (var line in s.Body)
{
if (opens.Contains(line)) { sb.AppendLine(); continue; }
sb.AppendLine(line.Trim().Length == 0 ? "" : " " + line);
if (line.Trim().Length > 0) wrote = true;
}
if (!wrote) sb.AppendLine(" ()");
sb.AppendLine();
}

sb.AppendLine("module Runner =");
sb.AppendLine(" open System");
sb.AppendLine(" open System.IO");
sb.AppendLine(" open System.Text.Json");
sb.AppendLine();
sb.AppendLine(" type Result = { Id: string; Status: string; Output: string }");
sb.AppendLine();
sb.AppendLine(" let samples : (string * (unit -> unit) * bool) list =");
if (wanted.Count == 0) sb.AppendLine(" []");
else
sb.AppendLine(" [ " + string.Join("\n ",
wanted.Select(s => $"(\"{s.Id}\", {s.Id}.run, {(s.Mode == Mode.Run ? "true" : "false")})"))
+ " ]");
sb.AppendLine("""

[<EntryPoint>]
let main argv =
let real = Console.Out
let results =
samples
|> List.map (fun (id, run, execute) ->
if not execute then { Id = id; Status = "compiled"; Output = "" }
else
let captured = new StringWriter()
Console.SetOut(captured)
let status =
try
run ()
"ok"
with e -> "threw " + e.GetType().Name + ": " + e.Message.Split('\n').[0]
Console.SetOut(real)
{ Id = id; Status = status; Output = captured.ToString() })
let path = if argv.Length > 0 then argv.[0] else "results.json"
File.WriteAllText(path, JsonSerializer.Serialize(results))
0
""");

File.WriteAllText(Path.Combine(dir, "Samples.fs"), sb.ToString());
File.WriteAllText(Path.Combine(dir, "generated-fsharp.fsproj"), $"""
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0</TargetFramework>
<AssemblyName>generatedfsharp</AssemblyName>
<TreatWarningsAsErrors>false</TreatWarningsAsErrors>
<WarningLevel>0</WarningLevel>
<NoWarn>FS0025;FS0049;FS0064;FS0193;FS1182</NoWarn>
</PropertyGroup>
<ItemGroup>
<Compile Include="Samples.fs" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="{wrapperFsproj}" />
</ItemGroup>
</Project>
""");
return offsets;
}
}
Loading
Loading