Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
e40ce6c
fix(profiles): XML and POM fact counters stay linear over unclosed ma…
bernardladenthin Nov 4, 2026
1869f79
feat(core): a changed prompt or fact file rewrites its summaries (reg…
bernardladenthin Nov 4, 2026
236bcd1
test(profiles): measure that every fact pattern reads linearly; fix t…
bernardladenthin Nov 4, 2026
16fba76
feat(core): fileNames, contentMatches, {a,b} globs and byte order marks
bernardladenthin Nov 4, 2026
b86771a
feat(profiles): enterprise kinds, shared OOP prompt, facts includes, …
bernardladenthin Nov 4, 2026
f5dc742
feat: route the enterprise kinds in the Luna example; measure them
bernardladenthin Nov 4, 2026
161974b
docs: every open point of the enterprise kinds in TODO.md; Gradle and…
bernardladenthin Nov 4, 2026
89601d3
feat(profiles): Java takes the shared prompt file-body-oop; the runs …
bernardladenthin Nov 4, 2026
a58faba
docs(todo): the branch is PR #269
bernardladenthin Nov 4, 2026
16d26a9
feat(core): scripts selected by their #! line, contentHeadChars, [raw…
bernardladenthin Nov 4, 2026
077be31
feat(profiles): counters for every kind of profile v3, DMN of its own…
bernardladenthin Nov 4, 2026
2fb0bdb
docs: the counters, markers, time limit and script selection; pathGlo…
bernardladenthin Nov 4, 2026
51fcd23
feat(prompts): the model reads a file's path; the OOP prompt names pr…
bernardladenthin Nov 4, 2026
f3ccec7
docs(profiles): java-ee-facts names its routing by a qualified annota…
bernardladenthin Nov 4, 2026
30f1498
docs(benchmark): the runs after the prompt changes; TODO keeps only w…
bernardladenthin Nov 4, 2026
fa86df9
fix: the review's findings on the branch
bernardladenthin Nov 4, 2026
98c6b77
feat!: regenerateOn, one switch per cause next to force; packages and…
bernardladenthin Nov 4, 2026
e9be510
docs: the findings of the reviews of PR #269 in TODO.md; one model di…
bernardladenthin Nov 4, 2026
ec97d6c
fix(indexer): the project's checksum folds its packages' checksums; t…
bernardladenthin Nov 4, 2026
4242a04
docs: Upgrading from 1.4.0 with the API breaks; how to keep old summa…
bernardladenthin Nov 4, 2026
5b53cf9
docs: examples need the next release, CONTRIBUTING as the build is, t…
bernardladenthin Nov 4, 2026
7c962c4
docs(core): the PIT scope is 83 classes
bernardladenthin Nov 4, 2026
b160aad
fix(selection): fileNames reject a directory everywhere, env's other …
bernardladenthin Nov 4, 2026
749ab05
fix(config): a JSON configuration with a byte order mark loads; a key…
bernardladenthin Nov 4, 2026
d038286
chore(benchmark): the agentic scripts take llama-server from LLAMA_SE…
bernardladenthin Nov 4, 2026
ddf16f2
fix(facts): an ignored region at the start of a line leaves nothing t…
bernardladenthin Nov 4, 2026
6d69d39
docs(todo): the reviews of 2026-11-04 keep only what waits for a deci…
bernardladenthin Nov 4, 2026
33f8e06
chore(spotbugs): AiConditionEvaluator prints its state, compares by i…
bernardladenthin Nov 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
4 changes: 2 additions & 2 deletions .github/plugin-it.sh
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@ reset
grep -q "AI index plan" "$LOGS/run4.log" || fail "the plan was not printed"
# The routing rule is matched BY ID here, which proves the condition tree routed rather than
# everything falling through to the fallback.
grep -q "| java | file-body-java |" "$LOGS/run4.log" \
grep -q "| java | file-body-oop |" "$LOGS/run4.log" \
|| fail "the .java routing rule did not match -- the condition tree did not route"
# If srcmorph.planOnly is ever renamed, the property stops binding, the run generates for real, and
# this is the assertion that catches it.
Expand Down Expand Up @@ -304,7 +304,7 @@ mvn -B --no-transfer-progress -f "$REPO_ROOT/srcmorph-maven-plugin/pom.xml" -P s
"net.ladenthin:srcmorph-maven-plugin:$VERSION:generate" -Dsrcmorph.planOnly=true \
> "$LOGS/run7-selftest.log" 2>&1 \
|| { cat "$LOGS/run7-selftest.log"; fail "the srcmorph-selftest profile did not plan"; }
grep -q "| java | file-body-java |" "$LOGS/run7-selftest.log" \
grep -q "| java | file-body-oop |" "$LOGS/run7-selftest.log" \
|| fail "the self-test profile routed no Java file to its java rule"
echo "ok: the self-test profile planned"

Expand Down
15 changes: 13 additions & 2 deletions .github/plugin-it/srcmorph.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,17 @@ srcMorph:
- "**/package-info.java"
aiVersion: "${it.aiVersion}"

# When a summary is rewritten. force: true rewrites every one. Otherwise a summary is rewritten when
# it is missing or empty, its aiVersion changed, or a change below whose switch is true (each one is
# by default); false keeps the summary and records the new value at its next rewrite.
force: false
regenerateOn:
source: true # C: the file's source, a package's files, the project's packages
prompt: true # P: the rule's prompts, its map and reduce prompts too
facts: true # N: the rule's fact counters
model: true # M: the rule's model (definition key, remoteModel, model file)
children: true # K: the summaries a package or the project is written from

aiDefinitions:
- key: fixture-model
modelPath: unused-with-the-mock-provider.gguf
Expand All @@ -44,7 +55,7 @@ srcMorph:
fieldGenerations:
# Extra.java: inline facts with an ignore counter, kept out of the facts line and out of the count.
- id: extra
promptKey: file-body-java
promptKey: file-body-oop
aiDefinitionKey: "${it.model}"
priority: 10
facts:
Expand All @@ -66,7 +77,7 @@ srcMorph:
- pathGlob: "**/Names.java"
- size: {max: 100000}
- id: java
promptKey: file-body-java
promptKey: file-body-oop
aiDefinitionKey: "${it.model}"
factsKey: java-facts
condition:
Expand Down
213 changes: 212 additions & 1 deletion CHANGELOG.md

Large diffs are not rendered by default.

16 changes: 11 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ Package root: `net.ladenthin.srcmorph`.
- **`config/`** — mutable JavaBeans (no Maven annotations) bound by Jackson from the one configuration
file of the CLI and the plugin, or built in plain Java code. The run's object is
**`SrcMorphConfiguration`**: one bean holding everything a run needs (`baseDirectory`,
`outputDirectory`, `subtrees`, `excludes`, `fileExtensions`, the size band, `force`, `planOnly`,
`outputDirectory`, `subtrees`, `excludes`, `fileExtensions`, the size band, `force`, `regenerateOn` (`AiRegenerateOn`: which changed header field rewrites a summary), `planOnly`,
`generationProvider`, `profileDirectory`, `promptDefinitions`, `aiDefinitions`, `fieldGenerations`, `factDefinitions`,
`pluginVersion`/`aiVersion`/`projectName`, `runRecordFile`: the JSON record of a file phase,
`engine.RunRecord`). The file around it is **`SrcMorphConfigurationFile`**
Expand Down Expand Up @@ -130,10 +130,15 @@ Package root: `net.ladenthin.srcmorph`.
The measured profile (prompts + fact counters) lives once in the repo-root `profiles/v3/`
(`profiles/README.md`); the plugin's self-test profile, the model-card benchmark, the examples and
the plugin-it fixture all name only keys and read it from there, so do not paste a prompt text or
a regex into a configuration again. `SqlFactPatternsTest`, `JavaStructureFactPatternsTest`,
`ShellFactPatternsTest`, `PerlFactPatternsTest` and `XmlFactPatternsTest` read the shipped `.tsv`
a regex into a configuration again. The `*FactPatternsTest` classes (`Sql`, `JavaStructure`,
`Shell`, `Perl`, `Xml`, `EnterpriseXml`, `Jvm`, `View`, `Contract`, `Ops`) read the shipped `.tsv`
files, never copies, and count through `AiFactExtractor.counts`, so the `[ignore]` lines (regions
blanked before counting: POD, heredocs, comments) apply as in production.
blanked before counting: POD, heredocs, comments) and the `[raw]` and `[distinct]` counters apply
as in production. `ProfileFactPatternsLinearityTest` runs every pattern of every `.tsv` of the
profile over repeated adversarial lines and counts the characters the matcher reads: four times
the input may read at most six times as many, so a quadratic pattern fails it on any machine, and
none may reach the time limit `AiFactExtractor` gives a pattern (`READS_PER_CHAR` per character
plus `READ_ALLOWANCE`). A new facts file is measured without a change to it.
- **`provider/`** — the AI backend abstraction: `AiGenerationProvider` (`Closeable`),
`AiGenerationProviderFactory` (looks up `"mock"` / `"llamacpp-jni"` / `"openai-compatible"`; a model
definition's own `provider` overrides the run's `generationProvider`), `MockAiGenerationProvider`,
Expand All @@ -151,6 +156,7 @@ Package root: `net.ladenthin.srcmorph`.
`AiTimeSupport`, `AiPathSupport`, `AiSourceExcludeFilter`, `AiProgressBar`, `AiSourceChunker`,
`AiSourceLines` (a source is read once with `\n` line endings: checksum, plan, prompt and chunks see
the same text),
`AiShebang` (the interpreter a `#!` line names: `fileInterpreters` and the `interpreters` condition),
`AiDeterministicSummary`, `AiGenerationTimeEstimator`, `Java8CompatibilityHelper`, `ConvertToRecord`,
`ReplaceWithLibrary` (marks a copy of a library class whose release is not out yet;
`ReplaceWithLibraryTest` fails once the library class is on the classpath, see `TODO.md`).
Expand All @@ -172,7 +178,7 @@ what the mapping unit tests and the `mock`-provider suite structurally cannot se
defects fixed in 1.2.0 were of exactly that shape. Its reflective completeness check fails when a knob
is neither swept nor listed in `NOT_SWEPT` with a reason, so a knob added later reds the class until
somebody decides how it is covered. **PIT mutation testing**: `mutationThreshold`
100 over an explicit `targetClasses` list in `srcmorph/pom.xml` — currently 78 entries (nested classes counted) across
100 over an explicit `targetClasses` list in `srcmorph/pom.xml` — currently 83 entries (nested classes counted) across
config/document/engine/indexer/prompt/provider/support, all killed at 100%. **All three modules are
PIT-gated now**: `srcmorph-cli` (24/24) and `srcmorph-maven-plugin` (80/80) carry their own
`pitest-maven` executions at the same threshold, and CI runs the goal reactor-wide. The natives come from `net.ladenthin:llama-platform` (a pom
Expand Down
13 changes: 7 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ Thank you for your interest in contributing to **srcmorph**, a prompt-driven sou

### Prerequisites

- Java 8+ (production code targets Java 8; CI builds on Java 8 via temurin)
- JDK 25 to build (the version in `.java-version`, which CI takes, Temurin); the production code of
`srcmorph` is compiled for Java 8 (`maven.compiler.release` 8), its tests for Java 21
- Maven 3.6.3+

### Common commands
Expand Down Expand Up @@ -86,13 +87,13 @@ Pull requests are expected to pass all CI checks (build, test, CodeQL) before me

## Coding standards

- **Java version:** Production code targets Java 8 (source/target = 1.8); do not use APIs unavailable on Java 8 in `src/main`.
- **Java version:** The production code of `srcmorph` is compiled with `--release 8`; do not use APIs unavailable on Java 8 in `src/main` (the compiler rejects them).
- **Encoding:** UTF-8 throughout.
- **License headers:** All `.java` files must include the Apache 2.0 license header wrapped in `// @formatter:off` / `// @formatter:on`. Copy the header from any existing source file.
- **License headers:** Every file carries an SPDX header (`SPDX-FileCopyrightText` and `SPDX-License-Identifier`), or an entry in `REUSE.toml` where it cannot hold one; `reuse lint` checks it. Copy the header from any existing source file.
- **Named constants:** Every meaningful string literal, header field name, and node type must be declared as a `public static final` or `private static final` constant with Javadoc.
- **Null safety:** Mark nullable parameters and return types explicitly; prefer early null/empty guards with a log warning over silent skips.
- **Records:** Use Java `record` types for immutable value carriers (note: records are a development-environment convenience; ensure source/target = 1.8 compatibility for emitted bytecode).
- **Logging:** Use `org.apache.maven.plugin.logging.Log` (not SLF4J), obtained via `getLog()` or constructor injection.
- **Value carriers:** Java 8 has no `record`; write a final class with final fields (Lombok's `@ToString` / `@EqualsAndHashCode` where the classes around it use them) and mark it `@ConvertToRecord` (`support.ConvertToRecord`) for the move to a later baseline.
- **Logging:** Use SLF4J (`org.slf4j.Logger`, one `private static final Logger LOGGER` per class) in the library and the CLI; the Maven plugin's Mojos log through Maven's `getLog()`.
- **Javadoc:** Use HTML entities for operators (`<`, `>`, `≤`, etc.); never use bare Unicode symbols in Javadoc.

Consult [CLAUDE.md](./CLAUDE.md) and [CODE_WRITING_GUIDE.md](./CODE_WRITING_GUIDE.md) at the root of the repository for the full set of conventions enforced on every production file.
Expand All @@ -103,7 +104,7 @@ Consult [CLAUDE.md](./CLAUDE.md) and [CODE_WRITING_GUIDE.md](./CODE_WRITING_GUID

> Every new feature or behavior change MUST include automated tests. Pull requests that add or change functionality without corresponding tests will be asked to add tests before merge. Bug fixes SHOULD include a regression test.

Tests are written with **JUnit 4** and **Hamcrest**. Use `MockAiGenerationProvider` for all tests that do not explicitly exercise the llama.cpp JNI backend. Tests that call the real JNI must guard with a JNI-availability check so the suite remains runnable on machines without the native library.
Tests are written with **JUnit 5 (Jupiter)** and **Hamcrest**. Use `MockAiGenerationProvider` (or a lambda `AiGenerationProvider`) for all tests that do not explicitly exercise the llama.cpp JNI backend. Tests that call the real JNI must guard with `NativeLlamaAvailability` (a JUnit assumption) so the suite remains runnable on machines without the native library.

See [TEST_WRITING_GUIDE.md](./TEST_WRITING_GUIDE.md) at the root of the repository for the full test conventions.

Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
[![Coverage Status](https://coveralls.io/repos/github/bernardladenthin/srcmorph/badge.svg?branch=main)](https://coveralls.io/github/bernardladenthin/srcmorph?branch=main)
[![codecov](https://codecov.io/gh/bernardladenthin/srcmorph/graph/badge.svg)](https://codecov.io/gh/bernardladenthin/srcmorph)
[![JaCoCo](https://img.shields.io/codecov/c/github/bernardladenthin/srcmorph?label=JaCoCo&logo=java)](https://codecov.io/gh/bernardladenthin/srcmorph)
[![PIT Mutation](https://img.shields.io/badge/PIT%20mutation-100%25%20(78%20classes)-brightgreen)](https://github.com/bernardladenthin/srcmorph/actions/workflows/publish.yml)
[![PIT Mutation](https://img.shields.io/badge/PIT%20mutation-100%25%20(83%20classes)-brightgreen)](https://github.com/bernardladenthin/srcmorph/actions/workflows/publish.yml)

**Quality:**
[![Quality Gate](https://sonarcloud.io/api/project_badges/measure?project=bernardladenthin_srcmorph&metric=alert_status)](https://sonarcloud.io/dashboard?id=bernardladenthin_srcmorph)
Expand Down Expand Up @@ -180,8 +180,10 @@ cd examples/build-your-own-jar
java -jar target/srcmorph.jar ../config_All.json
```

The examples follow this repository and need srcmorph 1.4.0 (for 1.3.0, take the examples of the
`v1.3.0` tag); against a reactor build, `mvn install` it and pass its version as `-Dsrcmorph.version`. Every `examples/config_*.json`/`.yaml` file works out of the box with the `mock` provider
The examples follow this repository: on `main` they need the next release (keys and facts markers
1.4.0 rejects); until it is out, `mvn install` the reactor and pass its version as
`-Dsrcmorph.version` (`1.5.0-SNAPSHOT`). For 1.4.0 or 1.3.0, take the examples of the `v1.4.0` or
`v1.3.0` tag. Every `examples/config_*.json`/`.yaml` file works out of the box with the `mock` provider
(no GGUF model required) — see [`examples/`](examples/) for the full set plus paired `run_*.sh`/`run_*.bat`
launcher scripts, and [`srcmorph-cli/README.md`](srcmorph-cli/README.md) for the config-file
reference. A walkthrough for a Windows laptop with Vulkan is in
Expand Down
2 changes: 2 additions & 0 deletions REUSE.toml
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,8 @@ path = [
"docs/ai-index-benchmark/remeasure-2026-11-02/*/*/search.tsv",
"docs/ai-index-benchmark/remeasure-2026-11-02/*/*.jsonl",
"docs/ai-index-benchmark/remeasure-2026-11-02/*/*/*/variant.json",
"docs/ai-index-benchmark/remeasure-2026-11-04/*/*/meta/*.json",
"docs/ai-index-benchmark/remeasure-2026-11-04b/*/*/meta/*.json",
]
precedence = "aggregate"
SPDX-FileCopyrightText = "2026 Bernard Ladenthin <bernard.ladenthin@gmail.com>"
Expand Down
38 changes: 33 additions & 5 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,32 @@ everything below is genuinely still open.

## Open

### Enterprise kinds of profile v3 (2026-11-04)

- **PR #269** (branch `feat/profiles-v3-kinds-and-safety`): CI green but the Claude review action;
the owner merges, then this line goes.
- The plugin's self-test profile routes only the kinds measured on this repository; the new ones are
routed in `examples/copilot-api-gateway/srcmorph-luna.yaml` and measured on the enterprise corpus
(`docs/ai-index-benchmark/remeasure-2026-11-04b/`), which is written for the benchmark. Once
measured on a corpus of real projects, the self-test can take them too.
- gpt-oss-20b named `ringbuf.c`'s headers instead of `malloc` and `memcpy` in two of six card runs
after the path line came into the user message (fallback prompt, none before): watch it in the
next round; with three more runs it is variation or a sentence for the fallback prompt.

### Reviews of 2026-11-04 (PR #269 at 98c6b77, the delta and the whole repository), open items

Two reviews, verified against the code; fixed on the branch except these, which wait for a decision:

- The read budget of fact and `contentMatches` patterns counts the matcher's `charAt` calls, which
may differ between JDK versions; `ProfileFactPatternsLinearityTest` runs on JDK 25 only (not
measured). Running it on the oldest supported JDK too would show it.
- In the plugin, `G` records the version of the user's project, not the plugin's
(`AbstractAiIndexMojo`); check whether that is meant.
- `${...}` in a configuration value has no escape, so a literal `${user.home}` in an inline prompt is
always replaced; the rate limiter keeps a window per server and limit, so two models on one server
with 20 and 10 requests a minute send up to 30 (its Javadoc says so; decide whether one window per
server is meant).

### After the release of 1.4.0

- `java-facts` `type declarations` counts private and protected nested types and `@interface`s since
Expand Down Expand Up @@ -79,9 +105,9 @@ Unreleased) except:

- The plan's chunk estimate cannot know a counted window (it loads no model), so for a local model its
ETA of a dense file stays an estimate.
- A prompt change is recorded (`P`, `N`), never acted on: an opt-in "rewrite what an older prompt
wrote" (compare `P`/`N` in `shouldWrite`) would make the hashes useful; `P` covers the map and
reduce prompts also for files that never used them.
- `P` covers the map and reduce prompts also for files that never used them, so with
`regenerateOn.prompt` (the default) a changed map or reduce prompt rewrites every file of the rule,
not only its chunked ones.
- The token count of a remote `llama-server` could use its `/tokenize`; OpenRouter has none.
- Benchmark: the copy filter misses `~~~` fences, indented code and near-copies; the invented-name
rate is taken inside backticks only and by substring; recall counts names, not faithfulness (a bare
Expand Down Expand Up @@ -164,8 +190,10 @@ Unreleased) except:

### Other open items

- **The Claude Code Review workflow fails on every pull request**: its `claude_code_oauth_token`
secret is empty. The owner sets it, or the workflow stays red.
- **The Claude Code Review workflow fails on every pull request**: it runs with the
`anthropic_api_key` secret (`claude_code_oauth_token` is empty) and the model call ends after
~300 ms with `is_error: true`, no turn, no cost: the key is refused or its account has no credit.
The owner renews the key or its credit, or the workflow stays red.

- **Use jllama's `ChatStreamAssembler` instead of srcmorph's own stream assembly.** The `stream`
setting of the `openai-compatible` provider needs a Server-Sent Events body assembled into a chat
Expand Down
2 changes: 1 addition & 1 deletion docs/ai-index-benchmark/HSMALL-VS-GPTOSS.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ mvn -o prepare-package -P ai-index-selftest -DskipTests \

Speed (exact prefill/decode tok/s) via a standalone calibrate POM parameterized by
`bench.model` / `bench.ctx` / `bench.ngl` / `bench.temp` / `bench.effort`, run with
`mvn ai-index:calibrate -f <pom>`; GPU adds `-Dbench.ngl=<N>
`mvn ai-index:calibrate -f <pom>` (the plugin of then, `llamacpp-ai-index-maven-plugin` 1.0.x; today the goal is `srcmorph:calibrate`); GPU adds `-Dbench.ngl=<N>
-Dllama.classifier=cuda13-windows-x86-64` with the CUDA toolkit on `PATH`. The `EXP-Granite-4.0-H-Small`
`<aiDefinition>` lives in the `ai-index-selftest` profile in `pom.xml`.

Expand Down
2 changes: 1 addition & 1 deletion docs/ai-index-benchmark/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ mvn prepare-package -P ai-index-selftest -DskipTests \
`EXP-*` model definitions, the `config`+`provider` subtree scope, and the
`package-info`/`module-info` excludes live in the self-test profile (`srcmorph-selftest` in
`srcmorph-maven-plugin/pom.xml` today, `ai-index-selftest` in the root `pom.xml` then, marked
`claude2026_06_26_00`); its v2 prompts are in the history of `profiles/` (commit `cd2e15f`). The model files are catalogued in `X:\Modelle\index.md`.
`claude2026_06_26_00`); its v2 prompts are in the history of `profiles/` (commit `cd2e15f`). The model files are catalogued in `M:\Modelle\index.md`.

> These are temporary experiment scaffolds on the `claude2026_06_26_00` branch, not production
> configuration.
Loading
Loading