Add an openai-compatible provider, tested against NativeServer, llama-server and OpenRouter

.github/workflows/publish.yml

@@ -196,6 +196,28 @@ jobs:
           cache: maven
       - name: Memory before tests
         run: free -h
+      # A real llama.cpp llama-server for OpenAiCompatibleLiveTest, serving the committed SmolLM2
+      # GGUF: the openai-compatible provider's HTTP path against a real server on every run, with no
+      # secret and no cost. The stub-server test pins the request; this one proves a real server
+      # accepts it. Pinned by tag and digest (b11946, the build it was first verified against).
+      # Dependabot does not move a pin inside a `docker run`, so bump it by hand with llama.cpp.
+      - name: Start llama-server for the openai-compatible live test
+        run: |
+          set -euo pipefail
+          docker run -d --name llama-server -p 127.0.0.1:8080:8080 \
+            -v "$GITHUB_WORKSPACE/srcmorph/src/test/resources:/models:ro" \
+            ghcr.io/ggml-org/llama.cpp:server-b11946@sha256:d880b2a0e478795ce99d0a275732f5a50e579cb234ae7d2d3815afc909fd02f2 \
+            -m /models/SmolLM2-135M-Instruct-Q3_K_M.gguf --host 0.0.0.0 --port 8080 -c 8192 -t 2
+          for _ in $(seq 1 60); do
+            if curl -fsS http://127.0.0.1:8080/health >/dev/null 2>&1; then
+              echo "llama-server is ready"
+              exit 0
+            fi
+            sleep 2
+          done
+          docker logs llama-server
+          echo "::error::llama-server did not become ready within 120 s"
+          exit 1
       - name: Test
         # Reactor-wide (no -pl): runs every module's tests. The `jcstress` profile is only
         # declared in srcmorph-maven-plugin/pom.xml, so it activates there and is a
@@ -205,9 +227,41 @@ jobs:
         # failure. Skipping is right on a developer machine with an unsupported OS/arch; here it
         # would mean a vanished native library or an unchecked-out model quietly restores a green
         # build with zero real inference — the exact shape that let a dead provider ship twice.
+        #
+        # OpenAiCompatibleLiveTest runs the openai-compatible provider against every API it supports,
+        # each in addition to the others: jllama's NativeServer (started by the test itself),
+        # the llama-server started above (openai.baseUrl), and OpenRouter (openai.openRouterModel,
+        # a cheap model, with the repository secret). A pull request from a fork or from Dependabot
+        # gets no secret, so there the OpenRouter case skips; the check below allows exactly that.
+        env:
+          OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
         run: >-
           mvn -e --batch-mode --no-transfer-progress -P jcstress
-          -Dnet.ladenthin.srcmorph.requireNativeLlama=true verify
+          -Dnet.ladenthin.srcmorph.requireNativeLlama=true
+          -Dnet.ladenthin.srcmorph.openai.baseUrl=http://127.0.0.1:8080/v1
+          -Dnet.ladenthin.srcmorph.openai.openRouterModel=openai/gpt-6-luna verify
+      # Each live case skips when its server is not configured, so a property that stops reaching the
+      # test JVM would turn it into a silent skip. Fail instead: all three must run, OpenRouter only
+      # when the secret is available to this run.
+      - name: Check that the openai-compatible live tests ran
+        env:
+          OPENROUTER_KEY_SET: ${{ secrets.OPENROUTER_API_KEY != '' }}
+        run: |
+          set -euo pipefail
+          report=srcmorph/target/surefire-reports/TEST-net.ladenthin.srcmorph.provider.OpenAiCompatibleLiveTest.xml
+          expected_skipped=0
+          if [ "$OPENROUTER_KEY_SET" != "true" ]; then
+            expected_skipped=1
+            echo "::warning::No OPENROUTER_API_KEY in this run (fork or Dependabot); the OpenRouter case skipped"
+          fi
+          if ! grep -q 'tests="3"' "$report" || ! grep -q "skipped=\"$expected_skipped\"" "$report"; then
+            echo "::error::OpenAiCompatibleLiveTest did not run against NativeServer, llama-server and OpenRouter as expected"
+            grep -o 'tests="[0-9]*" errors="[0-9]*" skipped="[0-9]*" failures="[0-9]*"' "$report" || true
+            exit 1
+          fi
+      - name: llama-server log
+        if: failure()
+        run: docker logs llama-server || true
       - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7
         with:
           name: jacoco-report

CHANGELOG.md

@@ -11,6 +11,27 @@ The release procedure (prompt template and step-by-step instructions) lives in [
 
 ## [Unreleased]
 
+### Added
+- **`openai-compatible` provider: a model can run on any OpenAI-compatible API.** OpenRouter (the
+  default `baseUrl`), a llama.cpp `llama-server`, jllama's `NativeServer` or any server answering
+  `POST <baseUrl>/chat/completions`. Four new per-model settings: `provider` (overrides the run's
+  `generationProvider` for this model, so one run can mix local and remote models), `remoteModel`,
+  `baseUrl` and `apiKeyEnv` (the *name* of the environment variable holding the key, e.g.
+  `OPENROUTER_API_KEY`; empty by default, so no key is sent unless a model names its variable and a
+  custom endpoint never receives the OpenRouter key by accident; the key never goes into a
+  configuration, and it is sent only over `https` or to a loopback `http` server, never in cleartext
+  to another machine). The request carries the same
+  prompts as the local provider and only the portable sampling knobs; the answer goes through the
+  same parsing, so the `.ai.md` comes out the same way. Remote models skip the GGUF checks before a
+  run and instead must name an `http(s)` `baseUrl`. Local stays the default: nothing leaves the
+  machine unless a model says `openai-compatible`.
+- **CI tests the provider against three real APIs on every run**, each in addition to the others:
+  jllama's `NativeServer` (started in-process by `OpenAiCompatibleLiveTest`), a llama.cpp
+  `llama-server` container (both on the committed SmolLM2 GGUF), and OpenRouter with
+  `openai/gpt-6-luna` and the `OPENROUTER_API_KEY` secret. A check on the Surefire report fails the
+  `test` job when any of them was skipped; only the OpenRouter case may skip, and only in a run
+  without the secret (a pull request from a fork or from Dependabot).
+
 ### Removed
 - **No more fat jars on the GitHub release.** Up to 1.3.0 every release attached 15 signed
   `srcmorph-cli-<version>-jar-with-dependencies[-<classifier>].jar` files (a CPU jar for every

CLAUDE.md

@@ -7,7 +7,8 @@ This document provides guidance for AI assistants working on this codebase.
 ## Project Overview
 
 **srcmorph** is a prompt-driven source-tree transformer: it walks a source tree and processes each
-file through a configurable local LLM prompt (via llama.cpp / GGUF models, no cloud calls), producing
+file through a configurable LLM prompt (by default local via llama.cpp / GGUF models; per model
+optionally an OpenAI-compatible API such as OpenRouter), producing
 layered output — per-file, then per-package, then per-project. Today it emits structured `.ai.md`
 Markdown summaries for AI-assisted code navigation; the same rule-routed pipeline is generic enough
 to eventually emit wikis, architecture docs, diagrams, or source-to-source transformations.
@@ -111,8 +112,11 @@ Framework-free: **no dependency on `org.apache.maven..`** anywhere (enforced by
 - **`prompt/`** — `AiPromptDefinition`, `AiPreparedPrompt`, `AiPromptSupport`,
   `AiPromptPreparationSupport`.
 - **`provider/`** — the AI backend abstraction: `AiGenerationProvider` (`Closeable`),
-  `AiGenerationProviderFactory` (looks up `"mock"` / `"llamacpp-jni"`), `MockAiGenerationProvider`,
-  `LlamaCppJniAiGenerationProvider`, `LlamaCppJniConfig` (built through `LlamaCppJniConfig.builder(modelPath)`; the
+  `AiGenerationProviderFactory` (looks up `"mock"` / `"llamacpp-jni"` / `"openai-compatible"`; a model
+  definition's own `provider` overrides the run's `generationProvider`), `MockAiGenerationProvider`,
+  `LlamaCppJniAiGenerationProvider`, `OpenAiCompatibleAiGenerationProvider` (HTTP to
+  `<baseUrl>/chat/completions`, key from the env var `apiKeyEnv` names; its pure logic lives in the
+  PIT-gated `OpenAiCompatibleProviderSupport`), `LlamaCppJniConfig` (built through `LlamaCppJniConfig.builder(modelPath)`; the
   positional constructor is private, and every value the caller does not name defaults to the matching
   `AiGenerationConfig.DEFAULT_*`) + `LlamaCppJniConfigFactory` (the pure 37-field
   mapping from a resolved `AiGenerationConfig` to the native binding's parameter objects — extracted
@@ -446,8 +450,9 @@ See [`../workspace/workflows/pull-request-workflow.md`](../workspace/workflows/p
 
 ## Key Design Principles
 
-1. **Local-first** — all AI inference runs locally via llama.cpp; no cloud API calls, no data leaves
-   the machine.
+1. **Local-first** — by default all AI inference runs locally via llama.cpp; no cloud API calls, no
+   data leaves the machine. A remote model is an explicit per-model opt-in (`provider`
+   `openai-compatible`), and its API key is only ever read from an environment variable.
 2. **Deterministic indexing** — the same source produces the same `.ai.md` skeleton (deterministic
    header); only the AI-generated body varies.
 3. **Incremental updates** — files with existing summaries are skipped unless `force=true`; checksums

README.md

@@ -74,8 +74,9 @@
 # srcmorph
 
 **srcmorph** is a prompt-driven source-tree transformer: it walks a source tree and processes each
-file through a configurable local LLM prompt — via [llama.cpp](https://github.com/ggerganov/llama.cpp)
-GGUF models, entirely local, no cloud calls — producing layered output: per-file, then per-package,
+file through a configurable LLM prompt — by default via [llama.cpp](https://github.com/ggerganov/llama.cpp)
+GGUF models, entirely local, no cloud calls; optionally, per model, through any OpenAI-compatible API
+(OpenRouter, a `llama-server`) — producing layered output: per-file, then per-package,
 then per-project. Today it emits structured, AI-readable `.ai.md` Markdown summaries for fast semantic
 code navigation; the same rule-routed pipeline (composable file-matching conditions, per-rule
 model + prompt, oversize handling) is generic enough to eventually emit wikis, architecture docs,

pom.xml

@@ -161,8 +161,9 @@ SPDX-License-Identifier: Apache-2.0
                 jackson release train and each transitively pulls jackson-core/jackson-annotations at a
                 matching version — pinning them together here keeps maven-enforcer's
                 dependencyConvergence rule satisfied without needing a separate jackson-core/
-                jackson-annotations/snakeyaml pin (nothing else in the reactor depends on jackson or
-                snakeyaml, so there is no other version to converge against).
+                jackson-annotations/snakeyaml pin. jackson-databind is also used directly by srcmorph's
+                OpenAiCompatibleAiGenerationProvider and transitively by net.ladenthin:llama; this one
+                managed version is what all three resolve to.
             -->
             <dependency>
                 <groupId>com.fasterxml.jackson.core</groupId>

srcmorph-maven-plugin/README.md

@@ -27,7 +27,8 @@ retrieval.
 - Build a single project index (one line + link per package) for top-down navigation, optionally with a one-call AI `#### Overview`
 - Run any phase independently — toggle file / package / project on or off
 - Exclude trivial or generated files with glob patterns
-- Uses local models via llama.cpp (no cloud dependency)
+- Uses local models via llama.cpp by default (no cloud dependency); a model can instead run on any
+  OpenAI-compatible API (OpenRouter, `llama-server`, jllama's `NativeServer`), chosen per model
 - Incremental updates (skips unchanged files)
 - Logs a rough per-file duration estimate before each generation (size, token count, expected time)
 - Optimized for AI-assisted code understanding
@@ -376,7 +377,8 @@ Run-level parameters (set in `<configuration>`):
 - `excludes` — glob patterns for source files to skip, matched against each file's path relative to
   the base dir with `/` separators, e.g. `**/package-info.java`, `**/generated/**` (default: none).
   `*` stays within one path segment, `**` spans directories, `?` is a single character.
-- `generationProvider` — AI backend: `mock` (default) or `llamacpp-jni`
+- `generationProvider` — AI backend: `mock` (default), `llamacpp-jni` or `openai-compatible`; a
+  model's own `provider` (below) overrides it for that model
 - `modelDirectory` (`srcmorph.modelDirectory`) — the directory a relative `modelPath` is resolved
   against; an absolute `modelPath` is used as it is, so models can live in one directory and still
   one of them elsewhere. Unset (default), a relative `modelPath` is relative to the working directory.
@@ -398,13 +400,18 @@ Run-level parameters (set in `<configuration>`):
 ### Per-model `<aiDefinition>` parameters
 
 Every model knob lives inside its `<aiDefinition>` (referenced by `aiDefinitionKey`), not as a top-level
-plugin parameter. Only `key` and `modelPath` are required; everything else has a default. The defaults
-below are the shipped values (`AiGenerationConfig.DEFAULT_*`).
+plugin parameter. Only `key` and `modelPath` are required (`modelPath` not for an `openai-compatible`
+model); everything else has a default. The defaults below are the shipped values
+(`AiGenerationConfig.DEFAULT_*`).
 
 | Element | Default | Description |
 |---|---|---|
 | `key` | *(required)* | Identifier referenced by a rule's `aiDefinitionKey` |
-| `modelPath` | *(required)* | Path to the GGUF model file: absolute, or relative to `modelDirectory` (without it, to the working directory) |
+| `modelPath` | *(required)* | Path to the GGUF model file: absolute, or relative to `modelDirectory` (without it, to the working directory). Not used by an `openai-compatible` model |
+| `provider` | *(empty)* | The backend of this model: `llamacpp-jni`, `openai-compatible` or `mock`; empty uses the run's `generationProvider`. Lets one run send some files to a local GGUF and others to a remote model |
+| `remoteModel` | *(empty)* | `openai-compatible` only: the model id requested (the `model` field, e.g. `openai/gpt-6-luna` on OpenRouter); empty sends none, which a single-model `llama-server` or `NativeServer` accepts |
+| `baseUrl` | `https://openrouter.ai/api/v1` | `openai-compatible` only: the API base; `<baseUrl>/chat/completions` is called. A local server is `http://127.0.0.1:<port>/v1`. Must be `http(s)`, checked before the run starts |
+| `apiKeyEnv` | *(empty)* | `openai-compatible` only: the **name** of the environment variable holding the API key, sent as `Authorization: Bearer` (for OpenRouter `OPENROUTER_API_KEY`). The key itself never goes into a configuration file. Empty (the default), or a variable that is unset or empty, sends no key: a key goes only where a model names it, so a custom `baseUrl` never receives your OpenRouter key by accident. A key is only sent over `https`, or over plain `http` to a loopback host (`localhost`, `127.x.x.x`, `[::1]`); any other plain-`http` `baseUrl` fails before anything is sent, so set `apiKeyEnv` empty for a keyless server on another machine |
 | `contextSize` | `32768` | Context window in tokens |
 | `maxOutputTokens` | `128` | Max generated tokens per call |
 | `threads` | `4` (fewer on a machine with fewer cores) | CPU threads for inference. Never set more than the machine has cores: ggml's threads wait on each other, and an oversubscribed run is orders of magnitude slower |
@@ -444,6 +451,24 @@ below are the shipped values (`AiGenerationConfig.DEFAULT_*`).
 | `drySequenceBreakers` | *(empty)* | DRY sequence-breaker strings; empty = the binding defaults |
 | `stopStrings` | *(empty)* | Extra stop strings that end generation |
 
+**An `openai-compatible` model** sends the source file to the server at `baseUrl`; with the default
+that is OpenRouter, i.e. a third party. It sends the same system prompt and user message as the
+local provider and only the portable sampling knobs: `maxOutputTokens`, `temperature`, `topP`,
+`topK` and `minP` (when positive), `repeatPenalty`, `seed` (when set), `stopStrings` and
+`reasoningEffort`. The knobs of a local llama.cpp context (threads, GPU, KV cache, DRY, prompt
+caching) are ignored. A non-2xx answer fails the file with the status and the server's error text;
+nothing is retried.
+
+```xml
+<aiDefinition>
+  <key>luna</key>
+  <provider>openai-compatible</provider>
+  <remoteModel>openai/gpt-6-luna</remoteModel>
+  <apiKeyEnv>OPENROUTER_API_KEY</apiKeyEnv>
+  <maxOutputTokens>1024</maxOutputTokens>
+</aiDefinition>
+```
+
 ## Prompt System
 Prompts are defined in the plugin configuration (`<promptDefinitions>`) and referenced by key
 from `<fieldGenerations>`. The self-test profile defines five:
@@ -492,7 +517,7 @@ identically, so the two can never disagree about what was measured.
 - Deterministic metadata (hash-based change detection)
 - Separation of concerns (header = metadata, body = summary)
 - AI-friendly structure (predictable and hierarchical)
-- Local-first (no external APIs required)
+- Local-first (no external APIs required; a remote model is an explicit, per-model opt-in)
 ## Known Limitations
 - Model output may require normalization (handled in code)
 - Large models increase runtime

srcmorph/pom.xml

@@ -117,6 +117,15 @@ SPDX-License-Identifier: Apache-2.0
             <groupId>org.checkerframework</groupId>
             <artifactId>checker-qual</artifactId>
         </dependency>
+        <dependency>
+            <!--
+                OpenAiCompatibleAiGenerationProvider builds its request JSON with Jackson. Declared
+                explicitly because the provider uses it directly; net.ladenthin:llama depends on the
+                same artifact, and the parent's dependencyManagement keeps both on one version.
+            -->
+            <groupId>com.fasterxml.jackson.core</groupId>
+            <artifactId>jackson-databind</artifactId>
+        </dependency>
         <dependency>
             <groupId>net.ladenthin</groupId>
             <artifactId>llama</artifactId>
@@ -732,6 +741,7 @@ SPDX-License-Identifier: Apache-2.0
                         <param>net.ladenthin.srcmorph.provider.AiGenerationProvider</param>
                         <param>net.ladenthin.srcmorph.provider.AiGenerationProviderFactory</param>
                         <param>net.ladenthin.srcmorph.provider.MockAiGenerationProvider</param>
+                        <param>net.ladenthin.srcmorph.provider.OpenAiCompatibleProviderSupport</param>
                         <param>net.ladenthin.srcmorph.config.AiOversizeStrategy</param>
                         <param>net.ladenthin.srcmorph.config.AiFactCounter</param>
                         <param>net.ladenthin.srcmorph.config.AiFactDefinition</param>