Skip to content

Add local tag suggestions with human review - #321

Open
manuc66 wants to merge 3 commits into
mainfrom
tag-suggestions
Open

manuc66 wants to merge 3 commits into
mainfrom
tag-suggestions

Conversation

@manuc66

@manuc66 manuc66 commented Oct 21, 2026

Copy link
Copy Markdown
Owner

What

Adds local tag suggestions with human review. After a few photos are tagged, the app proposes tags for the others, with a score and the evidence behind it. Nothing is assigned automatically: a suggestion only fills the label panel when the user accepts it.

This is the safe half of the "AI sorter" idea: a decision model that suggests tags, while the existing Publisher remains the only thing that fans files out into folders.

How it works

  • Local descriptor — a MobileNetV2 1.0 (224×224) ONNX model computes a 1280-value visual descriptor per photo. 14 MB, CPU only, no network. The model's pooled features are used instead of the ImageNet logits; the classifier weights are untouched. Source, licence and SHA-256 are documented in ImgTagFanOut/Assets/Models/README.md.
  • Cache — descriptors are stored in item_embeddings keyed by content hash and model version, like face analyses. A second scan only reads SQLite.
  • Three signals — file name/path, faces already named on the photo, and similarity with already-tagged photos (cosine kNN, only when the library has at least 3 distinct tags). Similarity is deliberately conservative: it needs a dominant share of the neighbourhood and a margin over the next tag.
  • Review — tag_suggestions stores at most three scored proposals per image with their source and evidence. Accepting assigns the tag and records the feedback; rejecting remembers the tag so it is not proposed again.
  • UI — a suggestion strip above the faces, a Suggest tags command with progress/cancel, and a Next suggestion command that walks the pending queue.

Notable implementation points

  • Tagged photos are processed first, so the comparison gallery is complete before an untagged photo is scored, independently of file order.
  • A missing model falls back to path and face signals; after repeated decode failures the model is disabled for the rest of the run.
  • The gallery is capped at 2000 descriptors to keep the cosine pass bounded.
  • The scan tolerates missing/unreadable files and saves its work every 50 images.

Verification

  • dotnet restore --locked-mode, dotnet build --no-restore: clean, 0 warnings.
  • dotnet test --no-build: 178 tests pass.
  • Coverage: 83.7% lines / 64.8% branches (floors: 78 / 55).
  • New tests include engine/scoring units, repository persistence, scan (cache invalidation, cancellation paths, order independence, missing model), the real ONNX model on the demo portraits, and headless UI tests that render the strip and accept a suggestion.

Notes / limits

  • The confidence is a ranking score, not a calibrated probability.
  • Visual suggestions need a cold start: at least 3 different tags in the library.
  • A rejected tag is permanent for that image (no reset yet).
  • No OCR yet; it can be added behind ITagSuggester later without touching the UI or the storage.

A small MobileNetV2 ONNX model describes each photo, and the suggestions
are stored per image with their evidence: the file name, a named face, or
a tagged photo that looks alike. Everything runs locally.

Embeddings are cached by content hash and model version, like face
analyses, so a second pass only reads SQLite. Rejected suggestions are
kept to avoid proposing the same tag again.
Tagged photos are processed first so the comparison gallery is complete
before an untagged photo is scored, whatever the file order. The gallery
is capped, the model is disabled after repeated decode failures, and a
missing model falls back to file names and named faces.

The scan never writes a tag: it only fills the pending suggestion rows.
The strip above the faces shows the scored proposals with the evidence
behind each one. Accepting assigns the tag and records the feedback;
rejecting hides it for good. A Next suggestion command walks the pending
queue, so reviewing a whole library does not mean clicking every photo.

The model, its source and its licence are documented next to the other
ONNX assets.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant