Optional ultrafast COCO evaluation backend in Detection reference scripts #9665

Proposal

I maintain ultrafast-pycocotools, a BSD-2-Clause Rust COCO evaluator. I would like to propose an explicit optional backend in the Detection reference scripts, selected with --coco-backend ultrafast, while retaining the default pycocotools path and existing dataset dependencies.

The proposed scope covers bbox, segmentation and keypoints, including the training/test-only CLI and engine.evaluate(). No torchvision core dependency or process-wide import replacement is needed.

This requires a real distributed adapter: ultrafast 0.1.11 evaluates and accumulates together, so assigning externally merged evalImgs does not recompute its curves. The implementation instead gathers prepared predictions, preserves the existing sorted image-ID / first-occurrence deduplication rule, and runs native evaluation during synchronization. Per-image records are requested explicitly so they remain inspectable. The per-batch evaluator timer consequently does not include this final work; I am not claiming a speed or memory improvement from unrelated standalone benchmarks.

A concrete implementation and tests are ready for a draft PR. Nine tests pass on macOS (Torch 2.14) and Linux (Torch 2.10 CPU), including every precision/recall/scores array and per-image matching record, repeated accumulation, pickle before synchronization, conflicting duplicate images, two-rank Gloo with an empty rank, CLI parsing and the engine evaluation path with fixed model outputs.

Would an optional evaluator backend in the reference scripts fit the project scope? I will keep the PR in draft while this proposal is discussed.