Repository navigation
Distance threshold query silently fails when using bit quantization #308
Description
This query seemed to ignore the 'v.distance <= :distanceThreshold' condition
FROM vec_history v
JOIN vec_history_mapping m USING (rowid)
WHERE v.embedding MATCH :vector
AND k = :limit
AND v.distance <= :distanceThreshold
ORDER BY v.distance
)
Table was created with
CREATE VIRTUAL TABLE vec_history USING vec0(
embedding FLOAT[${this.#embeddingSize}]
distance_metric=cosine
INDEXED BY rescore(
quantizer=bit,
oversample=50
)
);
Activity
[-]Distance threshold query silently fails[/-][+]Distance threshold query silently fails when using bit quantization[/+]Apply distance constraints in rescore KNN queries (fixes asg017#308)
realgetOff commentedon Aug 4, 2026
I dug into this and can confirm it, with a wider blast radius than the title suggests: it isn't specific to bit quantization, and it isn't specific to
rescore.Root cause
vec0BestIndexaccepts thedistanceconstraint and unconditionally setsomit = 1:That is a promise to SQLite that the vtab applies the constraint itself, so SQLite drops the
WHEREterm from the query plan and never re-checks it. But the constraint is only ever read back out ofidxStrin the FLAT chunk scan (vec0Filter_knn_chunks_iter).grep -n VEC0_IDXSTR_KIND_KNN_DISTANCE_CONSTRAINT sqlite-vec.creturns only that one function.rescore_knn,vec0Filter_knn_diskannandivf_query_knnnever look at it, so for those column types the predicate silently evaporates.It looks like a regression rather than an oversight: #166 landed the
distanceconstraints on 2026-02-13, and the new ANN backends (#276 rescore, #278 DiskANN, #277 IVF) were merged on 2026-03-31 without wiring them up.Reproduction
200 random
float[8]rows, same query vector,k = 20,AND distance <= 0.3:DiskANN matters most in practice here, since it is enabled by default (
SQLITE_VEC_ENABLE_DISKANN 1) whereas IVF is opt-in. All four operators (<,<=,>,>=) are affected, and bothquantizer=bitandquantizer=int8on the rescore path — so the issue title's "bit quantization" is incidental.An impossible constraint is also not honored:
AND distance < -1.0returns a full page of rows instead of none.Fix
I have a patch ready and will open a PR referencing this issue. The shape of it:
omit = 1has a single point of truth that every backend routes through;switchcases collapse into one loop calling the helper, which is a net -44 lines and leaves all 204 existing snapshots byte-identical;k;One honest caveat I'll document rather than paper over: on DiskANN a
distance >constraint can return fewer thankrows, because the filter runs after the graph search rather than over the candidate pool. Matching FLAT there would mean reaching intodiskann_search, which felt out of scope for a correctness fix.Verified with 4 new parametrized tests over flat / rescore-bit / rescore-int8 / diskann: 17 failed before the patch, 0 after, full suite 262 passed, and clean under ASan + UBSan.
Apply KNN distance constraints in every index backend
Apply KNN distance constraints in every index backend