Misc. bug: ngram-cache drafting reads before the start of draft for inputs shorter than 4 tokens #30724

Name and Version

version: 0.7.0-dev (build 50, commit 2a66703)
built with AppleClang 21.0.0.21000101 for Darwin arm64

Operating systems

Mac

Which llama.cpp modules do you know to be affected?

llama-server

Command line

head -c 2000 wiki.test.raw > short.txt
./llama-lookup-stats -m qwen2.5-0.5b-instruct-q4_k_m.gguf -f short.txt -c 128 --spec-draft-n-max 8 -ngl 0

Problem description & steps to reproduce

I noticed that common_ngram_cache_draft is currently reading before the start of draft when the input has fewer tokens than ngram_max. ngram_start_cd can become negative and wrap in get_token because the early return is currently only checking inp_size < LLAMA_NGRAM_STATIC.

My concern here is that in a release build the out of bounds tokens will become part of the n-gram key and the drafts will depend on whatever memory sits before draft.

From what I can tell, this bug has been around since 2024.

Steps to reproduce:

First build with -DCMAKE_BUILD_TYPE=Debug -DLLAMA_SANITIZE_ADDRESS=ON -DLLAMA_SANITIZE_UNDEFINED=ON. Then run the following command on any short text file that is longer than 128 tokenspbc:

./llama-lookup-stats -m qwen2.5-0.5b-instruct-q4_k_m.gguf -f short.txt -c 128 --spec-draft-n-max 8 -ngl 0

The text file I used was

head -c 2000 wiki.test.raw > short.txt

I think we can fix this by just skipping the ngram sizes that can't fit i.e just do a continue when ngram_start_cd < 0.

I am happy to open up a PR with the fix and a regression test.

First Bad Commit

50ccaf5 (#5479). The bug has been present since common/ngram-cache.cpp was added.

Relevant log output

.../c++/v1/__vector/vector.h:411:12: runtime error: addition of unsigned offset to 0x602000199650 overflowed to 0x602000199648
==54406==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000199648 at pc 0x00010700e32c bp 0x00016d39c0e0 sp 0x00016d39c0d8
READ of size 4 at 0x602000199648 thread T0
    #0 0x00010700e328 in get_token(...) ngram-cache.cpp:56
    #1 0x00010700d934 in common_ngram_cache_draft(...) ngram-cache.cpp:175
    #2 0x000102a63e68 in main lookup-stats.cpp:92

0x602000199648 is located 8 bytes before 4-byte region [0x602000199650,0x602000199654)