[leehack]

Summary

Three new exports let llamadart's caller-owned cancel byte stop native work in flight, instead of waiting for the current llama_decode or clip encode to finish. Fixes #88 (which absorbed #89).

Export	Use	Stops
llama_dart_cancel_flag_abort_callback(void *data)	llama_set_abort_callback(ctx, it, byte)	llama_decode / llama_encode on the CPU backend, after the current node; returns 2
llama_dart_mtmd_eval_callback(tensor, ask, user_data)	mtmd_context_params.cb_eval, replacing llama_dart_tts_eval_callback	Inside a TTS step: same answers as the TTS callback. Inside the call below: chunks the clip graph and ends it once the byte is set. Otherwise: requests no tensors.
llama_dart_mtmd_encode_chunk_cancellable(ctx, chunk, flag)	STT audio and vision encodes	Returns LLAMA_DART_MTMD_ENCODE_CANCELLED (2) when the byte stopped the encode or kept it from starting

This change only adds exports. The existing required exports and their behaviour are unchanged. validate_exports.py lists the new ones after #139's, so the required count goes from 93 to 96. llamadart starts using them in 0.18.

Clip encode: no upstream abort hook

At v0.7.0 (pinned) and v0.8.0 (8642541b), mtmd and clip expose no abort callback. Clip makes its own backends, so llama_set_abort_callback never reaches them. The only hook is cb_eval, so no upstream patch is needed. A thread-local scope that lives only for the call ties the call's byte to the callback, so a long-lived mtmd context never holds a pointer to a per-request byte.

Where the graph is chunked. Splitting the graph at a node splits any backend fusion that spans that node. Metal, Vulkan and CUDA fuse a MUL_MAT with the bias ADD after it, so the TTS callback's after-MUL_MAT boundaries would change GPU output. Boundaries here fall only after FLASH_ATTN_EXT / SOFT_MAX, once per 2.5e9 work units (the TTS budget).

Metal output. Metal fuses only within one command buffer, and its node ranges depend on the size of the graph it computes. So on Metal any chunking still changes which ops fuse. Qwen3-ASR's chunked encode differs from the unchunked one by 1.3% of the largest output magnitude. The difference goes to 0 with GGML_METAL_FUSION_DISABLE=1. CPU output is bit-identical. The difference is documented in the README.

Measured (M4 Max, pinned v0.7.0, tests/cancel_flag_test.cpp mtmd-encode)

Medians of 5. Latency runs from the byte store to the return. Host load average was 13–90 from parallel lanes, so the CPU latency spread is wide.

Model, backend	Plain encode	Cancellable, byte 0	Store → return	Output vs plain
SmolVLM-500M vision, CPU	356.6 ms	359.3 ms	12.0 ms (3.3–29.5)	identical
SmolVLM-500M vision, Metal	22.7 ms	25.1 ms	0.65 ms (0.53–1.01)	identical
Qwen3-ASR-0.6B audio (20 s), CPU	144.0 ms	146.3 ms	22.8 ms (1.4–40.9)	identical
Qwen3-ASR-0.6B audio (20 s), Metal	15.8 ms	17.2 ms	1.1 ms (0.66–4.09)	1.3% of max

After a cancel, the next encode on the same mtmd context returns 0 with the same output as before the cancel.

Teardown and barrier

• llama_dart_mtmd_encode_chunk_cancellable has the exception barrier and is a call in flight (llama_dart_exit_call), like llama_dart_mtmd_encode_chunk.
• The two callbacks touch only the caller's byte and the calling thread's scope, so they have no barrier and need no guard. The header lists them with the other barrier-free functions.
• The byte must stay readable until llama_set_abort_callback(ctx, NULL, NULL) returns or the context is freed, and until the cancellable encode returns.

Tests

tests/cancel_flag_test.cpp (new llamadart_cancel_flag_test):

• flag (every lane, MSVC included): the abort callback reads the byte; the eval callback asks for nothing outside a call; a byte set on entry returns 2 without touching the context; unit tests of the chunk/stop answers.
• decode (macOS and Linux, using the model the exit-teardown tests write): rc 2 with the byte set, seq_pos_max −1, and rc 0 again once it is cleared. A deterministic mid-decode stop: the store happens after node 3, the decode returns 2 and no computing node completes after the store. View nodes can still complete, because the CPU backend checks the abort callback only after a node that computes something. Clearing the callback restores full decodes.
• mtmd-encode (runs when LLAMADART_EXIT_TEARDOWN_TEST_MODEL and _MMPROJ are set): produces the measurements above.

Failing first: against stubs (abort returns false, the eval callback forwards to the TTS one, the cancellable call is the plain encode), flag, decode and mtmd-encode all failed their assertions.

Mutation check: 11 mutants, 11 killed.

Mutant	Killed by
abort callback always false	flag, decode
byte read without NULL check	flag
idle eval callback asks	flag, mtmd-encode
no entry check	flag
no thread-local scope	mtmd-encode (SmolVLM, ASR)
status ignores observed cancel	mtmd-encode (SmolVLM, ASR)
boundary after MUL_MAT	flag
cancel waits for a boundary	flag
budget never reset	flag
ask=false ignores cancel	flag
TTS branch removed from llama_dart_mtmd_eval_callback	local llamadart_tts_smoke (Qwen3-TTS 0.6B, Metal) through the new callback: unmutated PASS with RMS and chunks identical to the TTS callback; mutant fails "chunk exceeds a quarter"

Validation

• Local macOS arm64, post-v0.4.0 upstream: decode scenario passes (the first push asserted an exact node count, which failed there).
• Local iOS device arm64 (tools/build.py apple --target ios-device-arm64): builds and passes validate_exports.py (before the merge).
• After merging origin/main (Harden the Linux arm64 ISA validator; guard the 24.04 pin and glibc floor #136, [0.18] Add opt-in LFM3-Audio speech output to the TTS wrapper #139, [0.17.1] Retire carried patch 0003 on llama.cpp v0.8.0 #140) at 8e4d5c3, local macOS arm64 Debug: ctest 60/60 on pinned v0.7.0 and 60/60 on v0.8.0 (8642541b); validate_exports.py passes on both (96 exports, no __cxa_atexit import). The mtmd-encode scenarios also pass on v0.8.0. The Debug CPU timings there are not comparable with the table above.
• Before the merge, local macOS arm64 Debug: ctest 59/59; python3 -m unittest discover -s tests -p 'test_*.py' 279 OK (1 skipped).
• The new test runs in the wrapper-contract (Linux, every upstream row), MSVC, Windows arm64 and macOS exit-teardown lanes.

Not covered

• llama_set_abort_callback reaches only the CPU backend upstream, so a GPU decode still runs to its end.
• No mmproj in CI; the encode path is covered locally only.
• The existing TTS chunker comment says CPU and Metal fuse nothing past a MUL_MAT. At v0.7.0 Metal has MUL_MAT_ADD fusion. TTS behaviour is unchanged by this PR; the comment is stale.

CI results

Run	Head	Result
Validate Wrapper 40968666785	7a4d6aa	wrapper-contract (post-v0.4.0) failed on the exact node-count assertion, fixed in 1a782e0; run cancelled by the next push
Validate Wrapper 40974239474	1a782e0	64/64
Validate Wrapper 40984520101	8e4d5c3 (merge of main)	pending

HIP has no PR lane. The wrapper change there is host C++ only.

[leehack]

Independent audit at 8e4d5c3, with a delta re-audit at exact head b5fbb7c: no P1. The one P2 is resolved at b5fbb7c; it was the decode-abort contract on recurrent and hybrid models, which is now documented in the header and the README. CI passes 64/64.

• Additive: the change adds 3 exports (96 total). TTS output with the new callback is bit-identical.
• Sanitizers: nested and concurrent cancel scopes are TSan-clean, and the encode paths are ASan-clean.
• Teardown: exit teardown with a cancelled in-flight encode finishes in about 200 ms.
• Mutants: 16 of 19 killed.
• Merging before the v0.8.0 publish is safe.

P3s are tracked on #88:

• CI never runs the encode path, because there is no mmproj fixture.
• No CI test runs a TTS step under llama_dart_mtmd_eval_callback.
• Nothing tests llama_dart_exit_call in the cancellable encode (M17).
• Computed-index ops after a cancel in a multi-split clip graph carry a residual risk.

Evidence: llamadart-evidence/native-pr-141/.