Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
564fa15
feat(budget): add BeforeDispatch hook for reserved model calls
senamakel Oct 19, 2026
5389686
feat(budget): run before-dispatch hook on invoke and stream
senamakel Oct 19, 2026
1fd317f
style(model): reorder re-exports and reformat test response
senamakel Oct 19, 2026
7776e24
refactor(budget): collapse nested hook checks into let chains
senamakel Oct 19, 2026
45b6868
Merge pull request #110 from tinyhumansai/epic-236-228-dispatch-admis…
senamakel Oct 19, 2026
4423245
chore(deps): bump rustls-webpki, webpki-roots and getrandom
senamakel Oct 20, 2026
61d5406
test(budget): cover dispatch rechecks on retries and lazy streams
senamakel Oct 20, 2026
d678be2
fix(budget): run before-dispatch hook on every physical attempt
senamakel Oct 20, 2026
33d697c
docs(budget): clarify before-dispatch hook timing
senamakel Oct 20, 2026
5891eaa
style(budget_tests): reformat assertion to satisfy line width
senamakel Oct 20, 2026
e241da1
docs(budget): clarify BeforeDispatch idempotency requirement
senamakel Oct 20, 2026
9dbd069
feat(model): add dispatch-checked model wrapper
senamakel Oct 20, 2026
abea442
test(budget): assert budget state after retry and lazy stream dispatch
senamakel Oct 20, 2026
76c6a41
test(budget): cover budget refund on dispatch refusal
senamakel Oct 20, 2026
1d0db71
fix(budget): keep charge when a retry is refused after the first send
senamakel Oct 20, 2026
cedeced
style(tests): reformat budget tests to satisfy rustfmt
senamakel Oct 20, 2026
2356552
test(budget): cover retry refusal after the first physical send
senamakel Oct 20, 2026
cddf1ac
fix(budget): mark refused unbudgeted attempts before returning
senamakel Oct 20, 2026
10d442f
fix(budget): mark refusal before attempt on dispatch check failure
senamakel Oct 20, 2026
bf9cec1
test(budget): assert refused dispatches open no connection
senamakel Oct 20, 2026
76aab15
test(budget): cover nested dispatch hooks on openai retry
senamakel Oct 20, 2026
4b74f9c
test(budget): assert dispatch checks run on nested refund path
senamakel Oct 20, 2026
3308db0
test(budget): reformat assertions and relax accept timeout
senamakel Oct 20, 2026
9dbbbe8
test(budget): serve a real response in the nested retry test
senamakel Oct 20, 2026
ef8c343
test(budget): ignore send result when signalling invoke finished
senamakel Oct 20, 2026
5e9c1aa
refactor(budget): check unbudgeted admission alongside dispatch hook
senamakel Oct 20, 2026
e5df746
docs(budget): fix intra-doc link in BeforeDispatch comment
senamakel Oct 20, 2026
07de0cf
test(budget): cover dispatch rechecks for untracked and anthropic fal…
senamakel Oct 20, 2026
1123054
test(budget): reorder fallback dispatch assertions
senamakel Oct 20, 2026
c5d4e85
feat(llm): add tracks_physical_attempts to ChatModel
senamakel Oct 20, 2026
a7b012e
fix(budget): track untracked inner invocations in refusal check
senamakel Oct 20, 2026
1e333f1
feat(model): add tracks_physical_attempts to chat models
senamakel Oct 20, 2026
0aecbc7
test(llm): assert fallback tracks physical attempts
senamakel Oct 20, 2026
26a84e5
test(llm): opt retry test models into physical attempt tracking
senamakel Oct 20, 2026
e116be7
docs(budget): clarify physical attempt admission comment
senamakel Oct 20, 2026
6163672
docs(budget): rewrap doc comment for admit_extra_physical_attempt
senamakel Oct 20, 2026
3c486fc
test(llm): cover ollama warmup as a gated physical attempt
senamakel Oct 20, 2026
446a190
test(llm): simplify second connection handling in warmup budget test
senamakel Oct 20, 2026
31ba9bb
test(budget): annotate registry closure types in warmup test
senamakel Oct 20, 2026
52fe42f
fix(openai): account for warm-up requests in the attempt budget
senamakel Oct 20, 2026
963e9b8
test(budget): make warmup revocation deterministic
senamakel Oct 20, 2026
ec40279
test(llm): reorder assertions in warmup budget test
senamakel Oct 20, 2026
6c0107d
fix(openai): enforce budget before sending request
senamakel Oct 20, 2026
2ddd29f
style(tests): reflow budget test formatting
senamakel Oct 20, 2026
bdd21ae
feat(budget): add physical attempt hook to dispatch policy
senamakel Oct 20, 2026
1d4f317
refactor(budget): drop redundant physical attempt hook calls
senamakel Oct 20, 2026
29d2bc1
fix(budget): serialize unbudgeted physical attempt admission
senamakel Oct 20, 2026
fe36972
fix(budget): serialize physical attempt admission and reserve retry b…
senamakel Oct 20, 2026
e434f42
fix(budget): release host permit when stream reaches terminal state
senamakel Oct 20, 2026
0cfe2b6
test(budget): assert refused retries skip physical admission
senamakel Oct 20, 2026
3a8026e
fix(budget): track nested dispatch and attempt scopes
senamakel Oct 20, 2026
232f8aa
refactor(budget): rename dispatch refusal marker and hold attempt gates
senamakel Oct 20, 2026
42b61b7
test(budget): cover nested policy rechecks and dispatch delegation
senamakel Oct 20, 2026
fd8047f
style: apply rustfmt formatting to budget model and tests
senamakel Oct 20, 2026
3819ed1
fix(budget): check before-dispatch hook before first attempt
senamakel Oct 20, 2026
7921356
fix(llm): release dispatch and attempt scopes when streams terminate
senamakel Oct 20, 2026
c8fa756
feat(llm): add pre-invocation admission gate hook to ChatModel
senamakel Oct 20, 2026
abcab02
fix(budget): track untracked inner invocation via admission gate
senamakel Oct 20, 2026
1658997
chore: files changed crates/tinyinference-llm/src/model/budget.rs
senamakel Oct 20, 2026
247dbca
feat(model): propagate pre-invocation admission gate through wrappers
senamakel Oct 20, 2026
d514255
refactor(budget): remove pre-dispatch refusal check
senamakel Oct 20, 2026
5024440
test(budget): cover untracked inner flag when refused before attempt
senamakel Oct 20, 2026
71de656
fix(budget): move untracked invocation reset to dispatch refusal
senamakel Oct 20, 2026
cd3f324
test(budget): remove redundant untracked flag reset in refusal test
senamakel Oct 20, 2026
e2b82d3
refactor(budget): simplify nested attempt scope collection
senamakel Oct 20, 2026
f1ac157
refactor(budget): remove deferred fetch passthrough and reuse physica…
senamakel Oct 20, 2026
c7a4420
feat(llm): forward deferred fetch through dispatch-checked model
senamakel Oct 20, 2026
f456624
fix(budget): drop redundant admission gate check
senamakel Oct 20, 2026
553c235
fix(budget): drop pre-invocation gate from untracked inner check
senamakel Oct 20, 2026
b839de2
fix(budget): skip untracked-inner marking when admission gate exists
senamakel Oct 20, 2026
c606009
fix(budget): skip untracked-inner marking when admission gate exists
senamakel Oct 20, 2026
ae2a3bc
style(budget): reflow multi-line conditions onto single lines
senamakel Oct 20, 2026
3d985c8
fix(budget): allow refund when a pre-invocation admission gate refuses
senamakel Oct 20, 2026
2167350
chore: files changed crates/tinyinference-llm/src/model/budget_tests.rs
senamakel Oct 20, 2026
5cbbe5a
fix(budget): apply dispatch checks to deferred fetches
senamakel Oct 20, 2026
d9e0396
test(llm): assert no stray connection when warmup finishes first
senamakel Oct 20, 2026
52066d3
test(model): add budget tests for physical attempt models
senamakel Oct 20, 2026
74c3815
chore: files changed crates/tinyinference-llm/src/model/budget_tests.rs
senamakel Oct 20, 2026
ece3738
chore: files changed crates/tinyinference-llm/src/model/budget.rs
senamakel Oct 20, 2026
0ede537
chore: files changed crates/tinyinference-llm/src/model/budget_tests.rs
senamakel Oct 20, 2026
6ce7c7e
test(model): add budget refund test case
senamakel Oct 20, 2026
9894108
feat(model): add pre-invocation admission gate check
senamakel Oct 20, 2026
43b994c
refactor(model): remove has_pre_invocation_admission_gate check
senamakel Oct 20, 2026
5f6a5c5
feat(model): add pre-invocation admission gate check
senamakel Oct 20, 2026
e9bb68f
test(model): assert fallback invocations are zero on early refusal
senamakel Oct 21, 2026
8d8d318
test(model): add test for temporary wait mode retry refusal
senamakel Oct 21, 2026
0620a96
test(model): add admission check to budget tests
senamakel Oct 21, 2026
6fd7486
docs(model): clarify retry admission behavior
senamakel Oct 21, 2026
7136c19
test(model): assert budget cost equality in temporary wait mode
senamakel Oct 21, 2026
dc14cbd
fix(model): refund fallback pre-entry refusals
senamakel Oct 21, 2026
721380e
Merge commit 'dc14cbdb7d97bb87d852c150036d67a20f6426a9' into epic-809…
senamakel Oct 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions crates/tinyinference-llm/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@ pub use failure::{
};
pub use message::{AssistantMessage, ContentBlock, Message, MessageDelta};
pub use model::{
ChatModel, FINISH_REASON_LENGTH, InputModality, InputSource, ModelRequest, ModelResponse,
ModelStream, ModelStreamItem, canonical_finish_reason, context_window_for_model_id,
model_id_supports_vision,
BeforeDispatch, ChatModel, DispatchCheckedModel, FINISH_REASON_LENGTH, InputModality,
InputSource, ModelRequest, ModelResponse, ModelStream, ModelStreamItem,
canonical_finish_reason, context_window_for_model_id, model_id_supports_vision,
};
pub use network_guard::{allow_network_models, deny_network_models, network_models_denied};
pub use providers::{MockModel, ProviderKind, ProviderSpec};
Expand Down
Loading
Loading