feat(config): pass through global LiteLLM settings via a litellm: config block#138
Merged
Merged
Conversation
ade8500 to
29af484
Compare
…lock Add an optional `litellm:` mapping in .openkb/config.yaml as the single place to tune LiteLLM. Keys are forwarded to LiteLLM: `timeout` and `extra_headers` apply per request (the existing per-call mechanism, covering both the compiler and the agents-SDK paths), and the rest are set as litellm module globals (drop_params, num_retries, ssl_verify, ...). Resolves the Ollama UnsupportedParamsError in #137: `litellm: {drop_params: true}` lets LiteLLM drop params a provider rejects (e.g. parallel_tool_calls on Ollama). - config.resolve_litellm_settings(): validate the value is a mapping, drop non-string keys, pass values through verbatim (the user owns them). - cli._setup_llm_key(): the `litellm:` block is canonical — when it specifies `timeout` / `extra_headers` they route to the per-call stashes and replace the legacy top-level keys (an empty `extra_headers: {}` clears, not reverts); the rest go to cli._apply_litellm_settings(), which setattrs each onto litellm. Guards: skip+warn an unknown key, and refuse to overwrite a litellm function. Globals are applied process-wide and not reset. - Back-compat: the legacy top-level `timeout:` / `extra_headers:` keys still work (now undocumented; the `litellm:` block is the documented surface). - warnings go through the logger (stderr), not click.echo, so a typo can't corrupt piped stdout (e.g. `openkb query > out.txt`). - docs: config.yaml.example consolidated under `litellm:`; README drops the now-redundant top-level extra_headers section. Closes #137
29af484 to
63c0e1b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an optional
litellm:mapping to.openkb/config.yamlwhose keys are passed through verbatim onto thelitellmmodule (litellm.<key>):Why
Closes #137. A user on a local Ollama backend hit
litellm.UnsupportedParamsErrorbecause OpenKB sendsparallel_tool_calls, which Ollama doesn't support. The fix LiteLLM suggests islitellm.drop_params=True, but OpenKB only forwardedmodel(plus the existingextra_headers/timeoutwhitelist) — there was no way to set it from config, only theLITELLM_DROP_PARAMSenv var. This generalizes the existing per-key whitelist to a verbatim passthrough so any LiteLLM module global can be set from config.How
config.resolve_litellm_settings()— validates the value is a mapping, drops non-string keys, forwards values unchanged (no validation/coercion; the user owns them).cli._apply_litellm_settings()—setattrs each key ontolitellminside_setup_llm_key, so it covers every command and every call path routed through LiteLLM (these are module-level globals). Two guards:litellm.completion) is refused — overwriting a function with a config scalar would brick later calls.click.echo, so a typo can't corrupt piped stdout (e.g.openkb query > out.txt).config.yaml.example).Notes
litellm.drop_params=Trueper call during long-PDF indexing, so a user'sdrop_params: falsewon't make indexing strict — noted in the docs. (A separate upstream PageIndex change stops it from mutating the shared global.)Tests
tests/test_config.py(resolver: mapping/non-mapping/non-string-key + caplog on the warning paths) and a newtests/test_llm_config_passthrough.py(verbatim apply, unknown-key + callable guards, sticky no-reset, env-isolated end-to-end). Full suite green.