Skip to content

feat(config): pass through global LiteLLM settings via a litellm: config block#138

Merged
KylinMountain merged 1 commit into
mainfrom
feat/litellm-config-passthrough
Jun 25, 2026
Merged

feat(config): pass through global LiteLLM settings via a litellm: config block#138
KylinMountain merged 1 commit into
mainfrom
feat/litellm-config-passthrough

Conversation

@KylinMountain

Copy link
Copy Markdown
Collaborator

What

Adds an optional litellm: mapping to .openkb/config.yaml whose keys are passed through verbatim onto the litellm module (litellm.<key>):

litellm:
  drop_params: true     # let LiteLLM drop params a provider rejects
  num_retries: 3

Why

Closes #137. A user on a local Ollama backend hit litellm.UnsupportedParamsError because OpenKB sends parallel_tool_calls, which Ollama doesn't support. The fix LiteLLM suggests is litellm.drop_params=True, but OpenKB only forwarded model (plus the existing extra_headers/timeout whitelist) — there was no way to set it from config, only the LITELLM_DROP_PARAMS env var. This generalizes the existing per-key whitelist to a verbatim passthrough so any LiteLLM module global can be set from config.

How

  • config.resolve_litellm_settings() — validates the value is a mapping, drops non-string keys, forwards values unchanged (no validation/coercion; the user owns them).
  • cli._apply_litellm_settings()setattrs each key onto litellm inside _setup_llm_key, so it covers every command and every call path routed through LiteLLM (these are module-level globals). Two guards:
    • an unknown key (typo / litellm-version mismatch) is skipped with a warning instead of silently creating a dead attribute;
    • a key whose current value is callable (e.g. litellm.completion) is refused — overwriting a function with a config scalar would brick later calls.
  • Warnings go through the logger (stderr, like the sibling resolvers), not click.echo, so a typo can't corrupt piped stdout (e.g. openkb query > out.txt).
  • Process-wide and sticky on purpose: settings are applied, never reset (documented in the docstring, README, and config.yaml.example).

Notes

  • The currently-pinned PageIndex hardcodes litellm.drop_params=True per call during long-PDF indexing, so a user's drop_params: false won't make indexing strict — noted in the docs. (A separate upstream PageIndex change stops it from mutating the shared global.)

Tests

tests/test_config.py (resolver: mapping/non-mapping/non-string-key + caplog on the warning paths) and a new tests/test_llm_config_passthrough.py (verbatim apply, unknown-key + callable guards, sticky no-reset, env-isolated end-to-end). Full suite green.

@KylinMountain KylinMountain force-pushed the feat/litellm-config-passthrough branch 3 times, most recently from ade8500 to 29af484 Compare June 25, 2026 10:48
…lock

Add an optional `litellm:` mapping in .openkb/config.yaml as the single place to
tune LiteLLM. Keys are forwarded to LiteLLM: `timeout` and `extra_headers` apply
per request (the existing per-call mechanism, covering both the compiler and the
agents-SDK paths), and the rest are set as litellm module globals (drop_params,
num_retries, ssl_verify, ...). Resolves the Ollama UnsupportedParamsError in
#137: `litellm: {drop_params: true}` lets LiteLLM drop params a provider rejects
(e.g. parallel_tool_calls on Ollama).

- config.resolve_litellm_settings(): validate the value is a mapping, drop
  non-string keys, pass values through verbatim (the user owns them).
- cli._setup_llm_key(): the `litellm:` block is canonical — when it specifies
  `timeout` / `extra_headers` they route to the per-call stashes and replace the
  legacy top-level keys (an empty `extra_headers: {}` clears, not reverts); the
  rest go to cli._apply_litellm_settings(), which setattrs each onto litellm.
  Guards: skip+warn an unknown key, and refuse to overwrite a litellm function.
  Globals are applied process-wide and not reset.
- Back-compat: the legacy top-level `timeout:` / `extra_headers:` keys still work
  (now undocumented; the `litellm:` block is the documented surface).
- warnings go through the logger (stderr), not click.echo, so a typo can't
  corrupt piped stdout (e.g. `openkb query > out.txt`).
- docs: config.yaml.example consolidated under `litellm:`; README drops the
  now-redundant top-level extra_headers section.

Closes #137
@KylinMountain KylinMountain force-pushed the feat/litellm-config-passthrough branch from 29af484 to 63c0e1b Compare June 25, 2026 10:53
@KylinMountain KylinMountain merged commit b204ddf into main Jun 25, 2026
1 check passed
@KylinMountain KylinMountain deleted the feat/litellm-config-passthrough branch June 25, 2026 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Plans to support litellm config in .openkb/config.yaml? (Encountered UnsupportedParamsError with Ollama)

1 participant