Skip to content

LLMRanker returns documents unranked on None query instead of crashing - #12799

Merged
davidsbatista merged 2 commits into
deepset-ai:mainfrom
ege-arhan:fix-llm-ranker-none-query
Sep 22, 2026
Merged

davidsbatista merged 2 commits into
deepset-ai:mainfrom
ege-arhan:fix-llm-ranker-none-query

Conversation

@ege-arhan

Copy link
Copy Markdown
Contributor

Pipeline wiring can hand LLMRanker a None query when an upstream component emits nothing. Both run and run_async called query.strip() unconditionally, so that turned into AttributeError mid-pipeline.

The guard now treats anything that isn't a string the same as the existing empty-query path: log the warning and hand the deduplicated documents back untouched. No LLM call happens. The sync and async paths share the behavior, and I added regression coverage for None (both paths) plus a non-string int case on run.

Ran the ranker suite (49 passed, 3 live deselected) and ruff check/format clean on the touched files.

@ege-arhan
ege-arhan requested a review from a team as a code owner September 18, 2026 07:24
@ege-arhan
ege-arhan requested review from davidsbatista and removed request for a team September 18, 2026 07:24
@vercel

vercel Bot commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

@ege-arhan is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant

CLAassistant commented Sep 18, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@feiiiiii5 feiiiiii5 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified this in two throwaway checkouts of the repo (one at the PR base 52df9672a7, one at head dcad1fd048), with the package installed editable from each tree, Python 3.11, macOS. Only test/components/rankers/test_llm_ranker.py was run — not the full suite, and no mypy/hatch run.

The bug is real and reachable. At the base commit, calling LLMRanker.run with query=None fails exactly as the release note says:

E  AttributeError: 'NoneType' object has no attribute 'strip'
   haystack/components/rankers/llm_ranker.py:256

and with an int query, AttributeError: 'int' object has no attribute 'strip' at the same line.

The new tests do exercise the change rather than restate it. Copying this PR's test file into the base tree gives:

base 52df9672a7 -> 3 failed, 46 passed, 3 skipped
  test_run_non_string_query_returns_fallback[None]
  test_run_non_string_query_returns_fallback[123]
  TestLLMRankerAsync::test_run_async_none_query_returns_fallback
this head dcad1fd048 -> 49 passed, 3 skipped

which matches the "49 passed, 3 live deselected" you reported, and the three failures line up one-to-one with the two guarded call sites.

Two notes, neither blocking:

  1. For a non-string query the log line still reads Empty query provided to LLMRanker. Returning documents without reranking. (llm_ranker.py:257 and :324). Someone chasing a miswired upstream component will see "empty query" and look at the wrong thing — including the received type, e.g. f"Ignoring non-string query of type {type(query).__name__}..." for that branch, makes the warning actionable. Feel free to skip it.
  2. I checked whether the fallback (rather than raising) is the right shape, because query is typed str and a None here means something upstream is broken. Reusing the existing empty-query path is the consistent choice: that path already logs and returns documents unranked, and both call sites now behave the same. Raising would be a behaviour change for the empty-string case too, so it is a bigger question than this PR — flagging it only so a reviewer does not ask you to redesign mid-review.

Disclosure, per CONTRIBUTING's AI-assistance note: this comment was written by an AI agent working autonomously for @feiiiiii5, and no human read it before posting. Every number above is copied from the runs described, not summarised from memory; the checkouts, the single test module and its limits are stated so anyone can repeat or contradict it.

@vercel

vercel Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
haystack-docs Ignored Ignored Preview Sep 22, 2026 8:43am UTC

Request Review

@github-actions github-actions Bot added the type:documentation Improvements on the docs label Sep 21, 2026
@davidsbatista
davidsbatista enabled auto-merge (squash) September 21, 2026 15:07
auto-merge was automatically disabled September 22, 2026 07:13

Head branch was pushed to by a user without write access

@HaystackBot

Copy link
Copy Markdown
Contributor

Hi @ege-arhan, thanks a lot for your contribution! 🙏

We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.

To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.

@HaystackBot HaystackBot added the cla-pending PR is in draft until the contributor signs the CLA label Sep 22, 2026
@HaystackBot
HaystackBot marked this pull request as draft September 22, 2026 07:32
@ege-arhan
ege-arhan force-pushed the fix-llm-ranker-none-query branch from db06a0c to d85f05b Compare September 22, 2026 07:42
@HaystackBot
HaystackBot marked this pull request as ready for review September 22, 2026 07:52
@HaystackBot HaystackBot removed the cla-pending PR is in draft until the contributor signs the CLA label Sep 22, 2026
@HaystackBot

Copy link
Copy Markdown
Contributor

Thanks for signing the CLA, @ege-arhan! 🎉 This PR is now ready for review again and the reviewer has been re-assigned.

@github-actions

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/components/rankers
  llm_ranker.py
Project Total  

This report was generated by python-coverage-comment-action

@davidsbatista
davidsbatista merged commit 6df20e1 into deepset-ai:main Sep 22, 2026
30 checks passed
@ege-arhan
ege-arhan deleted the fix-llm-ranker-none-query branch September 22, 2026 10:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants