Skip to content

fix(metrics): let _mean ignore unmeasured scores like its siblings - #4510

Open
sclfcz wants to merge 1 commit into
Unstructured-IO:mainfrom
sclfcz:fix/mean-ignores-none-scores
Open

sclfcz wants to merge 1 commit into
Unstructured-IO:mainfrom
sclfcz:fix/mean-ignores-none-scores

Conversation

@sclfcz

@sclfcz sclfcz commented Sep 30, 2026 •

Copy link
Copy Markdown

Fixes #4509

What

_mean documents returning None when there is nothing to average, but unlike its neighbours _stdev and _pstdev it does not drop None values before averaging, so a score list with an unmeasured entry raised instead of being averaged:

call before after
_mean([0.9, None, 0.8]) TypeError: can't convert type 'NoneType' to numerator/denominator 0.85
_mean([None, None]) TypeError None
_mean([]) None unchanged
_mean([1.0, 2.0]) 1.5 unchanged

_stdev([0.9, None, 0.8]) → 0.071 and _pstdev → 0.05 already filter those values, and their type hint is List[Optional[float]]; _mean now matches them (hint included).

Testing

Added test_mean_ignores_unmeasured_scores to test_unstructured/metrics/test_utils.py covering the four rows above.

pytest test_unstructured/metrics/test_utils.py → 6 passed; reverting the filter fails exactly the new test and leaves the other five passing.

Review in cubic

_mean documents returning None when there is nothing to average, but it does not
drop None values the way _stdev and _pstdev do, so a score list containing an
unmeasured entry raised TypeError from statistics.mean instead of averaging the
scores that were measured.

@chrikrah chrikrah left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sclfcz I would merge this. The note below is non-blocking.

non-blocking: real callers aggregate a pandas column, so a missing score arrives as NaN, not None, and _stdev raises before _mean is reached. get_mean_grouping with one blank cct-accuracy cell fails identically before and after this change:

$ python probe_group.py   # 3 rows, cct-accuracy [0.9, None, 0.8], group_by="doctype"; torch and unstructured_inference stubbed
main ddf4453:  raised: ValueError inf or nan encountered in data
PR   43867d0:  raised: ValueError inf or nan encountered in data
# Python 3.13.15, pandas 2.3.3

Would you like pd.notna in _mean, _stdev and _pstdev as part of this change, or kept for a follow-up?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

_mean raises TypeError on a score list containing None, unlike _stdev and _pstdev

2 participants