Fix decimal scalar Mul/Div to rescale at the shared scale#8721
Conversation
checked_binary_numeric applied raw integer ops, so Mul results were off by 10^scale and Div dropped the scale entirely. Mul now rescales the raw product from 2s back to s and Div pre-scales the dividend, both truncating toward zero. Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Merging this PR will not alter performance
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
|
Superseded by the per-operator restructure starting at #8724. The scalar Mul and Div fixes from this PR will land with the decimal Mul and Div kernel PRs respectively, since each fix is only observable through the op it corrects. |
What
DecimalScalar::checked_binary_numericapplied the checked operator to the raw stored integers with no scale handling, so Mul results were off by10^scale(its own test asserted0.50 × 0.10 = "5.00"; mathematically0.05) and Div dropped the scale entirely (10.00 / 0.10 = "1.00"instead of100.00).This fixes Mul and Div to fixed-point semantics at the shared scale, truncating toward zero:
2s) is rescaled back tosvia a new truncating rescale helper. The product is computed in i256, so it can exceed the operand storage width as long as the rescaled result fits the precision.(a · 10^s) / bfors ≥ 0,a / (b · 10^{-s})for negative scales; division by zero returnsNone.The raw
DecimalValue::checked_mul/checked_divremain scale-oblivious and are now documented as such, pointing at the newchecked_mul_fixed_point/checked_div_fixed_point.The only production callers of the scalar op use Add (sum aggregate, stats), so this is a safe bug fix. It is also a prerequisite for native decimal array arithmetic (follow-up PR), whose conformance tests compare element-wise against this scalar implementation.
Note: the truncating rescale helper takes
i16scales because intermediate scales (e.g.2sfor the raw product) can exceed thei8range.Testing
0.15 × 0.15 → 0.02, toward-zero for negatives), negative-scale, wider-than-storage, and precision-overflow cases.cargo nextest run -p vortex-array,cargo clippy --all-targets --all-features,cargo +nightly fmt --all.🤖 Generated with Claude Code