Add native decimal arithmetic kernels#8722
Conversation
Mechanical move mirroring the compare split: dtype-generic lane machinery stays in numeric/mod.rs (bounds relaxed to T: Default so wider decimal types can reuse it), primitive-specific code moves to numeric/primitive.rs, tests to numeric/tests.rs. Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Add/Sub/Mul/Div over decimal arrays sharing a decimal dtype, executed as fixed-point arithmetic at the shared scale (truncating toward zero) in a working width wide enough that in-precision inputs cannot spuriously overflow intermediates. Overflow, precision violations, and division by zero error only on valid lanes, matching primitive semantics. widened_buffer moves to arrays/decimal so compare and numeric share it. Signed-off-by: Matt Katz <mhkatz97@gmail.com>
The conformance harness now accepts decimal arrays, checking array results element-wise against DecimalScalar::checked_binary_numeric for representative constants. Wire it into the decimal, chunked, and decimal-byte-parts compute tests, and add decimal cases to the binary_ops benchmark. Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Merging this PR will not alter performance
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
|
Superseded by the per-operator restructure: #8724 lands the infrastructure plus Add/Sub; Mul and Div follow as separate PRs, each bundled with its scalar fix. The full implementation from this PR is preserved on |
Adds array-level Add/Sub/Mul/Div over decimal arrays. Previously decimal arithmetic was rejected at two gates (
Binary::return_dtyperequires a primitive lhs;execute_numericstarts withPType::try_from), and there is no Arrow fallback — it simply errored.Stacked on #8721 (the scalar Mul/Div fix), which the conformance tests compare against.
Semantics: same-dtype fixed-point, truncating toward zero
Both operands must share one
DecimalDType(as compare requires since #8661;coerce_args/least_supertypealready coerce to a common decimal dtype), and the result keeps that dtype with unioned nullability. On stored integers at scales, precisionp:a ± b, checkeda × brescaled from2sback tos, truncating toward zero(a · 10^s) / b(s < 0:a / (b · 10^{-s})), integer truncationplus a
fits_in_precisionrange check. Overflow, precision violations, and division by zero error only on valid lanes; invalid lanes never error — matching the primitive kernel. SQL-style type-growth semantics (DataFusion/Spark) compose on top of these kernels via input casts in the engine conversion layers, planned as a follow-up that needs no kernel change.Implementation
numeric.rsis split intonumeric/{mod,primitive,decimal,tests}.rs, mirroring the compare split in Implement compare in vortex instead of falling back to arrow #8661. The generic checked lane loops move tonumeric/mod.rswith bounds relaxed toT: Defaultsoi256qualifies.numeric/decimal.rsfollowscompare/decimal.rs: operands canonicalize throughexecute::<DecimalArray>(so Chunked, DecimalByteParts, Dict, … work with no kernel registration) or extract constants, then widen once to a working type and run one-pass checked lanes.2pdigits (+|s|for negative scales), Divp + |s|, capped at i256. Near the 76-digit cap a raw product that overflows i256 errors even if the rescaled result would fit — documented limitation, only reachable forp > 38Mul.DecimalDataexplicitly supports wider-than-necessary storage, and this avoids a narrowing pass. Happy to add aBigCastnarrowing pass if preferred.DecimalScalar::checked_binary_numericfor constants0, 1, -1, 10^s, MAX_BY_PRECISION[p]in both operand orders; wired into the decimal, chunked, and decimal-byte-parts compute tests.widened_buffermoves fromcompare/decimal.rstoarrays/decimalso compare and numeric share it.Testing
60+60at p=2 in i8), constants on either side (incl. nullable/null/out-of-width constants), i256 working width, i256-intermediate-overflow at p=76, negative scales, empty, constant folding.add_decimal_i64,add_decimal_i128_nullable,mul_decimal_i64(widens to i128),div_decimal_i128_constantinbinary_ops.rs.cargo nextest run -p vortex-array(3130 passed),-p vortex-decimal-byte-parts(25 passed),cargo clippy --all-targets --all-features,cargo test --doc -p vortex-array,cargo +nightly fmt --all.🤖 Generated with Claude Code