Skip to content

perf: optimize cast_decimal128_to_utf8 in datafusion-comet-spark-expr#4924

Open
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:auto-opt/cast_decimal128_to_utf8-datafusion-comet-20260714-114428
Open

perf: optimize cast_decimal128_to_utf8 in datafusion-comet-spark-expr#4924
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:auto-opt/cast_decimal128_to_utf8-datafusion-comet-20260714-114428

Conversation

@andygrove

@andygrove andygrove commented Jul 14, 2026

Copy link
Copy Markdown
Member

Which issue does this PR close?

N/A

Rationale for this change

Optimize existing expression.

What changes are included in this PR?

Replaced the per-row u128 to_string heap allocation, format! machinery, and "0".repeat in the LEGACY Decimal128-to-string cast with an allocation-free stack digit buffer that renders the coefficient in 19-digit 64-bit chunks and appends slices directly to the reused output buffer.

How are these changes tested?

Existing tests + expanded unit tests.

Benchmark (criterion):

  • decimal128_leading_zeroes: 73.764% faster (base 197033ns -> cand 51693ns)
  • decimal128_wide: 20.876% faster (base 198585ns -> cand 157128ns)
  • decimal128_scientific: 49.361% faster (base 172590ns -> cand 87397ns)
  • decimal128_scale_2: 52.624% faster (base 155170ns -> cand 73513ns)

Full criterion output:

cast_decimal_to_string/decimal128_scale_2
                        time:   [73.783 µs 73.913 µs 74.059 µs]
                        change: [−52.799% −52.624% −52.452%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
cast_decimal_to_string/decimal128_leading_zeroes
                        time:   [51.381 µs 51.495 µs 51.615 µs]
                        change: [−73.817% −73.764% −73.700%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 22 outliers among 100 measurements (22.00%)
  16 (16.00%) low severe
  1 (1.00%) low mild
  3 (3.00%) high mild
  2 (2.00%) high severe
cast_decimal_to_string/decimal128_wide
                        time:   [157.06 µs 157.33 µs 157.66 µs]
                        change: [−20.987% −20.876% −20.752%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe
cast_decimal_to_string/decimal128_scientific
                        time:   [87.563 µs 87.699 µs 87.832 µs]
                        change: [−49.456% −49.361% −49.272%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) low severe
  1 (1.00%) high mild

@andygrove andygrove marked this pull request as ready for review July 14, 2026 18:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant