perf: Optimize array_has() for scalar needle#20374
Merged
Merged
Conversation
The previous implementation tested the cost of building an array_has() `Expr` (!), not actually evaluating the array_has() operation itself. Refactor things along the way.
Contributor
Author
|
Benchmarks: |
getChan
approved these changes
Feb 16, 2026
Jefffrey
approved these changes
Feb 19, 2026
Jefffrey
left a comment
Jefffrey
left a comment
Contributor
There was a problem hiding this comment.
Nice improvement; I have some potential ideas for further optimization, but these could be done in followups (if they are valid)
| for (i, (start, end)) in haystack.offsets().tuple_windows().enumerate() { | ||
| let length = end - start; | ||
| let offsets: Vec<usize> = haystack.offsets().collect(); | ||
| let mut matches = eq_bits.set_indices().peekable(); |
Contributor
There was a problem hiding this comment.
I wonder if we could get further gains by using set_slices() instead?
Contributor
Author
There was a problem hiding this comment.
Interesting idea. Most of the time, the bitmap should be sparse, so I'd guess that set_slices() won't be faster. It would also be a bit trickier to write the logic to handle contiguous slices of set bits. I'd prefer to defer this for now if that's okay.
Contributor
There was a problem hiding this comment.
Most of the time, the bitmap should be sparse, so I'd guess that
set_slices()won't be faster.
That sounds reasonable; we can keep this current approach 👍
Co-authored-by: Jeffrey Vo <jeffrey.vo.australia@gmail.com>
neilconway
changed the title
array_has() for scalar needle
Contributor
Author
|
Updated benchmark run: |
Contributor
|
Thanks @neilconway & @getChan 🚀 |
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Mar 4, 2026
## Which issue does this PR close? N/A ## Rationale for this change In #20374, `array_has` with a scalar needle was optimized to reconstruct matches more efficiently. Unfortunately, that code was incorrect for sliced arrays: `values()` returns the entire value buffer (including elements outside the visible slice), so we need to skip the corresponding indexes in the result bitmap. We could fix this by just skipping indexes, but it seems more robust and efficient to arrange to not compare the needle against elements outside the visible range in the first place. `array_position` has a similar behavior (introduced in #20532): it didn't have the buggy behavior, but it still did extra work for sliced arrays by comparing against elements outside the visible range. Benchmarking the revised code, there is no performance regression for unsliced arrays. ## What changes are included in this PR? * Fix `array_has` bug for sliced arrays with scalar needle * Improve `array_has` and `array_position` to not compare against elements outside the visible range of a sliced array * Add unit test for `array_has` bug * Add unit test to increase confidence in `array_position` behavior for sliced arrays ## Are these changes tested? Yes. ## Are there any user-facing changes? No.
neilconway
added a commit
to neilconway/datafusion
that referenced
this pull request
Mar 4, 2026
) ## Which issue does this PR close? N/A ## Rationale for this change In apache#20374, `array_has` with a scalar needle was optimized to reconstruct matches more efficiently. Unfortunately, that code was incorrect for sliced arrays: `values()` returns the entire value buffer (including elements outside the visible slice), so we need to skip the corresponding indexes in the result bitmap. We could fix this by just skipping indexes, but it seems more robust and efficient to arrange to not compare the needle against elements outside the visible range in the first place. `array_position` has a similar behavior (introduced in apache#20532): it didn't have the buggy behavior, but it still did extra work for sliced arrays by comparing against elements outside the visible range. Benchmarking the revised code, there is no performance regression for unsliced arrays. ## What changes are included in this PR? * Fix `array_has` bug for sliced arrays with scalar needle * Improve `array_has` and `array_position` to not compare against elements outside the visible range of a sliced array * Add unit test for `array_has` bug * Add unit test to increase confidence in `array_position` behavior for sliced arrays ## Are these changes tested? Yes. ## Are there any user-facing changes? No.
de-bgunter
pushed a commit
to de-bgunter/datafusion
that referenced
this pull request
Mar 24, 2026
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes apache#20377. ## Rationale for this change `compare_with_eq()` checks for matching array elements via a single pass across the entire flat values buffer, which is reasonably fast. The previous implementation then determined per-row results by creating a BooleanArray slice for each row and calling `true_count()` to check for any matches. It turns out that that's quite a lot of per-row work. Instead, we use `BooleanBuffer::set_indices()` to iterate over the set bits in the comparison result in a single forward pass. We walk this iterator in lockstep with the row offsets to determine whether each row contains a match, which does much less work per-row. This can be substantially faster, especially for short arrays. For example, for 10-element arrays of int64, it is 5-10x faster than the previous approach. 10-element string arrays are 1.8-5x faster. The improvement is smaller but non-zero for larger arrays (e.g., ~1.2x faster for 500 element arrays). ## What changes are included in this PR? In addition to the optimization, this commit adjusts the `array_has` benchmark code to actually benchmark `array_has` evaluation (!). The previous benchmark just constructed an `Expr`. ## Are these changes tested? Yes. Passes existing tests. Performance validated via several benchmark runs. ## Are there any user-facing changes? No. --------- Co-authored-by: Jeffrey Vo <jeffrey.vo.australia@gmail.com>
de-bgunter
pushed a commit
to de-bgunter/datafusion
that referenced
this pull request
Mar 24, 2026
) ## Which issue does this PR close? N/A ## Rationale for this change In apache#20374, `array_has` with a scalar needle was optimized to reconstruct matches more efficiently. Unfortunately, that code was incorrect for sliced arrays: `values()` returns the entire value buffer (including elements outside the visible slice), so we need to skip the corresponding indexes in the result bitmap. We could fix this by just skipping indexes, but it seems more robust and efficient to arrange to not compare the needle against elements outside the visible range in the first place. `array_position` has a similar behavior (introduced in apache#20532): it didn't have the buggy behavior, but it still did extra work for sliced arrays by comparing against elements outside the visible range. Benchmarking the revised code, there is no performance regression for unsliced arrays. ## What changes are included in this PR? * Fix `array_has` bug for sliced arrays with scalar needle * Improve `array_has` and `array_position` to not compare against elements outside the visible range of a sliced array * Add unit test for `array_has` bug * Add unit test to increase confidence in `array_position` behavior for sliced arrays ## Are these changes tested? Yes. ## Are there any user-facing changes? No.
freakyzoidberg
added a commit
to freakyzoidberg/datafusion
that referenced
this pull request
Jun 29, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there (an O(1) check); the all-valid fold has no such crossover. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~13x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~10x (8-elem) ... ~1x (512-elem); falls back beyond string elements: ~1.45x Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
freakyzoidberg
added a commit
to freakyzoidberg/datafusion
that referenced
this pull request
Jul 1, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~13x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~10x (8-elem) ... ~1x (512-elem); falls back beyond string elements: ~1.45x Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
freakyzoidberg
added a commit
to DataDog/datafusion
that referenced
this pull request
Jul 1, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~13x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~10x (8-elem) ... ~1x (512-elem); falls back beyond string elements: ~1.45x Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
freakyzoidberg
added a commit
to freakyzoidberg/datafusion
that referenced
this pull request
Jul 1, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~13x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~10x (8-elem) ... ~1x (512-elem); falls back beyond string elements: ~1.45x Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
freakyzoidberg
added a commit
to DataDog/datafusion
that referenced
this pull request
Jul 1, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~13x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~10x (8-elem) ... ~1x (512-elem); falls back beyond string elements: ~1.45x Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
freakyzoidberg
added a commit
to freakyzoidberg/datafusion
that referenced
this pull request
Jul 1, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~13x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~10x (8-elem) ... ~1x (512-elem); falls back beyond string elements: ~1.45x Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
freakyzoidberg
added a commit
to DataDog/datafusion
that referenced
this pull request
Jul 1, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~13x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~10x (8-elem) ... ~1x (512-elem); falls back beyond string elements: ~1.45x Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
freakyzoidberg
added a commit
to freakyzoidberg/datafusion
that referenced
this pull request
Jul 1, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~13x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~10x (8-elem) ... ~1x (512-elem); falls back beyond string elements: ~1.45x Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
freakyzoidberg
added a commit
to freakyzoidberg/datafusion
that referenced
this pull request
Jul 1, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. For string elements each row is a single pass over the row's values; for `Utf8View` this is view-aware -- the byte length and 4-byte prefix packed in the 128-bit view reject non-matches before touching the data buffer (what the `eq` kernel does, but without its per-row allocation), and an inline needle (<= 12 bytes) is matched by full-view equality with no materialization at all. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~15x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~9x (8-elem) ... ~1x (512-elem); falls back beyond Utf8 / LargeUtf8: ~2.5x Utf8View (view-aware): ~5x on short/inline strings Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
freakyzoidberg
added a commit
to freakyzoidberg/datafusion
that referenced
this pull request
Jul 1, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. For string elements each row is a single pass over the row's values; for `Utf8View` this is view-aware -- the byte length and 4-byte prefix packed in the 128-bit view reject non-matches before touching the data buffer (what the `eq` kernel does, but without its per-row allocation), and an inline needle (<= 12 bytes) is matched by full-view equality with no materialization at all. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~15x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~9x (8-elem) ... ~1x (512-elem); falls back beyond Utf8 / LargeUtf8: ~2.5x Utf8View (view-aware): ~5x on short/inline strings Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
freakyzoidberg
added a commit
to DataDog/datafusion
that referenced
this pull request
Jul 1, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. For string elements each row is a single pass over the row's values; for `Utf8View` this is view-aware -- the byte length and 4-byte prefix packed in the 128-bit view reject non-matches before touching the data buffer (what the `eq` kernel does, but without its per-row allocation), and an inline needle (<= 12 bytes) is matched by full-view equality with no materialization at all. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~15x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~9x (8-elem) ... ~1x (512-elem); falls back beyond Utf8 / LargeUtf8: ~2.5x Utf8View (view-aware): ~5x on short/inline strings Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
freakyzoidberg
added a commit
to freakyzoidberg/datafusion
that referenced
this pull request
Jul 2, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. For string elements each row is a single pass over the row's values; for `Utf8View` this is view-aware -- the byte length and 4-byte prefix packed in the 128-bit view reject non-matches before touching the data buffer (what the `eq` kernel does, but without its per-row allocation), and an inline needle (<= 12 bytes) is matched by full-view equality with no materialization at all. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~15x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~9x (8-elem) ... ~1x (512-elem); falls back beyond Utf8 / LargeUtf8: ~2.5x Utf8View (view-aware): ~5x on short/inline strings Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. A criterion benchmark for the array-needle path is included, covering null patterns and element types (i64, Utf8/LargeUtf8/Utf8View at short and long lengths), list length, and row count. Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
freakyzoidberg
added a commit
to DataDog/datafusion
that referenced
this pull request
Jul 2, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. For string elements each row is a single pass over the row's values; for `Utf8View` this is view-aware -- the byte length and 4-byte prefix packed in the 128-bit view reject non-matches before touching the data buffer (what the `eq` kernel does, but without its per-row allocation), and an inline needle (<= 12 bytes) is matched by full-view equality with no materialization at all. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~15x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~9x (8-elem) ... ~1x (512-elem); falls back beyond Utf8 / LargeUtf8: ~2.5x Utf8View (view-aware): ~5x on short/inline strings Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. A criterion benchmark for the array-needle path is included, covering null patterns and element types (i64, Utf8/LargeUtf8/Utf8View at short and long lengths), list length, and row count. Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
freakyzoidberg
added a commit
to freakyzoidberg/datafusion
that referenced
this pull request
Jul 2, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. For string elements each row is a single pass over the row's values; for `Utf8View` this is view-aware -- the byte length and 4-byte prefix packed in the 128-bit view reject non-matches before touching the data buffer (what the `eq` kernel does, but without its per-row allocation), and an inline needle (<= 12 bytes) is matched by full-view equality with no materialization at all. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~15x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~9x (8-elem) ... ~1x (512-elem); falls back beyond Utf8 / LargeUtf8: ~2.5x Utf8View (view-aware): ~5x on short/inline strings Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. The array-needle criterion benchmark used for these numbers is added separately (so it can be run against `main` to capture the before baseline). Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jul 6, 2026
freakyzoidberg
added a commit
to freakyzoidberg/datafusion
that referenced
this pull request
Jul 6, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. For string elements each row is a single pass over the row's values; for `Utf8View` this is view-aware -- the byte length and 4-byte prefix packed in the 128-bit view reject non-matches before touching the data buffer (what the `eq` kernel does, but without its per-row allocation), and an inline needle (<= 12 bytes) is matched by full-view equality with no materialization at all. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~15x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~9x (8-elem) ... ~1x (512-elem); falls back beyond Utf8 / LargeUtf8: ~2.5x Utf8View (view-aware): ~5x on short/inline strings Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. The array-needle criterion benchmark used for these numbers is added separately (so it can be run against `main` to capture the before baseline). Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
freakyzoidberg
added a commit
to freakyzoidberg/datafusion
that referenced
this pull request
Jul 6, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. For string elements each row is a single pass over the row's values; for `Utf8View` this is view-aware -- the byte length and 4-byte prefix packed in the 128-bit view reject non-matches before touching the data buffer (what the `eq` kernel does, but without its per-row allocation), and an inline needle (<= 12 bytes) is matched by full-view equality with no materialization at all. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~15x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~9x (8-elem) ... ~1x (512-elem); falls back beyond Utf8 / LargeUtf8: ~2.5x Utf8View (view-aware): ~5x on short/inline strings Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. The array-needle criterion benchmark used for these numbers is added separately (so it can be run against `main` to capture the before baseline). Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
freakyzoidberg
added a commit
to freakyzoidberg/datafusion
that referenced
this pull request
Jul 14, 2026
`array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar -- the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter -- execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) Add a fast path for primitive and string element types, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match). With all-valid elements each row is a single branchless OR-reduction over the native values. When primitive elements contain nulls -- whose backing values are arbitrary -- the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before the per-row reduction, so null slots never match regardless of their value. Past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) this bitmap's extra passes lose to the per-row kernel, so the element-null branch bails to it there; that length is measured over the visible (sliced) region, so a sliced array's hidden child elements don't route a small window to the slow path. The all-valid fold has no such crossover. For string elements each row is a single pass over the row's values; for `Utf8View` this is view-aware -- the byte length and 4-byte prefix packed in the 128-bit view reject non-matches before touching the data buffer (what the `eq` kernel does, but without its per-row allocation), and an inline needle (<= 12 bytes) is matched by full-view equality with no materialization at all. Nested and other element types keep using the per-row `eq` kernel. What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. Criterion microbenchmark, array needle on a 100k-row batch vs the per-row `eq` kernel (NOT end-to-end query time): i64, all-valid elements: ~15x (64-elem lists), ~1.9x at 1024-elem i64, ~30% null elements: ~9x (8-elem) ... ~1x (512-elem); falls back beyond Utf8 / LargeUtf8: ~2.5x Utf8View (view-aware): ~5x on short/inline strings Every shape improves with no regression. End-to-end impact depends on how much of a query `array_has` accounts for. For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction -- e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost) -- the overall speedup is single-digit percent. The array-needle criterion benchmark used for these numbers is added separately (so it can be run against `main` to capture the before baseline). Part of apache#18727. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mkleen
pushed a commit
to mkleen/datafusion
that referenced
this pull request
Jul 15, 2026
## Which issue does this PR close? - Part of apache#23334. > The numbers below come from the committed criterion benchmark added in apache#23335 (`cargo bench --bench array_has`) — **origin** = the per-row `eq` kernel (unoptimized `main` / apache#23335), **now** = with this optimization applied. Run the bench on `main` and on this branch to reproduce. Full disclosure - this was heavily assisted by AI, and I did my best to understand and justify every change here before submitting. ## Rationale for this change `array_has(array, element)` returns, for each row, whether the array contains the element. When the `element` (needle) is an array rather than a scalar, the needle argument is a column with one value per row, e.g. `array_has(t1.tags, t2.key)` in a join filter, execution goes through `array_has_dispatch_for_array` (the `ColumnarValue::Array` needle branch), which compared each row by invoking the Arrow `eq` kernel once per row. That kernel allocates a `BooleanArray` and pays downcast and dispatch overhead on every row. (The scalar-needle branch was optimized separately in apache#20374.) What this removes is the fixed per-row kernel overhead, not the element comparison itself, so the gain is largest for short lists and shrinks as lists grow. All numbers below are from the committed criterion benchmark (`cargo bench --bench array_has`, groups `array_has_array_null_patterns` / `array_has_array_by_size` / `array_has_array_by_rows`): the `array_has` UDF evaluated in isolation with an array needle, **origin** (the per-row `eq` kernel) vs **now**. "list length" is the number of elements in each row's array (not the row count). Not end-to-end query time. ### By data type and null pattern (list length 64, 10K rows) | element | element len | null pattern | origin | now | speedup | |-----------|----------------|----------------------|---------|---------|---------| | i64 | - | no nulls, found | 1.10 ms | 73 µs | 15.1x | | i64 | - | no nulls, not found | 1.07 ms | 72 µs. | 14.9x | | i64 | - | 30% nulls, found | 1.17 ms | 315 µs | 3.7x | | i64 | - | 30% nulls, not found | 1.10 ms | 274 µs | 4.0x | | i64 | - | all null | 1.10 ms | 272 µs | 4.0x | | i64 | - | collision | 1.10 ms | 270 µs | 4.1x | | Utf8 | short (inline) | no nulls | 2.57 ms | 1.01 ms | 2.5x | | Utf8 | short (inline) | 30% nulls | 3.37 ms | 1.52 ms | 2.2x | | Utf8 | long (>12B) | no nulls | 2.61 ms | 1.04 ms | 2.5x | | Utf8 | long (>12B) | 30% nulls | 3.31 ms | 1.52 ms | 2.2x | | Utf8 | - | all null | 1.26 ms | 256 µs | 4.9x | | LargeUtf8 | short (inline) | no nulls | 2.56 ms | 1.02 ms | 2.5x | | LargeUtf8 | short (inline) | 30% nulls | 3.20 ms | 1.54 ms | 2.1x | | LargeUtf8 | long (>12B) | no nulls | 2.67 ms | 1.05 ms | 2.6x | | LargeUtf8 | long (>12B) | 30% nulls | 3.42 ms | 1.59 ms | 2.2x | | LargeUtf8 | - | all null | 1.31 ms | 263 µs | 5.0x | | Utf8View | short (inline) | no nulls | 1.18 ms | 239 µs | 4.9x | | Utf8View | short (inline) | 30% nulls | 1.26 ms | 246 µs | 5.1x | | Utf8View | long (>12B) | no nulls | 2.86 ms | 1.17 ms | 2.4x | | Utf8View | long (>12B) | 30% nulls | 3.51 ms | 1.66 ms | 2.1x | | Utf8View | - | all null | 1.20 ms | 267 µs | 4.5x | The i64 null cases are uniform (~4x) whether the match is present, absent, the whole list is null, or the needle collides with a null slot's backing fill value — validity is folded in with one word-parallel op, so there is no per-row rescan and no null slot can match. Strings win ~2.1–2.5x mainly by dropping the per-row `BooleanArray` allocation. `Utf8View` additionally uses a view-aware compare: the byte length and 4-byte prefix packed into the 128-bit view reject non-matches before touching the data buffer, and an inline value (≤ 12 bytes) is matched by whole-view equality with no materialization at all — hence ~5x on short/inline strings. When long strings share a prefix (e.g. ARNs) the prefix can't reject, so `Utf8View` falls in line with the other string types (~2.1–2.4x). No string case regresses. ### By list length (i64, 30% element nulls, not found, 10K rows) | elems/row | origin | now | speedup | |-----------|---------|---------|--------------------------------------| | 8 | 1.03 ms | 111 µs | 9.3x | | 32 | 1.07 ms | 197 µs | 5.5x | | 128 | 1.18 ms | 446 µs | 2.6x | | 256 | 1.28 ms | 780 µs | 1.6x | | 512 | 1.54 ms | 1.44 ms | 1.1x | | 1024 | 2.17 ms | 2.15 ms | 1.0x (falls back to per-row kernel) | The element-null branch makes a few passes over the values; past a moderate average list length (`NULL_FAST_PATH_MAX_LEN`) the per-row kernel wins, so it bails to it there — no meaningful regression. That average is measured over the visible (sliced) region, so a sliced array's hidden child elements can't route a small window to the slow path. The all-valid fold has no such crossover. ### By row count (i64, 8 elems/row, 30% nulls, not found) | rows | origin | now | speedup | |------|-----------|----------|---------| | 10K | 1.04 ms | 111 µs | 9.4x | | 100K | 10.42 ms | 1.09 ms | 9.6x | | 1M | 102.68 ms | 10.91 ms | 9.4x | Invariant to the number of rows — the per-row overhead removed is a fixed cost, so absolute savings scale linearly with the column height. The remaining benchmarks in the suite (scalar `array_has`, `array_has_all`, `array_has_any` — paths this PR does not touch) are unchanged (median 0.99x, within measurement noise), confirming no regression outside the array-needle path. ### End-to-end (context) For a query dominated by an array-needle `array_has` join filter (a `NestedLoopJoinExec` with `filter=array_has(tags, key)` over 3000x3000 rows of 8-element lists) total time drops from 0.95s to 0.059s (~16x, identical results). For a workload where `array_has` is a smaller fraction, e.g. the ~6% of profile that motivated this (see apache#18070 / apache#18161, which fixed the join's deep-copy but left the per-row `array_has` cost), the overall speedup is single-digit percent. ## What changes are included in this PR? A fast path for primitive and string element types in `array_has_dispatch_for_array`, preserving the Arrow `eq` kernel semantics (total-order float equality; null elements never match): - **All-valid elements:** each row is a single branchless OR-reduction over the raw native value slice (auto-vectorizes; the common case). - **Element nulls:** a null slot's backing value is arbitrary, so the per-element equality bitmap is ANDed with the validity bitmap (one word-parallel op, no per-element branch) before reducing each row to "any bit set", a null slot can never match regardless of its value. This branch is processed in row chunks so the scratch buffer stays bounded, and past `NULL_FAST_PATH_MAX_LEN` average elements/row a length check over the visible (sliced) region bails to the per-row kernel (see the list-length table). - **String elements:** each row is a single pass over the row's values (compare, then consult validity only on a match). `Utf8View` compares the packed 128-bit views directly — length + 4-byte prefix reject non-matches before any data-buffer access, and an inline value (≤ 12 bytes) matches by whole-view equality with no materialization. - **Nested (and any other) element types** keep using the per-row `eq` kernel. The array-needle benchmarks used for the numbers above are added in apache#3 (null patterns, list length, and row count). ## Are these changes tested? Yes: - New unit tests for the array-needle path covering element nulls, the null-fill collision (needle equal to a null slot's backing value), total-order float equality (`NaN` / `-0.0`), sliced arrays (including a small visible window over a large backing child), `LargeList` offsets, empty rows, a multi-chunk input, and a long-list input that exercises the per-row fallback, each cross-checked against the original per-row `eq` kernel as an oracle. - Existing `array_has` / `array_contains` / `join_lists` sqllogictest suites pass. ## Are there any user-facing changes? No. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
array_has()for scalar needle #20377.Rationale for this change
compare_with_eq()checks for matching array elements via a single pass across the entire flat values buffer, which is reasonably fast. The previous implementation then determined per-row results by creating a BooleanArray slice for each row and callingtrue_count()to check for any matches. It turns out that that's quite a lot of per-row work.Instead, we use
BooleanBuffer::set_indices()to iterate over the set bits in the comparison result in a single forward pass. We walk this iterator in lockstep with the row offsets to determine whether each row contains a match, which does much less work per-row.This can be substantially faster, especially for short arrays. For example, for 10-element arrays of int64, it is 5-10x faster than the previous approach. 10-element string arrays are 1.8-5x faster. The improvement is smaller but non-zero for larger arrays (e.g., ~1.2x faster for 500 element arrays).
What changes are included in this PR?
In addition to the optimization, this commit adjusts the
array_hasbenchmark code to actually benchmarkarray_hasevaluation (!). The previous benchmark just constructed anExpr.Are these changes tested?
Yes. Passes existing tests. Performance validated via several benchmark runs.
Are there any user-facing changes?
No.