Skip to content

fix(array): don't panic on unsupported arrow types#8564

Merged
AdamGS merged 1 commit into
vortex-data:developfrom
miniex:fix/arrow-unsupported-dtype
Jun 24, 2026
Merged

fix(array): don't panic on unsupported arrow types#8564
AdamGS merged 1 commit into
vortex-data:developfrom
miniex:fix/arrow-unsupported-dtype

Conversation

@miniex

@miniex miniex commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Summary

DType::from_arrow hit the unimplemented! fallback arm for unsupported Arrow types (Duration, Interval, FixedSizeBinary), which surfaced in Python as a PanicException instead of a clean error. I added a fallible TryFromArrowType conversion that returns a VortexError on that arm, kept the infallible FromArrowType as a thin delegate for existing trusted callers, and routed the ArrowSession leaf path and the Python entry points through it so unsupported types raise a clean ValueError.

The error keeps the not supported marker so it lands in the fuzzer's existing unsupported-type skip path; the Duration/FixedSizeBinary generation exclusions can be lifted once the fuzzer is on develop.

Closes: #8346

Testing

Added an rstest in vortex-array/src/dtype/arrow.rs checking Duration/Interval/FixedSizeBinary error cleanly as a leaf and nested in a field/schema, plus a parametrized regression test in vortex-python/test/test_array.py for the issue's repro; both of the issue's Python snippets now raise ValueError instead of panicking. cargo nextest run -p vortex-array (3060 tests) and uv run --all-packages pytest vortex-python/test/test_array.py pass; fmt --all, clippy --all-targets --all-features, cargo doc, basedpyright and ruff are clean.


I'm Korean, so sorry if any wording reads a little awkward.

`DType::from_arrow` hit `unimplemented!` for unsupported arrow types (duration,
interval, fixed-size binary), surfacing as a panic. add a fallible
`TryFromArrowType` conversion that returns a `VortexError`, and route the session
and python entry points through it so unsupported types raise a clean `ValueError`
instead.

Closes vortex-data#8346

Signed-off-by: Han Damin <miniex@daminstudio.net>
@miniex miniex requested a review from a team June 24, 2026 02:01

@AdamGS AdamGS left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems good, just running CI to see if everything else is good.

@AdamGS AdamGS added the changelog/fix A bug fix label Jun 24, 2026
@codspeed-hq

codspeed-hq Bot commented Jun 24, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 2 improved benchmarks
❌ 2 regressed benchmarks
✅ 1581 untouched benchmarks
⏩ 4 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation chunked_varbinview_canonical_into[(1000, 10)] 154.8 µs 190.9 µs -18.93%
Simulation slice_empty_vortex 310 ns 368.3 ns -15.84%
Simulation chunked_varbinview_into_canonical[(1000, 10)] 204.4 µs 168.7 µs +21.17%
Simulation bitwise_not_vortex_buffer_mut[128] 273.6 ns 244.4 ns +11.93%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing miniex:fix/arrow-unsupported-dtype (136c0a6) with develop (4668b6e)

Open in CodSpeed

Footnotes

  1. 4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@robert3005

robert3005 commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Shouldn't this be migrated to the session arrow conversion? it's a bigger change

@AdamGS AdamGS merged commit 2a19323 into vortex-data:develop Jun 24, 2026
78 of 81 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DType::from_arrow panics with unimplemented! for Duration, Interval and FixedSizeBinary instead of returning an error

3 participants