Skip to content

Commit 1983656

Browse files
dfa1claude
andcommitted
fix(cli): uber-jar bundles libzstd natives for all platforms
The CLI is an application, not a library: it shipped the FFM binding's classes but not the natives, silently depending on a system libzstd. zstd-platform (runtime) now rides in the shade — osx/linux/windows x x86_64/aarch64 all present in the jar, and the uber-jar reads a vortex.zstd file end-to-end (verified via count and export). Library modules keep zstd as the documented two- artifact opt-in. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 02123e5 commit 1983656

3 files changed

Lines changed: 13 additions & 0 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ claim measured against the Rust (JNI) oracle; behavior divergences documented in
3131

3232
### Changed
3333

34+
- The CLI uber-jar is now fully self-contained for `vortex.zstd` files: it bundles the FFM binding's native `libzstd` for all six platforms (osx/linux/windows × x86_64/aarch64) via `zstd-platform` — previously it shipped the binding's classes but relied on a system libzstd. The library modules keep zstd optional. ([pending-cli](https://github.com/dfa1/vortex-java/commits/main))
3435
- The zstd binding stays a two-artifact opt-in (`io.github.dfa1.zstd:zstd` + `zstd-platform`, both `optional`; no JNI anywhere), and touching `vortex.zstd` without it now fails with an actionable `VortexException` naming the two artifacts to add, instead of a raw `NoClassDefFoundError`. Default write/read paths never touch the binding (measured). ([ae9f95cc](https://github.com/dfa1/vortex-java/commit/ae9f95cc))
3536
- The little-endian `ValueLayout` constants moved from `PTypeIO.LE_*` to `VortexFormat.LE_*` — endianness is a property of the wire format, not of ptypes, and `VortexFormat` is where format facts live. Six classes that carried private copies (including reversed-name duplicates like `SHORT_LE`) now share the single source; nothing outside `VortexFormat` defines a `withOrder(LITTLE_ENDIAN)` layout. ([c060d34f](https://github.com/dfa1/vortex-java/commit/c060d34f))
3637
- `Chunk.columns()` returns an order-preserving `SequencedMap<ColumnName, Chunk.Column>` — one map instead of two parallel string-keyed maps, with each column's `Array` and `DType` traveling together in the `Column` carrier. `column(String)` stays as boundary sugar (plus a `column(ColumnName)` overload); iteration order is the schema/projection order, now guaranteed even for 1–2 column chunks. Typed names originate in `ScanIterator` from the file's already-certified schema. ([f8ad15d1](https://github.com/dfa1/vortex-java/commit/f8ad15d1))

cli/pom.xml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,14 @@
3737
<groupId>io.github.dfa1.zstd</groupId>
3838
<artifactId>zstd</artifactId>
3939
</dependency>
40+
<dependency>
41+
<!-- The CLI is an application, not a library: the uber-jar must read vortex.zstd
42+
files on a machine with no system libzstd, so the all-platform natives the
43+
FFM binding loads are bundled here (the library modules keep zstd optional). -->
44+
<groupId>io.github.dfa1.zstd</groupId>
45+
<artifactId>zstd-platform</artifactId>
46+
<scope>runtime</scope>
47+
</dependency>
4048
<dependency>
4149
<!-- Required by hardwood-core (via vortex-parquet) for ZSTD-compressed
4250
Parquet files; listed here explicitly so the shade plugin bundles

docs/reference.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -278,6 +278,10 @@ CSV import is CLI-only — types are inferred from the data.
278278

279279
## CLI
280280

281+
The `-all` uber-jar is self-contained: pure-Java code plus the native `libzstd` for all
282+
supported platforms (the FFM loader picks the right one at runtime) — no system libraries
283+
required.
284+
281285
The `cli` module ships a fat jar with subcommands for inspecting and querying Vortex files.
282286

283287
```bash

0 commit comments

Comments
 (0)