feat(core,reader,writer): strict field-name policy both ways

dfa1 · claude · dfa1 · commit c993a3552f95 · 2026-07-04T17:52:32.000+02:00
Wire-legal is a floor, not a policy: like a JSON library refusing
to write the "" key it could technically parse, vortex-java now
rejects footgun field names on BOTH sides of the boundary.

- StructBuilder.field and VortexWriter reject blank names and
  control characters (IllegalArgumentException; NUL additionally
  SIGABRTs the reference toolchain's Arrow FFI, so it must never
  be written).
- PostscriptParser rejects files carrying blank or control
  character field names (VortexException naming the producing
  pipeline as the likely bug) — a loud, actionable failure beats
  propagating unusable names into name-keyed APIs and SQL
  identifiers.
- Printable names of any shape ($$$$$, spaces inside, emoji)
  remain legal and round-trip intact both directions (measured
  against vortex-jni; pinned by ColumnNameEdgeCasesIntegrationTest,
  which also pins that a Rust-written "" file is rejected by
  policy).

Deliberate divergence documented in docs/compatibility.md.

Co-Authored-By: Claude Fable 5 &lt;noreply@anthropic.com&gt;
diff --git a/core/src/main/java/io/github/dfa1/vortex/core/model/DType.java b/core/src/main/java/io/github/dfa1/vortex/core/model/DType.java
@@ -164,11 +164,28 @@ private StructBuilder() {
 
         /// Adds a named field to the struct under construction.
         ///
-        /// @param name the field name; must be non-`null` and not previously added
+        /// Field names follow the write-side policy, stricter than the wire format on purpose:
+        /// blank names and control characters are rejected as footguns (a foreign file may
+        /// carry them and the reader tolerates it, but vortex-java refuses to produce them —
+        /// same spirit as a JSON library refusing to write a `""` key it would happily parse).
+        ///
+        /// @param name the field name; non-`null`, non-blank, no control characters, not
+        ///             previously added
         /// @param type the field type
         /// @return this builder
-        /// @throws IllegalArgumentException if `name` duplicates a previously added field
+        /// @throws IllegalArgumentException if `name` is blank, contains a control character,
+        ///         or duplicates a previously added field
         public StructBuilder field(String name, DType type) {
+            if (name.isBlank()) {
+                throw new IllegalArgumentException(
+                        "blank field name (wire-legal, but a footgun vortex-java refuses to write)");
+            }
+            for (int i = 0; i < name.length(); i++) {
+                if (Character.isISOControl(name.charAt(i))) {
+                    throw new IllegalArgumentException(
+                            "field name contains control character U+%04X".formatted((int) name.charAt(i)));
+                }
+            }
             if (fields.putIfAbsent(name, type) != null) {
                 throw new IllegalArgumentException("duplicate field name: " + name);
             }
diff --git a/core/src/test/java/io/github/dfa1/vortex/core/model/DTypeStructBuilderTest.java b/core/src/test/java/io/github/dfa1/vortex/core/model/DTypeStructBuilderTest.java
@@ -88,4 +88,14 @@ void builder_isNotReusable_afterMutation_byField() {
         assertThat(resultX.fieldNames()).containsExactly("x");
         assertThat(resultY.fieldNames()).containsExactly("y");
     }
+
+    @org.junit.jupiter.params.ParameterizedTest
+    @org.junit.jupiter.params.provider.ValueSource(strings = {"", " ", "   ", "a\nb", "nul\u0000here"})
+    void field_footgunName_throwsIllegalArgumentException(String name) {
+        // Given / When / Then — the friendly path enforces the write-side name policy up front:
+        // blank and control-character names are wire-legal footguns vortex-java refuses to write
+        org.assertj.core.api.Assertions.assertThatThrownBy(
+                        () -> DType.structBuilder().field(name, DType.I64))
+                .isInstanceOf(IllegalArgumentException.class);
+    }
 }
diff --git a/docs/compatibility.md b/docs/compatibility.md
@@ -35,8 +35,8 @@ resolves only the standalone decoders in `reader`; no encoder class is loaded.
 | `vortex.onpair` experimental string encoding | Rust 0.74.0 | ❌ Not registered. Files using it fail to decode unless `Registry.allowUnknown()` is enabled. |
 | `vortex.variant` arbitrary nested objects | Rust (`vortex.parquet.variant`) | ⚠️ Java encodes/decodes variant columns of **typed scalar** values (constant / chunked-of-constants core, optional shredded child); Java↔Rust round-trip verified. Arbitrary nested JSON objects and real path-based shredding need the `vortex.parquet.variant` physical encoding — deferred ([ADR 0014](../adr/0014-variant-encoding-strategy.md)). |
 | Arrow extension array import affecting Variant shape | Rust 0.74.0 (#8125) | Untested. Re-run integration fixtures against v0.74.0 once published. |
-| Duplicate struct field names | Rust writer rejects ("StructLayout must have unique field names"); Rust reader tolerates foreign files (first-match access) | ⚠️ Deliberate divergence on read: Java rejects such files with `VortexException("duplicate field name in file schema")` instead of tolerating them — the name-keyed `Chunk` API cannot represent both columns, and silent column loss is worse than a loud failure on a file the reference writer refuses to produce. Java's writer mirrors the Rust writer's rejection. Empty column name `""` is legal on both sides (pinned by `ColumnNameEdgeCasesIntegrationTest`). |
-| NUL (`U+0000`) in a field name | Aborts the Rust toolchain: Arrow FFI schema export hits a panic-cannot-unwind in `arrow-rs` (`ffi_stream::get_schema`) and SIGABRTs the process (measured against vortex-jni 0.75.0) | ⚠️ Java's writer rejects NUL in field names (`IllegalArgumentException`) so we never emit a file the canonical reader dies on; Java's reader tolerates such files (pure-Java path, no C strings). All other names are opaque UTF-8 and round-trip intact both directions — including `""`, whitespace-only, `$`-runs, newlines, and emoji (measured). |
+| Duplicate struct field names | Rust writer rejects ("StructLayout must have unique field names"); Rust reader tolerates foreign files (first-match access) | ⚠️ Deliberate divergence on read: Java rejects such files with `VortexException("duplicate field name in file schema")` instead of tolerating them — the name-keyed `Chunk` API cannot represent both columns, and silent column loss is worse than a loud failure on a file the reference writer refuses to produce. Java's writer mirrors the Rust writer's rejection. |
+| Blank / control-character field names | Wire-legal; the Rust writer produces `""` and whitespace-only names. NUL (`U+0000`) additionally aborts the Rust toolchain: Arrow FFI schema export hits a panic-cannot-unwind in `arrow-rs` (`ffi_stream::get_schema`) and SIGABRTs the process (measured against vortex-jni 0.75.0) | ⚠️ Deliberate strictness BOTH ways: vortex-java's writer refuses blank and control-character field names (`IllegalArgumentException`), and its reader rejects files carrying them (`VortexException` naming the producing pipeline as the likely bug) — the JSON-`""`-key principle: wire-legal is a floor, not a policy. Printable names of any shape (`$`-runs, spaces inside, emoji) are legal and round-trip intact both directions (measured; pinned by `ColumnNameEdgeCasesIntegrationTest`). |
 
 ## Encodings
 
diff --git a/integration/src/test/java/io/github/dfa1/vortex/integration/ColumnNameEdgeCasesIntegrationTest.java b/integration/src/test/java/io/github/dfa1/vortex/integration/ColumnNameEdgeCasesIntegrationTest.java
@@ -1,51 +1,43 @@
 package io.github.dfa1.vortex.integration;
 
-import dev.vortex.api.DataSource;
-import dev.vortex.api.Scan;
-import dev.vortex.api.ScanOptions;
 import dev.vortex.api.Session;
 import dev.vortex.api.VortexWriter;
 import dev.vortex.arrow.ArrowAllocation;
 import dev.vortex.jni.NativeLoader;
-import io.github.dfa1.vortex.core.model.DType;
-import io.github.dfa1.vortex.core.model.PType;
+import io.github.dfa1.vortex.core.error.VortexException;
 import io.github.dfa1.vortex.reader.VortexReader;
-import io.github.dfa1.vortex.reader.array.LongArray;
 import org.apache.arrow.c.ArrowArray;
 import org.apache.arrow.c.ArrowSchema;
 import org.apache.arrow.c.Data;
 import org.apache.arrow.memory.BufferAllocator;
 import org.apache.arrow.vector.BigIntVector;
 import org.apache.arrow.vector.VectorSchemaRoot;
-import org.apache.arrow.vector.ipc.ArrowReader;
 import org.apache.arrow.vector.types.pojo.ArrowType;
 import org.apache.arrow.vector.types.pojo.Field;
 import org.apache.arrow.vector.types.pojo.Schema;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.io.TempDir;
 
 import java.io.IOException;
-import java.nio.channels.FileChannel;
 import java.nio.file.Path;
-import java.nio.file.StandardOpenOption;
-import java.util.ArrayList;
 import java.util.HashMap;
 import java.util.List;
-import java.util.Map;
 
 import static org.assertj.core.api.Assertions.assertThat;
 import static org.assertj.core.api.Assertions.catchThrowable;
 
 /// Cross-compatibility for column-name edge cases nobody advertises, measured against the
 /// Rust (JNI) reference:
 ///
-/// - `""` is a legal column name: round-trips in BOTH directions (Rust writes/Java reads and
-///   Java writes/Rust reads).
+/// - Blank names (`""`, whitespace-only) are wire-legal — the Rust writer produces them — but
+///   vortex-java refuses them BOTH ways by policy: the writer never emits them and the reader
+///   rejects files carrying them with a message pointing at the producing pipeline. Stricter
+///   than the wire on purpose, like a JSON library refusing a `""` key it could technically
+///   parse (see `VortexWriterTest` / `DTypeStructBuilderTest` / `PostscriptParserDTypeGuardsTest`).
 /// - Duplicate field names are legal in Rust's in-memory `StructFields`
 ///   (`vortex-array/src/dtype/struct_.rs`, first-match name access) but REJECTED by its file
 ///   writer: "StructLayout must have unique field names" — the wire contract both writers
-///   must enforce. Reader-side handling of foreign duplicate-name files is tracked in
-///   TODO.md §"Column identity".
+///   enforce; our reader rejects such files too.
 class ColumnNameEdgeCasesIntegrationTest {
 
     private static final Session SESSION = Session.create();
@@ -56,45 +48,28 @@ class ColumnNameEdgeCasesIntegrationTest {
     }
 
     @Test
-    void jniWritesEmptyColumnName_javaReadsIt(@TempDir Path tmp) throws IOException {
-        // Given — the Rust (JNI) writer produces a file whose first column is named ""
+    void jniWritesEmptyColumnName_javaRejectsItByPolicy(@TempDir Path tmp) throws IOException {
+        // Given — the Rust (JNI) writer legitimately produces a file whose first column is
+        // named "" (the wire format permits it)
         Path file = tmp.resolve("jni_empty_name.vortex");
         Schema schema = new Schema(List.of(
                 Field.notNullable("", new ArrowType.Int(64, true)),
                 Field.notNullable("x", new ArrowType.Int(64, true))));
         writeJni(file, schema, new long[][]{{1, 2}, {30, 40}});
 
         // When
-        Map<String, long[]> result = javaReadAllLongColumns(file);
-
-        // Then — the empty name is a plain map key, values intact
-        assertThat(result.keySet()).containsExactlyInAnyOrder("", "x");
-        assertThat(result.get("")).containsExactly(1, 2);
-        assertThat(result.get("x")).containsExactly(30, 40);
-    }
-
-    @Test
-    void javaWritesEmptyColumnName_jniReadsIt(@TempDir Path tmp) throws IOException {
-        // Given — the Java writer produces a file with an empty column name (the Struct record
-        // constructor performs no name validation; only StructBuilder rejects duplicates)
-        Path file = tmp.resolve("java_empty_name.vortex");
-        var dtype = new DType.Struct(
-                List.of("", "x"),
-                List.of(new DType.Primitive(PType.I64, false), new DType.Primitive(PType.I64, false)),
-                false);
-        try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE);
-             var writer = io.github.dfa1.vortex.writer.VortexWriter.create(
-                     ch, dtype, io.github.dfa1.vortex.writer.WriteOptions.defaults())) {
-            writer.writeChunk(Map.of("", new long[]{1, 2}, "x", new long[]{30, 40}));
-        }
-
-        // When — the Rust (JNI) reader scans it
-        List<String> result = new ArrayList<>();
-        long rows = jniReadFieldNames(file, result);
+        Throwable result = catchThrowable(() -> {
+            try (var reader = VortexReader.open(file)) {
+                assertThat(reader).isNotNull();
+            }
+        });
 
-        // Then — the reference reader accepts the file and reports the empty-named field
-        assertThat(rows).isEqualTo(2);
-        assertThat(result).containsExactly("", "x");
+        // Then — deliberate strictness beyond the wire: a blank name is almost certainly a bug
+        // in the producing pipeline, and rejecting it loudly beats propagating an unusable name
+        // into name-keyed APIs and SQL identifiers
+        assertThat(result)
+                .isInstanceOf(VortexException.class)
+                .hasMessageContaining("blank field name in file schema");
     }
 
     @Test
@@ -140,48 +115,4 @@ private static void writeJni(Path file, Schema schema, long[][] columns) throws
             }
         }
     }
-
-    /// Scans the whole file with the Java reader, materializing every column as `long[]`.
-    private static Map<String, long[]> javaReadAllLongColumns(Path file) throws IOException {
-        Map<String, long[]> out = new HashMap<>();
-        try (var reader = VortexReader.open(file);
-             var iter = reader.scan(io.github.dfa1.vortex.reader.ScanOptions.all())) {
-            while (iter.hasNext()) {
-                try (var chunk = iter.next()) {
-                    for (var entry : chunk.columns().entrySet()) {
-                        LongArray col = (LongArray) entry.getValue();
-                        long[] values = new long[(int) col.length()];
-                        for (int i = 0; i < values.length; i++) {
-                            values[i] = col.getLong(i);
-                        }
-                        out.put(entry.getKey(), values);
-                    }
-                }
-            }
-        }
-        return out;
-    }
-
-    /// Scans the file with the Rust (JNI) reader, returning the row count and collecting the
-    /// Arrow schema field names.
-    private static long jniReadFieldNames(Path file, List<String> namesOut) throws IOException {
-        long rows = 0;
-        DataSource ds = DataSource.open(SESSION, file.toAbsolutePath().toUri().toString());
-        Scan scan = ds.scan(ScanOptions.of());
-        while (scan.hasNext()) {
-            var partition = scan.next();
-            try (ArrowReader reader = partition.scanArrow(ALLOCATOR)) {
-                while (reader.loadNextBatch()) {
-                    var root = reader.getVectorSchemaRoot();
-                    rows += root.getRowCount();
-                    if (namesOut.isEmpty()) {
-                        for (Field field : root.getSchema().getFields()) {
-                            namesOut.add(field.getName());
-                        }
-                    }
-                }
-            }
-        }
-        return rows;
-    }
 }
diff --git a/reader/src/main/java/io/github/dfa1/vortex/reader/PostscriptParser.java b/reader/src/main/java/io/github/dfa1/vortex/reader/PostscriptParser.java
@@ -254,6 +254,22 @@ private static DType convertDType(io.github.dfa1.vortex.core.fbs.FbsDType fbs, i
                         // file, and the name-keyed Chunk API would silently drop a column.
                         throw new VortexException("duplicate field name in file schema: " + name);
                     }
+                    // Same strict name policy as the write side: blank and control-character
+                    // names are wire-legal but almost certainly a bug in the producing
+                    // pipeline — reject with a message that says so rather than propagate
+                    // unusable names into name-keyed APIs and SQL identifiers.
+                    if (name.isBlank()) {
+                        throw new VortexException("blank field name in file schema (field index "
+                                + i + "): wire-legal, but rejected by policy — likely a bug in "
+                                + "the pipeline that produced this file");
+                    }
+                    for (int c = 0; c < name.length(); c++) {
+                        if (Character.isISOControl(name.charAt(c))) {
+                            throw new VortexException(
+                                    "field name in file schema contains control character U+%04X (field index %d)"
+                                            .formatted((int) name.charAt(c), i));
+                        }
+                    }
                     names.add(name);
                 }
                 for (int i = 0; i < s.dtypesLength(); i++) {
diff --git a/reader/src/test/java/io/github/dfa1/vortex/reader/PostscriptParserDTypeGuardsTest.java b/reader/src/test/java/io/github/dfa1/vortex/reader/PostscriptParserDTypeGuardsTest.java
@@ -47,6 +47,31 @@ void convertDType_structNamesDtypesArityMismatch_throwsVortexException() {
                 .hasMessageContaining("names/dtypes length mismatch");
     }
 
+    @org.junit.jupiter.params.ParameterizedTest
+    @org.junit.jupiter.params.provider.ValueSource(strings = {"", " ", "   "})
+    void convertDType_blankFieldName_throwsVortexException(String name) {
+        // Given — a file schema carrying a blank field name (wire-legal; the Rust writer can
+        // produce it) — rejected by policy with a message pointing at the producing pipeline
+        MemorySegment dtype = structDType(new String[]{name}, 1);
+
+        // When / Then
+        assertThatThrownBy(() -> PostscriptParser.parseBlobs(minimalFooter(), flatLayout(), dtype))
+                .isInstanceOf(VortexException.class)
+                .hasMessageContaining("blank field name in file schema");
+    }
+
+    @Test
+    void convertDType_controlCharacterFieldName_throwsVortexException() {
+        // Given — a newline inside a file schema's field name: wire-legal, but it poisons
+        // SQL/CSV/log renderings downstream, so the reader rejects it like the writer does
+        MemorySegment dtype = structDType(new String[]{"a\nb"}, 1);
+
+        // When / Then
+        assertThatThrownBy(() -> PostscriptParser.parseBlobs(minimalFooter(), flatLayout(), dtype))
+                .isInstanceOf(VortexException.class)
+                .hasMessageContaining("control character U+000A");
+    }
+
     // ── FlatBuffer builders ─────────────────────────────────────────────────────
 
     /// Builds a struct dtype blob with the given field names and `dtypeCount` null-typed fields.
diff --git a/writer/src/main/java/io/github/dfa1/vortex/writer/VortexWriter.java b/writer/src/main/java/io/github/dfa1/vortex/writer/VortexWriter.java
@@ -135,17 +135,26 @@ private VortexWriter(
         // names"): duplicate-name schemas are constructible via the DType.Struct record (only
         // StructBuilder validates), so guard here — colChunks below is name-keyed and would
         // silently collapse the duplicates anyway.
+        // Write-side name policy — stricter than the wire format on purpose (a JSON parser
+        // accepts a "" key; a good JSON library still refuses to encourage writing one). The
+        // reader stays tolerant: foreign files with blank or control-character names are legal
+        // and must open. Mirrored in DType.StructBuilder.field for earlier, friendlier failure.
         var uniqueNames = new java.util.HashSet<String>();
         for (String name : schema.fieldNames()) {
             if (!uniqueNames.add(name)) {
                 throw new IllegalArgumentException("duplicate field name: " + name);
             }
-            // A NUL byte in a field name aborts the reference toolchain's Arrow FFI export
-            // (panic-cannot-unwind in arrow-rs ffi_stream::get_schema, SIGABRT — measured
-            // 2026-07-04): never emit a file the canonical reader dies on. Everything else,
-            // including "" and whitespace-only names, is legal and round-trips.
-            if (name.indexOf('\u0000') >= 0) {
-                throw new IllegalArgumentException("field name contains NUL (U+0000)");
+            if (name.isBlank()) {
+                throw new IllegalArgumentException(
+                        "blank field name (wire-legal, but a footgun vortex-java refuses to write)");
+            }
+            for (int i = 0; i < name.length(); i++) {
+                // NUL aborts the reference toolchain's Arrow FFI export (SIGABRT in arrow-rs,
+                // measured 2026-07-04); other control characters poison SQL/CSV/log renderings.
+                if (Character.isISOControl(name.charAt(i))) {
+                    throw new IllegalArgumentException(
+                            "field name contains control character U+%04X".formatted((int) name.charAt(i)));
+                }
             }
         }
         this.channel = channel;
diff --git a/writer/src/test/java/io/github/dfa1/vortex/writer/VortexWriterTest.java b/writer/src/test/java/io/github/dfa1/vortex/writer/VortexWriterTest.java
