Skip to content

Commit 3bcd988

Browse files
dfa1claude
andcommitted
feat(reader): route untrusted slices/sizes through IoBounds (ADR 0003 Phase E)
Migrate the file-structure and decode-side bounds operations that take offsets/lengths from parsed file bytes so a malformed file throws VortexException, not a raw JDK exception: - asSlice → IoBounds.slice: VortexReader (trailer/postscript/stats/segment), VortexHttpReader, Trailer magic, ScanIterator stats, FlatSegmentDecoder buffer descriptors, PostscriptParser blob slice. - FlatSegmentDecoder and ScanIterator stats now guard the trailing u32 fbLen read (checkRange) and segLen narrowing (toIntSize) — both were previously unguarded; a crafted fbLen leaked IndexOutOfBoundsException. - Math.toIntExact(storage.length()) → IoBounds.toIntSize in the Date/Time/ Timestamp/Uuid extension decoders (ArithmeticException → VortexException on a > 2 GB declared length). reader.array .limited() re-slices are left raw: offset 0, rows < length, bounded by construction — not untrusted input (ADR 0003 Phase E item 5). Existing MalformedTrailer/Footer/ZipBomb security tests stay green (no behaviour change on already-validated paths). Follow-ups: checkstyle ban on raw asSlice, checkCount guards on new T[(int)n] alloc sites, and the Objects.checkIndex sweep of consumer-access getters. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent e9af80d commit 3bcd988

11 files changed

Lines changed: 39 additions & 22 deletions

docs/adr/0003-vortex-exception-sanitization.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -365,9 +365,12 @@ widens the parse surface.
365365
4. Collapse the ~14 consumer-access `getX(i)` guards onto
366366
`Objects.checkIndex(i, length)` (separate commit — different error class,
367367
no `IoBounds`).
368-
5. Checkstyle `RegexpSingleline` rejecting raw `.asSlice(` in
369-
`reader` / `reader.array` / `reader.decode` / `core.proto` packages
370-
(mirrors the existing `<p>`-blocking rule), so new raw slices can't regress.
368+
5. Checkstyle `RegexpSingleline` rejecting raw `.asSlice(` in the
369+
file-structure (`reader` root) and `reader.decode` packages — the layers
370+
that slice on offsets parsed from file bytes. `reader.array` is excluded:
371+
its only `asSlice` calls are inside `limited(rows)`, re-slicing an already
372+
validated segment at offset 0 with `rows < length` (bounded by construction,
373+
no untrusted offset), so wrapping them adds noise without closing a gap.
371374
6. `BoundsTypingSecurityTest`: crafted file with out-of-range slice offset,
372375
oversize declared length, and non-monotonic VarBin offsets each produce a
373376
`VortexException`, never a raw JDK exception.

reader/src/main/java/io/github/dfa1/vortex/reader/FlatSegmentDecoder.java

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package io.github.dfa1.vortex.reader;
22

33
import io.github.dfa1.vortex.core.DType;
4+
import io.github.dfa1.vortex.core.IoBounds;
45
import io.github.dfa1.vortex.reader.array.Array;
56
import io.github.dfa1.vortex.encoding.EncodingId;
67
import io.github.dfa1.vortex.fbs.Buffer;
@@ -44,12 +45,15 @@ public FlatSegmentDecoder(ReadRegistry registry) {
4445
/// @return the decoded [Array] for this segment
4546
public Array decode(MemorySegment seg, List<String> encodingSpecs,
4647
DType dtype, long rowCount, SegmentAllocator arena) {
47-
int segLen = (int) seg.byteSize();
48+
int segLen = IoBounds.toIntSize(seg.byteSize());
4849
ByteBuffer bb = seg.asByteBuffer().order(ByteOrder.LITTLE_ENDIAN);
4950

51+
// The trailing u32 length field must itself be in range before we read it.
52+
IoBounds.checkRange(segLen - 4L, 4, segLen);
5053
int fbLen = bb.getInt(segLen - 4);
51-
int fbStart = segLen - 4 - fbLen;
52-
ByteBuffer fbBuf = bb.slice(fbStart, fbLen).order(ByteOrder.LITTLE_ENDIAN);
54+
long fbStart = segLen - 4L - fbLen;
55+
IoBounds.checkRange(fbStart, fbLen, segLen);
56+
ByteBuffer fbBuf = bb.slice((int) fbStart, fbLen).order(ByteOrder.LITTLE_ENDIAN);
5357
var fbArray = io.github.dfa1.vortex.fbs.Array.getRootAsArray(fbBuf);
5458

5559
int numBuffers = fbArray.buffersLength();
@@ -58,7 +62,7 @@ public Array decode(MemorySegment seg, List<String> encodingSpecs,
5862
for (int i = 0; i < numBuffers; i++) {
5963
Buffer bufDesc = fbArray.buffers(i);
6064
dataOffset += bufDesc.padding();
61-
bufs[i] = seg.asSlice(dataOffset, bufDesc.length());
65+
bufs[i] = IoBounds.slice(seg, dataOffset, bufDesc.length());
6266
dataOffset += bufDesc.length();
6367
}
6468

reader/src/main/java/io/github/dfa1/vortex/reader/PostscriptParser.java

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package io.github.dfa1.vortex.reader;
22

33
import io.github.dfa1.vortex.core.DType;
4+
import io.github.dfa1.vortex.core.IoBounds;
45
import io.github.dfa1.vortex.core.PType;
56
import io.github.dfa1.vortex.core.VortexException;
67
import io.github.dfa1.vortex.fbs.Binary;
@@ -115,7 +116,7 @@ static ParsedFile parseBlobs(ByteBuffer footerBuf, ByteBuffer layoutBuf, ByteBuf
115116
}
116117

117118
private static ByteBuffer slice(MemorySegment seg, long offset, long length) {
118-
return seg.asSlice(offset, length).asByteBuffer().order(ByteOrder.LITTLE_ENDIAN);
119+
return IoBounds.slice(seg, offset, length).asByteBuffer().order(ByteOrder.LITTLE_ENDIAN);
119120
}
120121

121122
static Footer convertFooter(io.github.dfa1.vortex.fbs.Footer f) {

reader/src/main/java/io/github/dfa1/vortex/reader/ScanIterator.java

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package io.github.dfa1.vortex.reader;
22

33
import io.github.dfa1.vortex.core.DType;
4+
import io.github.dfa1.vortex.core.IoBounds;
45
import io.github.dfa1.vortex.core.PType;
56
import io.github.dfa1.vortex.core.VortexException;
67
import io.github.dfa1.vortex.encoding.EncodingId;
@@ -788,9 +789,10 @@ private ArrayStats readFlatStats(Layout flat) {
788789

789790
// Stats FlatBuffer lives in the segment's last 4+fbLen bytes; reading the whole
790791
// segment as a ByteBuffer would fail for segments larger than 2 GB (ByteBuffer cap).
792+
IoBounds.checkRange(segLen - 4L, 4, segLen);
791793
int fbLen = seg.get(LE_INT, segLen - 4);
792794
long fbStart = segLen - 4L - fbLen;
793-
ByteBuffer fbBuf = seg.asSlice(fbStart, fbLen).asByteBuffer().order(ByteOrder.LITTLE_ENDIAN);
795+
ByteBuffer fbBuf = IoBounds.slice(seg, fbStart, fbLen).asByteBuffer().order(ByteOrder.LITTLE_ENDIAN);
794796
var fbArray = io.github.dfa1.vortex.fbs.Array.getRootAsArray(fbBuf);
795797

796798
io.github.dfa1.vortex.fbs.ArrayNode root = fbArray.root();

reader/src/main/java/io/github/dfa1/vortex/reader/Trailer.java

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
package io.github.dfa1.vortex.reader;
22

3+
import io.github.dfa1.vortex.core.IoBounds;
34
import io.github.dfa1.vortex.core.VortexException;
45
import io.github.dfa1.vortex.core.VortexFormat;
56

@@ -32,7 +33,7 @@ static Trailer parse(MemorySegment trailerSeg, long bodyBytes) {
3233
int version = Short.toUnsignedInt(trailerSeg.get(LE_SHORT, 0));
3334
int postscriptLen = Short.toUnsignedInt(trailerSeg.get(LE_SHORT, 2));
3435

35-
MemorySegment magicSlice = trailerSeg.asSlice(4, VortexFormat.MAGIC_SIZE);
36+
MemorySegment magicSlice = IoBounds.slice(trailerSeg, 4, VortexFormat.MAGIC_SIZE);
3637
if (magicSlice.mismatch(VortexFormat.MAGIC) != -1) {
3738
throw new VortexException(
3839
"invalid magic bytes [%02x %02x %02x %02x]".formatted(

reader/src/main/java/io/github/dfa1/vortex/reader/VortexHttpReader.java

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package io.github.dfa1.vortex.reader;
22

33
import io.github.dfa1.vortex.core.DType;
4+
import io.github.dfa1.vortex.core.IoBounds;
45
import io.github.dfa1.vortex.core.VortexException;
56
import io.github.dfa1.vortex.core.VortexFormat;
67
import io.github.dfa1.vortex.fbs.Postscript;
@@ -90,7 +91,7 @@ public static VortexHttpReader open(URI uri, ReadRegistry registry, HttpClient c
9091
MemorySegment tailSeg = MemorySegment.ofArray(tail);
9192
long trailerOff = tailLen - VortexFormat.TRAILER_SIZE;
9293
long bodyBytes = fileSize - VortexFormat.TRAILER_SIZE;
93-
Trailer trailer = Trailer.parse(tailSeg.asSlice(trailerOff, VortexFormat.TRAILER_SIZE), bodyBytes);
94+
Trailer trailer = Trailer.parse(IoBounds.slice(tailSeg, trailerOff, VortexFormat.TRAILER_SIZE), bodyBytes);
9495

9596
// HTTP-specific: postscript may extend past the prefetched tail and need a larger fetch.
9697
long psOffInTail = trailerOff - trailer.postscriptLen();
@@ -100,7 +101,7 @@ public static VortexHttpReader open(URI uri, ReadRegistry registry, HttpClient c
100101
.formatted(trailer.postscriptLen(), TAIL_SIZE));
101102
}
102103

103-
ByteBuffer postscriptBuf = tailSeg.asSlice(psOffInTail, trailer.postscriptLen())
104+
ByteBuffer postscriptBuf = IoBounds.slice(tailSeg, psOffInTail, trailer.postscriptLen())
104105
.asByteBuffer().order(ByteOrder.LITTLE_ENDIAN);
105106

106107
var ps = Postscript.getRootAsPostscript(postscriptBuf);

reader/src/main/java/io/github/dfa1/vortex/reader/VortexReader.java

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package io.github.dfa1.vortex.reader;
22

33
import io.github.dfa1.vortex.core.DType;
4+
import io.github.dfa1.vortex.core.IoBounds;
45
import io.github.dfa1.vortex.core.VortexException;
56
import io.github.dfa1.vortex.core.VortexFormat;
67

@@ -78,11 +79,11 @@ private static VortexReader parse(
7879
MemorySegment seg, long size, Arena arena, ReadRegistry registry
7980
) {
8081
long bodyBytes = size - VortexFormat.TRAILER_SIZE;
81-
var trailerSeg = seg.asSlice(bodyBytes, VortexFormat.TRAILER_SIZE);
82+
var trailerSeg = IoBounds.slice(seg, bodyBytes, VortexFormat.TRAILER_SIZE);
8283
Trailer trailer = Trailer.parse(trailerSeg, bodyBytes);
8384

8485
long postscriptOffset = bodyBytes - trailer.postscriptLen();
85-
var postscriptBuf = seg.asSlice(postscriptOffset, trailer.postscriptLen())
86+
var postscriptBuf = IoBounds.slice(seg, postscriptOffset, trailer.postscriptLen())
8687
.asByteBuffer().order(ByteOrder.LITTLE_ENDIAN);
8788

8889
PostscriptParser.ParsedFile parsed;
@@ -211,15 +212,15 @@ private ArrayStats readFlatStats(Layout flat) {
211212
if (segLen < 4) {
212213
return ArrayStats.empty();
213214
}
214-
MemorySegment seg = fileSegment.asSlice(spec.offset(), segLen);
215+
MemorySegment seg = IoBounds.slice(fileSegment, spec.offset(), segLen);
215216
int fbLen = seg.get(LE_INT, segLen - 4);
216217
// Reject negative fbLen (signed int from untrusted bytes) or any value that would push
217218
// fbStart below 0 → asSlice(negative, ...) throws IndexOutOfBoundsException without this guard.
218219
if (fbLen < 0 || fbLen > segLen - 4) {
219220
return ArrayStats.empty();
220221
}
221222
long fbStart = segLen - 4L - fbLen;
222-
var fbBuf = seg.asSlice(fbStart, fbLen).asByteBuffer().order(ByteOrder.LITTLE_ENDIAN);
223+
var fbBuf = IoBounds.slice(seg, fbStart, fbLen).asByteBuffer().order(ByteOrder.LITTLE_ENDIAN);
223224
var fbArray = io.github.dfa1.vortex.fbs.Array.getRootAsArray(fbBuf);
224225
var root = fbArray.root();
225226
if (root == null) {
@@ -234,15 +235,15 @@ public io.github.dfa1.vortex.reader.array.Array decodeFlatSegment(
234235
DType dtype, long rowCount,
235236
java.lang.foreign.SegmentAllocator arena
236237
) {
237-
MemorySegment seg = fileSegment.asSlice(spec.offset(), spec.length()).asReadOnly();
238+
MemorySegment seg = IoBounds.slice(fileSegment, spec.offset(), spec.length()).asReadOnly();
238239
return new FlatSegmentDecoder(registry)
239240
.decode(seg, footer.arraySpecs(), dtype, rowCount, arena);
240241
}
241242

242243
/// Zero-copy read-only slice of the memory-mapped file covering the given spec.
243244
@Override
244245
public MemorySegment rawSegment(SegmentSpec spec) {
245-
return fileSegment.asSlice(spec.offset(), spec.length()).asReadOnly();
246+
return IoBounds.slice(fileSegment, spec.offset(), spec.length()).asReadOnly();
246247
}
247248

248249
@Override

reader/src/main/java/io/github/dfa1/vortex/reader/extension/DateExtensionDecoder.java

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package io.github.dfa1.vortex.reader.extension;
22

33
import io.github.dfa1.vortex.core.DType;
4+
import io.github.dfa1.vortex.core.IoBounds;
45
import io.github.dfa1.vortex.core.PType;
56
import io.github.dfa1.vortex.encoding.TimeUnit;
67
import io.github.dfa1.vortex.reader.array.Array;
@@ -60,7 +61,7 @@ public LocalDate decode(Array storage, long i) {
6061
/// @param storage signed-integer storage array (optionally wrapped in `MaskedArray`)
6162
/// @return list of decoded dates in row order; `null` entries mark invalid rows
6263
public List<LocalDate> decodeAll(Array storage) {
63-
int n = Math.toIntExact(storage.length());
64+
int n = IoBounds.toIntSize(storage.length());
6465
List<LocalDate> out = new ArrayList<>(n);
6566
if (storage instanceof MaskedArray masked) {
6667
for (long i = 0; i < n; i++) {

reader/src/main/java/io/github/dfa1/vortex/reader/extension/TimeExtensionDecoder.java

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package io.github.dfa1.vortex.reader.extension;
22

33
import io.github.dfa1.vortex.core.DType;
4+
import io.github.dfa1.vortex.core.IoBounds;
45
import io.github.dfa1.vortex.core.VortexException;
56
import io.github.dfa1.vortex.reader.array.Array;
67
import io.github.dfa1.vortex.reader.array.MaskedArray;
@@ -72,7 +73,7 @@ public LocalTime decode(DType.Extension ext, Array storage, long i) {
7273
/// @param storage signed-integer storage array (optionally wrapped in `MaskedArray`)
7374
/// @return list of decoded times in row order; `null` entries mark invalid rows
7475
public List<LocalTime> decodeAll(DType.Extension ext, Array storage) {
75-
int n = Math.toIntExact(storage.length());
76+
int n = IoBounds.toIntSize(storage.length());
7677
List<LocalTime> out = new ArrayList<>(n);
7778
if (storage instanceof MaskedArray masked) {
7879
for (long i = 0; i < n; i++) {

reader/src/main/java/io/github/dfa1/vortex/reader/extension/TimestampExtensionDecoder.java

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package io.github.dfa1.vortex.reader.extension;
22

33
import io.github.dfa1.vortex.core.DType;
4+
import io.github.dfa1.vortex.core.IoBounds;
45
import io.github.dfa1.vortex.core.VortexException;
56
import io.github.dfa1.vortex.reader.array.Array;
67
import io.github.dfa1.vortex.reader.array.MaskedArray;
@@ -95,7 +96,7 @@ public Optional<ZoneId> timezone(DType.Extension ext) {
9596
/// @param storage signed-integer storage array (optionally wrapped in `MaskedArray`)
9697
/// @return list of decoded instants in row order; `null` entries mark invalid rows
9798
public List<Instant> decodeAll(DType.Extension ext, Array storage) {
98-
int n = Math.toIntExact(storage.length());
99+
int n = IoBounds.toIntSize(storage.length());
99100
List<Instant> out = new ArrayList<>(n);
100101
if (storage instanceof MaskedArray masked) {
101102
for (long i = 0; i < n; i++) {

0 commit comments

Comments
 (0)