Found by Raincloud triage round 4 (#205) on uci-online-retail, customerid (u16?). Same silent-corruption class as #210 (null rows decode as plausible values), in the lazy RunEnd path.
RunEndEncodingDecoder unwraps the values child's MaskedArray and passes only the raw data to LazyRunEnd*Array, which has no slot for per-run validity:
Array valuesArr = ctx.decodeChild(1, ctx.dtype(), numRuns);
Array valuesData = valuesArr instanceof MaskedArray m ? m.inner() : valuesArr; // strips validity
...
case I16, U16 -> new LazyRunEndShortArray(ctx.dtype(), n, (ShortArray) valuesData, endsData, offset);
Array tree (chunk 0): vortex.runend(u16?) → values: fastlanes.for(u16?) → bitpacked(u16?) validity_child bool (1843 null runs). Ground truth vs Java:
row 622 rust=None java=12346 (= FoR reference base)
row 499785 rust=None java=0 (= raw underlying value)
row 0 rust=17850 java=17850 OK
135,080 wrong cells. The differing fillers (12346 vs 0) confirm it's the run-values' validity being dropped (chunk1 uses vortex.primitive(u16?) with its own validity child — same drop).
Fix direction: LazyRunEnd*Array must carry the run-values' validity and mark expanded rows null (a null run → all its rows null), mirroring the #210 approach; wrap the result in MaskedArray when the values child carried validity.
Found by Raincloud triage round 4 (#205) on
uci-online-retail,customerid(u16?). Same silent-corruption class as #210 (null rows decode as plausible values), in the lazy RunEnd path.RunEndEncodingDecoderunwraps the values child'sMaskedArrayand passes only the raw data toLazyRunEnd*Array, which has no slot for per-run validity:Array tree (chunk 0):
vortex.runend(u16?) → values: fastlanes.for(u16?) → bitpacked(u16?) validity_child bool (1843 null runs). Ground truth vs Java:135,080 wrong cells. The differing fillers (12346 vs 0) confirm it's the run-values' validity being dropped (chunk1 uses
vortex.primitive(u16?)with its own validity child — same drop).Fix direction:
LazyRunEnd*Arraymust carry the run-values' validity and mark expanded rows null (a null run → all its rows null), mirroring the #210 approach; wrap the result inMaskedArraywhen the values child carried validity.