Skip to content

AlpRdEncodingDecoder drops left_parts validity — nulls decode as 0.0 (silent loss) #234

Description

@dfa1

Found by Raincloud triage round 5 (#205) on bi-euro2016, columns polarity_confidence (109047 nulls) and subjectivity_confidence (26201 nulls), both f64?. Same silent-null-loss class as the fixed #210/#225/#226, but in a decoder none of those touched: vortex.alprd (ALP real-double).

ALPRD reconstructs f64 = f(left_parts, right_parts); nullability rides on the nullable left_parts child. The decoder does not propagate that child's validity, so null rows decode to 0.0.

Array tree (chunk 0):

polarity_confidence: vortex.alprd(f64?, len=131072)
  metadata: right_bit_width: 50
  left_parts: fastlanes.bitpacked(u16?, len=131072)
    validity_child: vortex.bool(bool, len=131072) [sum=125297]   # 5775 nulls, dropped
  right_parts: fastlanes.bitpacked(u64, len=131072)

[polarity_confidence row 26] java='0.0' parquet=None (also rows 55, 81, 128, 143).

Contrast: plain vortex.alp over a bitpacked-with-validity_child child decodes nulls correctly (fixed in #210), so this is specific to the vortex.alprd path. Fix: AlpRdEncodingDecoder must lift the left_parts MaskedArray validity and wrap the reconstructed f64 output in a MaskedArray, mirroring AlpEncodingDecoder's #210 fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions