Skip to content

GeoPackage reader reads NULL integer, real and boolean values as 0, 0.0 and false #3409

Description

@jiayuasu

The GeoPackage data source reads INTEGER, SMALLINT, MEDIUMINT, FLOAT, DOUBLE, REAL and BOOLEAN columns in ValuesMapper with ResultSet.getInt / getFloat / getDouble / getBoolean. For a SQL NULL those JDBC getters return 0, 0.0 or false, and the mapper never checks wasNull(). NULL cells therefore come back as real zeros and false, and since the columns are declared nullable nothing signals the substitution. This is the silent counterpart of #3352, where NULL DATE/DATETIME values threw an NPE.

Reproduce

A GeoPackage layer with a numeric/boolean column that has a NULL in some rows, for example written with GDAL:

# nulls.geojson: two points, properties {name, score, weight, flag}; row 2 has score/weight/flag = null
ogr2ogr -f GPKG nulls.gpkg nulls.geojson -nln test_features   # score MEDIUMINT, weight REAL, flag BOOLEAN
df = spark.read.format("geopackage").option("tableName", "test_features").load("nulls.gpkg")
df.show()
df.filter("score IS NULL").count()

Actual

|fid|name    |score|weight|flag |
|1  |all set |7    |1.5   |true |
|2  |all null|0    |0.0   |false|

df.filter("score IS NULL").count() returns 0.

Expected

NULL cells come back as null, as they already do for TEXT, BLOB, geometry and DATE/DATETIME columns, and score IS NULL returns 1. Today filters and aggregates on numeric or boolean columns that contain NULLs give wrong results.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions