The GeoPackage data source reads INTEGER, SMALLINT, MEDIUMINT, FLOAT, DOUBLE, REAL and BOOLEAN columns in ValuesMapper with ResultSet.getInt / getFloat / getDouble / getBoolean. For a SQL NULL those JDBC getters return 0, 0.0 or false, and the mapper never checks wasNull(). NULL cells therefore come back as real zeros and false, and since the columns are declared nullable nothing signals the substitution. This is the silent counterpart of #3352, where NULL DATE/DATETIME values threw an NPE.
Reproduce
A GeoPackage layer with a numeric/boolean column that has a NULL in some rows, for example written with GDAL:
# nulls.geojson: two points, properties {name, score, weight, flag}; row 2 has score/weight/flag = null
ogr2ogr -f GPKG nulls.gpkg nulls.geojson -nln test_features # score MEDIUMINT, weight REAL, flag BOOLEAN
df = spark.read.format("geopackage").option("tableName", "test_features").load("nulls.gpkg")
df.show()
df.filter("score IS NULL").count()
Actual
|fid|name |score|weight|flag |
|1 |all set |7 |1.5 |true |
|2 |all null|0 |0.0 |false|
df.filter("score IS NULL").count() returns 0.
Expected
NULL cells come back as null, as they already do for TEXT, BLOB, geometry and DATE/DATETIME columns, and score IS NULL returns 1. Today filters and aggregates on numeric or boolean columns that contain NULLs give wrong results.
The GeoPackage data source reads INTEGER, SMALLINT, MEDIUMINT, FLOAT, DOUBLE, REAL and BOOLEAN columns in
ValuesMapperwithResultSet.getInt/getFloat/getDouble/getBoolean. For a SQL NULL those JDBC getters return 0, 0.0 or false, and the mapper never checkswasNull(). NULL cells therefore come back as real zeros andfalse, and since the columns are declared nullable nothing signals the substitution. This is the silent counterpart of #3352, where NULL DATE/DATETIME values threw an NPE.Reproduce
A GeoPackage layer with a numeric/boolean column that has a NULL in some rows, for example written with GDAL:
Actual
df.filter("score IS NULL").count()returns 0.Expected
NULL cells come back as null, as they already do for TEXT, BLOB, geometry and DATE/DATETIME columns, and
score IS NULLreturns 1. Today filters and aggregates on numeric or boolean columns that contain NULLs give wrong results.