Iceberg jobs with with Hive 4.0.1 failure #537

Affected Stackable version

all supported

Affected Apache Spark-on-Kubernetes version

No response

Current and expected behavior

Hive 4.0.1 has removed some deprecated apis but the spark iceberg runtime has not been updated.

Spark jobs using the Hive 4.0.1 catalog to write using the iceberg format will fail with the error message:

spark Caused by: org.apache.thrift.TApplicationException: Invalid method name: 'get_table' 

Discovered during the upgrade of the anomaly detection demo.

Possible solution

No response

Additional context

No response

Environment

No response

Would you like to work on fixing this bug?

None