Repository navigation
cannot configure a secret class only for tls validation #529
Description
Affected Stackable version
No response
Affected Apache Spark-on-Kubernetes version
probably all
Current and expected behavior
Given a secret configuration like this:
---
apiVersion: secrets.stackable.tech/v1alpha1
kind: SecretClass
metadata:
name: s3-ca-class
spec:
backend:
k8sSearch:
searchNamespace:
pod: {}
---
kind: Secret
apiVersion: v1
metadata:
name: s3-ca
namespace: stackable-operators
labels:
secrets.stackable.tech/class: s3-ca-class
data:
ca.crt: ...
type: Opaqueand an application such as:
apiVersion: spark.stackable.tech/v1alpha1
kind: SparkApplication
metadata:
name: pyspark-pi
namespace: stackable-operators
spec:
sparkConf:
"spark.driver.extraJavaOptions": "-Djavax.net.debug=ssl"
mode: cluster
driver:
config:
logging:
enableVectorAgent: null
logFileDirectory:
s3:
bucket:
inline:
bucketName: spark-object-storage-3894f55e-3f37-49c0-b37c-1636954f50ca
connection:
inline:
accessStyle: Path
credentials:
secretClass: s3-credentials-class
host: rook-ceph-rgw-ocs-storagecluster-cephobjectstore.openshift-storage.svc
port: 443
tls:
verification:
none: {}
prefix: eventlogs/
executor:
config:
logging:
enableVectorAgent: null
replicas: 1
mainApplicationFile: 'local:///stackable/spark/examples/src/main/python/pi.py'
sparkImage:
custom: 'redacted: use a 3.5.2 image'
productVersion: 3.5.2
pullPolicy: Always
Results in the following error:
javax.net.ssl|ERROR|21|Thread-3|2025-02-26 15:34:53.208 UTC|null:-1|Fatal (CERTIFICATE_UNKNOWN): PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target (
5238"throwable" : {
5239sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
5240at java.base/sun.security.validator.PKIXValidator.doBuild(Unknown Source)
5241at java.base/sun.security.validator.PKIXValidator.engineValidate(Unknown Source)
5242at java.base/sun.security.validator.Validator.validate(Unknown Source)
5243at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(Unknown Source)
5244at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)
5245at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(Unknown Source)
5246at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(Unknown Source)
5247at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(Unknown Source)
Also clarify why ignoring TLS validation is prohibited for applications but not for the history server.
Possible solution
No response
Additional context
No response
Environment
No response
Would you like to work on fixing this bug?
None
Activity
razvan commentedon Feb 27, 2025
This PR tries to simulate the problematic environment as good as possible. Currently, the modified spark-history test is successful but that is because a certificate is also provided.
Remove the tls certificate from the
minio-tls-eventlogsecret to reproduce the problem.nightkr commentedon Feb 27, 2025
Part of this should come with stackabletech/secret-operator#557, but on second thought it sounds like it would still try to use the TLS certificate even if it's missing...