[SPARK-53958][BUILD] Simplify Jackson deps management by using BOM#52668
Closed
pan3793 wants to merge 3 commits into
Closed
[SPARK-53958][BUILD] Simplify Jackson deps management by using BOM#52668pan3793 wants to merge 3 commits into
pan3793 wants to merge 3 commits into
Conversation
pan3793
commented
Oct 20, 2025
| <scalafmt.validateOnly>true</scalafmt.validateOnly> | ||
| <scalafmt.changedOnly>true</scalafmt.changedOnly> | ||
| <fasterxml.jackson.version>2.19.2</fasterxml.jackson.version> | ||
| <fasterxml.jackson.databind.version>2.19.2</fasterxml.jackson.databind.version> |
Member
Author
There was a problem hiding this comment.
in case that jackson-databind has a standalone patched version, the jackson-bom also has a new release, for example
https://mvnrepository.com/artifact/com.fasterxml.jackson/jackson-bom/2.13.4.20221013
Member
Author
|
I hit some Jackson deps issues (#52664 (comment)) in upgrading Avro to 1.12.1, and haven't figured out the root cause, so I am looking into this new simple approach to address the issues. cc @LuciferYang @sarutak @dongjoon-hyun WDYT? |
Member
There was a problem hiding this comment.
Could you revert all elimination of <exclusion> tags from this PR?
I believe this PR should work without touching the existing exclusion rules in order to verify that BOM is actually working correctly, @pan3793 .
Member
Author
|
@dongjoon-hyun, either w/ or w/o the elimination of |
dongjoon-hyun
approved these changes
Oct 21, 2025
dongjoon-hyun
left a comment
dongjoon-hyun
left a comment
Member
There was a problem hiding this comment.
+1, LGTM. Thank you for revising this PR, @pan3793 .
Merged to master for Apache Spark 4.1.0-preview3.
huangxiaopingRD
pushed a commit
to huangxiaopingRD/spark
that referenced
this pull request
Nov 25, 2025
### What changes were proposed in this pull request? Currently, it's relatively complicated with dependency version management due to the different resolution mechanisms between Maven and SBT. Take Jackson deps as an example, to ensure all Jackson deps have the same version on the classpath, for both Maven and SBT, we must exclude all transitive Jackson deps and re-declare them manually, otherwise SBT may pick the higher version from transitive deps then breaks the building. This PR proposes to use BOM to simplify the above process, see technical details at https://github.com/FasterXML/jackson-bom Maven has built-in support for BOM, SBT also has a plugin https://github.com/heremaps/here-sbt-bom ### Why are the changes needed? Simplify dependency management. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass GHA. Manually checked the output of `build/sbt dependencyTree`, all Jackson deps are correctly resolved to the expected version - 2.19.2. ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#52668 from pan3793/SPARK-53958. Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
dongjoon-hyun
added a commit
to apache/orc
that referenced
this pull request
Mar 4, 2026
### What changes were proposed in this pull request? This PR aims to upgrade `spark.jackson.version` to 2.21.1 in bench module. ### Why are the changes needed? From 2.21, we need to use BOM. https://cowtowncoder.medium.com/jackson-2-21-released-ce0b6582ae67 To use the latest version like Apache Spark. - apache/spark#52668 (Apache Spark 4.1.0) - apache/spark#53886 (Apache Spark 4.2.0-preview2) - apache/spark#54626 ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: `Gemini 3.1 Pro (High)` on `Antigravity`. Closes #2570 from dongjoon-hyun/ORC-2122. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
dongjoon-hyun
added a commit
to apache/orc
that referenced
this pull request
Mar 4, 2026
### What changes were proposed in this pull request? This PR aims to upgrade `spark.jackson.version` to 2.21.1 in bench module. ### Why are the changes needed? From 2.21, we need to use BOM. https://cowtowncoder.medium.com/jackson-2-21-released-ce0b6582ae67 To use the latest version like Apache Spark. - apache/spark#52668 (Apache Spark 4.1.0) - apache/spark#53886 (Apache Spark 4.2.0-preview2) - apache/spark#54626 ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: `Gemini 3.1 Pro (High)` on `Antigravity`. Closes #2570 from dongjoon-hyun/ORC-2122. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit 8167cbc) Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
uros-b
pushed a commit
that referenced
this pull request
Jun 24, 2026
### What changes were proposed in this pull request?
This PR uses `io.grpc:grpc-bom` to manage `io.grpc` artifact versions, replacing the per-artifact `<version>${io.grpc.version}</version>` declarations across `sql/connect/common`, `sql/connect/server`, and `udf/worker/grpc`. Mirrors the existing `jackson-bom` (#52668) and `jjwt-bom` (#56155) imports.
### Why are the changes needed?
Simplify dependency management.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs. No-op on dependency resolution — `io.grpc` resolves identically (`1.76.0`, direct and transitive) before/after, verified with `build/mvn help:effective-pom` and `build/mvn dependency:tree`.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
Closes #56741 from gillespeiffer-db/grpc-bom.
Authored-by: gillespeiffer-db <263405958+gillespeiffer-db@users.noreply.github.com>
Signed-off-by: Uros Bojanic <221401595+uros-b@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Currently, it's relatively complicated with dependency version management due to the different resolution mechanisms between Maven and SBT.
Take Jackson deps as an example, to ensure all Jackson deps have the same version on the classpath, for both Maven and SBT, we must exclude all transitive Jackson deps and re-declare them manually, otherwise SBT may pick the higher version from transitive deps then breaks the building.
This PR proposes to use BOM to simplify the above process, see technical details at https://github.com/FasterXML/jackson-bom
Maven has built-in support for BOM, SBT also has a plugin https://github.com/heremaps/here-sbt-bom
Why are the changes needed?
Simplify dependency management.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Pass GHA.
Manually checked the output of
build/sbt dependencyTree, all Jackson deps are correctly resolved to the expected version - 2.19.2.Was this patch authored or co-authored using generative AI tooling?
No.