Remove log4j dependencies #367
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This allows keeping the core dependencies separate from the overrides. On the other hand, the overrides must target specific packages, which in case of some benchmarks requires listing specific Netty components, because we cannot simply override the version of the netty-all package.
This is to avoid pulling in the log4j-1.2.17.jar. The functionality should be replaced by reload4j, which gets pulled in by slf4j-log4j12 starting with version 1.7.36. Because reload4j did not find Spark's default logging configuration file (log4j.properties), it was placed in the apache-spark benchmark resource directory so that reload4j can find it. This appears to provide a reasonable logging configuration again. The commons-logging package should be subsumed by the jcl-over-slf4j logging bridge.
lbulej
force-pushed
the
topic/remove-log4j
branch
from
f21b0d3 to
e2fc973
Compare
|
I have moved Spark's log4j configuration file to a place where |
lbulej
changed the title
|
With a few minor updates, the suite seems to be free of Overall, logging seems to be a somewhat messy issue, especially with libraries using different logging solutions. We should be able to get somewhat "unified" control over dependencies that use |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Attempt to completely remove
log4jfromapache-sparkdependencies. Complements #365 which updatedslf4j-log4j12to pull inreload4j, but did not remove direct the Spark dependency onlog4j. Withlog4jremoved, there seem to be some rough edges (missing appenders), probably configuration related. If this can be sorted out, we can do a minimal-change maintenance release, otherwise we'll see if the problems go away on its own after updating most of the dependencies.