[SPARK-14754][SPARK CORE] Metrics as logs are not coming through slf4j #12697
Conversation
mihir6692
changed the title
|
Can one of the admins verify this patch? |
| MetricsSystem.checkMinimalPollingPeriod(pollUnit, pollPeriod) | ||
|
|
||
| val reporter: Slf4jReporter = Slf4jReporter.forRegistry(registry) | ||
| .outputTo(LoggerFactory.getLogger("org.apache.spark.metrics")) |
There was a problem hiding this comment.
Why not the class name for the logger name here, per your JIRA?
Updated import style as per conventions
Update class name for logger
|
I have added 2 more commits and updated the pull request. Please have a look. |
| MetricsSystem.checkMinimalPollingPeriod(pollUnit, pollPeriod) | ||
|
|
||
| val reporter: Slf4jReporter = Slf4jReporter.forRegistry(registry) | ||
| .outputTo(LoggerFactory.getLogger("org.apache.spark.metrics.sink.Slf4jSink")) |
There was a problem hiding this comment.
Sure, it can be classOf[Slf4jSink].getName for simplicity and to avoid forgetting to update it if for some reason this moves.
There was a problem hiding this comment.
For Slf4jSink.scala :-
Its not about class path or class level in package. It is just a name. Ex.
If you keep name like Spark.log4j, it would still work. ( and use the
same Spark.log4j in log4j.properties). So it won't matter even if we
move class to some other folder or package.
EDIT :-
I understood what you are trying to say. I will update it with new commit. Thanks for guidance.
|
gentle ping @mihir6692 |
1 similar comment
|
gentle ping @mihir6692 |
## What changes were proposed in this pull request? This PR proposes to close stale PRs. What I mean by "stale" here includes that there are some review comments by reviewers but the author looks inactive without any answer to them more than a month. I left some comments roughly a week ago to ping and the author looks still inactive in these PR below These below includes some PR suggested to be closed and a PR against another branch which seems obviously inappropriate. Given the comments in the last three PRs below, they are probably worth being taken over by anyone who is interested in it. Closes apache#7963 Closes apache#8374 Closes apache#11192 Closes apache#11374 Closes apache#11692 Closes apache#12243 Closes apache#12583 Closes apache#12620 Closes apache#12675 Closes apache#12697 Closes apache#12800 Closes apache#13715 Closes apache#14266 Closes apache#15053 Closes apache#15159 Closes apache#15209 Closes apache#15264 Closes apache#15267 Closes apache#15871 Closes apache#15861 Closes apache#16319 Closes apache#16324 Closes apache#16890 Closes apache#12398 Closes apache#12933 Closes apache#14517 ## How was this patch tested? N/A Author: hyukjinkwon <gurwls223@gmail.com> Closes apache#16937 from HyukjinKwon/stale-prs-close.
4 participants
What changes were proposed in this pull request?
Two changes need to be done :
Slf4jsink.scala
Added class name for logging. (Reference : https://dropwizard.github.io/metrics/3.1.0/manual/core/#man-core-reporters-slf4j )
log4j.properties.template
Added log configuration in log4j.properties.template for in support of above changes.
How was this patch tested?
It is tested with manual testing. I build spark with make-distribution.sh and then tried few example job to print Metrics in log files.