Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get rid of ADAMSparkRegistrator spam #4186

Closed
lbergelson opened this issue Jan 17, 2018 · 4 comments · Fixed by #4314
Closed

get rid of ADAMSparkRegistrator spam #4186

lbergelson opened this issue Jan 17, 2018 · 4 comments · Fixed by #4314
Labels

Comments

@lbergelson
Copy link
Member

18/01/17 08:24:55 INFO org.bdgenomics.adam.serialization.ADAMKryoRegistrator: Did not find Spark internal class. This is expected for Spark 1.

Appears hundreds of times in the log output since we upgraded ADAM. This is despite the fact that we're not using spark 1. Maybe there's a way to silence a specific logger?

@lbergelson
Copy link
Member Author

I've been told that this is solvable by bumping the dataproc version from 1.1 -> 1.2. We need to update this on jenkins assuming it works fine with spark 2.2.

@lbergelson
Copy link
Member Author

It might still be worth silencing this for people who are running spark 2.0.0 which is what we are doing on jenkins. (and what we officially support in our build.gradle). Ultimately we need to upgrade to spark 2.2.x #2555

@cwhelan
Copy link
Member

cwhelan commented Jan 22, 2018

This is still an issue when running with --spark-runner LOCAL, ie not in dataproc.

@lbergelson
Copy link
Member Author

That makes sense because it uses the bundled spark 2.0.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants