[ADAM-891] Mark SparkContext as @transient. #894
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Closes bigdatagenomics#891. In org.bdgenomics.adam.rdd.ADAMContext, the val sparkContext: SparkContext is not marked as @transient, which causes serialization issues. SparkContexts are not serializable and should only be called from the driver.
|
Test FAILed. Build result: FAILUREGitHub pull request #894 of commit a4be852 automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/894/merge^{commit} # timeout=10 > git branch -a --contains 55dfb83 # timeout=10 > git rev-parse remotes/origin/pr/894/merge^{commit} # timeout=10Checking out Revision 55dfb83 (origin/pr/894/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 55dfb8363337194e825c5ddfbebecadffd197becFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.4.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.4.1,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'Test FAILed. |
|
Should ADAMContext be serializable? |
|
Jenkins, retest this please. |
|
Test PASSed. |
|
LGTM, ok to merge via Merge pull request button? |
|
@ryan-williams I think it needs to be serializable as we have closures in the class that need to be serialized (e.g., for BAM->ADAM conversion). It might make sense to pull the things that need to be serializable out into another object, but I digress. |
|
Makes sense, gtk, thx |
heuermh
added a commit
that referenced
this pull request
Dec 11, 2015
[ADAM-891] Mark SparkContext as @transient.
|
Thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #891. In org.bdgenomics.adam.rdd.ADAMContext, the val sparkContext:
SparkContext is not marked as @transient, which causes serialization issues.
SparkContexts are not serializable and should only be called from the driver.