Recompute analysis results in dockerized environment #24
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We only need it for installation time
=== Do not change lines below ===
{
"chain": [],
"cmd": "sh -c 'rm -rf container/image; docker build -t remodnav:latest container && python -m datalad_container.adapters.docker save remodnav:latest container/image && echo '\"'\"'**/*json annex.largefiles=nothing\\nrepositories annex.largefiles=nothing\\n**/VERSION annex.largefiles=nothing'\"'\"' > container/image/.gitattributes'",
"dsid": "c5a79271-7d24-42aa-a0cf-38d84fd15eaa",
"exit": 0,
"extra_inputs": [],
"inputs": [
"container/Dockerfile"
],
"outputs": [
"container/image"
],
"pwd": "."
}
^^^ Do not change lines above ^^^
This can be used with `containers-run` and normal "make" targets, but from the Docker-Makefile, and with their execution actually taking place inside the container. For example ``` datalad containers-run -n docker-make main.pdf ```
The creation of a containerized environment for the analysis became necessary, because 3+ years after the "final" results have been computed originally, it is getting difficult to recreate a matching computational environment. Even with pinned versions of essential software dependencies, issues of incompatibilities with modern Python versions slowly arise. The container setup used for this recomputation is the result of a detailed exploration on the effect of software versions and deployment methods. A reports is provided at #20 Importantly, the employed setup is NOT capable of yielded exactly identical results. While all statistical scores reported in the paper remain indeed identical, there is a visually small change to one histogram panel in Fig 4. The change is illustrated at #20 (comment) Given the overall state of reproducibility, and the anticipated longevity of the containerized computation, we decided that this small difference with respect to the journal publication is tolerable. This changeset support a DataLad-based re-execution (for verification): ``` datalad rerun <commitsha> ``` After this changeset, a complete manuscript can be compiled, also via DataLad via a: ``` datalad containers-run -n docker-make main.pdf ``` By default this uses the local Python installation via `python` to orchestrate Docker. If python is available via a different name, overide, for example, via: ``` datalad -c datalad.run.subsitutions.python=python3 rerun <commitsha> ``` Closes #20 === Do not change lines below === { "chain": [], "cmd": "{python} -m datalad_container.adapters.docker run container/image sh -c \"mkdir /tmp/dockertmp; HOME=/tmp/dockertmp make -f Docker-Makefile clean results_def.tex && rm -rf /tmp/dockertmp\"", "dsid": "c5a79271-7d24-42aa-a0cf-38d84fd15eaa", "exit": 0, "extra_inputs": [ "container/image" ], "inputs": [ "remodnav/remodnav/tests/data/anderson_etal", "data/studyforrest-data-eyemovementlabels/sub-*/*.tsv", "data/raw_eyegaze/sub-*/ses-movie/func/*_recording-eyegaze_physio.tsv.gz", "data/raw_eyegaze/sub-*/beh/*_recording-eyegaze_physio.tsv.gz" ], "outputs": [ "img", "results_def.tex" ], "pwd": "." } ^^^ Do not change lines above ^^^
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The creation of a containerized environment for the analysis became
necessary, because 3+ years after the "final" results have been computed
originally, it is getting difficult to recreate a matching computational
environment.
Even with pinned versions of essential software dependencies, issues
of incompatibilities with modern Python versions slowly arise.
The container setup used for this recomputation is the result of a
detailed exploration on the effect of software versions and deployment
methods. A reports is provided at
#20
Importantly, the employed setup is NOT capable of yielded exactly
identical results. While all statistical scores reported in the paper
remain indeed identical, there is a visually small change to one
histogram panel in Fig 4. The change is illustrated at
#20 (comment)
Given the overall state of reproducibility, and the anticipated
longevity of the containerized computation, we decided that this small
difference with respect to the journal publication is tolerable.
This changeset support a DataLad-based re-execution (for verification):
After this changeset, a complete manuscript can be compiled, also
via DataLad via a:
Closes #20
TODO: