Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor docker usage and support input files #20

Merged
merged 5 commits into from
Sep 6, 2024
Merged

Conversation

jrm5100
Copy link
Contributor

@jrm5100 jrm5100 commented Sep 6, 2024

Running cargo run --release --example docker

Does an ls on the /volA mount in one execution then cats the contents of the file in the next execution, showing support for:

  • Input files (non-local kinds like s3 are still todo!)
  • Persistent volumes across executions
Reply: Reply { executions: Some(NonEmpty { head: ExecutionResult { status: 0, stdout: "test_input.txt\n", stderr: "" }, tail: [ExecutionResult { status: 0, stdout: "Hello, world from an input\n", stderr: "" }] }) }
Reply: Reply { executions: Some(NonEmpty { head: ExecutionResult { status: 0, stdout: "test_input.txt\n", stderr: "" }, tail: [ExecutionResult { status: 0, stdout: "Hello, world from an input\n", stderr: "" }] }) }
Reply: Reply { executions: Some(NonEmpty { head: ExecutionResult { status: 0, stdout: "test_input.txt\n", stderr: "" }, tail: [ExecutionResult { status: 0, stdout: "Hello, world from an input\n", stderr: "" }] }) }
Reply: Reply { executions: Some(NonEmpty { head: ExecutionResult { status: 0, stdout: "test_input.txt\n", stderr: "" }, tail: [ExecutionResult { status: 0, stdout: "Hello, world from an input\n", stderr: "" }] }) }
Reply: Reply { executions: Some(NonEmpty { head: ExecutionResult { status: 0, stdout: "test_input.txt\n", stderr: "" }, tail: [ExecutionResult { status: 0, stdout: "Hello, world from an input\n", stderr: "" }] }) }
Reply: Reply { executions: Some(NonEmpty { head: ExecutionResult { status: 0, stdout: "test_input.txt\n", stderr: "" }, tail: [ExecutionResult { status: 0, stdout: "Hello, world from an input\n", stderr: "" }] }) }
Reply: Reply { executions: Some(NonEmpty { head: ExecutionResult { status: 0, stdout: "test_input.txt\n", stderr: "" }, tail: [ExecutionResult { status: 0, stdout: "Hello, world from an input\n", stderr: "" }] }) }
Reply: Reply { executions: Some(NonEmpty { head: ExecutionResult { status: 0, stdout: "test_input.txt\n", stderr: "" }, tail: [ExecutionResult { status: 0, stdout: "Hello, world from an input\n", stderr: "" }] }) }
Reply: Reply { executions: Some(NonEmpty { head: ExecutionResult { status: 0, stdout: "test_input.txt\n", stderr: "" }, tail: [ExecutionResult { status: 0, stdout: "Hello, world from an input\n", stderr: "" }] }) }
Reply: Reply { executions: Some(NonEmpty { head: ExecutionResult { status: 0, stdout: "test_input.txt\n", stderr: "" }, tail: [ExecutionResult { status: 0, stdout: "Hello, world from an input\n", stderr: "" }] }) }

@jrm5100
Copy link
Contributor Author

jrm5100 commented Sep 6, 2024

I'd imagine there may be a lot of room to make it more efficient (especially for large files) but at least we have a working example here so we could theoretically do a samtools flagstat or something.

Specifically we re-copy all of the inputs before each execution, even though those in the volumes could be checked to ensure they weren't modified first.

Cargo.lock Outdated Show resolved Hide resolved
Copy link
Collaborator

@claymcleod claymcleod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending @peterhuene's comment, this looks good.

@jrm5100 jrm5100 merged commit 51e275c into main Sep 6, 2024
6 checks passed
@jrm5100 jrm5100 deleted the more-complete-docker branch September 6, 2024 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants