Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workflows #788

Merged
merged 156 commits into from
Mar 21, 2018
Merged

Workflows #788

merged 156 commits into from
Mar 21, 2018

Conversation

ShaiberAlon
Copy link
Contributor

I think it is stable enough to be merged with master

ShaiberAlon and others added 30 commits March 1, 2018 10:43
This makes it much easier to understand the directory structure for
each workflow.
Since we use these functions in workflows that are not in the anvio
repository (like `export_locus.snake`), then I wish to keep them for
now, and maybe forever.
of course, we can't do anything with them, but just listing them is
also nice.
and also added flag references_mode to the config to be more verbose
about it.
this option is there so we could check if a param was set by the user.
Now you can save a default config file for a workflow (but the only the
contigs default config is actually accurate)
You can list the available workflows

In addition, contigs workflow now works and has been tested.
let's use this flag to let user define a config file path. one stone two birds.
- no more output file argument, because now the get-default-config requires a filename argument.

- removing the entire if block to export default config into the base class. if the parameter set, it will save the default config for whicever workflow is set, and exit.
ShaiberAlon and others added 22 commits March 15, 2018 09:28
I commented out the dry-run steps because they actually start running
things and I don't want that yet.
For now, it doesn't do anything yet, but at least the purpose is clear.
also, if the user set the value to a boolean False then don't return
the default. This fixes that behavior
any other value will result in a negative result. maybe we should put a
warning that the value should be either boolean true or false.
if user asked to save workflow graph we will also set `dry_run_only` to
True
@meren, please tell me if this makes sense to you.
The idea is that this would allow us to run:
```
anvi-run-snakemake-workflow -w contigs -c config.json
--additional-params '--clustetr "-n {threads} -log {log}" --resources
nodes=60'
This is how I had to solve this. following an advice in the discussion
here: https://bugs.python.org/issue9334
@ShaiberAlon
Copy link
Contributor Author

@meren, I tested it on barhal to see that the --additional-params works as expected and allows to use (for example): --additional-params --cluster 'clusterize -log {log} -n {threads}' --resources nodes=40 --jobs 10, and it worked smoothly. So this is ready to use.

@meren
Copy link
Member

meren commented Mar 21, 2018

This is very happy news, Alon! Thank you very much.

@meren meren merged commit c119b5c into master Mar 21, 2018
@ShaiberAlon
Copy link
Contributor Author

:-) Thanks!

@ShaiberAlon ShaiberAlon deleted the workflows branch August 10, 2018 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants