-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[MRG] Speed up copy_local_genomes.py
with symbolic links
#181
Conversation
copy_local_genomes.py
with symbolic linkscopy_local_genomes.py
with symbolic links
copy_local_genomes.py
with symbolic linkscopy_local_genomes.py
with symbolic links
@ctb symlinks are working. I suspect the issue you observed was symlinking relative paths, not absolute paths, maybe? The problem now after this PR is that the Snakefile always expects gzipped files to process |
cool, thanks! the sticking point for the compression are the Anyhoo, rather than fix the code to figure out if the file is gzipped and then do the right thing, I just decided that compressing files is good and we should be doing it in the first place :) |
Perfect! The PR is now ready for the review :) Symlinking will only work now if the Fasta files are already compressed. |
copy_local_genomes.py
with symbolic linkscopy_local_genomes.py
with symbolic links
Observed behavior: Using symlinks will work, but the original filename will show in prefetch results, not the new one in the symlink. |
if it's just in the prefetch CSV could you separate out the doc change into its own PR, or do you mind if I do? I think that can/should be merged quickly but I might not get to the symlink PR for a bit. |
Co-authored-by: C. Titus Brown <titus@idyll.org>
Co-authored-by: C. Titus Brown <titus@idyll.org>
Co-authored-by: C. Titus Brown <titus@idyll.org>
Co-authored-by: C. Titus Brown <titus@idyll.org>
This reverts commit fee1aee.
genome_grist.copy_local_genomes
takes forever to copy and compress FASTA files. This PR resolves #138 by adding the command line option--sym
to use symbolic links.--sym
defaults to False, might be changed later after discussing the PR.Also upgrading private genomes config file to the latest version in docs (
samples
instead ofsample
).