free space check incorrectly fails on HPCC

[mr-c]

|| This is the script 'load-into-counting.py' in khmer.
|| You are running khmer version 1.1
[...]
Estimated memory usage is 2.2e+11 bytes (n_tables x min_tablesize)
--------
ERROR: Not enough free space on disk for saved table files;       Need at least 215.040499418 GB more.
       Table size: 215.0 GB
       Free space: 0.0 GB

It turns out that I didn't have write permissions to the directory I wanted to output to so perhaps that was the culprit.

[SensibleSalmon]

testing; case: can't write to /home/root (surprise)

(env)bocajnotnef@GCU-Caconym:~/git/khmer$ load-into-counting.py /home/root/table tests/test-data/test-large.fa 

|| This is the script 'load-into-counting.py' in khmer.
|| You are running khmer version 2.0-rc1+116.g2dacd90
|| You are also using screed version 0.9
||
|| If you use this script in a publication, please cite EACH of the following:
||
||   * MR Crusoe et al., 2014. http://dx.doi.org/10.6084/m9.figshare.979190
||   * Q Zhang et al., http://dx.doi.org/10.1371/journal.pone.0101271
||   * A. Döring et al. http://dx.doi.org:80/10.1186/1471-2105-9-11
||
|| Please see http://khmer.readthedocs.org/en/latest/citations.html for details.


PARAMETERS:
 - kmer size =    32        (-k)
 - n tables =     4         (-N)
 - max tablesize = 1e+06    (-x)

Estimated memory usage is 4e+06 bytes (n_tables x max_tablesize)
--------

** WARNING: tablesize is default!
** You probably want to increase this with -M/--max-memory-usage!
** Please read the docs!

ERROR: File /home/root/table does not have write permission; exiting

Note: this is a directory that is not a mountpoint.

[SensibleSalmon]

scenario: mounted SD card to /home/root as root--still don't have write permissions:

(env)bocajnotnef@GCU-Caconym:~/git/khmer$ sudo umount /media/bocajnotnef/Mercury 
(env)bocajnotnef@GCU-Caconym:~/git/khmer$ sudo mount /dev/mmcblk0p1 /home/root/
(env)bocajnotnef@GCU-Caconym:~/git/khmer$ sudo ls /home/root/
System Volume Information  testdir
(env)bocajnotnef@GCU-Caconym:~/git/khmer$ load-into-counting.py /home/root/table tests/test-data/test-large.fa 

|| This is the script 'load-into-counting.py' in khmer.
|| You are running khmer version 2.0-rc1+116.g2dacd90
|| You are also using screed version 0.9
||
|| If you use this script in a publication, please cite EACH of the following:
||
||   * MR Crusoe et al., 2014. http://dx.doi.org/10.6084/m9.figshare.979190
||   * Q Zhang et al., http://dx.doi.org/10.1371/journal.pone.0101271
||   * A. Döring et al. http://dx.doi.org:80/10.1186/1471-2105-9-11
||
|| Please see http://khmer.readthedocs.org/en/latest/citations.html for details.


PARAMETERS:
 - kmer size =    32        (-k)
 - n tables =     4         (-N)
 - max tablesize = 1e+06    (-x)

Estimated memory usage is 4e+06 bytes (n_tables x max_tablesize)
--------

** WARNING: tablesize is default!
** You probably want to increase this with -M/--max-memory-usage!
** Please read the docs!

ERROR: File /home/root/table does not have write permission; exiting

[mr-c]

What is this /home/root business about?

[SensibleSalmon]

It didn't exist before the testing. I mkdir'd it--for some reason it
seemed like the most logical place? I switched to a directory in /tmp
later.

On Sat, Jul 18, 2015, 22:49 Michael R. Crusoe notifications@github.com
wrote:

What is this /home/root business about?

—
Reply to this email directly or view it on GitHub
#668 (comment).

[ctb]

So in any case this doesn't seem to have anything to do with the original error that @mr-c reported.

[SensibleSalmon]

I was testing against the possibility that it was the lack of write permissions that caused the error.

[SensibleSalmon]

So, have tried in ~, /mnt/research/ged and /mnt/scratch/ged-testing/ and cannot reproduce the issue

That is, it's working.

I'm using tests/test-data/test-large.fa as my source.

[mr-c]

Cool. Is this with the new fix for checking the free space in the target
directory instead of the current directory?

On Mon, Jul 20, 2015 at 10:08 AM Jake Fenton notifications@github.com
wrote:

So, have tried in ~, /mnt/research/ged and /mnt/scratch/ged-testing/ and
cannot reproduce the issue

That is, it's working.

I'm using tests/test-data/test-large.fa as my source.

—
Reply to this email directly or view it on GitHub
#668 (comment).

Michael R. Crusoe: Programmer & Bioinformatician crusoe@ucdavis.edu
The lab for Data Intensive Biology; University of California, Davis
https://impactstory.org/MichaelRCrusoe http://twitter.com/biocrusoe

[SensibleSalmon]

Actually no, I did all my testing off of master which doesn't have the fix implemented--That is, I was unable to reproduce the issue.

[mr-c]

Can you reproduce using the the HPCC's version of khmer via module load khmer ?

On Mon, Jul 20, 2015 at 10:52 AM Jake Fenton notifications@github.com
wrote:

Actually no, I did all my testing off of master which doesn't have the fix
implemented--That is, I was unable to reproduce the issue.

—
Reply to this email directly or view it on GitHub
#668 (comment).

Michael R. Crusoe: Programmer & Bioinformatician crusoe@ucdavis.edu
The lab for Data Intensive Biology; University of California, Davis
https://impactstory.org/MichaelRCrusoe http://twitter.com/biocrusoe

[SensibleSalmon]

Nope. Home, /mnt/research and /mnt/scratch all appear to be writable. (Which is interesting since module load khmer gives me khmer 1.3, which shouldn't have any schmancy fixes)

[SensibleSalmon]

will be fixed by #1170

[mr-c]

Fixed in #1170