update_db crash

[pgcudahy]

Hi Jody, hope you're well. I'm having trouble setting up a new install of NTM-profiler. I did

conda create -n ntmprofiler_v0.3.0 ntm-profiler=0.3.0 -y
conda activate ntmprofiler_v0.3.0
mkdir -p ntm_profiler/v0.3.0
cd ntm_profiler/v0.3.0
ntm-profiler update_db --logging DEBUG

And get

[03:54:39] DEBUG    ntm-db                                                                                                                                                                                                                                                                                                                   ntm-profiler:186
           DEBUG    Directory ntm-db does not exist, cloning repo                                                                                                                                                                                                                                                                            ntm-profiler:193
           DEBUG    Running command: git clone https://github.com/ntm-db/ntm-db.git                                                                                                                                                                                                                                                              utils.py:386
[03:54:45] DEBUG    Running command: git checkout master                                                                                                                                                                                                                                                                                         utils.py:386
           DEBUG    Running command: zip -F species/sourmash-split.zip --out species/sourmash.sbt.zip                                                                                                                                                                                                                                            utils.py:386
           INFO                                                                                                                                                                                                                                                                                                                              ntm-profiler:205
                    Creating species DB                                                                                                                                                                                                                                                                                                                      
           DEBUG    Running command: ntm-profiler create_species_db -p ntmdb --sourmash_db sourmash.sbt.zip --accessions accessions.csv --load                                                                                                                                                                                                   utils.py:386
[03:54:47] DEBUG    Moving to Mycobacterium_abscessus                                                                                                                                                                                                                                                                                        ntm-profiler:211
           INFO                                                                                                                                                                                                                                                                                                                              ntm-profiler:214
                    Creating DB for M                                                                                                                                                                                                                                                                                                                        
           DEBUG    Running command: ntm-profiler create_resistance_db --prefix M --csv resistance_variants.csv --barcode barcode.bed --load                                                                                                                                                                                                     utils.py:386
[03:54:48] ERROR    Traceback (most recent call last):                                                                                                                                                                                                                                                                                           utils.py:391
                      File "/home/pgc29/.conda/envs/ntmprofiler_v0.3.0/bin/ntm-profiler", line 390, in <module>                                                                                                                                                                                                                                              
                        args.func(args)                                                                                                                                                                                                                                                                                                                      
                      File "/home/pgc29/.conda/envs/ntmprofiler_v0.3.0/bin/ntm-profiler", line 177, in create_resistance_db                                                                                                                                                                                                                                  
                        pp.create_db(args,extra_files=extra_files)                                                                                                                                                                                                                                                                                           
                      File "/home/pgc29/.conda/envs/ntmprofiler_v0.3.0/lib/python3.10/site-packages/pathogenprofiler/db.py", line 417, in create_db                                                                                                                                                                                                          
                        hgvs_variants = [r for r in csv.DictReader(open(args.csv))]                                                                                                                                                                                                                                                                          
                    FileNotFoundError: [Errno 2] No such file or directory: 'resistance_variants.csv'                                                                                                                                                                                                                                                        
                    Cleaning up after failed run                                                                                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                                                                                                             
Traceback (most recent call last):
  File "/home/pgc29/.conda/envs/ntmprofiler_v0.3.0/bin/ntm-profiler", line 390, in <module>
    args.func(args)
  File "/home/pgc29/.conda/envs/ntmprofiler_v0.3.0/bin/ntm-profiler", line 216, in main_update_db
    pp.run_cmd(f'ntm-profiler create_resistance_db --prefix {species.replace(" ","_")} --csv resistance_variants.csv {barcode_arg} --load')
  File "/home/pgc29/.conda/envs/ntmprofiler_v0.3.0/lib/python3.10/site-packages/pathogenprofiler/utils.py", line 392, in run_cmd
    raise ValueError("Command Failed:\n%s\nstderr:\n%s" % (cmd,result.stderr.decode()))
ValueError: Command Failed:
/bin/bash -c set -o pipefail; ntm-profiler create_resistance_db --prefix M --csv resistance_variants.csv --barcode barcode.bed --load
stderr:
Traceback (most recent call last):
  File "/home/pgc29/.conda/envs/ntmprofiler_v0.3.0/bin/ntm-profiler", line 390, in <module>
    args.func(args)
  File "/home/pgc29/.conda/envs/ntmprofiler_v0.3.0/bin/ntm-profiler", line 177, in create_resistance_db
    pp.create_db(args,extra_files=extra_files)
  File "/home/pgc29/.conda/envs/ntmprofiler_v0.3.0/lib/python3.10/site-packages/pathogenprofiler/db.py", line 417, in create_db
    hgvs_variants = [r for r in csv.DictReader(open(args.csv))]
FileNotFoundError: [Errno 2] No such file or directory: 'resistance_variants.csv'
Cleaning up after failed run

Cleaning up after failed run
           ERROR                                                                                                                                                                                                                                                                                                                              ntm-profiler:55
                                                                                                                                                                                                                                                                                                                                                             
                    ################################# ERROR #######################################                                                                                                                                                                                                                                                          
                                                                                                                                                                                                                                                                                                                                                             
                    This run has failed. Please check all arguments and make sure all input files                                                                                                                                                                                                                                                            
                    exist. If no solution is found, please open up an issue at                                                                                                                                                                                                                                                                               
                    https://github.com/jodyphelan/NTM-Profiler/issues/new and paste or attach the                                                                                                                                                                                                                                                            
                    contents of the error log (1d14c959-8c57-4537-9d0a-364d402ae24d.errlog.txt)                                                                                                                                                                                                                                                              
                                                                                                                                                                                                                                                                                                                                                             
                    ###############################################################################                                                                                                                                                                                                                                                          
                                                                                                                          

The error log is

ntm-profiler error report

• OS: linux
• ntm-profiler version: 0.3.0
• pathogen-profiler version: 3.1.0
• Program call:

{'no_clean': False, 'logging': 'DEBUG', 'repo': 'https://github.com/ntm-db/ntm-db.git', 'branch': 'master', 'commit': None, 'dir': '.', 'temp': '.', 'func': <function main_update_db at 0x149324f12a70>, 'software_name': 'ntm-profiler', 'tmp_prefix': 'cb8c3dd4-29ec-49e7-9543-58eb6d414fce', 'files_prefix': './cb8c3dd4-29ec-49e7-9543-58eb6d414fce'}

Traceback:

  File "/home/pgc29/.conda/envs/ntmprofiler_v0.3.0/bin/ntm-profiler", line 390, in <module>
    args.func(args)
  File "/home/pgc29/.conda/envs/ntmprofiler_v0.3.0/bin/ntm-profiler", line 216, in main_update_db
    pp.run_cmd(f'ntm-profiler create_resistance_db --prefix {species.replace(" ","_")} --csv resistance_variants.csv {barcode_arg} --load')
  File "/home/pgc29/.conda/envs/ntmprofiler_v0.3.0/lib/python3.10/site-packages/pathogenprofiler/utils.py", line 392, in run_cmd
    raise ValueError("Command Failed:\n%s\nstderr:\n%s" % (cmd,result.stderr.decode()))

Value:

Command Failed:
/bin/bash -c set -o pipefail; ntm-profiler create_resistance_db --prefix M --csv resistance_variants.csv --barcode barcode.bed --load
stderr:
Traceback (most recent call last):
  File "/home/pgc29/.conda/envs/ntmprofiler_v0.3.0/bin/ntm-profiler", line 390, in <module>
    args.func(args)
  File "/home/pgc29/.conda/envs/ntmprofiler_v0.3.0/bin/ntm-profiler", line 177, in create_resistance_db
    pp.create_db(args,extra_files=extra_files)
  File "/home/pgc29/.conda/envs/ntmprofiler_v0.3.0/lib/python3.10/site-packages/pathogenprofiler/db.py", line 417, in create_db
    hgvs_variants = [r for r in csv.DictReader(open(args.csv))]
FileNotFoundError: [Errno 2] No such file or directory: 'resistance_variants.csv'
Cleaning up after failed run

[jodyphelan]

Thanks @pgcudahy , I'm going to be releasing a new version shortly that will have this fixed!