Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Doubts processing COVID-19 samples #12

Open
vanessassilva opened this issue Aug 15, 2020 · 4 comments
Open

Doubts processing COVID-19 samples #12

vanessassilva opened this issue Aug 15, 2020 · 4 comments

Comments

@vanessassilva
Copy link

vanessassilva commented Aug 15, 2020 •
edited
Loading

Hi Pierre Bost,

I downloaded the nine FASTQ files for series GSE145926, via SRA in the original format, conform reported from paper. So I end up with two files the each sample. Did you use the 18 FASTQ files of COVID-19 samples to analysis? if so, after doing all the preparation of the files, how long did it take to run the scripts? When I run Viral-Track Scanning.R script, reported me the error:

samtools index: "/home/vanessa/star_cv/output_viral_track//SRR11181955_2//SRR11181955_2Aligned.sortedByCoord.out.bam" is in a format that cannot be usefully indexed
Indexing of the bam file for SRR11181955_2 is done
samtools idxstats: fail to load index for "/home/vanessa/star_cv/output_viral_track//SRR11181955_2//SRR11181955_2Aligned.sortedByCoord.out.bam"
Computing stat file for the bam file for SRR11181955_2 is done
Error in read.table(temp_chromosome_count_path, header = F, row.names = 1) :
no lines available in input
Execution halted

What am I doing wrong? What solution do you suggest?

Another question, about the analysis with MetaCell, this analysis was carried out separated of the scripts, correct?

Best regards,
Vanessa
@yangjywhu
Copy link

I have the same problem：

[Mon Aug 31 19:42:08 2020]
rule viraltrack:
    input: result/viraltrack/umi_tools/p1_R2_extraced.fastq.gz, ref/hg38_WIV04/STARindex/STARindex, script/Viral-Track/Viral_Track_scanning.R, script/Viral-Track/Virusite_annotation_file.txt
    output: result/viraltrack/scanning/p1_R2/Parameters.txt, result/viraltrack/scanning/p1_R2/Files_to_process.txt, result/viraltrack/scanning/p1_R2/outs
    jobid: 1
    wildcards: sample=p1, read=R2

Loading of the libraries.... ... done ! Warning message:
Output directory does not exist ! Creating it ! 
1 Fastq files are going to be processed ! 
Mapping p1_R2_extraced.fastq file 
Aug 31 19:42:40 ..... started STAR run
Aug 31 19:42:40 ..... loading genome
Aug 31 20:19:08 ..... started 1st pass mapping
Mapping ofp1_R2_extraced.fastq done ! 
All fastq files have been mapped successfully 
Starting the BAM file analysis 
samtools index: "result/viraltrack/scanning/p1_R2/outs/p1_R2_extraced//p1_R2_extraced_Aligned.sortedByCoord.out.bam" is in a format that cannot be usefully indexed
Indexing of the bam file for p1_R2_extraced is done 
samtools idxstats: fail to load index for "result/viraltrack/scanning/p1_R2/outs/p1_R2_extraced//p1_R2_extraced_Aligned.sortedByCoord.out.bam"
Computing stat file for the bam file for p1_R2_extraced is done 
Error in read.table(temp_chromosome_count_path, header = F, row.names = 1) : 
  no lines available in input
Execution halted
[Mon Aug 31 20:19:37 2020]
Error in rule viraltrack:
    jobid: 1
    output: result/viraltrack/scanning/p1_R2/Parameters.txt, result/viraltrack/scanning/p1_R2/Files_to_process.txt, result/viraltrack/scanning/p1_R2/outs
    shell:
        
        echo 'N_thread=1
Output_directory="result/viraltrack/scanning/p1_R2/outs"
Index_genome="ref/hg38_WIV04/STARindex/STARindex"
Viral_annotation_file="script/Viral-Track/Virusite_annotation_file.txt"
Name_run="cov2sc_p1_R2"
Load_STAR_module=FALSE
Load_samtools_module=FALSE
Load_stringtie_module=FALSE
Minimal_read_mapped=5
Single_cell_metadata="script/config.yaml"
' > result/viraltrack/scanning/p1_R2/Parameters.txt
        echo "result/viraltrack/umi_tools/p1_R2_extraced.fastq.gz" > result/viraltrack/scanning/p1_R2/Files_to_process.txt
        set +u; source ~/miniconda3/bin/activate viraltrack; set -u
        Rscript script/Viral-Track/Viral_Track_scanning.R result/viraltrack/scanning/p1_R2/Parameters.txt result/viraltrack/scanning/p1_R2/Files_to_process.txt
        set +u; conda deactivate; set -u
        
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job viraltrack since they might be corrupted:
result/viraltrack/scanning/p1_R2/Parameters.txt, result/viraltrack/scanning/p1_R2/Files_to_process.txt, result/viraltrack/scanning/p1_R2/outs
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

@PierreBSC
Copy link
Owner

Dear Vanessa and Jiayi,

Thank you for your feedback. This is indeed weird : first according to Jiayi's log file, STAR only performs a single-pass mapping and not a two-pass mapping, but this should only be secondary and not causes this problem.
Can you check that the resulting bam exists and is not empty ?
Best

Pierre

@yangjywhu
Copy link

yangjywhu commented Sep 1, 2020 •
edited
Loading

Hello @PierreBSC,

The output directory is:

yangjiayi@localhost:~/project/cov2sc/result/viraltrack/scanning/p1_R2/outs/p1_R2_extraced
$ ll
total 52K
-rw-r--r--. 1 yangjiayi zhoulab    0 Sep  1 00:25 Count_chromosomes.txt
-rw-r--r--. 1 yangjiayi zhoulab    0 Aug 31 23:16 p1_R2_extraced_Aligned.sortedByCoord.out.bam
-rw-r--r--. 1 yangjiayi zhoulab  29K Sep  1 00:25 p1_R2_extraced_Log.out
-rw-r--r--. 1 yangjiayi zhoulab 5.1K Sep  1 00:25 p1_R2_extraced_Log.progress.out
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:22 p1_R2_extraced__STARgenome
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:22 p1_R2_extraced__STARpass1
drwx------. 3 yangjiayi zhoulab 4.0K Sep  1 00:24 p1_R2_extraced__STARtmp

Unfortunately, the bam file is empty, but Log.progress.out is not empty, and it tell me Finished 1st pass mapping
But, the bug is reported in Log.out:

BAMoutput.cpp:27:BAMoutput: exiting because of *OUTPUT FILE* error: could not create output file result/viraltrack/scanning/p1_R2/outs/p1_R2_extraced/p1_R2_extraced__STARtmp//BAMsort/19/46
SOLUTION: check that the path exists and you have write permission for this file. Also check ulimit -n and increase it to allow more open files.

So i check p1_R2_extraced__STARtmp//BAMsort.

yangjiayi@localhost:~/project/cov2sc/result/viraltrack/scanning/p1_R2/outs/p1_R2_extraced
$ ll p1_R2_extraced__STARtmp
total 5.2M
drwx------. 22 yangjiayi zhoulab 4.0K Sep  1 00:25 BAMsort
-rw-r--r--.  1 yangjiayi zhoulab  109 Sep  1 00:25 readFilesIn.info
-rwx------.  1 yangjiayi zhoulab  177 Sep  1 00:25 readsCommand_read1
-rw-r--r--.  1 yangjiayi zhoulab 5.2M Sep  1 00:22 SJ.start_gap.tsv
prw-------.  1 yangjiayi zhoulab    0 Sep  1 00:25 tmp.fifo.read1
yangjiayi@localhost:~/project/cov2sc/result/viraltrack/scanning/p1_R2/outs/p1_R2_extraced
$ ll p1_R2_extraced__STARtmp/BAMsort/
total 80K
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 0
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 1
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 10
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 11
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 12
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 13
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 14
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 15
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 16
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 17
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 18
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 19
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 2
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 3
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 4
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 5
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 6
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 7
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 8
drwx------. 2 yangjiayi zhoulab 4.0K Sep  1 00:25 9
yangjiayi@localhost:~/project/cov2sc/result/viraltrack/scanning/p1_R2/outs/p1_R2_extraced
$ ll p1_R2_extraced__STARtmp/BAMsort/19
total 0
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 0
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 1
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 10
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 11
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 12
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 13
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 14
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 15
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 16
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 17
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 18
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 19
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 2
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 20
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 21
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 22
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 23
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 24
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 25
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 26
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 27
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 28
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 29
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 3
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 30
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 31
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 32
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 33
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 34
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 35
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 36
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 37
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 38
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 39
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 4
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 40
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 41
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 42
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 43
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 44
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 45
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 5
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 6
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 7
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 8
-rw-r--r--. 1 yangjiayi zhoulab 0 Sep  1 00:25 9

Working directory is ~/project/cov2sc and i have permission on it.
Thank you for your attention on this problem

Jiayi Yang

@yangjywhu
Copy link

yangjywhu commented Sep 15, 2020 •
edited
Loading

hi @PierreBSC
This problem still occurs, and I can’t use Viral-Track. Could you give me some suggestions?
Output Directory is update above.

Thanks
Jiayi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@PierreBSC @yangjywhu @vanessassilva