[BUG] htsjdk.samtools.util.RuntimeIOException: java.io.IOException: Invalid file pointer: 101939186246823 for gnomAD.r2.1.1.GRCh38.PASS.AC.AF.only.vcf.gz

[CloXD]

Hello,
first of all, thanks for the great pipeline.
I'm trying to process two samples, one normal and one tumour, using the following command:

nextflow run nf-core/sarek -r 2.7.1 --input ./samples.tsv -profile singularity --tools Mutect2,Strelka,Manta,TIDDIT,ASCAT,ControlFREEC,snp
Eff,VEP --genome GRCh38  

the samples.tsv (names simplified) is like:

TestSample	XY	0	TestSample_0	1	/data/exome/fastq/testA_1.fastq.gz	/data/exome/fastq/testA_2.fastq.gz
TestSample	XY	1	TestSample_1	1	/data/exome/fastq/testB_1.fastq.gz	/data/exome/fastq/testB_2.fastq.gz

From the reports, it looks like both MantaSingle and Mutect2Single are failing, but I get only a Mutec2Single error log:

Error executing process > 'Mutect2Single (WCMC4_1_C_1-chr6_61371373-95020790)'

Caused by:
  Process `Mutect2Single (WCMC4_1_C_1-chr6_61371373-95020790)` terminated with an error exit status (3)

Command executed:

  # Get raw calls
  gatk --java-options "-Xmx7g"       Mutect2       -R Homo_sapiens_assembly38.fasta      -I WCMC4_1_C_1.recal.bam  -tumor WCMC4_1_C_1       -L chr6_61371373-95020790.bed              --germline-resource gnomAD.r2.1.1.GRCh38.PASS.AC.AF.only.vcf.gz              -O chr6_61371373-95020790_WCMC4_1_C_1.vcf

Command exit status:
  3

Command output:
  (empty)

Command error:
  15:04:28.146 INFO  Mutect2 - Inflater: IntelInflater
  15:04:28.146 INFO  Mutect2 - GCS max retries/reopens: 20
  15:04:28.146 INFO  Mutect2 - Requester pays: disabled
  15:04:28.146 INFO  Mutect2 - Initializing engine
  15:04:28.731 INFO  FeatureManager - Using codec VCFCodec to read file file://gnomAD.r2.1.1.GRCh38.PASS.AC.AF.only.vcf.gz
  15:04:28.890 INFO  FeatureManager - Using codec BEDCodec to read file file://chr6_61371373-95020790.bed
  15:04:28.896 INFO  IntervalArgumentCollection - Processing 33649418 bp from intervals
  15:04:28.951 INFO  Mutect2 - Done initializing engine
  15:04:29.045 INFO  NativeLibraryLoader - Loading libgkl_utils.so from jar:file:/opt/conda/envs/nf-core-sarek-2.7.1/share/gatk4-4.1.7.0-0/gatk-package-4.1.7.0-local.jar!/com/intel/gkl/native/libgkl_utils.so
  15:04:29.056 INFO  NativeLibraryLoader - Loading libgkl_pairhmm_omp.so from jar:file:/opt/conda/envs/nf-core-sarek-2.7.1/share/gatk4-4.1.7.0-0/gatk-package-4.1.7.0-local.jar!/com/intel/gkl/native/libgkl_pairhmm_omp.so
  15:04:29.095 INFO  IntelPairHmm - Using CPU-supported AVX-512 instructions
  15:04:29.095 INFO  IntelPairHmm - Flush-to-zero (FTZ) is enabled when running PairHMM
  15:04:29.096 INFO  IntelPairHmm - Available threads: 80
  15:04:29.096 INFO  IntelPairHmm - Requested threads: 4
  15:04:29.096 INFO  PairHMM - Using the OpenMP multi-threaded AVX-accelerated native PairHMM implementation
  15:04:29.163 INFO  ProgressMeter - Starting traversal
  15:04:29.163 INFO  ProgressMeter -        Current Locus  Elapsed Minutes     Regions Processed   Regions/Minute
  15:04:29.575 INFO  VectorLoglessPairHMM - Time spent in setup for JNI call : 0.0
  15:04:29.575 INFO  PairHMM - Total compute time in PairHMM computeLogLikelihoods() : 0.0
  15:04:29.575 INFO  SmithWatermanAligner - Total compute time in java Smith-Waterman : 0.00 sec
  15:04:29.576 INFO  Mutect2 - Shutting down engine
  [February 2, 2022 3:04:29 PM UTC] org.broadinstitute.hellbender.tools.walkers.mutect.Mutect2 done. Elapsed time: 0.03 minutes.
  Runtime.totalMemory()=2185232384
  htsjdk.samtools.util.RuntimeIOException: java.io.IOException: Invalid file pointer: 101939186246823 for gnomAD.r2.1.1.GRCh38.PASS.AC.AF.only.vcf.gz
  	at htsjdk.tribble.readers.TabixIteratorLineReader.readLine(TabixIteratorLineReader.java:48)
  	at htsjdk.tribble.TabixFeatureReader$FeatureIterator.readNextRecord(TabixFeatureReader.java:170)
  	at htsjdk.tribble.TabixFeatureReader$FeatureIterator.(TabixFeatureReader.java:159)
  	at htsjdk.tribble.TabixFeatureReader.query(TabixFeatureReader.java:133)
  	at org.broadinstitute.hellbender.engine.FeatureDataSource.refillQueryCache(FeatureDataSource.java:567)
  	at org.broadinstitute.hellbender.engine.FeatureDataSource.queryAndPrefetch(FeatureDataSource.java:536)
  	at org.broadinstitute.hellbender.engine.FeatureManager.getFeatures(FeatureManager.java:353)
  	at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:173)
  	at org.broadinstitute.hellbender.tools.walkers.mutect.Mutect2Engine.isActive(Mutect2Engine.java:389)
  	at org.broadinstitute.hellbender.engine.AssemblyRegionIterator.loadNextAssemblyRegion(AssemblyRegionIterator.java:136)
  	at org.broadinstitute.hellbender.engine.AssemblyRegionIterator.next(AssemblyRegionIterator.java:112)
  	at org.broadinstitute.hellbender.engine.AssemblyRegionIterator.next(AssemblyRegionIterator.java:35)
  	at org.broadinstitute.hellbender.engine.AssemblyRegionWalker.processReadShard(AssemblyRegionWalker.java:192)
  	at org.broadinstitute.hellbender.engine.AssemblyRegionWalker.traverse(AssemblyRegionWalker.java:173)
  	at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1048)
  	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)
  	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
  	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
  	at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:163)
  	at org.broadinstitute.hellbender.Main.mainEntry(Main.java:206)
  	at org.broadinstitute.hellbender.Main.main(Main.java:292)
  Caused by: java.io.IOException: Invalid file pointer: 101939186246823 for gnomAD.r2.1.1.GRCh38.PASS.AC.AF.only.vcf.gz
  	at htsjdk.samtools.util.BlockCompressedInputStream.seek(BlockCompressedInputStream.java:385)
  	at htsjdk.tribble.readers.TabixReader$IteratorImpl.next(TabixReader.java:427)
  	at htsjdk.tribble.readers.TabixIteratorLineReader.readLine(TabixIteratorLineReader.java:46)
  	... 20 more

Any help is appreciated.

singularity version: 3.8.5-2.el8
Workflow repository: https://github.com/nf-core/sarek, revision 2.7.1 (commit hash 68b9930)
Workflow profile: singularity
Workflow container: nfcore/sarek:2.7.1
Container engine: singularity
Nextflow version: version 21.10.6, build 5660 (21-12-2021 16:55 UTC)

[FriederikeHanssen]

Hi @CloXD ! Could you please post the full command? The above can't be all, since --tools is not included, as well as the .nextflow.log if possible.

[CloXD]

Sorry, I modified the previous post.

[CloXD]

nextflow.log.zip
Here's the nextflow log.
I'm cleaning everything and re-running the whole pipeline to give you a full log.

nextflow.zip
Hi @FriederikeHanssen , here's the complete log ( not the one using the -resume ).
It's plenty of warnings like

WARN  c.a.i.InstanceMetadataServiceResourceFetcher - Fail to retrieve token 

Could it be the cause of the problem? Am I missing some configuration?
Sorry for the inconvenience, It's my first experience with nextflow.
Thanks in advance,
Claudio

[CloXD]

Solved on slack, It was necessary to setup an aws account and the aws CLI tool

https://nfcore.slack.com/archives/CGFUX04HZ/p1643893577527149

[brucemoran]

Hi, I can't see or sign up to the Slack IDKY.

Can you outline what you did to solve here as I am seeing this error also?

Thanks, Bruce.

[CloXD]

Hi Bruce,
at last it was enough to install and setup the aws cli https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html#install-next-steps
Afterwards the error should be solved