Exit: Error: Directory Hierarchy of rawdata '/home/genome/Caulobacter/HiC_data/' is not correct. No '.fastq(.gz)' files detected

[xiaoaozqd]

Dear,
When I use Hic-Pro (HiC-Pro_2.10.0) on (linux 3.10.0-693.21.1.el7.x86_64 #1 x86_64 GNU/Linux) with the code "HiC-Pro -i /home/genome/Caulobacter/HiC_data/ -o /home/genome/Caulobacter/HiC_data_out -c /home/genome/Caulobacter/config-hicpro.txt", there is an error "Exit: Error: Directory Hierarchy of rawdata '/home/genome/Caulobacter/HiC_data/' is not correct. No '.fastq(.gz)' files detected".
while the fastq files just in this file
"ll /home/genome/Caulobacter/HiC_data/
总用量 9965776
-rw-rw-r-- 1 zengqd zengqd 5102474162 7月 27 20:12 SRR824846_1.fastq
-rw-rw-r-- 1 zengqd zengqd 5102474162 7月 27 20:12 SRR824846_2.fastq
"
my config-hicpro.txt as:
cat config-hicpro.txt

Please change the variable settings below if necessary

#########################################################################

Paths and Settings - Do not edit !

#########################################################################

TMP_DIR = tmp
LOGS_DIR = logs
BOWTIE2_OUTPUT_DIR = bowtie_results
MAPC_OUTPUT = hic_results
RAW_DIR = rawdata

#######################################################################

SYSTEM AND SCHEDULER - Start Editing Here !!

#######################################################################
N_CPU = 40
LOGFILE = hicpro.log

JOB_NAME =
JOB_MEM =
JOB_WALLTIME =
JOB_QUEUE =
JOB_MAIL =

#########################################################################

Data

#########################################################################

PAIR1_EXT = _R1
PAIR2_EXT = _R2

#######################################################################

Alignment options

#######################################################################

FORMAT = phred33
MIN_MAPQ = 0

BOWTIE2_IDX_PATH = /home/genome/Caulobacter
BOWTIE2_GLOBAL_OPTIONS = --very-sensitive -L 30 --score-min L,-0.6,-0.2 --end-to-end --reorder
BOWTIE2_LOCAL_OPTIONS = --very-sensitive -L 20 --score-min L,-0.6,-0.2 --end-to-end --reorder

#######################################################################

Annotation files

#######################################################################

REFERENCE_GENOME = bacteria
GENOME_SIZE = bacteria.size
CAPTURE_TARGET =

#######################################################################

Allele specific analysis

#######################################################################

ALLELE_SPECIFIC_SNP =

#######################################################################

Digestion Hi-C

#######################################################################

GENOME_FRAGMENT = /home/genome/Caulobacter/bacteria_dpnii.bed
LIGATION_SITE = AAGCTAGCTT
MIN_FRAG_SIZE =
MAX_FRAG_SIZE =
MIN_INSERT_SIZE =
MAX_INSERT_SIZE =

#######################################################################

Hi-C processing

#######################################################################

MIN_CIS_DIST =
GET_ALL_INTERACTION_CLASSES = 1
GET_PROCESS_SAM = 0
RM_SINGLETON = 1
RM_MULTI = 1
RM_DUP = 1

#######################################################################

Contact Maps

#######################################################################

#BIN_SIZE = 20000 40000 100000 150000 500000 1000000
BIN_SIZE = 100000
MATRIX_FORMAT = upper

#######################################################################

Normalization

#######################################################################
MAX_ITER = 100
FILTER_LOW_COUNT_PERC = 0.02
FILTER_HIGH_COUNT_PERC = 0
EPS = 0.1

Could you kindhearted help me to fix this problem?

[nservant]

Hi
Your input file is not correct.
You need to have one folder per sample, ie.
HiC_data
++ sample1
++ ++ SRR824846_1.fastq
++ ++ SRR824846_2.fastq

You will also need to update the config file with
PAIR1_EXT = _1
PAIR2_EXT = _2

And please, check to the index files are
/home/genome/Caulobacter/bacteria*

Best

[xiaoaozqd]

Dear nservant,
So appreciation for your help!
It's worked!
Best wishes!
Qingdong

[nservant]

Hi, Which HiC-Pro version are you using ? Could you show me the data organization, and the config file. Thanks N Le 09/03/2019 à 18:59, GitHub-Lujianjun a écrit :

…

Dear Qingdong: I also have the some quesion you had described before. I have tried many times according to @navrant's advises, but still not work! The only difference from you is that my HiC data are compressed in " *fastq.gz". BTW, I'm curious about that "PAIR1_EXT = _1" and "PAIR1_EXT = _2",why not the full name like "PAIR1_EXT = SRR824846_1" and "PAIR1_EXT =SRR824846_2" ? Do you have any ideas for me? Thankyou so much! JJLu. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#168 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKR2azZ-hxHvSk0RsfoDHlU2etli47Grks5vU_aYgaJpZM4Vjc5r>.

[ViriatoII]

The same happens to me, but they are there..

bin/HiC-Pro -i /gpfs/project/projects/qggp/Potatodenovo/data/HiC/sample1 -o /gpfs/project/projects/qggp/Potatodenovo/results/10x/HiC-pro -c config-hicpro.txt -p

ll /gpfs/project/projects/qggp/Potatodenovo/data/HiC/sample1
-rwxrwx--- 1 riesd QGGP 46656175972 May 4 04:19 HiC_R1.fastq
-rwxrwx--- 1 riesd QGGP 46640555968 May 4 04:34 HiC_R2.fastq

[porpheria]

Dear Viriatoll, I was also facing similar kind of issue recently. When I checked the log file of HiC-Pro, I noticed that my system created a hidden system/temp folder with the same directory name, and HiC-Pro actually looking for those fastq files on that hidden system/temp folder. Deleting those hidden system/temp directories solved my issue. You can also try the same, to resolve your issue. Have a good day. Sourav Ghosh

…

On Tue, Jul 9, 2019 at 4:59 PM ViriatoII ***@***.***> wrote: The same happens to me, but they are there.. ll /gpfs/project/projects/qggp/Potatodenovo/data/HiC/sample1 -rwxrwx--- 1 riesd QGGP 46656175972 May 4 04:19 HiC_R1.fastq -rwxrwx--- 1 riesd QGGP 46640555968 May 4 04:34 HiC_R2.fastq — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#168?email_source=notifications&email_token=ABVBVQYWPEEDJK2MVEASNJ3P6SRT7A5CNFSM4FMNZZV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZQRFNQ#issuecomment-509678262>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABVBVQZH7ZEV6A6NHFYRNMTP6SRT7ANCNFSM4FMNZZVQ> .

[nservant]

Please run HiC-Pro with
bin/HiC-Pro -i /gpfs/project/projects/qggp/Potatodenovo/data/HiC/ -o /gpfs/project/projects/qggp/Potatodenovo/results/10x/HiC-pro -c config-hicpro.txt -p

[wuxiaopei0509]

I encounter the same issue, I have only one sample,and construst the reads directory like
02Hi-C_reads
++ Sample1
++ ++ Hi-C_R1.fq.gz
++ ++ Hi-C_R2.fq.gz
but it also occured 'Exit: Error: Directory Hierarchy of rawdata '~/02Hi-C_reads/' is not correct. No '.fastq(.gz)' files detected ',why does it happen?
Thank you !

[JeffreyMaurer]

This is the first result when I google the issue. I think the issue for wuxiaopei0509 is that HiC Pro doesn't accept gzipped fastq files.

I could be wrong, but I didn't find anything contradictory in the documentation.