No filtering was done! Exiting...

[mictadlo]

Hi,
Thank you for sharing your cafe pipeline. However, I ran into the following problems:

> awk -F'\t' '{print "(null)\t"$0}' Orthogroups.GeneCount.tsv > tmp.tsv
> sed -i.bak 's|(null)|Desc|g' tmp.tsv

> head tmp.tsv
Desc	Orthogroup	Atha	Cann	Csat	Mgut	Natt	Nlab	Nobt	Noto	Nqld	Nsyl	Ntab	Ntom	Ptri	Slyc	Smel	Stub	Vvin	Total
Desc	OG0000000	0	0	0	0	0	3	0	0	982	0	0	1	0	0	0	986
Desc	OG0000001	0	9	0	0	5	23	8	59	3	31	267	14	0	6	113	540

> python2.7 ~/scripts/cafe/cafetutorial_clade_and_size_filter.py -i tmp.tsv -s -o cafe.input.tsv
No filtering was done! Exiting...

> conda activate cafe
> cafe
# load -i tmp.tsv reports/log_run1.txt
Failed to load file

What did I miss?

Thank you in advance

[harish0201]

Ah! It's been some time that I've looked at this, so apologies for the confusion ;)

In the tmp.tsv, as per CAFE's format, the first column should be the (null) with the header being Desc. So what you've shared should look like:

head tmp.tsv
Desc	Orthogroup	Atha	Cann	Csat	Mgut	Natt	Nlab	Nobt	Noto	Nqld	Nsyl	Ntab	Ntom	Ptri	Slyc	Smel	Stub	Vvin	Total
(null)	OG0000000	0	0	0	0	0	3	0	0	982	0	0	1	0	0	0	986
(null)	OG0000001	0	9	0	0	5	23	8	59	3	31	267	14	0	6	113	540

IIRC, this is what the awk portion actually does, only the "first column header" should be the "Desc".

Once this is done, you can run the cafetutorial_clade_and_size_filter.py script that CAFE has.

In any case, what it does is, it's going to remove the OGs/Groups if any of the species have more than 100 proteins in them.

Once this is done, you'd need to create a rooted tree. I recommend using Timetree.org or other age approaches (ape package in R is a good started)

Unfortunately, the way I've arranged the CAFE thing is essentially a guideline as to how I do it, rather than a complete pipeline because I kept running into issues :(

I might automate it some day, hopefully, but the use case for me is fairly too low to justify the time sunk into it.

[mictadlo]

Thank you. The new file looks like this:

> head tmp.tsv
Desc	Orthogroup	Atha	Cann	Csat	Mgut	Natt	Nlab	Nobt	Noto	Nqld	Nsyl	Ntab	Ntom	Ptri	Slyc	Smel	Stub	Vvin	Total
(null)	OG0000000	0	0	0	0	0	3	0	0	982	0	0	1	0	0	0	0	0	986
(null)	OG0000001	0	9	0	0	5	23	8	59	3	31	267	14	0	6	113	2	0	540
(null)	OG0000002	0	128	3	20	21	21	8	42	21	36	70	37	3	25	38	39	22	534
(null)	OG0000003	1	66	12	20	29	17	12	42	21	30	60	21	5	19	36	84	42	517

The age (28 MYA) has been provided to me by a collaborator. Unfortunately, I got a new error:

> make_ultrametric.py -r 28 SpeciesTree_rooted.txt
> head SpeciesTree_rooted.txt.ultrametric.tre
(((Ptri:21.5258,(Csat:19.4873,Atha:19.4873):2.03847):2.54693,Vvin:24.0727):3.92728,(Mgut:23.5518,((Cann:4.94278,((Stub:1.91372,Slyc:1.91372):1.99847,Smel:3.91219):1.0306):2.54854,((Natt:2.59231,((Nsyl:1.44399,Ntab:1.44399):0.826096,(Nlab:0.508663,Nqld:0.508663):1.76142):0.322218):0.530061,((Ntom:1.26249,Noto:1.26249):1.11482,Nobt:2.37732):0.745047):4.36896):16.0605):4.44819);

python2.7 ~/scripts/cafe/cafetutorial_clade_and_size_filter.py -i tmp.tsv -s -o cafe.input.tsv
> cafe
# load -i cafe.input.tsv
-----------------------------------------------------------
Family information: cafe.input.tsv
Log: stdout
The number of families is 35104
Root Family size : 1 ~ 124
Family size : 0 ~ 149
P-value: 0.01
Num of Threads: 1
Num of Random: 1000
# tree (((Ptri:21.5258,(Csat:19.4873,Atha:19.4873):2.03847):2.54693,Vvin:24.0727):3.92728,(Mgut:23.5518,((Cann:4.94278,((Stub:1.91372,Slyc:1.91372):1.99847,Smel:3.91219):1.0306):2.54854,((Natt:2.59231,((Nsyl:1.44399,Ntab:1.44399):0.826096,(Nlab:0.508663,Nqld:0.508663):1.76142):0.322218):0.530061,((Ntom:1.26249,Noto:1.26249):1.11482,Nobt:2.37732):0.745047):4.36896):16.0605):4.44819);
(((Ptri:21.5258,(Csat:19.4873,Atha:19.4873):2.03847):2.54693,Vvin:24.0727):3.92728,(Mgut:23.5518,((Cann:4.94278,((Stub:1.91372,Slyc:1.91372):1.99847,Smel:3.91219):1.0306):2.54854,((Natt:2.59231,((Nsyl:1.44399,Ntab:1.44399):0.826096,(Nlab:0.508663,Nqld:0.508663):1.76142):0.322218):0.530061,((Ntom:1.26249,Noto:1.26249):1.11482,Nobt:2.37732):0.745047):4.36896):16.0605):4.44819)
No species 'total' was found in the tree

What did I miss?

Thank you in advance,

Michal

[harish0201]

Hey!

You have to remove the "Total" column, that won't be needed. The error specifies that :)

[mictadlo]

Thank you,, I have forgotten to do awk -F'\t' '{$NF=""; print $0}' tmp.tsv | rev | sed 's/^\s*//g' | rev | tr ' ' '\t' > mod.tsv. Unfortunately, I got now Tree must be binary:

> awk -F'\t' '{$NF=""; print $0}' tmp.tsv | rev | sed 's/^\s*//g' | rev | tr ' ' '\t' > mod.tsv
> python2.7 ~/scripts/cafe/cafetutorial_clade_and_size_filter.py -i mod.tsv -s -o cafe.input.tsv  
>  cafe
# load -i cafe.input.tsv
-----------------------------------------------------------
Family information: cafe.input.tsv
Log: stdout
The number of families is 27493
Root Family size : 1 ~ 124
Family size : 0 ~ 149
P-value: 0.01
Num of Threads: 1
Num of Random: 1000
# tree (((Ptri:21.5258,(Csat:19.4873,Atha:19.4873):2.03847):2.54693,Vvin:24.0727):3.92728,(Mgut:23.5518,((Cann:4.94278,((Stub:1.91372,Slyc:1.91372):1.99847,Smel:3.91219):1.0306):2.54854,((Natt:2.59231,((Nsyl:1.44399,Ntab:1.44399):0.826096,(Nlab:0.508663,Nqld:0.508663):1.76142):0.322218):0.530061,((Ntom:1.26249,Noto:1.26249):1.11482,Nobt:2.37732):0.745047):4.36896):16.0605):4.44819);
(((Ptri:21.5258,(Csat:19.4873,Atha:19.4873):2.03847):2.54693,Vvin:24.0727):3.92728,(Mgut:23.5518,((Cann:4.94278,((Stub:1.91372,Slyc:1.91372):1.99847,Smel:3.91219):1.0306):2.54854,((Natt:2.59231,((Nsyl:1.44399,Ntab:1.44399):0.826096,(Nlab:0.508663,Nqld:0.508663):1.76142):0.322218):0.530061,((Ntom:1.26249,Noto:1.26249):1.11482,Nobt:2.37732):0.745047):4.36896):16.0605):4.44819)
# lambda -s -t ((1,1)1,(1,1):1,1)
Tree must be binary

What did I miss?

Thank you in advance,

Michal

[harish0201]

The lambda tree that you've pasted should look like the tree that you've generated (Ptri.....)

or rather taking a small portion of your ultrametric tree: (Ptri:21.5258,(Csat:19.4873,Atha:19.4873):2.03847):2.54693,Vvin:24.0727)

the lambda tree would look like: (1,(2,2)),1) assuming Csat is your species of interest. My recommendation is go through the CAFE tutorial once though, because it covers this in more detail.