Conflict with two diamond versions

[g1o]

In my server there are two diamond versions, one for my conda enviroment and other in /usr/local/bin/diamond

The diamond makedb, with the default config.json, somehow used one version to index the databases but another when comparing the species 0 vs 0 and species 1 vs 1. However, it was fine comparing the species 1 vs 0 or species 0 vs 1.

This was solved by removing the other diamond version. But changing the config.json to the absolute path of one diamond could work too.

I am including this as an issue as using two versions in the same run was unexpected.
Also, there should be a check in case of a failure that ends with an empty 0 vs 0 or 1 vs 1 diamond/blast files, or at least a better BIG warning in this extreme case. Instead of just :
WARNING: Too few hits between species 0 and species 0 to normalise the scores, these hits will be ignored
WARNING: Too few hits between species 1 and species 1 to normalise the scores, these hits will be ignored
To something like:
WARNING: THIS IS UNCOMMON, there is ZERO hits between species 0 and species 0 to normalise the scores, these hits will be ignored, check the diamond/blast command for this one
WARNING: THIS IS UNCOMMON, there is ZERO hits between species 1 and species 1 to normalise the scores, these hits will be ignored, check the diamond/blast command for this one

OrthoFinder version 2.5.4:
Running diamond all-versus-all

Using 64 thread(s)
2021-09-05 22:36:46 : This may take some time....

ERROR: external program called by OrthoFinder returned an error code: -4

Command: diamond blastp -d ~/ortho/orthofinder/OrthoFinder/Results_Sep05/WorkingDirectory/diamondDBSpecies0 -q ~/ortho/orthofinder/OrthoFinder/Results_Sep05/WorkingDirectory/Species0.fa -o ~/ortho/orthofinder/OrthoFinder/Results_Sep05/WorkingDirectory/Blast0_0.txt --more-sensitive -p 1 --quiet -e 0.001 --compress 1

stdout

stderr

ERROR: external program called by OrthoFinder returned an error code: -4

Command: diamond blastp -d ~/ortho/orthofinder/OrthoFinder/Results_Sep05/WorkingDirectory/diamondDBSpecies1 -q ~/ortho/orthofinder/OrthoFinder/Results_Sep05/WorkingDirectory/Species1.fa -o ~/ortho/orthofinder/OrthoFinder/Results_Sep05/WorkingDirectory/Blast1_1.txt --more-sensitive -p 1 --quiet -e 0.001 --compress 1

stdout

stderr

2021-09-05 22:39:38 : Done all-versus-all sequence search

Running OrthoFinder algorithm

[davidemms]

Hi

Thanks for reporting this. I've looked carefully at the code and so far I can't see a way for different versions to be called for the database construction and database search. Perhaps there's something I'm missing, but I've confirmed that the same line of code with the same set of environment variables is executing both commands. The operating system will search the directories in the path variable in order for 'diamond' so this should return the same result each time.

The way to confirm this is to edit the config.json file for how orthofinder calls diamond to this:

    "diamond":{
    "program_type": "search",
    "db_cmd": "which diamond >> /tmp/version_db.txt ; diamond makedb --in INPUT -d OUTPUT",
    "search_cmd": "which diamond >> /tmp/version_search.txt ; diamond blastp -d DATABASE -q INPUT -o OUTPUT --more-sensitive -p 1 --quiet -e 0.001 --compress 1"
    },

This uses the linux 'which' command so that each time diamond is called it will write the path to the executable used for the database construction to "/tmp/version_db.txt" and for the search to "/tmp/version_search.txt". This will confirm which executable is used in each case.

If a version mismatch isn't the cause of the problem then the question still remains what was. Could orthofinder have been trying to use the version of diamond that you later deleted, and there was a problem with this version? Or is there another potential cause for the issue?

I've implemented your suggestion for updating the warning message.

Best wishes
David

[g1o]

It was probably a very unusual behaviour within my conda enviroment.
I tried to reproduce the problem and use which diamond in the config,json. But this time there was no error and the path was the same.
Thanks for implementing the warning message, it may be useful in strange errors like this.