Bad answer to (procedures treating cataracts)

[cbizon]

Query:
https://github.com/NCATSTranslator/testing/blob/main/ars-requests/not-none/1.2/cataractTreatment.json
Results:
https://arax.ncats.io/?r=1ef9d36e-dd59-4248-b0fc-fb588a387010

The query is "procedure that treats cataracts". RTX KG2 is retuning a bunch of answers relating to kidneys, with very general (low IC) terms for the procedure (e.g. "Therapeutic Procedure").

Then AC happily says, "hey what do a bunch of kidney diseases and eye diseases have in common?" and finds some garbage high level node like "disease by anatomical region" and merges everything together.

Then the ranker looks at that, says "great, so many nodes!" and gives a high score.

So I think that there are multiple things that could be done here, affecting different components:

1. RTX I think shouldn't be returning those results, I have an issue into them
2. Should strider try to verify the subclass of and filter in cases when it thinks the KPs are wrong? How much trust vs verify do we need in strider?
3. AC should probably be tuned; I doubt that disease by anatomical feature should ever be considered an enrichment?
4. Ranker should downweight this answer based on the low IC of the "disease by anatomical feature"

[cbizon]

@kennethmorton thoughts?

[cbizon]

After discussion on aragorn today, we decided the following:

1. Strider really has to trust the KPs. If they give us errors, we're just in trouble, but it's going to be too slow to double check everything all the time
2. We need to go back to AC and check the denominators, the disease by anatomy is not a good grouping. We have an issue about this already.
3. Ranker should make use of node information content to modulate scores.

[cbizon]

4. We need to start a collection of queries to keep an eye on.

[cbizon]

I don't think that this is an aragorn issue per-se, but holding it open here to coordinate some of the other work.