phase 1: add qualifier info from x-bte operations to /meta_knowledge_graph responses

[colleenXu]

From bullet 4 of the changelog:

In MetaKnowledgeGraph component, added MetaEdge.qualifiers as list of new MetaQualifier types https://github.com/NCATSTranslator/ReasonerAPI/pull/387/files

With TRAPI 1.4, tools can specify qualifiers in their /meta_knowledge_graph endpoint responses (edges section). We'll want to use this format to advertise the qualifier info in some of our x-bte operations. This applies to all 3 kinds of /meta_knowledge_graph endpoints: the api-specific (v1/smartapi), team-specific (v1/team) and general-ARA (v1)

• timing: phase 1. Other ARAs use our api-specific and team-specific endpoints as KPs, and this will give them the info they need to query us more effectively
• priority: medium

example of expected output

BTE should take the DGIdb x-bte operations listed below (yaml here) and create something like this in in the /meta_knowledge_graph output (v1/smartapi for dgidb, v1/team for Service Provider, and v1):

• activator
• agonist
• allosteric_modulator
• antagonist
• blocker
• inhibitor
• inverse_agonist
• modulator
• partial_agonist
• positive_modulator

{
  "nodes": {...},
  "edges": [
    {
      "subject": "biolink:SmallMolecule",
      "predicate": "biolink:affects",
      "object": "biolink:Gene",
      "knowledge_types": [
        "lookup"
      ],
      "qualifiers": [
        {
          "qualifier_type_id": "biolink:qualified_predicate",
          "applicable_values": [
            "biolink:causes"
          ]
        },
        {
          "qualifier_type_id": "biolink:object_aspect_qualifier",
          "applicable_values": [
            "activity",
            "activity_or_abundance"
          ]
        },
        {
          "qualifier_type_id": "biolink:object_direction_qualifier",
          "applicable_values": [
            "increased",
            "decreased"
          ]
        },
        {
          "qualifier_type_id": "biolink:causal_mechanism_qualifier",
          "applicable_values": [
            "activation",
            "agonism",
            "antagonism",
            "molecular_channel_blockage",
            "inhibition",
            "inverse_agonism"
          ]
        }
      ]
    }
  ]
}

Other notes:

• Maybe relevant: older discussion of our /meta_knowledge_graph endpoints and qualifier info: here and reply
• Potential complication: my understanding is that we should squash multiple x-bte operations into 1 TRAPI MetaEdge object (as shown in the example I provided). But is each MetaEdge required to be a unique combo of subject-node-category / predicate / object-node-category (meta-triple)? Or can there be multiple MetaEdges with the same "meta-triple" that represent different qualifier sets (individual x-bte operations)?

[colleenXu]

Update notes: opening post was last updated 4/4 evening

• this issue has been updated to include ONLY phase 1 tasks. Related tasks were moved to a phase 3 issue
• TRAPI 1.4 may still change. When those changes happen, they should be on this branch and documented in this branch's changelog

[rjawesome]

Note: I wasn't sure what to do with the operations from supportedLookups() (called in bte trapi, src/controllers/meta_knowledge_graph.js) because they only seem to have subject/object/predicate and no additional info

[colleenXu]

@rjawesome That's a question for @tokebe....

[colleenXu]

And regarding

• Potential complication: my understanding is that we should squash multiple x-bte operations into 1 TRAPI MetaEdge object (as shown in the example I provided). But is each MetaEdge required to be a unique combo of subject-node-category / predicate / object-node-category (meta-triple)? Or can there be multiple MetaEdges with the same "meta-triple" that represent different qualifier sets (individual x-bte operations)?

We may ask the TRAPI team for guidance (and they may change the spec)...

The squashing of qualifier-sets introduces ambiguity for those who ingest this info

I explain more in the "keep in mind" note of #597 (comment), copied below

• keep in mind the difference between x-bte annotation with qualifiers vs TRAPI KP MetaEdges with qualifiers:
  • when we parse a x-bte operation, the qualifier info is the exact set of type_ids/values for the associations/data
  • but when we parse a TRAPI KP MetaEdge, there is ambiguity and multiple applicable_values elements are possible....so this isn't an exact-matching scenario.
    • ambiguity: associations might exist with no qualifiers or a set of type_id/value pairs. That set may have 2 pairs,...up to the total number of unique type_ids, because each type_id can only appear once in a qualifier-set.
    • For each type_id, its value could be one of the elements in its applicable_values set

And the example TRAPI-MetaEdge given is confusing because it includes the qualifier_type_id subject_aspect_qualifier twice, with slightly different applicable_values sets.

[colleenXu]

I have asked the TRAPI team about the "potential complications" in my opening post and post above (internal Translator Slack link).

We haven't gotten a response yet.

---

I've advised Jackson: if the current PR code turns each unique operation (based on subject-type / predicate / qualifier-set / object-type) into a TRAPI-MetaEdge, I'm fine with that for now even though it doesn't meet the requirements in the opening post. We're waiting for their response...

[tokebe]

@rjawesome's implementation looks good so far, based on conversation in the Translator Slack I'm going to wait until after the Architecture meeting to give any additional feedback -- It's looking like we might need to have qualifier set merging.

[colleenXu]

Copying over the questions we've posed in Slack

My initial ask: has 3 questions

Q1: What does it mean for a MetaEdge to be considered "unique"? And in the edges section, does each MetaEdge have to be unique?

• we dunno if only the S-P-O content is considered vs other info like specific qualifier-sets, different edge-attribute info, different association values (new property specified in the PR as well). etc....
• It looks like specific qualifier-sets isn't considered...and that qualifier set info is merged/flattened instead....which leads to Q2

Q2: We have feedback, which is that the MetaQualifiers spec introduces ambiguities for us because it doesn't specify the exact qualifier-sets in the edge data. If we (as an ARA) get the MetaEdge in the attached file from a TRAPI KP...

• We cannot tell if this resource has edges with a particular set of qualifier-type/value pairs (2, 3 or 4 qualifier-types). For example, it's kinda looks like the qualifier-set "causes decreased activity through inhibition" could exist in the edge data and that it's reasonable for us to query with these qualifier-constraints...
  • However, this resource actually doesn't have any data with that qualifier-set (we know because it's one of Service Provider's)
  • what does exist is "causes decreased activity_or_abundance through inhibition". The reason why applicable_values includes activity is that other qualifier-sets in the data use it

Q3: we're confused by the example JSON because subject_aspect_qualifier is listed twice in the same MetaQualifier...

Jackson's restating of Q2

To summarize:

• MetaEdges containing all possible qualifiers for a given S-P-O combination leads to ambiguity where certain subsets of the qualifiers listed are simply not present in the actual KG
• This is caused by two or more separate supported qualifier sets overlapping in a given MetaEdge
• Our understanding is that making MetaEdges unique by S-P-O and qualifier set would resolve this

Our questions:

• Is this ambiguity intended/acceptable, or does the spec need to be fixed?
• Is our solution to this ambiguity an acceptable implementation of the spec?

Colleen's restating of Q1 (removed from Slack)

I think Q1 gets at more fundamental expectations, so I'd like to know if it has concrete answers or if it still needs discussion.

I'll try restating Q1:

For the MetaKnowledgeGraph of a tool (KP or ARA), should there only be 1 MetaEdge per unique combo of SubjectCategory - Predicate - ObjectCategory (S-P-O)?

What should a tool do when 1 S-P-O combo represents edges with a variety of...

• ID-namespace combos. Ex: subject-CHEBI / object-MONDO and subject-UMLS / object-DOID
• edge-attribute sets
  • some edges may have edge-attributes that other edges don't

And with TRAPI 1.4, 1 S-P-O combo may also have variety in...

• association values: I'm not sure if this is possible, but it might be?
• existence of qualifiers: some data may not have qualifiers
• qualifier-sets (when data does have qualifiers) <- this is where the convo seems to have focused on so far...

[colleenXu]

Other references:

• Slack poll
• Slack discussion
• TRAPI repo issue
• UPDATE: PR for Q3 was updated and merged to 1.4 branch of TRAPI

[tokebe]

@rjawesome It looks like the consortium requires that edges are not unique by qualifiers -- in other words, if subject-predicate-object are the same, merge all the qualifiers.

[rjawesome]

I believe that was done already in the last couple commits on the bte pr

[tokebe]

Sorry, you're right.

[colleenXu]

Now (after deploying TRAPI 1.4.0 code to ITRB prod): based on a quick look at https://bte.transltr.io/v1/team/Service Provider/meta_knowledge_graph, the x-bte operations appear to be correctly transformed into this TRAPI endpoint's output.

Closing this issue.