Introduce fhir-term-graph module

[JohnTimm]

The fhir-term-graph module will have an implementation of the FHIRTermServiceProvider interface built on top of a graph database (JanusGraph). This will enable the ability to efficiently support code systems with a large number of concepts (e.g. SNOMED-CT).

Current feature branch:
https://github.com/IBM/FHIR/tree/fhir-term-graph

The feature will be configurable.

[lmsurpre]

The graph loader definitely needs some better docs.

On my first attempt to load it, I passed the following args:

SNOMED
-base /Users/lmsurpre/SnomedCT_USEditionRF2_PRODUCTION_20210301T120000Z/Full
-concept Terminology/sct2_Concept_Full_US1000124_20210301.txt
-desc Terminology/sct2_Description_Full-en_US1000124_20210301.txt
-relation Terminology/sct2_Relationship_Full_US1000124_20210301.txt
-lang Refset/der2_cRefset_LanguageFull-en_US1000124_20210301.txt
-config src/test/resources/conf/janusgraph-berkeleyje-lucene.properties

It did a bunch of work and then bombed out with this:

An error occurred: java.io.FileNotFoundException: /Users/lmsurpre/SnomedCT_USEditionRF2_PRODUCTION_20210301T120000Z/Full/../Refset/Language/Refset/der2_cRefset_LanguageFull-en_US1000124_20210301.txt (No such file or directory)

Apparently it wants something like this instead:

SNOMED
-base /Users/lmsurpre/SnomedCT_USEditionRF2_PRODUCTION_20210301T120000Z/Full/Terminology
-concept sct2_Concept_Full_US1000124_20210301.txt
-desc sct2_Description_Full-en_US1000124_20210301.txt
-relation sct2_Relationship_Full_US1000124_20210301.txt
-lang der2_cRefset_LanguageFull-en_US1000124_20210301.txt
-config src/test/resources/conf/janusgraph-berkeleyje-lucene.properties

Are there different release formats for SNOMED CT? Could we just point it at /Users/lmsurpre/SnomedCT_USEditionRF2_PRODUCTION_20210301T120000Z/Full and let it figure out the rest?

[lmsurpre]

What does the terminology system update scenario look like? Is it idempotent (i.e. loading the same source files twice is the same loading it once)?
What happens if we import this version of the SNOMED CT US edition, then import the international version later. They just co-exist or does it meld them together based on the individual concepts?
And I have similar questions for the UMLS loader as well. A new version of UMLS may contain some source systems that changed and others that didn't.

[lmsurpre]

I opened #2176 for the missing doc

[lmsurpre]

With fhir-term-graph enabled as described above, I noticed the following output during startup:

[2021-03-30T21:25:17.347+0000] 0000002e SystemErr     R   SLF4J: Class path contains multiple SLF4J bindings.
[2021-03-30T21:25:17.348+0000] 0000002e SystemErr     R   SLF4J: Found binding in [wsjar:file:/Users/lmsurpre/.m2/repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
[2021-03-30T21:25:17.348+0000] 0000002e SystemErr     R   SLF4J: Found binding in [wsjar:file:/Users/lmsurpre/.m2/repository/org/slf4j/slf4j-nop/1.7.30/slf4j-nop-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
[2021-03-30T21:25:17.348+0000] 0000002e SystemErr     R   SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
[2021-03-30T21:25:17.439+0000] 0000002e SystemErr     R   SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]

I'm not positive that its related to the graph stuff, but I went ahead and opened #2177 for it.

[lmsurpre]

After loading a berkelydb/lucene graph as described above, I moved that to my server and updated my fhir-server-config with this:

        "term": {
            "graphTermServiceProvider": {
                "enabled": true,
                "timeLimit": 30000,
                "configuration": {
                    "storage.backend": "berkeleyje",
                    "storage.directory": "data/graph",
                    "index.search.backend": "lucene",
                    "index.search.directory": "data/searchindex"
                }
            }
        },

Unfortunately, calls to $lookup are now failing with a stacktrace like this:

[2021-03-30T21:33:08.978+0000] 00000020 FHIRResource  E   An error occurred during the CodeSystem lookup operation
                                 com.ibm.fhir.exception.FHIROperationException: An error occurred during the CodeSystem lookup operation  [probeId=7f-0-0-1-a6a2c6c3-d13f-4937-9ea7-79c93c9f4a5c]
	at com.ibm.fhir.operation.term.LookupOperation.doInvoke(LookupOperation.java:52)
	at com.ibm.fhir.server.operation.spi.AbstractOperation.invoke(AbstractOperation.java:56)
	at com.ibm.fhir.server.util.FHIRRestHelper.doInvoke(FHIRRestHelper.java:1119)
	at com.ibm.fhir.server.resources.Operation.invoke(Operation.java:212)
	...
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: ele-1: All FHIR elements must have a @value or children
	at com.ibm.fhir.model.util.ValidationSupport.requireValueOrChildren(ValidationSupport.java:378)
	at com.ibm.fhir.model.type.String.<init>(String.java:30)
	at com.ibm.fhir.model.type.String$Builder.build(String.java:197)
	at com.ibm.fhir.model.type.String.string(String.java:58)
	at com.ibm.fhir.term.graph.provider.GraphTermServiceProvider.createConcept(GraphTermServiceProvider.java:558)
	at com.ibm.fhir.term.graph.provider.GraphTermServiceProvider.createConcept(GraphTermServiceProvider.java:546)
	at com.ibm.fhir.term.graph.provider.GraphTermServiceProvider.getConcept(GraphTermServiceProvider.java:601)
	at com.ibm.fhir.term.graph.provider.GraphTermServiceProvider.getConcept(GraphTermServiceProvider.java:122)
	at com.ibm.fhir.term.service.FHIRTermService.lookup(FHIRTermService.java:289)
	at com.ibm.fhir.operation.term.LookupOperation.doInvoke(LookupOperation.java:48)
	... 59 more

I opened #2178 for making loading idempotent (to avoid the partially imported content) and #2179 for being more resilient.

[lmsurpre]

After recreating the graph db, I was able to successfully load SnomedCT_USEditionRF2_PRODUCTION_20210301T120000Z and exercise the following queries:

• https://localhost:9443/fhir-server/api/v4/CodeSystem/$lookup?system=http://snomed.info/sct&code=64766004
• https://localhost:9443/fhir-server/api/v4/CodeSystem/$validate-code?url=http://snomed.info/sct&code=64766004
• https://localhost:9443/fhir-server/api/v4/ValueSet/$validate-code?url=http://snomed.info/sct?fhir_vs=isa/64766004&system=http://snomed.info/sct&code=64766004
• https://localhost:9443/fhir-server/api/v4/ValueSet/$expand

  {
      "resourceType": "ValueSet",
      "id": "snomed-valueset",
      "url": "http://ibm.com/fhir/ValueSet/snomed-valueset",
      "version": "1",
      "name": "SNOMED-CT ValueSet",
      "status": "active",
      "compose": {
          "include": [
              {
                  "system": "http://snomed.info/sct",
                  "filter": [
                      {
                          "property": "concept",
                          "op": "is-a",
                          "value": "195967001"
                      }
                  ]
              }
          ]
      }
  }
  

• https://localhost:9443/fhir-server/api/v4/ValueSet/$expand?url=http://snomed.info/sct?fhir_vs=isa/195967001
• https://localhost:9443/fhir-server/api/v4/CodeSystem/$subsumes?system=http://snomed.info/sct&codeA=10675911000119109&codeB=195967001
• https://localhost:9443/fhir-server/api/v4/$closure

  {
    "resourceType" : "Parameters",
    "parameter" : [
      {
        "name" : "name",
        "valueString" : "asthma-closure"
      },
      {
        "name" : "concept",
        "valueCoding" : {
          "system" : "http://snomed.info/sct",
          "code" : "195967001"
        }
      }
    ]
  }
  

• https://localhost:9443/fhir-server/api/v4/ValueSet/$expand

  {
      "resourceType": "ValueSet",
      "id": "snomed-valueset",
      "url": "http://ibm.com/fhir/ValueSet/snomed-valueset",
      "version": "1",
      "name": "SNOMED-CT ValueSet",
      "status": "active",
      "compose": {
          "include": [
              {
                  "system": "http://snomed.info/sct",
                  "filter": [
                      {
                          "property": "concept",
                          "op": "is-a",
                          "value": "195967001"
                      },
                      {
                          "property": "concept",
                          "op": "is-not-a",
                          "value": "5281000124103"
                      }
                  ]
              }
          ]
      }
  }
  

[ryadav007]

after loading successfully snomed ct codes and starting the fhir server it gives this issue
java.lang.Error: java.lang.IllegalArgumentException: Could not find implementation class: org.janusgraph.diskstorage.cql.CQLStoreManager at org.linuxforhealth.fhir.term.graph.impl.FHIRTermGraphImpl.<init>(FHIRTermGraphImpl.java:43) at org.linuxforhealth.fhir.term.graph.factory.FHIRTermGraphFactory.open(FHIRTermGraphFactory.java:51) at org.linuxforhealth.fhir.term.graph.provider.GraphTermServiceProvider.<init>(GraphTermServiceProvider.java:89) at org.linuxforhealth.fhir.server.listener.FHIRServletContextListener.configureTermServiceCapabilities(FHIRServletContextListener.java:432) at org.linuxforhealth.fhir.server.listener.FHIRServletContextListener.contextInitialized(FHIRServletContextListener.java:295) at com.ibm.ws.webcontainer.webapp.WebApp.notifyServletContextCreated(WebApp.java:2466) at com.ibm.ws.webcontainer31.osgi.webapp.WebApp31.notifyServletContextCreated(WebApp31.java:510) at com.ibm.ws.webcontainer.webapp.WebApp.initialize(WebApp.java:1064) at com.ibm.ws.webcontainer.webapp.WebApp.initialize(WebApp.java:6704) at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost.startWebApp(DynamicVirtualHost.java:472) at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost.startWebApplication(DynamicVirtualHost.java:467) at com.ibm.ws.webcontainer.osgi.WebContainer.startWebApplication(WebContainer.java:1188) at com.ibm.ws.webcontainer.osgi.WebContainer.startModule(WebContainer.java:977) at com.ibm.ws.app.manager.module.internal.ModuleHandlerBase.deployModule(ModuleHandlerBase.java:99) at com.ibm.ws.app.manager.module.internal.DeployedModuleInfoImpl.installModule(DeployedModuleInfoImpl.java:49) at com.ibm.ws.app.manager.module.internal.SimpleDeployedAppInfoBase.deployModules(SimpleDeployedAppInfoBase.java:597) at com.ibm.ws.app.manager.module.internal.SimpleDeployedAppInfoBase.installApp(SimpleDeployedAppInfoBase.java:511) at com.ibm.ws.app.manager.module.internal.DeployedAppInfoBase.deployApp(DeployedAppInfoBase.java:349) at com.ibm.ws.app.manager.war.internal.WARApplicationHandlerImpl.install(WARApplicationHandlerImpl.java:65) at com.ibm.ws.app.manager.internal.statemachine.StartAction.execute(StartAction.java:161) at com.ibm.ws.app.manager.internal.statemachine.ApplicationStateMachineImpl.enterState(ApplicationStateMachineImpl.java:1367) at com.ibm.ws.app.manager.internal.statemachine.ApplicationStateMachineImpl.run(ApplicationStateMachineImpl.java:910) at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:245) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.IllegalArgumentException: Could not find implementation class: org.janusgraph.diskstorage.cql.CQLStoreManager at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:75) at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:527) at org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:491) at org.janusgraph.graphdb.configuration.builder.GraphDatabaseConfigurationBuilder.build(GraphDatabaseConfigurationBuilder.java:64) at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:176) at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:147) at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:127) at org.linuxforhealth.fhir.term.graph.impl.FHIRTermGraphImpl.open(FHIRTermGraphImpl.java:53) at org.linuxforhealth.fhir.term.graph.impl.FHIRTermGraphImpl.<init>(FHIRTermGraphImpl.java:40) ... 25 more Caused by: java.lang.ClassNotFoundException: org.janusgraph.diskstorage.cql.CQLStoreManager at com.ibm.ws.classloading.internal.AppClassLoader.findClassCommonLibraryClassLoaders(AppClassLoader.java:734) at com.ibm.ws.classloading.internal.AppClassLoader.findClass(AppClassLoader.java:319) at com.ibm.ws.classloading.internal.AppClassLoader.findOrDelegateLoadClass(AppClassLoader.java:706) at com.ibm.ws.classloading.internal.AppClassLoader.loadClass(AppClassLoader.java:578) at com.ibm.ws.classloading.internal.AppClassLoader.loadClass(AppClassLoader.java:545) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Class.java:315) at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:71) ... 33 more