Validator fails to load valusets and codes

[jhollinger]

Hi,

We are unable to get the validator to load the valusets or codes. We've followed the instructions extremely carefully and followed every piece of advice online to no avail. At this point we're completely out of ideas and completely desperate. Thank you for any insight!

Log output. As you can see it finishes loading the database in 0 time at all, which I suspect is wrong (since there's so much data):

LOADING SCENARIO FILES AT /srv/validator_configuration/scenarios
LOADING VOCABULARY DATABASES FROM THE FOLLOWING RESOURCES: VALUESETS - /srv/validator_configuration/vocabulary/valueset_repository CODES - /srv/validator_configuration/vocabulary/code_repository
Dec 15, 2020 10:29:44 PM org.apache.catalina.startup.HostConfig deployDescriptor
INFO: Deployment of deployment descriptor [/usr/local/tomcat/conf/Catalina/localhost/referenceccdaservice.xml] has finished in [11,138] ms
Dec 15, 2020 10:29:44 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-apr-8080"]
Dec 15, 2020 10:29:44 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 11296 ms

conf/Catalina/localhost/referenceccdaservice.xml

<Context reloadable="true">
    <Parameter name="vocabulary.localCodeRepositoryDir" value="/srv/validator_configuration/vocabulary/code_repository" override="true"/>
    <Parameter name="vocabulary.localValueSetRepositoryDir" value="/srv/validator_configuration/vocabulary/valueset_repository" override="true"/>
    <Parameter name="referenceccda.configFile" value="/srv/validator_configuration/ccdaReferenceValidatorConfig.xml" override="true"/>
    <Parameter name="referenceccda.isDynamicVocab" value="false" override="true"/>
    <Parameter name="referenceccda.configFolder" value="/srv/validator_configuration" override="true"/>
    <Parameter name="content.scenariosDir" value="/srv/validator_configuration/scenarios" override="true"/>
</Context>

Valueset files

ls -1 /srv/validator_configuration/vocabulary/valueset_repository/VSAC/
Language.xlsx
PostalCode.xlsx
StateValueSet.xlsx
c_cda_release_20200713.xlsx
country.xlsx

The c_cda_release_20200713.xlsx files was obtained from VSAC. I also tried downloading all of the files individually but it didn't seem to matter.

Code System files

ls -1 /srv/validator_configuration/vocabulary/code_repository/*/
/srv/validator_configuration/vocabulary/code_repository/CDT/:
CDT.xlsx

/srv/validator_configuration/vocabulary/code_repository/CPT/:
CPT.txt

/srv/validator_configuration/vocabulary/code_repository/HCPCS/:

/srv/validator_configuration/vocabulary/code_repository/ICD10CM/:
ICD10CM.txt

/srv/validator_configuration/vocabulary/code_repository/ICD10PCS/:
ICD10PCS.txt

/srv/validator_configuration/vocabulary/code_repository/ICD9CM_DX/:

/srv/validator_configuration/vocabulary/code_repository/ICD9CM_SG/:

/srv/validator_configuration/vocabulary/code_repository/LOINC/:
LOINC.csv

/srv/validator_configuration/vocabulary/code_repository/RXNORM/:
RXNCONSO.RRF

/srv/validator_configuration/vocabulary/code_repository/SNOMED-CT/:
SNOMED-CT.txt

[Plow74]

Hello Jordan. Thank you for reaching out to us for help. What you're seeing in the logs looks suspisious, but a quick question for clarification...
Do you have a referenceccdaservice..log in your tomcat's logs directory? That log should be a bit more verbose about what is going on during startup. For example, you should see it walking through all the files the loader is parsing.

08:21:00,732 INFO [VocabularyLoadRunner:92] Loading vocabularies at: C:/Users/bharris/engineering/environments/site/mdht/Environment/VocabularyConfiguration/Vocabulary/code_repository...
08:21:00,733 INFO [VocabularyLoadRunner:44] Loading files in : CDT...
08:21:02,427 INFO [VocabularyLoadRunner:44] Loading files in : CPT...
08:21:03,783 INFO [VocabularyLoadRunner:44] Loading files in : ICD10CM...
08:21:07,012 INFO [VocabularyLoadRunner:44] Loading files in : ICD10PCS...
08:21:09,369 INFO [VocabularyLoadRunner:44] Loading files in : ICD9CM_DX...
08:21:09,867 INFO [VocabularyLoadRunner:44] Loading files in : ICD9CM_SG...
08:21:09,942 INFO [VocabularyLoadRunner:44] Loading files in : LOINC...
08:21:09,946 INFO [LoincLoader:60] Loading latest LOINC CSV / updating indexes
08:22:08,976 INFO [VocabularyLoadRunner:44] Loading files in : RXNORM...
08:22:31,838 INFO [VocabularyLoadRunner:44] Loading files in : SNOMED-CT...
08:23:33,470 INFO [VocabularyLoadRunner:94] Vocabularies loaded...
08:23:33,470 INFO [VocabularyLoadRunner:99] Loading value sets at: C:/Users/bharris/engineering/environments/site/mdht/Environment/VocabularyConfiguration/Vocabulary/valueset_repository...
08:23:33,471 INFO [VocabularyLoadRunner:44] Loading files in : VSAC...
08:23:33,471 INFO [VsacLoader:35] Loading Value Set File: Ability.xlsx
08:23:33,621 INFO [VsacLoader:35] Loading Value Set File: ActNoImmunizationReason.xlsx
08:23:33,678 INFO [VsacLoader:35] Loading Value Set File: ActPriority.xlsx
08:23:33,733 INFO [VsacLoader:35] Loading Value Set File: ActStatus.xlsx
08:23:33,786 INFO [VsacLoader:35] Loading Value Set File: ADLResultType.xlsx
08:23:33,832 INFO [VsacLoader:35] Loading Value Set File: AdministrationUnitDoseForm.xlsx
08:23:33,889 INFO [VsacLoader:35] Loading Value Set File: AdministrativeGenderHL7V3.xlsx
08:23:33,936 INFO [VsacLoader:35] Loading Value Set File: AdvanceCarePlanningServicesGrouping.xlsx
08:23:33,981 INFO [VsacLoader:35] Loading Value Set File: AdvanceDirectiveContentTypeSCT.xlsx

Another thing to check would be to make sure the configuration file in conf/Catalina/localhost/referenceccdaservice.xml is still there after Tomcat starts. I think there are times where that file might be removed - if I remember correctly, it could be during a hot deploy. Anyway, let me know how you make out.

Thanks again!

[drbgfc]

ETT GG link:
https://groups.google.com/u/2/g/edge-test-tool/c/m5WOrSqdFsE

[jhollinger]

@Plow74 I looked around and finally found a log file that looked like your example (seems it's not being written to stdout). It was crashing when it got to the HCPCS folder, complaining that it couldn't find a HCPCS Bean. I deleted the code_repository/HCPCS directory and now it's working like a charm. Thanks!

[jhollinger]

Here's the trace in case you're curious. But again, works fine when the HCPCS dir is removed. I'm using the latest .war file (1.0.49). I don't see a HCPCS loader in https://github.com/onc-healthit/code-validator-api/tree/master/src/main/java/org/sitenv/vocabularies/loader/code, so maybe it's been removed and this is simply a case of outdated docs in https://github.com/onc-healthit/code-validator-api/tree/master/codevalidator-api/docs?

19:25:28,158 INFO  [VocabularyLoadRunner:92] Loading vocabularies at: /srv/validator_configuration/vocabulary/code_repository...
19:25:28,160 INFO  [VocabularyLoadRunner:44] Loading files in : ICD10CM...
19:25:31,407 INFO  [VocabularyLoadRunner:44] Loading files in : CDT...
19:25:32,019 INFO  [VocabularyLoadRunner:44] Loading files in : ICD10PCS...
19:25:34,397 INFO  [VocabularyLoadRunner:44] Loading files in : CPT...
19:25:35,406 INFO  [VocabularyLoadRunner:44] Loading files in : RXNORM...
19:25:40,685 INFO  [VocabularyLoadRunner:44] Loading files in : ICD9CM_DX...
19:25:40,685 INFO  [VocabularyLoadRunner:44] Loading files in : ICD9CM_SG...
19:25:40,685 INFO  [VocabularyLoadRunner:44] Loading files in : SNOMED-CT...
19:26:13,046 ERROR [VocabularyLoadRunner:107] Failed to load configured vocabulary directory.
org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'HCPCS' is defined
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinition(DefaultListableBeanFactory.java:698)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getMergedLocalBeanDefinition(AbstractBeanFactory.java:1175)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:284)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
        at org.springframework.beans.factory.config.ServiceLocatorFactoryBean$ServiceLocatorInvocationHandler.invokeServiceLocatorMethod(ServiceLocatorFactoryBean.java:373)
        at org.springframework.beans.factory.config.ServiceLocatorFactoryBean$ServiceLocatorInvocationHandler.invoke(ServiceLocatorFactoryBean.java:363)
        at com.sun.proxy.$Proxy69.getVocabularyLoader(Unknown Source)
        at org.sitenv.vocabularies.loader.VocabularyLoadRunner.load(VocabularyLoadRunner.java:41)
        at org.sitenv.vocabularies.loader.VocabularyLoadRunner.loadDirectory(VocabularyLoadRunner.java:32)
        at org.sitenv.vocabularies.loader.VocabularyLoadRunner.afterPropertiesSet(VocabularyLoadRunner.java:93)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1637)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1574)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:545)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:772)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:839)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:538)
        at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:446)
        at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:328)
        at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5126)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5653)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:1007)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:983)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:639)
        at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:712)
        at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:2001)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

[BrandonBoone]

Looks like HCPCS support was removed: #83