Skip to content

Commit

Permalink
#2421 - Add terminology clear-cache operations (#2422)
Browse files Browse the repository at this point in the history
* #2421 - Add terminology clear-cache operations

The terminology subsystem caches terminology resources by URL and/or URL
+ version. If the user tries to update a CodeSystem or ValueSet resource
keeping the same URL + version, the cache does not get cleared and some
operations (e.g. validate-code) do not return the expected results after
update. This change adds an operation ($clear-cache) that can be called
on either terminology resource and will clear the appropriate caches.

Signed-off-by: Corey Sanders <corey.thecolonel@gmail.com>

* Address test failure due to unexpected test interactions.

Signed-off-by: Corey Sanders <corey.thecolonel@gmail.com>

* Fix issue with operation ordering on clean database

All my testing has been done on a dirty database. There are automated
build failures that appear to be related to test order and a resource
not existing yet on a clean database. Flipping the order of the resource
puts and initial clear cache operations so that the resource is
guaranteed to be in place.

Signed-off-by: Corey Sanders <corey.thecolonel@gmail.com>

* Address code format issues

Signed-off-by: Corey Sanders <corey.thecolonel@gmail.com>

* Split term cache operations into a separate module

Per discussion with the team, the cache operations are being split into
a separate module that will not be included in the default server build.
Since it isn't automatically included, the build automation scripts
needed to be updated.

Signed-off-by: Corey Sanders <corey.thecolonel@gmail.com>

* Update POM version for new module

Signed-off-by: Corey Sanders <corey.thecolonel@gmail.com>

* Fix Windows integration, update README

Signed-off-by: Corey Sanders <corey.thecolonel@gmail.com>

* Apply suggested change

Signed-off-by: Corey Sanders <corey.thecolonel@gmail.com>

* Apply suggested change

Signed-off-by: Corey Sanders <corey.thecolonel@gmail.com>

Co-authored-by: Lee Surprenant <lmsurpre@us.ibm.com>

* change package name from javax.json to jakarta.json

Signed-off-by: Corey Sanders <corey.thecolonel@gmail.com>

Co-authored-by: Lee Surprenant <lmsurpre@us.ibm.com>
  • Loading branch information
2 people authored and tbieste committed Jun 9, 2021
1 parent a8abc0a commit 8769b79
Show file tree
Hide file tree
Showing 24 changed files with 1,176 additions and 1 deletion.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ The IBM FHIR Server is modular and extensible. The following tables provide an o
|fhir-operation-document|Basic support for the Composition `$document` operation defined at https://www.hl7.org/fhir/operation-composition-document.html |false|
|fhir-operation-healthcheck|The `$healthcheck` operation checks for a valid connection to the database and returns the server status|false|
|fhir-operation-term|[Terminology service](https://www.hl7.org/fhir/terminology-service.html) operations which use the default fhir-term TerminologyServiceProvider to implement $expand, $lookup, $subsumes, $closure, $validate and $translate|false|
|fhir-operation-term-cache|Add-on module that provides operations for clearing the terminology subsystem caches for non-production scenarios|false|
|fhir-operation-validate|An implementation of the FHIR resource [$validate operation](https://www.hl7.org/fhir/R4/operation-resource-validate.html)|false|
|fhir-operation-everything|An implementation of the FHIR patient [`$everything`](https://www.hl7.org/fhir/operation-patient-everything.html) operation|false|
|fhir-operation-erase|A hard delete operation for resource instances referred to as the `$erase` operation. See the [README.md](operation/fhir-operation-erase/README.md)|false|
Expand Down
1 change: 1 addition & 0 deletions build/audit/kafka/pre-integration-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ config(){
mkdir -p $USERLIB
find ${WORKSPACE}/conformance -iname 'fhir-ig*.jar' -not -iname 'fhir*-tests.jar' -not -iname 'fhir*-test-*.jar' -exec cp -f {} ${USERLIB} \;
cp -pr ${WORKSPACE}/operation/fhir-operation-test/target/fhir-operation-*-tests.jar ${USERLIB}
cp -pr ${WORKSPACE}/operation/fhir-operation-term-cache/target/fhir-operation-*.jar ${USERLIB}
echo "Finished copying fhir-server dependencies..."

# Move over the test configurations
Expand Down
1 change: 1 addition & 0 deletions build/docker/copy-test-operations.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,6 @@ mkdir -p $USERLIB
echo "Copying test artifacts to install location..."
find ${WORKSPACE}/conformance -iname 'fhir-ig*.jar' -not -iname 'fhir*-tests.jar' -not -iname 'fhir*-test-*.jar' -exec cp -f {} ${USERLIB} \;
cp -pr ${WORKSPACE}/operation/fhir-operation-test/target/fhir-operation-*-tests.jar ${USERLIB}
cp -pr ${WORKSPACE}/operation/fhir-operation-term-cache/target/fhir-operation-*.jar ${USERLIB}

echo "Finished copying test operations."
1 change: 1 addition & 0 deletions build/notifications/kafka/pre-integration-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ config(){
mkdir -p $USERLIB
find ${WORKSPACE}/conformance -iname 'fhir-ig*.jar' -not -iname 'fhir*-tests.jar' -not -iname 'fhir*-test-*.jar' -exec cp -f {} ${USERLIB} \;
cp -pr ${WORKSPACE}/operation/fhir-operation-test/target/fhir-operation-*-tests.jar ${USERLIB}
cp -pr ${WORKSPACE}/operation/fhir-operation-term-cache/target/fhir-operation-*.jar ${USERLIB}
echo "Finished copying fhir-server dependencies..."

# Move over the test configurations
Expand Down
1 change: 1 addition & 0 deletions build/persistence/postgres/pre-integration-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ copy_server_config(){
echo "Copying test artifacts to install location..."
find ${WORKSPACE}/conformance -iname 'fhir-ig*.jar' -not -iname 'fhir*-tests.jar' -not -iname 'fhir*-test-*.jar' -exec cp -f {} ${USERLIB} \;
cp -pr ${WORKSPACE}/operation/fhir-operation-test/target/fhir-operation-*-tests.jar ${USERLIB}
cp -pr ${WORKSPACE}/operation/fhir-operation-term-cache/target/fhir-operation-*.jar ${USERLIB}
echo "Finished copying fhir-server dependencies..."
}

Expand Down
3 changes: 3 additions & 0 deletions build/pre-integration-test.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,9 @@ If (!(Test-Path -Path $USERLIB_DIR) ) {
}
Copy-Item $CP_ITEM -Destination $USERLIB_DST

$CP_ITEM=[string]$DIR_WORKSPACE + '\operation\fhir-operation-term-cache\target\fhir-operation-term-cache-*.jar'
Copy-Item $CP_ITEM -Destination $USERLIB_DST

# Start up the fhir server
$DATE_PS=[System.TimeZoneInfo]::ConvertTimeBySystemTimeZoneId((Get-Date), 'Greenwich Standard Time').ToString('t')
Write-Host '>>> Current time: ' $DATE_PS
Expand Down
1 change: 1 addition & 0 deletions build/pre-integration-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ cp -p ${WORKSPACE}/fhir-server/liberty-config/configDropins/disabled/datasource-
echo "Copying test artifacts to install location..."
rm -rf ${SIT}/wlp/usr/servers/fhir-server/userlib/fhir-operation-*-tests.jar
cp -pr ${WORKSPACE}/operation/fhir-operation-test/target/fhir-operation-*-tests.jar ${SIT}/wlp/usr/servers/fhir-server/userlib/
cp -pr ${WORKSPACE}/operation/fhir-operation-term-cache/target/fhir-operation-*.jar ${SIT}/wlp/usr/servers/fhir-server/userlib/
find ${WORKSPACE}/conformance -iname 'fhir-ig*.jar' -not -iname 'fhir*-tests.jar' -not -iname 'fhir*-test-*.jar' -exec cp -f {} ${SIT}/wlp/usr/servers/fhir-server/userlib/ \;

# Start up the fhir server
Expand Down
1 change: 1 addition & 0 deletions fhir-parent/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@
<module>../operation/fhir-operation-apply</module>
<module>../operation/fhir-operation-convert</module>
<module>../operation/fhir-operation-term</module>
<module>../operation/fhir-operation-term-cache</module>
<module>../operation/fhir-operation-reindex</module>
<module>../operation/fhir-operation-bulkdata</module>
<module>../operation/fhir-operation-everything</module>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
/*
* (C) Copyright IBM Corp. 2021
*
* SPDX-License-Identifier: Apache-2.0
*/

package com.ibm.fhir.server.test.terminology;

import static org.testng.Assert.assertEquals;

import java.io.ByteArrayInputStream;

import javax.ws.rs.core.Response;

import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

import com.ibm.fhir.model.format.Format;
import com.ibm.fhir.model.parser.FHIRParser;
import com.ibm.fhir.model.resource.Parameters;

/**
* These tests exercise the $clear-cache operation on a CodeSystem.
*/
public class CodeSystemClearCacheOperationTest extends TerminologyOperationTestBase {

private static final String SNOMED_CT = "http://snomed.info/sct";
// Test Specific
public static final String TEST_GROUP_NAME = "terminology";
public static final boolean DEBUG = false;

// URLs to call against the instance
public static final String BASE_VALID_URL = "/CodeSystem";

public static final String CODE_SYSTEM_ID = "test";
public static final String CODE_SYSTEM_URL = "http://ibm.com/fhir/CodeSystem/test";
public static final String CODE_SYSTEM_VERSION = "1.0.0";

@BeforeClass
public void setup() throws Exception {
Response response = doPut("CodeSystem", CODE_SYSTEM_ID, "testdata/CodeSystem-test.json");
assertEquals(response.getStatusInfo().getFamily(), Response.Status.Family.SUCCESSFUL);
}

@Test(groups = { TEST_GROUP_NAME })
public void testClearCacheParameterVariations() throws Exception {
Response response;

response = doGet(BASE_VALID_URL + "/$clear-cache");
assertEquals(response.getStatus(), Response.Status.OK.getStatusCode());

response = doGet(BASE_VALID_URL + "/$clear-cache", "url", CODE_SYSTEM_URL);
assertEquals(response.getStatus(), Response.Status.OK.getStatusCode());

response = doGet(BASE_VALID_URL + "/$clear-cache", "url", CODE_SYSTEM_URL, "codeSystemVersion", CODE_SYSTEM_VERSION);
assertEquals(response.getStatus(), Response.Status.OK.getStatusCode());

clearCache();
}

@Test(groups = { TEST_GROUP_NAME })
public void testClearCacheValidateCode() throws Exception {
Response response;
Parameters parameters;

// Make sure the resource is there
response = doPut("CodeSystem", CODE_SYSTEM_ID, "testdata/CodeSystem-test.json");
assertEquals(response.getStatusInfo().getFamily(), Response.Status.Family.SUCCESSFUL);

// Once to make sure it is gone from the cache
clearCache();

// Twice to see what happens when it is empty
clearCache();

// Subsumes to reload the cache
parameters = validateCode(SNOMED_CT, "K");
assertEquals(getBooleanParameterValue(parameters, "result"), Boolean.TRUE);

// Update the resource
response = doPut("CodeSystem", CODE_SYSTEM_ID, "testdata/CodeSystem-test-updated.json");
assertEquals(response.getStatus(), Response.Status.OK.getStatusCode());

parameters = validateCode(SNOMED_CT, "K");
assertEquals(getBooleanParameterValue(parameters, "result"), Boolean.TRUE);

// After cache is cleared, subsumes outcome should be updated
clearCache();

parameters = validateCode(SNOMED_CT, "K");
assertEquals(getBooleanParameterValue(parameters, "result"), Boolean.FALSE);
}

private Parameters validateCode(String system, String code) throws Exception {
Response response = doGet(BASE_VALID_URL + "/" + CODE_SYSTEM_ID + "/$validate-code", "system", system, "code", code);
String responseBody = response.readEntity(String.class);
assertEquals(response.getStatus(), Response.Status.OK.getStatusCode(), responseBody);
return FHIRParser.parser(Format.JSON).parse(new ByteArrayInputStream(responseBody.getBytes()));
}

private void clearCache() {
Response response = doGet(BASE_VALID_URL + "/" + CODE_SYSTEM_ID + "/$clear-cache");
assertEquals(response.getStatus(), Response.Status.OK.getStatusCode());
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
/*
* (C) Copyright IBM Corp. 2021
*
* SPDX-License-Identifier: Apache-2.0
*/

package com.ibm.fhir.server.test.terminology;

import static org.testng.Assert.assertEquals;
import static org.testng.Assert.assertTrue;

import java.io.ByteArrayInputStream;
import java.util.Properties;

import jakarta.json.JsonObject;
import javax.ws.rs.client.Entity;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.core.Response;

import org.testng.annotations.BeforeClass;

import com.ibm.fhir.core.FHIRMediaType;
import com.ibm.fhir.model.format.Format;
import com.ibm.fhir.model.parser.FHIRParser;
import com.ibm.fhir.model.parser.exception.FHIRParserException;
import com.ibm.fhir.model.resource.Parameters;
import com.ibm.fhir.model.resource.Resource;
import com.ibm.fhir.model.test.TestUtil;
import com.ibm.fhir.server.test.FHIRServerTestBase;

public abstract class TerminologyOperationTestBase extends FHIRServerTestBase {

public static final String FORMAT = "application/json";

private final String tenantName = "default";
private final String dataStoreId = "default";

@BeforeClass
public void setup() throws Exception {
Properties testProperties = TestUtil.readTestProperties("test.properties");
setUp(testProperties);
}

public Response doPut(String resourceType, String id, String resourcePath) throws Exception {
JsonObject jsonObject = TestUtil.readJsonObject(resourcePath);
Entity<JsonObject> entity = Entity.entity(jsonObject, FHIRMediaType.APPLICATION_FHIR_JSON);

Response response = getWebTarget().path(resourceType + "/" + id).request().put(entity, Response.class);
String responseBody = response.readEntity(String.class);
assertEquals(response.getStatusInfo().getFamily(), Response.Status.Family.SUCCESSFUL, responseBody);
return response;
}

public Response doGet(String path, String... params) {

WebTarget target = getWebTarget();
target = target.path(path);

// When the path is passed in with the parameters, the ?, &, etc.
// get escaped and it causes failures, so we are doing some
// hacking here.
if (params != null && params.length > 0) {
assert (params.length % 2 == 0);
for (int i = 0; i < params.length; i += 2) {
target = target.queryParam(params[i], params[i + 1]);
}
}

return target.request(FORMAT).header("X-FHIR-TENANT-ID", tenantName).header("X-FHIR-DSID", dataStoreId).get(Response.class);
}

public Resource parseResource(String responseBody) throws FHIRParserException {
Resource resource = FHIRParser.parser(Format.JSON).parse(new ByteArrayInputStream(responseBody.getBytes()));
return resource;
}

public Parameters.Parameter getParameter(Resource resource, String propertyName) {
assertTrue(resource instanceof Parameters);
Parameters parameters = (Parameters) resource;
return parameters.getParameter().stream().filter(p -> p.getName().getValue().equals(propertyName)).reduce((a, b) -> {
throw new IllegalStateException("More than one parameter found with the same name '" + propertyName + "'");
}).get();
}

public Boolean getBooleanParameterValue(Resource resource, String propertyName) {
return ((com.ibm.fhir.model.type.Boolean) getParameter(resource, propertyName).getValue()).getValue();
}
}
Loading

0 comments on commit 8769b79

Please sign in to comment.