IBM FHIR Server 4.9.0

This version of the IBM FHIR Server supports HL7 FHIR Specification version v4.0.1.

This release focuses on whole-system search and reindex performance.

Added

• Dedicated tables for whole-system-search parameters _security, _profile, and _tag
  • see migration considerations for info on populating the new tables for existing resources
• Drive reindex using client side checkpointing for improved throughput
• Support for Pattern Value Constraint with $this in the Path
• Support for slice-specific constraints
• Quantity search now considers the quantity comparator field
• Support revocation of tenant keys (Db2)
• Custom operations for clearing terminology caches
• Export to Azure Blob storage
• Configurable "fail fast" validation
• fhir-smart support for user/ scopes
  • The interceptor now allows interactions with resource types covered by user/<resourceType>.<permission> scopes in the access token, but operators will still want to add their own permissions model on top.
• Support static configuration of external FHIR base URL (in favor of originalRequestUriHeader)
  • This introduces config property fhirServer/core/externalBaseUrl for statically setting the server's base url

Changed

• Improve reindex performance by skipping resources with no search parameter changes
• whole-system-search: use global parameter tables in whole system search join
• expand support for $reindex to the Type and Instance levels
• token search for |<code> should only match codes with no systems
• improve error handling for remote term service poor responses
• replace Java Streams with conventional loop in FHIRPathEvaluator for improved perf
• improve performance for date range searches
• fhir-notification: special handling for resources over 1M
• Use the "skippableUpdate" optimization during bulk import
• Make schema drop idempotent
• Upgrade to the latest SPL snapshot
• Dockerfile should ensure directories are writable by root group (gid=0)

Deprecated

• the old query builder
• the (base)/Resource and (base)/DomainResource endpoints

Removed

• fhir-cli
• the exportPublic + ACL-based approach to exported object expiry

Fixed

• Strange behavior with lt and gt prefixes when searching by date / partial dateTime
• _sort not working correctly with id value
• Whole-system search using _id parameter performs poorly
• poor performance for whole-system search with _lastUpdated
• Concurrency issue during apply FHIRPathPatchReplace
• reindex process slows after some hours when Postgres vacuuming falls behind
  • this change includes a postgresql-only schema change that makes autovacuuming more aggressive for tables that are heavily updated during reindex
• OperationDefinition.name and OperationDefinition.code inconsistent between FHIRRestHelpers and FHIROperationRegistry
• NPE thrown in FHIRPathUtil.getResourceNodes(FHIRPathTree tree, FHIRPathNode node) when FHIRPathNode is not in FHIRPathTree
• FHIRPath conformsTo loops can lead to StackOverflow
• Reindex Returns 500 when the Client sends a bad timestamp
• Vanity URLs with AWS result in bucket NOT_FOUND on $import
• When multiple tenants exist, Drop Tenant Results in an incorrect FHIRDATA.PARAMETER_NAMES association and drop.
• $import Exit Status on Matrixed Resources results in NPE
• fhir-bucket ignores --schema-name when running scan and load
• fhir-persistance-schema-cli update-schema not updating functions.
• Under certain conditions, the root cause of a BundledRequestException is never logged
• Discriminator incorrectly generated for profile constraints
• When using _total=none, a "next" link is generated when there isn't a "next" page
• non-fast bulk export writes to many files instead of honoring settings
• file based export resulting in multiple files is not reported correctly by $bulkdata-status

Security

• fhir-smart interceptor should enforce scope permissions up front

Migration Considerations

To migrate from a prior version of the IBM FHIR Server to version 4.9.0:

1. Analyze configuration changes and prepare for the upgrade
   • fhirServer/core/externalBaseUrl is introduced as an alternative to constructing the baseUrl from the value of the configured "originalRequestUriHeaderName"
   • fhirServer/bulkdata/storageProviders/(source)/exportPublic has been removed
   • fhirServer/notifications/common/maxNotificationSizeBytes and fhirServer/notifications/common/maxNotificationSizeBehavior were introduced for controlling the notification behavior for large messages
   • fhirServer/validation/failFast was introduced to make it possible to ask the validator to fail on the first error
   • fhirServer/search/enableLegacyWholeSystemSearchParams was introduced to allow whole-system param queries to continue working after migration and before a complete reindex. This parameter should be added and set to true before migration.
2. Stop the server (or otherwise prevent write requests).
3. Perform a database back up.
4. Run the fhir-persistence-schema-4.9.0-cli.jar --update-schema command. If using a non-admin user (recommended), also perform the --grant-to action.
5. Deploy the new server.
6. Perform re-indexing.
   • Needed for #264. See special considerations for more info.
7. Remove fhirServer/search/enableLegacyWholeSystemSearchParams (or explicitly set it to false)

Special considerations for upgrading to 4.9.0:

• The schema migration for 4.9.0 adds columns to the whole-system logical_resources table and populates these columns for each resource in the database. This can be a lengthy process (9 hours on our PostgreSQL database with ~80 million resources). We have tested on-line migration (serving read-only requests while the update is live) and encountered no issue, but care should be taken to avoid potential impact to end users.
• Starting in 4.9.0, the $reindex operation will skip resources with no search parameter changes. However, this is driven by a new column in the database that stores the hash of the parameter values and so the first reindex after upgrading will populate that column for all resources.
• The location of _security, _tag, and _profile search parameter values is updated in 4.9.0. To ensure that queries continue to work as expected during migration, please use the newly introduced fhirServer/search/enableLegacyWholeSystemSearchParams parameter. When set to true, ingestion and reindexing will extract the value to both locations and searches will look in the old location. When set to false (the default), ingestion and reindexing and search will use only the new location which is expected to yield improved performance for select queries.

Using the Release

Run or extend the Docker image from https://hub.docker.com/r/ibmcom/ibm-fhir-server or download fhir-install-4.9.0.zip and install as described at https://ibm.github.io/FHIR/guides/FHIRServerUsersGuide.

Executable jar files for the fhir-persistence-schema, fhir-swagger-generator, fhir-bucket, and fhir-path projects are also available:

• fhir-persistence-schema-4.9.0-cli.jar
• fhir-swagger-generator-4.9.0-cli.jar
• fhir-bucket-4.9.0-cli.jar
• fhir-path-4.9.0-cli.jar

Usage is documented in the corresponding README files:

• https://github.com/IBM/FHIR/tree/master/fhir-persistence-schema
• https://github.com/IBM/FHIR/tree/master/fhir-swagger-generator
• https://github.com/IBM/FHIR/tree/master/fhir-bucket
• https://github.com/IBM/FHIR/tree/master/fhir-path

For Maven users, all project binaries are posted to Maven Central with a version of 4.9.0.