Investigate changes in FHIR 4.3.0 (R4B)

[lmsurpre]

Current thought is to support R4B in-place (non-optional).

Goals: look for backwards-breaking changes that could affect this plan.
Examples: changed field names, resource type names, etc.

Ensure we can handle application/fhir+json;fhirVersion=x.y.z clause appropriately for both 4.0.1 and 4.1.0 (#896)

[lmsurpre]

I havn't checked this against the list published by HL7, but after an initial attempt at #2066 I found the following changes (ignoring valueset binding version changes from 4.0.1 to 4.1.0):

Resource Types

deleted:    EffectEvidenceSynthesis.java
deleted:    SubstanceNucleicAcid.java
deleted:    SubstancePolymer.java
deleted:    SubstanceProtein.java
deleted:    SubstanceSourceMaterial.java
deleted:    MedicinalProductContraindication.java
deleted:    MedicinalProductInteraction.java
deleted:    MedicinalProductUndesirableEffect.java
deleted:    RiskEvidenceSynthesis.java
deleted:    SubstanceReferenceInformation.java

renamed:    MedicinalProduct.java -> MedicinalProductDefinition.java
renamed:    MedicinalProductAuthorization.java -> RegulatedAuthorization.java
renamed:    MedicinalProductIngredient.java -> Ingredient.java
renamed:    MedicinalProductManufactured.java -> ManufacturedItemDefinition.java
renamed:    MedicinalProductPackaged.java -> PackagedProductDefinition.java
renamed:    MedicinalProductPharmaceutical.java -> AdministrableProductDefinition.java
renamed:    SubstanceSpecification.java -> SubstanceDefinition.java
renamed:    MedicinalProductIndication.java -> ClinicalUseDefinition.java

modified:   ActivityDefinition.java
modified:   AllergyIntolerance.java
modified:   Bundle.java
modified:   ClaimResponse.java
modified:   Composition.java
modified:   Condition.java
modified:   Contract.java
modified:   CoverageEligibilityResponse.java
modified:   Device.java
modified:   DiagnosticReport.java
modified:   DocumentReference.java
modified:   DomainResource.java
modified:   EnrollmentResponse.java
modified:   Evidence.java
modified:   EvidenceVariable.java
modified:   ExplanationOfBenefit.java
modified:   Group.java
modified:   InsurancePlan.java
modified:   Library.java
modified:   Measure.java
modified:   MeasureReport.java
modified:   NamingSystem.java
modified:   Observation.java
modified:   Parameters.java
modified:   PaymentReconciliation.java
modified:   PlanDefinition.java
modified:   Questionnaire.java
modified:   ResearchSubject.java
modified:   RiskAssessment.java
modified:   ServiceRequest.java
modified:   StructureDefinition.java
modified:   Substance.java

new file:   Citation.java
new file:   ClinicalUseIssue.java
new file:   EvidenceReport.java
new file:   NutritionProduct.java
new file:   SubscriptionStatus.java
new file:   SubscriptionTopic.java

Data Types

deleted:    SubstanceAmount.java

modified:   Address.java
modified:   Attachment.java
modified:   Duration.java
modified:   Element.java
modified:   ElementDefinition.java
modified:   MarketingStatus.java
modified:   ProductShelfLife.java
modified:   Signature.java

new file:   CodeableReference.java
new file:   RatioRange.java

Code Subtypes

deleted:    ExposureState.java

renamed:    EvidenceVariableType.java -> EvidenceVariableHandling.java

modified:   ConceptSubsumptionOutcome.java
modified:   DocumentConfidentiality.java
modified:   FHIRAllTypes.java
modified:   FHIRDefinedType.java
modified:   FHIRResourceType.java
modified:   FHIRVersion.java
modified:   RemittanceOutcome.java
modified:   ResourceType.java
modified:   Status.java

new file:   CharacteristicCombination.java
new file:   ClinicalUseDefinitionType.java
new file:   ClinicalUseIssueType.java
new file:   CriteriaNotExistsBehavior.java
new file:   MethodCode.java
new file:   NutritionProductStatus.java
new file:   ReportRelationshipType.java
new file:   SubscriptionNotificationType.java
new file:   SubscriptionTopicFilterBySearchModifier.java

[lmsurpre]

Code Subtype breaking changes:

EffectEvidenceSynthesis:  deleted.  was only used in EffectEvidenceSynthesis and that went away, so this makes sense

EvidenceVariableType: removes a value, adds two.  used only by EvidenceVariable in R4

ConceptSubsumptionOutcome:  flat-out missing. see https://jira.hl7.org/browse/FHIR-34183

FHIRAllTypes:  changes coincide with the spec changes; used by DataRequirement, OperationDefinition, and ParameterDefinition in R4
FHIRDefinedType: changes coincide with the spec changes; used by StructureDefinition, TestScript, and ElementDefinition in R4
FHIRResourceType: changes coincide with the spec changes; used by ExampleScenario in R4
ResourceType: changes coincide with the spec changes; used by ExampleScenario in R4; used in MessageDefinition, CapabilityStatement, GraphDefinition, ImplementationGuide, SearchParameter, Questionnaire, OperationDefinition, CompartmentDefinition, and Reference in R4

FHIRVersion: removes "4.0.1" and replaces it with "4.1.0"; used by CapabilityStatement, StructureDefinition, and ImplementationGuide.  This seems WRONG to me...should only be additive.  TODO: open hl7 jira tracker for this.

RemittanceOutcome: removes QUEUED; used by ExplanationOfBenefit, ClaimResponse, CoverageEligibilityResponse, EnrollmentResponse, and PaymentReconciliation.  this one seems problematic.

[lmsurpre]

Data Types breaking changes:

SubstanceAmount: deleted; was only used in SubstancePolymer

Duration: expression change.  `code.exists() implies ((system = %ucum) and value.exists())` to `value.exists() implies ((system = %ucum) and code.exists())`
ElementDefinition: regex fix; added explicit start (^) and end ($) matching
MarketingStatus: country and dateRange are no longer required / may be null (java breaking, not rest breaking)
ProductShelfLife: big changes; used in DeviceDefinition and MedicinalProductPackaged

notable mention:

Element: now explicitly allows empty Parameters resource (not a breaking change i think)

[lmsurpre]

Resource type breaking changes:

All renamed resources have substantial changes:
* MedicinalProduct
* MedicinalProductAuthorization
* MedicinalProductIndication
* MedicinalProductIngredient
* MedicinalProductManufactured
* MedicinalProductPackaged
* MedicinalProductPharmaceutical
* SubstanceSpecification

Additionally:
AllergyIntolerance: 2 constraint expression updates
Bundle: 1 constraint expression update
ClaimResponse: RemittanceOutcome no longer allows 'queued' (as mentioned above)
Composition: valueset url change.  new canonical url looks invalid to me:  `http://terminology.hl7.org/ValueSet/v3-Confidentiality|2.0.0|2.0.0` TODO: open hl7 jira tracker for this.
Condition: 1 constraint expression update
CoverageEligibilityResponse: RemittanceOutcome no longer allows 'queued' (as mentioned above)
DomainResource: 1 constraint expression update
EnrollmentResponse:  RemittanceOutcome no longer allows 'queued' (as mentioned above)
Evidence: big changes
EvidenceVariable: big changes
ExplanationOfBenefit: RemittanceOutcome no longer allows 'queued' (as mentioned above)
PaymentReconciliation: RemittanceOutcome no longer allows 'queued' (as mentioned above)
Questionnaire: 1 constraint expression fix
RiskAssessment:  1 constraint expression fix

I chose not to consider the addition of new valid reference targets a "breaking" change, but its possible for that to break folks if they encounter a resource and try processing it without realizing the new type is allowed.

notable mentions:

InsurancePlan: lost a `checkValueSetBinding(cost.applicability, "applicability", "http://hl7.org/fhir/ValueSet/insuranceplan-applicability", "http://terminology.hl7.org/CodeSystem/applicability", "in-network", "out-of-network", "other")`...not sure why
ActivityDefinition: new choice type allowed for subject
PlanDefinition: new choice type allowed for subject

The new choice types won't prevent R4 resources from being valid in R4B, but could prevent R4 clients from successfully working with instances of these types that were created with the new choice type in R4B.

[lmsurpre]

ideas:

A: totally separate versions, no migration

4.x - R4.  only support 4.0.1

5.x - R4B.  only supports 4.3.0?  maybe continue to support content-type with mime 4.0.1 but still validate with R4B constraints

6.x - R5?

Concerns:

4 and 5 will fight over which is "latest"

• maven users that use LATEST could pick up 5.x by accident
  • can a maven bom help here?

java package naming

• if we do it in-place (same artifact id or not) and user has them both, the packages will conflict
• HAPI has fhir version built into their package names. but what do they do for R4B?

B: totally separate versions but ensuring our 4.x will continue to work with the same database

4.x - R4.  only support 4.0.1

5.x - R4B.  only supports 4.3.0?  maybe continue to support content-type with mime 4.0.1 but still validate with R4B constraints

6.x - R5?

How long before we are allowed to make schema changes that will break R4?

Do we want to ensure that our R4 server doesn't try writing data to an R4B schema?

• how?
  • proposal A: if the version of the schema is beyond the version in the code, then run in read-only mode
  • proposal B: add additional metadata to the FHIRSchemaVersion to indicate level of compatibility
  • proposal C: add additional metadata to the FHIRSchemaVersion to indicate level of compatibility
  • proposal D: don't ensure it, but just document how to configure an R4 server to be read-only (already-supported)

C: different api endpoints in the same deployment

* /api/v4/ <- serving R4
* /api/v5/ <- serving R4B
   * could be /api/v4b/ to avoid confusion?

Two flavors of this idea:
1. same schema
2. two different schemas

D: try supporting both in parallel

both versions on the same endpoint:
* /api/v4/ <- serving R4B + removed/deleted R4 resource types

Maybe
 * based on some config we can even use R4 constraints
 * use the fhirversion part from the ContentType/Accept headers to drive behavior

Two flavors of this idea:
1. same schema
2. two different schemas

---

any special considerations for audit?

What about changes to:

• search parameters?
• compartment definitions?
• operation definitions?

Current naming convention: marketing version and module version always in sync

IBM FHIR Server 5.0.0
module versions: 5.0.0

To be naming options:

conflicting priorities

• avoid confusion for which version of server is for which version of the spec
• accurately reflect major breaking changes in fhir-model and elsewhere

A: jump a bunch of versions, but keep current versioning convention in place

marketing version: IBM FHIR Server 4.20
module version: 4.20.0

make very clear in release notes that this version adds support for R4B

B: break the marketing version from the module version

marketing version: IBM FHIR Server R4B 2022-01
module version: 5.0.0

alternative marketing version: IBM FHIR Server 2022-1 for FHIR R4B

what do we use for the git tag? paul suggesting to keep it in line with the module version, so that would be 5.0.0

incrementing number or month?

• leaning toward an incrementing number because we can do multiple releases in a month
• people will assume its a month but maybe thats ok?

single digit (2022-1) or multiple digits (2022-01)?

• two-digit looks more like a date, so maybe go with one-digit if its an incrementing number?

benefit: the new name is sufficiently different that we could still release 4.x versions with old naming scheme

• IBM FHIR Server R4B 2022-01
• IBM FHIR Server 4.11.0 (IBM FHIR Server R4 2022-01 follow the new naming better but team agrees its actually too similar...favors old naming style for followup R4 releases)

i think we could release them in either order. only place i think it matters is the docker 'latest' tag

• extra "rolling" tags are allowed in docker, could be added manually to start
  • 4.x, r4
  • 5.x, r4b, 2022-01

rough plan:

• do an initial release from an r4b branch
  • this will help us kick the tires / run through a "release from a different branch" exercise
• continue with 4.x releases as needed
• when ready, move the r4b branch to main
  • from this point on, hopefully we wouldn't need another R4 (4.x) release, but if we do need it we'll have at least gone through that exercise of releasing form a different branch

[lmsurpre]

After lots of team discussion, we're leaning toward the following choices:
B: totally separate versions but ensuring our 4.x will continue to work with the same database
B: break the marketing version from the module version

That would mean we'd shoot for a release from the r4b branch sometime in january/february.
Will open separate tickets for the remaining work.

[lmsurpre]

ProductShelfLife: big changes; used in DeviceDefinition and MedicinalProductPackaged

MedicinalProductPackaged was renamed, so we won't need to deal with old data there (we won't perform any automated migration for any of the "renamed" resource types).
However, DeviceDefinition exists in both 4.0.1 AND 4.3.0 and so this is a breaking change for this resource type. Its not a real common one and so I think this is still acceptable, but we should definitely make this one clear in our release notes.

UPDATE: Lloyd is going to discuss this one with the FHiR management workgroup...they may fix this to avoid the breaking change to DeviceDefinition: https://chat.fhir.org/#narrow/stream/179166-implementers/topic/R4B.20compatibilty/near/269339301

We'll still have backwards-compatibility issues with Evidence and EvidenceVariable though...so we'll still need to warn users about that.

[lmsurpre]

I missed the significance of this one in my initial analysis

FHIRAllTypes: changes coincide with the spec changes; used by DataRequirement, OperationDefinition, and ParameterDefinition in R4

The issue is that DataRequirement is used by TriggerDefinition, GuidanceResponse, Library, Parameters, PlanDefinition, ResearchElementDefinition, StructureMap and Task ...so this one has a potentially wide blast radius. I'm gonna make the same change that I made to the ResourceType code subtype to the FHIRAllTypes one so that we can at least parse any old resources that are using one of the retired codes.