-
Notifications
You must be signed in to change notification settings - Fork 160
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Duplicate values for one resource in xx_resource_token_refs #3763
Comments
Deduplicating the parameter values before we store them requires visiting all the ExtractedParameterValue values and converting them into values we intend to store in the database. This deduplication can't be done at the ExtractedParameterValue level because an extracted value may contain multiple values. This process is already performed by the remote-index client, so I propose that we do some refactoring and replace the current parameter persistence code with the newer (hopefully faster) remote index implementation. This will allow all parameter values to be collected as part of the transaction and should lead to improved throughput. |
This refactor requires a new project is created. The new project will be shared by both fhir-persistence-jdbc and fhir-remote-index. This avoids polluting fhir-persistence-jdbc with Kafka stuff, and fhir-remote-index with all the dependencies required for fhir-persistence-jdbc. |
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
The new search parameter persistence code in fhir-persistence-params supports both PostgreSQL and Derby. For now, Db2 will use the old mechanism. |
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
Tried running the system integration test suite and did not find any duplicate values for a resource in patient_resource_token_refs. |
Describe the bug
The xx_resource_token_refs table contains duplicate entries. Although this doesn't impact the results of FHIR search interactions, it does lead to increased storage costs:
The same is true for xx_str_values.
Parameter tables are heavily indexed, so having duplicates here is significant.
Environment
Which version of IBM FHIR Server? 5.0.0
To Reproduce
Expected behavior
Parameter tables should not have duplicates. Note, some parameter values may be stored more than once if there are composites involved, but a parameter value should not be stored more than once where composite_id is null.
Additional context
N/A
The text was updated successfully, but these errors were encountered: