New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add experimental support for Citus #3437

Closed

punktilious opened this issue Mar 7, 2022 · 1 comment

Assignees

Labels

Collaborator

punktilious commented Mar 7, 2022

Is your feature request related to a problem? Please describe.
Citus is a distributed database extension for PostgreSQL.

Add distribution/replication information to the schema so that this metadata can be applied when the tables are created. Filter the creation of foreign key relationships according to the distribution rules.

Describe the solution you'd like
The schema tool should contain support for Citus databases.

Describe alternatives you've considered
Apply the Citus logic manually after the schema is created.

Acceptance Criteria

GIVEN [a precondition]
AND [another precondition]
WHEN [test step]
AND [test step]
THEN [verification step]
AND [verification step]

Additional context
Does not support migration from an earlier version. The first Citus install must be FHIR schema version V0026.

punktilious added the enhancement label

punktilious added a commit that referenced this issue


          issue #3437 experimental support for Citus database

c4a11be

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 schema build changes to support Citus distribution

7244dce

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 citus requires some custom ingestion logic

a7f2379

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 distribute add_any_resource function in Citus

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 merge latest main

d0eaa36

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 distributed schema using shard_key

68819ab

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 remote index kafka consumer

0752bbb

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 remote index support for profile, tags and security

6343ab9

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 remote index wait until server transaction commits

a8acdb8

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

lmsurpre assigned punktilious

punktilious added a commit that referenced this issue


          issue #3437 add logical_resource_ident and ref param support

7125f67

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 fix reference unit tests for new ref_values table

f705e5a

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 use logical_resource_ident for absolute references

0c1702b

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 compilation issues from shard update

8c5cc75

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 fix remote index processing for new logical_resource_iden…

ca0bfa0

…t recs

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 tidied javadoc

246897f

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 fixed issues using Citus

24bb883

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 use identity cache for more ids

3e6699d

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 merge with main for R4B

19daa4c

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 do not apply permissions to tables not created

fcf2c1e

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 initial derby unit test for fhir-remote-index

f2687cc

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 more unit test coverage for fhir-remote-index

f44b4cb

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 delete parameters from new xx_ref_values

4de35a5

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 allow reindex to be forced after database migration

40e2454

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 use logical_resource_ident for read and vread with citus

c3ab574

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 fixed postgres and db2 add_any_resource to use lr ident

fe03611

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 db-type arg now required when running schema tool

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 fixed schema migration for Db2

7ca7633

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 fixed missing grant for Db2

89e14ad

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 new inserts need mt_id for Db2 multitenant

fde6f80

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

lmsurpre mentioned this issue

Documentation for postgresql citus support #3698

Closed

punktilious added a commit that referenced this issue


          issue #3437 addressed review comments. Support whole system token val…

7a9221a

…ues. Remove ref_version_id

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 fixing drop column for ref_version_id

dec8d3a

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 tab to space and fixed comments

dd08e87

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 fixed fhir-bucket and corrected DropView comment

f37233b

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 corrected DropView comment

5f7e25d

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 javadoc review comments

150e3c3

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 fix usage of logicalResourceId and resourceId for remote …

f85055f

…index

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 skip parameter storage for remote index with derby and db2

5aa9325

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 add instanceIdentifier to remote index messages and other…

c446a15

… review fixes

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3713 use distributed add_any_resource function for citus (#3718)

583903a

* issue #3713 use distributed add_any_resource function for citus

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

* issue #3437 fix usage of logicalResourceId and resourceId for remote index

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 inject instanceIdentifier when sending remote index payload

6c8d712

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

punktilious added a commit that referenced this issue


          issue #3437 more schema design documentation

59e3c24

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

Member

lmsurpre commented Aug 4, 2022 •

edited

Loading

Robin ran the fhir-server-test integration tests against a server configured with the citus backend and all the tests were passed.

Interestingly, this was using single-node PostgreSQL that is configured to work like Citus by specifying dataSourceType=citus in the fhir-server-config.
On a real citus, there remain some limitations on the search API including _include, _revinclude, chaining, and reverse chaining (_has)

lmsurpre closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment