You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In this issue, I wanted to list some of the things we need in order to enable Citus MX on all clusters. Assuming that we enable this feature on Citus 11.0, all the clusters upgrading from earlier version would sync metadata to all nodes.
Sync metadata to all nodes by default / sync always [3 weeks]
Probably requires lots of regression test changes
Likely to find Citus bugs
Maybe merge mx- schedules as all the schedules are now MX
Local tables: All tables are auto-converted to Citus local tables
Coordinator is always added to the metadata
Automatically convert Local tables to Citus Local Tables at CREATE TABLE time
Some features that are not very relevant to Citus MX. Can we drop support for them?
Replication factor > 1
Append/range partitioned tables: We currently do not sync metadata for these
Upgrades / downgrades
Sync metadata to all the nodes
Make sure that the coordinator is added to the metadata [regarding local tables]
Any problems in case other extension exists? postgres_fdw or dblink?
Should we stop metadata sync on downgrades?
Can we have --check-upgrade type of UDF?
Connection scaling
(Marco) Random thing that bugs me a bit about MX which I'm not sure I mentioned: XXX did transactions with many inserts across different shards in large batches, causing every transaction to span across all nodes, which also means that even with outbound pgbouncers and a million nodes you still cannot handle more than max_connections concurrent transactions
PG 14 connection scaling benchmarks
Testing
Can we run more queries from the workers on the regression tests?
Local tables: All tables are auto-converted to Citus local tables #5305
This relates to the discussion in #3851 . I don't think it is strictly necessary for an MVP as long as we have a fallback mechanism for views and functions.
It does seem like a "should have", given the general problem of local tables in MX.
Github seems to add a new feature, if you click on a task item, it opens an issue. So, 5305 opened accidentally. I agree that 3851 has more information
In this issue, I wanted to list some of the things we need in order to enable Citus MX on all clusters. Assuming that we enable this feature on Citus 11.0, all the clusters upgrading from earlier version would sync metadata to all nodes.
CREATE TABLE
timeSequences:
Distributed views
Foreign tables Metadata syncing fails when there are foreign distributed tables #4947
Observibility
Function propagation is simplified
Propagate pg_dist_object Propagate pg_dist_object for metadata nodes #5065
Some features that are not very relevant to Citus MX. Can we drop support for them?
Upgrades / downgrades
Connection scaling
Testing
Multi-user testing
Various MX specific deadlocks&bugs requiring some testing
get_all_active_transactions
->get_all_active_distributed_executions
or such #5283Remove code
create_distributed_function
syncing the metadata can be removedShouldSyncTableMetadata
ErrorIfAnyMetadataNodeOutOfSync
anymore? All the metadata nodes should always be synced?The text was updated successfully, but these errors were encountered: