Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server: ensure SQL pods have unique instance IDs within the tenant #64474

Closed
knz opened this issue Apr 30, 2021 · 2 comments
Closed

server: ensure SQL pods have unique instance IDs within the tenant #64474

knz opened this issue Apr 30, 2021 · 2 comments
Assignees
Labels
A-cc-enablement Pertains to current CC production issues or short-term projects A-cli-server CLI commands that pertain to CockroachDB server processes A-multitenancy Related to multi-tenancy C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) X-server-triaged-202105

Comments

@knz
Copy link
Contributor

knz commented Apr 30, 2021

In the serverless / multi-tenant architecture, we need to identify different SQL pods for the same tenant.

This separation is provided by the "Instance ID" abstraction in the source code. We use the instance ID at least in 3 places:

  • when logging events, to distinguish servers under a single tenant.
  • to compute the unique_rowid() function in SQL
  • for table leases

As of this writing, the instance ID is not fully implemented and is fixed to a constant value. This prevents running multiple SQL pods for a single tenant.

We need to complete this by allocating a separate instance ID to each tenant process.

There are three design constraints:

  • the value must remain 31 bits (like NodeID), and preferably use fewer bits than that.
  • since we envision that SQL pods will be restarted very frequently, we can't blindly increment a sequence. Some form of ID reuse should be implemented.
  • the instance ID must be known very early during server initialization, before the SQL subsystem is ready to serve queries. This means that we cannot look up the instance ID inside a SQL table from the pod process directly.

The third point alone suggests that the SQL pod process should request its SQL instance ID using a RPC to the nearest KV node.

Epic CRDB-2576

@knz knz added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-multitenancy Related to multi-tenancy A-cli-server CLI commands that pertain to CockroachDB server processes labels Apr 30, 2021
@knz
Copy link
Contributor Author

knz commented Apr 30, 2021

cc @dhartunian

@exalate-issue-sync
Copy link

Assigned to rima

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-cc-enablement Pertains to current CC production issues or short-term projects A-cli-server CLI commands that pertain to CockroachDB server processes A-multitenancy Related to multi-tenancy C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) X-server-triaged-202105
Projects
None yet
Development

No branches or pull requests

3 participants