#### Hardware
- 3 regions
-
- Per region, 3+ AZs with 3+ VMs evenly distributed across them
-
- Region-specific app instances and load balancers
- Each load balancer redirects to CockroachDB nodes in its region.
- When CockroachDB nodes are unavailable in a region, the load balancer redirects to nodes in other regions.
#### Cluster
-Each node is started with the [`--locality`](cockroach-start.html#locality) flag specifying its region and AZ combination. For example, the following command starts a node in the west1 AZ of the us-west region:
+Each node is started with the [`--locality`](cockroach-start.html#locality) flag specifying its region and AZ combination. For example, the following command starts a node in the `west1` AZ of the `us-west` region:
{% include copy-clipboard.html %}
~~~ shell
@@ -22,7 +20,7 @@ $ cockroach start \
--locality=region=us-west,zone=west1 \
--certs-dir=certs \
--advertise-addr=`-e` | Execute SQL statements directly from the command line, without opening a shell. This flag can be set multiple times, and each instance can contain one or more statements separated by semi-colons.
If an error occurs in any statement, the command exits with a non-zero status code and further statements are not executed. The results of each statement are printed to the standard output (see `--format` for formatting options). `--format` | How to display table rows printed to the standard output. Possible values: `tsv`, `csv`, `table`, `raw`, `records`, `sql`, `html`.
**Default:** `table` for sessions that [output on a terminal](cockroach-sql.html#session-and-output-types); `tsv` otherwise
This flag corresponds to the `display_format` [client-side option](#client-side-options) for use in interactive sessions. -`--geo-partitioned-replicas` | Start a 9-node demo cluster with the [Geo-Partitioned Replicas](topology-geo-partitioned-replicas.html) topology pattern applied to the [`movr`](movr.html) database. +`--geo-partitioned-replicas` | Start a 9-node demo cluster with [geo-partitioning](#start-a-multi-region-demo-cluster-with-automatic-geo-partitioning) applied to the [`movr`](movr.html) database. `--global` | This experimental flag is used to simulate a [multi-region cluster](simulate-a-multi-region-cluster-on-localhost.html) which sets the [`--locality` flag on node startup](cockroach-start.html#locality) to three different regions. It also simulates the network latency that would occur between them given the specified localities. In order for this to operate as expected, with 3 nodes in each of 3 regions, you must also pass the `--nodes 9` argument. `--insecure` | Set this to `false` to start the demo cluster in secure mode using TLS certificates to encrypt network communication. `--insecure=false` gives you an easy way test out CockroachDB [authorization features](authorization.html) and also creates a password (`admin`) for the `root` user for logging into the DB Console.
**Env Variable:** `COCKROACH_INSECURE`
**Default:** `false` `--max-sql-memory` | For each demo node, the maximum in-memory storage capacity for temporary SQL data, including prepared queries and intermediate data rows during query execution. This can be a percentage (notated as a decimal or with `%`) or any bytes-based unit, for example:
`--max-sql-memory=.25`
`--max-sql-memory=25%`
`--max-sql-memory=10000000000 ----> 1000000000 bytes`
`--max-sql-memory=1GB ----> 1000000000 bytes`
`--max-sql-memory=1GiB ----> 1073741824 bytes`
**Default:** `128MiB` @@ -446,7 +446,9 @@ You can also use this URL to connect an application to the demo cluster. $ cockroach demo --geo-partitioned-replicas ~~~ -This command starts a 9-node demo cluster with the `movr` database preloaded, and [partitions](partitioning.html) and [zone constraints](configure-replication-zones.html) applied to the primary and secondary indexes. For more information, see the [Geo-Partitioned Replicas](topology-geo-partitioned-replicas.html) topology pattern. +This command starts a 9-node demo cluster with the `movr` database preloaded, and [partitions](partitioning.html) and [zone constraints](configure-replication-zones.html) applied to the primary and secondary indexes. + +{% include {{page.version.version}}/sql/use-multiregion-instead-of-partitioning.md %} ### Shut down and restart nodes in a multi-node demo cluster diff --git a/v21.1/computed-columns.md b/v21.1/computed-columns.md index 8ac36a164a9..0c70b456693 100644 --- a/v21.1/computed-columns.md +++ b/v21.1/computed-columns.md @@ -6,12 +6,9 @@ toc: true A computed column exposes data generated from other columns by a [scalar expression](scalar-expressions.html) included in the column definition. A stored computed column (set with the `STORED` SQL keyword) is calculated when a row is inserted or updated, and stores the resulting value of the scalar expression in the primary index similar to a regular column. A virtual computed column (set with the `VIRTUAL` SQL keyword) is not stored, and the value of the scalar expression is computed during queries as needed. - ## Why use computed columns? -Computed columns are especially useful when used with [partitioning](partitioning.html), [`JSONB`](jsonb.html) columns, or [secondary indexes](indexes.html). - -- **Partitioning** requires that partitions are defined using columns that are a prefix of the [primary key](primary-key.html). In the case of geo-partitioning, some applications will want to collapse the number of possible values in this column, to make certain classes of queries more performant. For example, if a users table has a country and state column, then you can make a stored computed column locality with a reduced domain for use in partitioning. For more information, see the [partitioning example](#create-a-table-with-geo-partitions-and-a-computed-column) below. +Computed columns are especially useful when used with [`JSONB`](jsonb.html) columns or [secondary indexes](indexes.html). - **JSONB** columns are used for storing semi-structured `JSONB` data. When the table's primary information is stored in `JSONB`, it's useful to index a particular field of the `JSONB` document. In particular, computed columns allow for the following use case: a two-column table with a `PRIMARY KEY` column and a `payload` column, whose primary key is computed as some field from the `payload` column. This alleviates the need to manually separate your primary keys from your JSON blobs. For more information, see the [`JSONB` example](#create-a-table-with-a-jsonb-column-and-a-stored-computed-column) below. @@ -72,10 +69,6 @@ Parameter | Description {% include {{ page.version.version }}/computed-columns/virtual.md %} -### Create a table with geo-partitions and a computed column - -{% include {{ page.version.version }}/computed-columns/partitioning.md %} The `locality` values can then be used for geo-partitioning. - ### Create a table with a secondary index on a computed column {% include {{ page.version.version }}/computed-columns/secondary-index.md %} @@ -96,4 +89,3 @@ For more information, see [`ADD COLUMN`](add-column.html). - [Information Schema](information-schema.html) - [`CREATE TABLE`](create-table.html) - [`JSONB`](jsonb.html) -- [Define Table Partitions (Enterprise)](partitioning.html) diff --git a/v21.1/cost-based-optimizer.md b/v21.1/cost-based-optimizer.md index 3850ab0ff3a..52a26bf021d 100644 --- a/v21.1/cost-based-optimizer.md +++ b/v21.1/cost-based-optimizer.md @@ -193,509 +193,12 @@ To make the optimizer prefer lookup joins to merge joins when performing foreign - Hint usage should be reconsidered with each new release of CockroachDB. Due to improvements in the optimizer, hints specified to work with an older version may cause decreased performance in a newer version. -## Preferring the nearest index +## Examples -Given multiple identical [indexes](indexes.html) that have different locality constraints using [replication zones](configure-replication-zones.html), the optimizer will prefer the index that is closest to the gateway node that is planning the query. In a properly configured geo-distributed cluster, this can lead to performance improvements due to improved data locality and reduced network traffic. - -{{site.data.alerts.callout_info}} -This feature is only available to users with an [enterprise license](enterprise-licensing.html). For insight into how to use this feature to get low latency, consistent reads in multi-region deployments, see the [Duplicate Indexes](topology-follower-reads.html) topology pattern. -{{site.data.alerts.end}} - -This feature enables scenarios such as: - -- Reference data such as a table of postal codes that can be replicated to different regions, and queries will use the copy in the same region. See [Example - zone constraints](#zone-constraints) for more details. -- Optimizing for local reads (potentially at the expense of writes) by adding leaseholder preferences to your zone configuration. See [Example - leaseholder preferences](#leaseholder-preferences) for more details. - -To take advantage of this feature, you need to: - -1. Have an [enterprise license](enterprise-licensing.html). -2. Determine which data consists of reference tables that are rarely updated (such as postal codes) and can therefore be easily replicated to different regions. -3. Create multiple [secondary indexes](indexes.html) on the reference tables. **Note that these indexes must include (in key or using [`STORED`](create-index.html#store-columns)) *every* column that you wish to query**. For example, if you run `SELECT * from db.table` and not every column of `db.table` is in the set of secondary indexes you created, the optimizer will have no choice but to fall back to the primary index. -4. Create [replication zones](configure-replication-zones.html) for each index. - -With the above pieces in place, the optimizer will automatically choose the index nearest the gateway node that is planning the query. - -{{site.data.alerts.callout_info}} -The optimizer does not actually understand geographic locations, i.e., the relative closeness of the gateway node to other nodes that are located to its "east" or "west". It is matching against the [node locality constraints](configure-replication-zones.html#descriptive-attributes-assigned-to-nodes) you provided when you configured your replication zones. -{{site.data.alerts.end}} - -### Examples - -#### Inverted join examples +### Inverted join examples {% include {{ page.version.version }}/sql/inverted-joins.md %} -#### Zone constraints - -We can demonstrate the necessary configuration steps using a local cluster. The instructions below assume that you are already familiar with: - -- How to [start a local cluster](start-a-local-cluster.html). -- The syntax for [assigning node locality when configuring replication zones](configure-replication-zones.html#descriptive-attributes-assigned-to-nodes). -- Using [the built-in SQL client](cockroach-sql.html). - -First, start 3 local demo nodes as shown below. The `--demo-locality` flag uses the [`--locality`](cockroach-start.html#locality) internally flag to put the nodes each in a different region as denoted by `region=usa`, `region=eu`, and `region=apac`. - -{% include copy-clipboard.html %} -~~~ shell -./cockroach demo --nodes 3 --demo-locality=region=usa:region=eu:region=apac -~~~ - -Create a test database and table. The table will have 3 indexes into the same data. Later, we'll configure the cluster to associate each of these indexes with a different datacenter using replication zones. - -{% include copy-clipboard.html %} -~~~ sql -> CREATE DATABASE IF NOT EXISTS test; -~~~ - -{% include copy-clipboard.html %} -~~~ sql -> USE test; -~~~ - -{% include copy-clipboard.html %} -~~~ sql -CREATE TABLE postal_codes ( - id INT PRIMARY KEY, - code STRING, - INDEX idx_eu (id) STORING (code), - INDEX idx_apac (id) STORING (code) -); -~~~ - -Next, we modify the replication zone configuration via SQL so that: - -- Nodes in the USA will use the primary key index. -- Nodes in the EU will use the `postal_codes@idx_eu` index (which is identical to the primary key index). -- Nodes in APAC will use the `postal_codes@idx_apac` index (which is also identical to the primary key index). - -{% include copy-clipboard.html %} -~~~ sql -ALTER TABLE postal_codes CONFIGURE ZONE USING constraints='["+region=usa"]'; -~~~ - -{% include copy-clipboard.html %} -~~~ sql -ALTER INDEX postal_codes@idx_eu CONFIGURE ZONE USING constraints='["+region=eu"]'; -~~~ - -{% include copy-clipboard.html %} -~~~ sql -ALTER INDEX postal_codes@idx_apac CONFIGURE ZONE USING constraints='["+region=apac"]'; -~~~ - -To verify this feature is working as expected, we'll query the database from each of our local nodes as shown below. Each node has been configured to be in a different region, and it should now be using the index pinned to that region. - -{{site.data.alerts.callout_info}} -In a geo-distributed scenario with a cluster that spans multiple datacenters, it may take time for the optimizer to fetch schemas from other nodes the first time a query is planned; thereafter, the schema should be cached locally. - -For example, if you have 11 nodes, you may see 11 queries with high latency due to schema cache misses. Once all nodes have cached the schema locally, the latencies will drop. - -This behavior may also cause the [Statements page of the DB Console](ui-statements-page.html) to show misleadingly high latencies until schemas are cached locally. -{{site.data.alerts.end}} - -As expected, the node in the USA region uses the primary key index. - -{% include copy-clipboard.html %} -~~~ sql -EXPLAIN SELECT * FROM postal_codes WHERE id=1; -~~~ - -~~~ - info ---------------------------------- - distribution: local - vectorized: true - - • scan - estimated row count: 1 (100% of the table; stats collected 22 seconds ago) - table: postal_codes@primary - spans: [/1 - /1] -(7 rows) - -Time: 1ms total (execution 1ms / network 0ms) -~~~ - -Find the connection string for the nodes in the demo cluster. - -{% include copy-clipboard.html %} -~~~ sql -\demo ls -~~~ - -~~~ -node 1: - (webui) http://127.0.0.1:8080/demologin?password=demo65395&username=demo - (sql) postgres://demo:demo65395@127.0.0.1:26257?sslmode=require - (sql/unix) postgres://demo:demo65395@?host=%2Fvar%2Ffolders%2Fr5%2F7q2jzh4n0rj5lgcwc47st58m0000gp%2FT%2Fdemo064093534&port=26257 - -node 2: - (webui) http://127.0.0.1:8081/demologin?password=demo65395&username=demo - (sql) postgres://demo:demo65395@127.0.0.1:26258?sslmode=require - (sql/unix) postgres://demo:demo65395@?host=%2Fvar%2Ffolders%2Fr5%2F7q2jzh4n0rj5lgcwc47st58m0000gp%2FT%2Fdemo064093534&port=26258 - -node 3: - (webui) http://127.0.0.1:8082/demologin?password=demo65395&username=demo - (sql) postgres://demo:demo65395@127.0.0.1:26259?sslmode=require - (sql/unix) postgres://demo:demo65395@?host=%2Fvar%2Ffolders%2Fr5%2F7q2jzh4n0rj5lgcwc47st58m0000gp%2FT%2Fdemo064093534&port=26259 -~~~ - -Note the `(sql)` connection string for nodes 2 and 3. - -Open a new terminal and connect to node 2, which is in the `eu` region, and run the same `EXPLAIN` query. Use the connection string from node 2 in your demo cluster, as it will differ from this example. - -{% include copy-clipboard.html %} -~~~ shell -cockroach sql --url 'postgres://demo:demo65395@127.0.0.1:26258?sslmode=require' --database=test -e 'EXPLAIN SELECT * FROM postal_codes WHERE id=1;' -~~~ - -The node in the EU uses the `idx_eu` index. - -~~~ ---------------------------------- - distribution: local - vectorized: true - - • scan - estimated row count: 1 (100% of the table; stats collected 5 minutes ago) - table: postal_codes@idx_eu - spans: [/1 - /1] -(7 rows) - -Time: 11ms -~~~ - -Connect to node 3, which is in the `apac` region, and run the same `EXPLAIN` query. Use the connection string from node 3 in your demo cluster, as it will differ from this example. - -{% include copy-clipboard.html %} -~~~ shell -$ cockroach sql --url 'postgres://demo:demo65395@127.0.0.1:26259?sslmode=require' --database=test -e 'EXPLAIN SELECT * FROM postal_codes WHERE id=1;' -~~~ - -The node in APAC uses the `idx_apac` index. - -~~~ ---------------------------------- - distribution: local - vectorized: true - - • scan - estimated row count: 1 (100% of the table; stats collected 9 minutes ago) - table: postal_codes@idx_apac - spans: [/1 - /1] -(7 rows) - -Time: 1ms -~~~ - -You'll need to make changes to the above configuration to reflect your [production environment](recommended-production-settings.html), but the concepts will be the same. - -#### Leaseholder preferences - -If you provide [leaseholder preferences](configure-replication-zones.html#lease_preferences) in addition to replication zone constraints, the optimizer will attempt to take your leaseholder preferences into account as well when selecting an index for your query. There are several factors to keep in mind: - -- Zone constraints are always respected (hard constraint), whereas lease preferences are taken into account as "additional information" -- as long as they do not contradict the zone constraints. - -- The optimizer does not consider the real-time location of leaseholders when selecting an index; it is pattern matching on the text values passed in the configuration (e.g., the [`ALTER INDEX`](alter-index.html) statements shown below). For the same reason, the optimizer only matches against the first locality in your `lease_preferences` array. - -- The optimizer may use an index that satisfies your leaseholder preferences even though that index has moved to a different node/region due to [leaseholder rebalancing](architecture/replication-layer.html#leaseholder-rebalancing). This can cause slower performance than you expected. Therefore, you should only use this feature if you’re confident you know where the leaseholders will end up based on your cluster's usage patterns. We recommend thoroughly testing your configuration to ensure the optimizer is selecting the index(es) you expect. - -In this example, we'll set up an authentication service using the access token / refresh token pattern from [OAuth 2](https://www.digitalocean.com/community/tutorials/an-introduction-to-oauth-2). To support fast local reads in our geo-distributed use case, we will have 3 indexes into the same authentication data: one for each region of our cluster. We configure each index using zone configurations and lease preferences so that the optimizer will use the local index for better performance. - -The instructions below assume that you are already familiar with: - -- How to [start a local cluster](start-a-local-cluster.html). -- The syntax for [assigning node locality when configuring replication zones](configure-replication-zones.html#descriptive-attributes-assigned-to-nodes). -- Using [the built-in SQL client](cockroach-sql.html). - -First, start 3 local nodes as shown below. Use the [`--locality`](cockroach-start.html#locality) flag to put them each in a different region. - -{% include copy-clipboard.html %} -~~~ shell -$ cockroach start --locality=region=us-east --insecure --store=/tmp/node0 --listen-addr=localhost:26257 \ - --http-port=8888 --join=localhost:26257,localhost:26258,localhost:26259 --background -~~~ - -{% include copy-clipboard.html %} -~~~ shell -$ cockroach start --locality=region=us-central --insecure --store=/tmp/node1 --listen-addr=localhost:26258 \ - --http-port=8889 --join=localhost:26257,localhost:26258,localhost:26259 --background -~~~ - -{% include copy-clipboard.html %} -~~~ shell -$ cockroach start --locality=region=us-west --insecure --store=/tmp/node2 --listen-addr=localhost:26259 \ - --http-port=8890 --join=localhost:26257,localhost:26258,localhost:26259 --background -~~~ - -{% include copy-clipboard.html %} -~~~ shell -$ cockroach init --insecure --host=localhost --port=26257 -~~~ - -From the SQL client, add your organization name and enterprise license: - -{% include copy-clipboard.html %} -~~~ sh -$ cockroach sql --insecure --host=localhost --port=26257 -~~~ - -{% include copy-clipboard.html %} -~~~ sql -> SET CLUSTER SETTING cluster.organization = 'FooCorp - Local Testing'; -~~~ - -{% include copy-clipboard.html %} -~~~ sql -> SET CLUSTER SETTING enterprise.license = 'xxxxx'; -~~~ - -Create an authentication database and table: - -{% include copy-clipboard.html %} -~~~ sql -> CREATE DATABASE if NOT EXISTS auth; -~~~ - -{% include copy-clipboard.html %} -~~~ sql -> USE auth; -~~~ - -{% include copy-clipboard.html %} -~~~ sql -> CREATE TABLE token ( - token_id VARCHAR(100) NULL, - access_token VARCHAR(4000) NULL, - refresh_token VARCHAR(4000) NULL - ); -~~~ - -Create the indexes for each region: - -{% include copy-clipboard.html %} -~~~ sql -> CREATE INDEX token_id_west_idx ON token (token_id) STORING (access_token, refresh_token); -~~~ - -{% include copy-clipboard.html %} -~~~ sql -> CREATE INDEX token_id_central_idx ON token (token_id) STORING (access_token, refresh_token); -~~~ - -{% include copy-clipboard.html %} -~~~ sql -> CREATE INDEX token_id_east_idx ON token (token_id) STORING (access_token, refresh_token); -~~~ - -Enter zone configurations to distribute replicas across the cluster as follows: - -- For the "East" index, store 2 replicas in the East, 2 in Central, and 1 in the West. Further, prefer that the leaseholders for that index live in the East or, failing that, in the Central region. -- Follow the same replica and leaseholder patterns for each of the Central and West regions. - -The idea is that, for example, `token_id_east_idx` will have sufficient replicas (2/5) so that even if one replica goes down, the leaseholder will stay in the East region. That way, if a query comes in that accesses the columns covered by that index from the East gateway node, the optimizer will select `token_id_east_idx` for fast reads. - -{{site.data.alerts.callout_info}} -The `ALTER TABLE` statement below is not required since it's later made redundant by the `token_id_west_idx` index. In production, you might go with the `ALTER TABLE` to put your table's lease preferences in the West, and then create only 2 indexes (for East and Central); however, the use of 3 indexes makes the example easier to understand. -{{site.data.alerts.end}} - -{% include copy-clipboard.html %} -~~~ sql -> ALTER TABLE token CONFIGURE ZONE USING - num_replicas = 5, constraints = '{+region=us-east: 1, +region=us-central: 2, +region=us-west: 2}', lease_preferences = '[[+region=us-west], [+region=us-central]]'; -~~~ - -{% include copy-clipboard.html %} -~~~ sql -> ALTER INDEX token_id_east_idx CONFIGURE ZONE USING num_replicas = 5, - constraints = '{+region=us-east: 2, +region=us-central: 2, +region=us-west: 1}', lease_preferences = '[[+region=us-east], [+region=us-central]]'; -~~~ - -{% include copy-clipboard.html %} -~~~ sql -> ALTER INDEX token_id_central_idx CONFIGURE ZONE USING num_replicas = 5, - constraints = '{+region=us-east: 2, +region=us-central: 2, +region=us-west: 1}', lease_preferences = '[[+region=us-central], [+region=us-east]]'; -~~~ - -{% include copy-clipboard.html %} -~~~ sql -> ALTER INDEX token_id_west_idx CONFIGURE ZONE USING num_replicas = 5, - constraints = '{+region=us-west: 2, +region=us-central: 2, +region=us-east: 1}', lease_preferences = '[[+region=us-west], [+region=us-central]]'; -~~~ - -Next let's [check our zone configurations](show-zone-configurations.html) to make sure they match our expectation: - -{% include copy-clipboard.html %} -~~~ sql -> SHOW ZONE CONFIGURATIONS; -~~~ - -The output should include the following: - -~~~ - target | raw_config_sql ----------------------------------------------------+--------------------------------------------------------------------------------------- - RANGE default | ALTER RANGE default CONFIGURE ZONE USING - | range_min_bytes = 134217728, - | range_max_bytes = 536870912, - | gc.ttlseconds = 90000, - | num_replicas = 3, - | constraints = '[]', - | lease_preferences = '[]' - DATABASE system | ALTER DATABASE system CONFIGURE ZONE USING - | range_min_bytes = 134217728, - | range_max_bytes = 536870912, - | gc.ttlseconds = 90000, - | num_replicas = 5, - | constraints = '[]', - | lease_preferences = '[]' - RANGE meta | ALTER RANGE meta CONFIGURE ZONE USING - | range_min_bytes = 134217728, - | range_max_bytes = 536870912, - | gc.ttlseconds = 3600, - | num_replicas = 5, - | constraints = '[]', - | lease_preferences = '[]' - RANGE system | ALTER RANGE system CONFIGURE ZONE USING - | range_min_bytes = 134217728, - | range_max_bytes = 536870912, - | gc.ttlseconds = 90000, - | num_replicas = 5, - | constraints = '[]', - | lease_preferences = '[]' - RANGE liveness | ALTER RANGE liveness CONFIGURE ZONE USING - | range_min_bytes = 134217728, - | range_max_bytes = 536870912, - | gc.ttlseconds = 600, - | num_replicas = 5, - | constraints = '[]', - | lease_preferences = '[]' - TABLE system.public.replication_constraint_stats | ALTER TABLE system.public.replication_constraint_stats CONFIGURE ZONE USING - | gc.ttlseconds = 600, - | constraints = '[]', - | lease_preferences = '[]' - TABLE system.public.replication_stats | ALTER TABLE system.public.replication_stats CONFIGURE ZONE USING - | gc.ttlseconds = 600, - | constraints = '[]', - | lease_preferences = '[]' - TABLE auth.public.token | ALTER TABLE auth.public.token CONFIGURE ZONE USING - | num_replicas = 5, - | constraints = '{+region=us-central: 2, +region=us-east: 1, +region=us-west: 2}', - | lease_preferences = '[[+region=us-west], [+region=us-central]]' - INDEX auth.public.token@token_id_east_idx | ALTER INDEX auth.public.token@token_id_east_idx CONFIGURE ZONE USING - | num_replicas = 5, - | constraints = '{+region=us-central: 2, +region=us-east: 2, +region=us-west: 1}', - | lease_preferences = '[[+region=us-east], [+region=us-central]]' - INDEX auth.public.token@token_id_central_idx | ALTER INDEX auth.public.token@token_id_central_idx CONFIGURE ZONE USING - | num_replicas = 5, - | constraints = '{+region=us-central: 2, +region=us-east: 2, +region=us-west: 1}', - | lease_preferences = '[[+region=us-central], [+region=us-east]]' - INDEX auth.public.token@token_id_west_idx | ALTER INDEX auth.public.token@token_id_west_idx CONFIGURE ZONE USING - | num_replicas = 5, - | constraints = '{+region=us-central: 2, +region=us-east: 1, +region=us-west: 2}', - | lease_preferences = '[[+region=us-west], [+region=us-central]]' -(11 rows) -~~~ - -Now that we've set up our indexes the way we want them, we need to insert some data. The first statement below inserts 10,000 rows of placeholder data; the second inserts a row with a specific UUID string that we'll later query against to check which index is used. - -{{site.data.alerts.callout_info}} -On a freshly created cluster like this one, you may need to wait a moment after adding the data to give [automatic statistics](#table-statistics) time to update. Then, the optimizer can generate a query plan that uses the expected index. -{{site.data.alerts.end}} - -{% include copy-clipboard.html %} -~~~ sql -> INSERT - INTO - token (token_id, access_token, refresh_token) - SELECT - gen_random_uuid()::STRING, - gen_random_uuid()::STRING, - gen_random_uuid()::STRING - FROM - generate_series(1, 10000); -~~~ - -{% include copy-clipboard.html %} -~~~ sql -> INSERT - INTO - token (token_id, access_token, refresh_token) - VALUES - ( - '2E1B5BFE-6152-11E9-B9FD-A7E0F13211D9', - '49E36152-6152-11E9-8CDC-3682F23211D9', - '4E0E91B6-6152-11E9-BAC1-3782F23211D9' - ); -~~~ - -Finally, we [`EXPLAIN`](explain.html) a [selection query](selection-queries.html) from each node to verify which index is being queried against. For example, when running the query shown below against the `us-west` node, we expect it to use the `token_id_west_idx` index. - -{% include copy-clipboard.html %} -~~~ sh -$ cockroach sql --insecure --host=localhost --port=26259 --database=auth # "West" node -~~~ - -{% include copy-clipboard.html %} -~~~ sql -> EXPLAIN - SELECT - access_token, refresh_token - FROM - token - WHERE - token_id = '2E1B5BFE-6152-11E9-B9FD-A7E0F13211D9'; -~~~ - -~~~ - info ------------------------------------------------------------------------------------------------- - distribution: local - vectorized: true - - • scan - estimated row count: 1 (100% of the table; stats collected 34 seconds ago) - table: token@token_id_west_idx - spans: [/'2E1B5BFE-6152-11E9-B9FD-A7E0F13211D9' - /'2E1B5BFE-6152-11E9-B9FD-A7E0F13211D9'] -(7 rows) - -Time: 1ms total (execution 1ms / network 0ms) -~~~ - -Similarly, queries from the `us-east` node should use the `token_id_east_idx` index (and the same should be true for `us-central`). - -{% include copy-clipboard.html %} -~~~ sh -$ cockroach sql --insecure --host=localhost --port=26257 --database=auth # "East" node -~~~ - -{% include copy-clipboard.html %} -~~~ sql -> EXPLAIN - SELECT - access_token, refresh_token - FROM - token - WHERE - token_id = '2E1B5BFE-6152-11E9-B9FD-A7E0F13211D9'; -~~~ - -~~~ - info ------------------------------------------------------------------------------------------------- - distribution: local - vectorized: true - - • scan - estimated row count: 1 (0.01% of the table; stats collected 18 seconds ago) - table: token@token_id_west_idx - spans: [/'2E1B5BFE-6152-11E9-B9FD-A7E0F13211D9' - /'2E1B5BFE-6152-11E9-B9FD-A7E0F13211D9'] -(7 rows) - -Time: 1ms total (execution 1ms / network 0ms) -~~~ - -You'll need to make changes to the above configuration to reflect your [production environment](recommended-production-settings.html), but the concepts will be the same. - ## See also - [`JOIN` expressions](joins.html) diff --git a/v21.1/create-table-as.md b/v21.1/create-table-as.md index 5082fdfa634..a69554a3ebf 100644 --- a/v21.1/create-table-as.md +++ b/v21.1/create-table-as.md @@ -92,7 +92,7 @@ table td:first-child { The default rules for [column families](column-families.html) apply. -The [primary key](primary-key.html) of tables created with `CREATE TABLE ... AS` is not automatically derived from the query results. You must specify new primary keys at table creation. For examples, see [Specify a primary key](create-table-as.html#specify-a-primary-key) and [Specify a primary key for partitioning](create-table-as.html#specify-a-primary-key-for-partitioning). +The [primary key](primary-key.html) of tables created with `CREATE TABLE ... AS` is not automatically derived from the query results. You must specify new primary keys at table creation. For examples, see [Specify a primary key](create-table-as.html#specify-a-primary-key). ## Examples @@ -296,57 +296,6 @@ You can define the [column families](column-families.html) of a new table create (1 row) ~~~ -### Specify a primary key for partitioning - -If you are [partitioning](partitioning.html) a table based on a [primary key](primary-key.html), the primary key must be properly defined. To change the primary key after table creation, you can use an [`ALTER TABLE ... ALTER PRIMARY KEY`](alter-primary-key.html) statement. - -Suppose that you want to [geo-partition](demo-low-latency-multi-region-deployment.html) the `drivers` table that you created with the following statement: - -{% include copy-clipboard.html %} -~~~ sql -> CREATE TABLE drivers (id, city, name) AS VALUES (gen_random_uuid(), 'new york', 'Harry Potter'), (gen_random_uuid(), 'seattle', 'Evelyn Martin'); -~~~ - -{% include copy-clipboard.html %} -~~~ sql -> SHOW CREATE TABLE drivers; -~~~ -~~~ - table_name | create_statement -+------------+----------------------------------------------+ - drivers | CREATE TABLE drivers ( - | id UUID NULL, - | city STRING NULL, - | name STRING NULL, - | FAMILY "primary" (id, city, name, rowid) - | ) -(1 row) -~~~ - -In order for this table to be properly geo-partitioned with the other tables in the `movr` dataset, the table must have a composite primary key defined that includes the unique row identifier (`id`, in this case) and the row locality identifier (`city`). Use the following statement to change the primary key to a composite primary key: - -{% include copy-clipboard.html %} -~~~ sql -> CREATE TABLE drivers_pk (id, city, name, PRIMARY KEY (id, city)) AS SELECT id, city, name FROM drivers; -~~~ - -{% include copy-clipboard.html %} -~~~ sql -> SHOW CREATE TABLE drivers_pk; -~~~ -~~~ - table_name | create_statement -+------------+----------------------------------------------------------+ - drivers_pk | CREATE TABLE drivers_pk ( - | id UUID NOT NULL, - | city STRING NOT NULL, - | name STRING NULL, - | CONSTRAINT "primary" PRIMARY KEY (id ASC, city ASC), - | FAMILY "primary" (id, city, name) - | ) -(1 row) -~~~ - ## See also - [Selection Queries](selection-queries.html) diff --git a/v21.1/disaster-recovery.md b/v21.1/disaster-recovery.md index e2803f7927b..8abb2117aae 100644 --- a/v21.1/disaster-recovery.md +++ b/v21.1/disaster-recovery.md @@ -160,6 +160,10 @@ When using Kubernetes, recovery actions happen automatically in many cases and n ### Multi-region survivability planning +{{site.data.alerts.callout_success}} +New in v21.1: By default, every [multi-region database](multiregion-overview.html) has a [zone-level survival goal](multiregion-overview.html#survival-goals) associated with it. The survival goal setting provides an abstraction that handles the low-level details of replica placement to ensure your desired fault tolerance. The information below is still useful for legacy deployments. +{{site.data.alerts.end}} + The table below shows the replication factor (RF) needed to achieve the listed fault tolerance (e.g., survive 1 failed node) for a multi-region, cloud-deployed cluster with 3 availability zones (AZ) per region and one node in each AZ: {{site.data.alerts.callout_danger}} diff --git a/v21.1/follower-reads.md b/v21.1/follower-reads.md index b861a98001d..4ccaf259927 100644 --- a/v21.1/follower-reads.md +++ b/v21.1/follower-reads.md @@ -6,7 +6,7 @@ toc: true Follower reads are a mechanism that CockroachDB uses to provide faster reads in situations where you can afford to read data that may be slightly less than current (using [`AS OF SYSTEM TIME`](as-of-system-time.html)). Normally, reads have to be serviced by a replica's [leaseholder](architecture/overview.html#architecture-leaseholder). This can be slow, since the leaseholder may be geographically distant from the gateway node that is issuing the query. -A follower read is a read taken from the closest [replica](architecture/overview.html#architecture-replica), regardless of the replica's leaseholder status. This can result in much better latency in [geo-distributed, multi-region deployments](topology-patterns.html#multi-region-patterns). +A follower read is a read taken from the closest [replica](architecture/overview.html#architecture-replica), regardless of the replica's leaseholder status. This can result in much better latency in [geo-distributed, multi-region deployments](topology-patterns.html#multi-region). The shortest interval at which [`AS OF SYSTEM TIME`](as-of-system-time.html) can serve follower reads is 4.8 seconds. In prior versions of CockroachDB, the interval was 48 seconds. diff --git a/v21.1/get-started-with-enterprise-trial.md b/v21.1/get-started-with-enterprise-trial.md index 97bb89ac980..620869f00d1 100644 --- a/v21.1/get-started-with-enterprise-trial.md +++ b/v21.1/get-started-with-enterprise-trial.md @@ -5,7 +5,7 @@ toc: true license: true --- -Congratulations on starting your CockroachDB Enterprise Trial! With it, you'll not only get access to CockroachDB's core capabilities like [high availability](frequently-asked-questions.html#how-does-cockroachdb-survive-failures) and [`SERIALIZABLE` isolation](frequently-asked-questions.html#how-is-cockroachdb-strongly-consistent), but also our Enterprise-only features like distributed [`BACKUP`](backup.html) & [`RESTORE`](restore.html), [geo-partitioning](partitioning.html), and [cluster visualization](enable-node-map.html). +Congratulations on starting your CockroachDB Enterprise Trial! With it, you'll not only get access to CockroachDB's core capabilities like [high availability](frequently-asked-questions.html#how-does-cockroachdb-survive-failures) and [`SERIALIZABLE` isolation](frequently-asked-questions.html#how-is-cockroachdb-strongly-consistent), but also our Enterprise-only features like distributed [`BACKUP`](backup.html) & [`RESTORE`](restore.html), [multi-region capabilities](multiregion-overview.html), and [cluster visualization](enable-node-map.html). ## Install CockroachDB diff --git a/v21.1/global-tables.md b/v21.1/global-tables.md new file mode 100644 index 00000000000..d8693398f61 --- /dev/null +++ b/v21.1/global-tables.md @@ -0,0 +1,97 @@ +--- +title: Global Tables +summary: Guidance on using Global Tables in a multi-region deployment. +toc: true +redirect_from: topology-duplicate-indexes.html +--- + +In a [multi-region deployment](multiregion-overview.html), the [`GLOBAL` table locality](multiregion-overview.html#global-tables) is a good choice for tables with the following requirements: + +- Read latency must be low, but write latency can be much higher. +- Reads must be up-to-date for business reasons or because the table is referenced by [foreign keys](foreign-key.html). +- Rows in the table, and all latency-sensitive reads, **cannot** be tied to specific regions. + +In general, this pattern is suited well for reference tables that are rarely updated. + +{{site.data.alerts.callout_info}} +Tables with the `GLOBAL` locality can survive zone or region failures, depending on the database-level [survival goal](multiregion-overview.html#survival-goals) setting. +{{site.data.alerts.end}} + +{{site.data.alerts.callout_info}} +`GLOBAL` tables (and the other [multi-region capabilities](multiregion-overview.html)) require an [Enterprise license](https://www.cockroachlabs.com/get-cockroachdb). +{{site.data.alerts.end}} + +## Prerequisites + +### Fundamentals + +{% include {{ page.version.version }}/topology-patterns/multiregion-fundamentals.md %} + +### Cluster setup + +{% include {{ page.version.version }}/topology-patterns/multi-region-cluster-setup.md %} + +## Configuration + +### Summary + +To use this pattern, you tell CockroachDB to set the [table locality](multiregion-overview.html#table-locality) to `GLOBAL`. + +{% include {{page.version.version}}/sql/global-table-description.md %} + +### Steps + +{% include {{page.version.version}}/topology-patterns/multiregion-db-setup.md %} + +Next, create a [`GLOBAL` table](multiregion-overview.html#global-tables) by issuing the following statement: + +{% include copy-clipboard.html %} +~~~ sql +CREATE TABLE postal_codes ( + id INT PRIMARY KEY, + code STRING +) LOCALITY GLOBAL; +~~~ + +Alternatively, you can set an existing table's locality to `GLOBAL` using [`ALTER TABLE ... SET LOCALITY`](set-locality.html): + +{% include copy-clipboard.html %} +~~~ sql +> ALTER TABLE postal_codes SET LOCALITY GLOBAL; +~~~ + +{{site.data.alerts.callout_success}} +A good way to check that your [table locality settings](multiregion-overview.html#table-locality) are having the expected effect is by monitoring how the performance metrics of a workload change as the settings are applied to a running cluster. For a tutorial showing how table localities can improve performance metrics across a multi-region cluster, see [Low Latency Reads and Writes in a Multi-Region Cluster](demo-low-latency-multi-region-deployment.html). +{{site.data.alerts.end}} + +## Characteristics + +### Latency + +Global tables support low-latency, global reads of read-mostly data using an extension to CockroachDB's standard transaction protocol called [non-blocking transactions](architecture/transaction-layer.html#non-blocking-transactions). + +#### Reads + +Thanks to the [non-blocking transaction](architecture/transaction-layer.html#non-blocking-transactions) protocol extension, reads against `GLOBAL` tables access a consistent local replica and therefore never leave the region. This keeps read latency low. + +#### Writes + +Writes incur higher latencies than reads, since they require a "commit-wait" step to ensure consistency. For more information about how this works, see [non-blocking transactions](architecture/transaction-layer.html#non-blocking-transactions). + +### Resiliency + +Because the `test` database does not specify a [survival goal](multiregion-overview.html#survival-goals), it uses the default [`ZONE` survival goal](multiregion-overview.html#surviving-zone-failures). With the default settings, an entire zone can fail without interrupting access to the database. + +For more information about how to choose a database survival goal, see [When to use `ZONE` vs. `REGION` survival goals](when-to-use-zone-vs-region-survival-goals.html). + +## Alternatives + +- If rows in the table, and all latency-sensitive queries, can be tied to specific geographies, consider the [`REGIONAL` Table Locality Pattern](regional-tables.html) pattern. + +## Tutorial + +For a step-by-step demonstration showing how CockroachDB's multi-region capabilities (including `GLOBAL` and `REGIONAL` tables) give you low-latency reads in a distributed cluster, see the tutorial on [Low Latency Reads and Writes in a Multi-Region Cluster](demo-low-latency-multi-region-deployment.html). + +## See also + +{% include {{ page.version.version }}/topology-patterns/see-also.md %} diff --git a/v21.1/licensing-faqs.md b/v21.1/licensing-faqs.md index 9bad9eace9d..07277ffbd9f 100644 --- a/v21.1/licensing-faqs.md +++ b/v21.1/licensing-faqs.md @@ -57,10 +57,9 @@ Feature | BSL | CCL (free) | CCL (paid) **[Core changefeed](stream-data-out-of-cockroachdb-using-changefeeds.html#create-a-core-changefeed)** | | ✓ | **[Enterprise changefeed](stream-data-out-of-cockroachdb-using-changefeeds.html#configure-a-changefeed-enterprise)** | | | ✓ **[Table-level zone configuration](configure-replication-zones.html#replication-zone-levels)** | ✓ | | -**[Geo-partitioning](topology-geo-partitioned-replicas.html)** | | | ✓ +**[Multi-Region Capabilities](multiregion-overview.html)** | | | ✓ **[Follower reads](follower-reads.html)** | | | ✓ **[Node map](enable-node-map.html)** | | | ✓ -**[Locality-aware index selection](cost-based-optimizer.html#preferring-the-nearest-index)** | | | ✓ **[Encryption at rest](encryption.html#encryption-at-rest-enterprise)** | | | ✓ **[Role-based access management](authorization.html#roles)** | ✓ | | **[Password and certificate authentication](authentication.html)** | ✓ | | diff --git a/v21.1/multi-region-use-case.md b/v21.1/multi-region-use-case.md index c13affd2585..e98563dab6e 100644 --- a/v21.1/multi-region-use-case.md +++ b/v21.1/multi-region-use-case.md @@ -34,7 +34,7 @@ Limiting latency improves the user experience, and it can also help avoid proble To reduce database latency in a distributed CockroachDB deployment, data can be [geo-partitioned](topology-geo-partitioned-replicas.html). Geo-partitioning enables you to control where specific rows of data are stored. Limiting database operations to specific partitions can reduce the distance requests need to travel between the client and the database. {{site.data.alerts.callout_info}} -Geo-partitioned replicas can dramatically improve latency in multi-region deployments, but at the [cost of resiliency](topology-geo-partitioned-replicas.html#resiliency). Geo-partitioned replicas are resilient to availability zone failures, but not regional failures. +Geo-partitioned replicas can dramatically improve latency in multi-region deployments, but at the [cost of resiliency](topology-geo-partitioned-replicas.html). Geo-partitioned replicas are resilient to availability zone failures, but not regional failures. {{site.data.alerts.end}} If you are building an application, it's likely that the end user will not be making requests to the database directly. Instead, the user makes requests to the application, and the application makes requests to the database on behalf of the user. To limit the latency between the application and the database, you need to design and deploy your application such that: diff --git a/v21.1/multiregion-overview.md b/v21.1/multiregion-overview.md index cdbe0ebc0ac..bf1b0c1b8e8 100644 --- a/v21.1/multiregion-overview.md +++ b/v21.1/multiregion-overview.md @@ -1,12 +1,12 @@ --- -title: Multi-region Overview -summary: Learn how to use CockroachDB's improved multi-region user experience. +title: Multi-region Capabilities Overview +summary: Learn how to use CockroachDB's improved multi-region capabilities. toc: true --- ## Overview -New in v21.1: CockroachDB has improved multi-region capabilities that make it easier to run global applications. It is intended that these capabilities will supersede the current set of [Multi-region Topology Patterns](topology-patterns.html#multi-region-patterns). The capabilities described here are still in development; they are available in [testing releases of v21.1](../releases/#testing-releases). +New in v21.1: CockroachDB has improved multi-region capabilities that make it easier to run global applications. It is intended that these capabilities will supersede the current set of [Multi-region Topology Patterns](topology-patterns.html#multi-region). The capabilities described here are still in development; they are available in [testing releases of v21.1](../releases/#testing-releases). To take advantage of these improved capabilities, you will need to understand the following concepts: @@ -134,18 +134,12 @@ Table locality settings are used for optimizing latency under different read/wri ### Regional tables -Regional tables work well when your application requires low-latency reads and writes for an entire table from a single region. - -For _regional_ tables, access to the table will be fast in the table's "home region" and slower in other regions. In other words, CockroachDB optimizes access to data in regional tables from a single region. By default, a regional table's home region is the [database's primary region](#database-regions), but that can be changed to use any region added to the database. - -For instructions showing how to set a table's locality to `REGIONAL BY TABLE`, see [`ALTER TABLE ... SET LOCALITY`](set-locality.html#regional-by-table) - -{{site.data.alerts.callout_info}} -By default, all tables in a multi-region database are _regional_ tables that use the database's primary region. Unless you know your application needs different performance characteristics than regional tables provide, there is no need to change this setting. -{{site.data.alerts.end}} +{% include {{page.version.version}}/sql/regional-table-description.md %} ### Regional by row tables +{% include {{page.version.version}}/sql/regional-by-row-table-description.md %} + In _regional by row_ tables, individual rows are optimized for access from different regions. This setting divides a table and all of [its indexes](#indexes-on-regional-by-row-tables) into [partitions](partitioning.html), with each partition optimized for access from a different region. Like [regional tables](#regional-tables), _regional by row_ tables are optimized for access from a single region. However, that region is specified at the row level instead of applying to the whole table. Use regional by row tables when your application requires low-latency reads and writes at a row level where individual rows are primarily accessed from a single region. For example, a users table in a global application may need to keep some users' data in specific regions due to regulations (such as GDPR), for better performance, or both. @@ -156,13 +150,7 @@ For instructions showing how to set a table's locality to `REGIONAL BY ROW`, see ### Global tables - _Global_ tables are optimized for low-latency reads from every region in the database. The tradeoff is that writes will incur higher latencies from any given region, since writes have to be replicated across every region to make the global low-latency reads possible. - -Use global tables when your application has a "read-mostly" table of reference data that is rarely updated, and needs to be available to all regions. - -For an example of a table that can benefit from the _global_ table locality setting in a multi-region deployment, see the `promo_codes` table from the [MovR application](movr.html). - -For instructions showing how to set a table's locality to `GLOBAL`, see [`ALTER TABLE ... SET LOCALITY`](set-locality.html#global) +{% include {{page.version.version}}/sql/global-table-description.md %} ## Additional Features diff --git a/v21.1/partition-by.md b/v21.1/partition-by.md index cddbff7c87d..bb8b93afcc4 100644 --- a/v21.1/partition-by.md +++ b/v21.1/partition-by.md @@ -6,6 +6,8 @@ toc: true `PARTITION BY` is a subcommand of [`ALTER TABLE`](alter-table.html) and [`ALTER INDEX`](alter-index.html) that is used to partition, re-partition, or un-partition a table or secondary index. After defining partitions, [`CONFIGURE ZONE`](configure-zone.html) is used to control the replication and placement of partitions. +{% include {{page.version.version}}/sql/use-multiregion-instead-of-partitioning.md %} + {{site.data.alerts.callout_info}} [Partitioning](partitioning.html) is an [enterprise-only](enterprise-licensing.html) feature. If you are looking for the `PARTITION BY` used in SQL window functions, see [Window Functions](window-functions.html). {{site.data.alerts.end}} diff --git a/v21.1/partitioning.md b/v21.1/partitioning.md index b8a9d4d9937..d067ff759b0 100644 --- a/v21.1/partitioning.md +++ b/v21.1/partitioning.md @@ -12,6 +12,8 @@ Table partitioning is an [enterprise-only](enterprise-licensing.html) feature. F ## Why use table partitioning +{% include {{page.version.version}}/sql/use-multiregion-instead-of-partitioning.md %} + Table partitioning helps you reduce latency and cost: - **Geo-partitioning** allows you to keep user data close to the user, which reduces the distance that the data needs to travel, thereby **reducing latency**. To geo-partition a table, define location-based partitions while creating a table, create location-specific zone configurations, and apply the zone configurations to the corresponding partitions. diff --git a/v21.1/query-replication-reports.md b/v21.1/query-replication-reports.md index f43e2ca9e9b..a1ae1b2376a 100644 --- a/v21.1/query-replication-reports.md +++ b/v21.1/query-replication-reports.md @@ -417,7 +417,9 @@ The `system.replication_critical_localities` report contains which of your local That said, a locality being critical is not necessarily a bad thing as long as you are aware of it. What matters is that [you configure the topology of your cluster to get the resiliency you expect](topology-patterns.html). -By default, the [movr demo cluster](cockroach-demo.html#start-a-multi-region-demo-cluster-with-automatic-geo-partitioning) has some ranges in critical localities. This is expected because it follows the [geo-partitioned replicas topology pattern](topology-geo-partitioned-replicas.html), which ties data for latency-sensitive queries to specific geographies at the cost of data unavailability during a region-wide failure. +By default, the [movr demo cluster](cockroach-demo.html#start-a-multi-region-demo-cluster-with-automatic-geo-partitioning) has some ranges in critical localities. This is expected because it ties data for latency-sensitive queries to specific geographies at the cost of data unavailability during a region-wide failure. + +{% include {{page.version.version}}/sql/use-multiregion-instead-of-partitioning.md %} {% include copy-clipboard.html %} ~~~ sql diff --git a/v21.1/regional-tables.md b/v21.1/regional-tables.md new file mode 100644 index 00000000000..d71fb9d06a0 --- /dev/null +++ b/v21.1/regional-tables.md @@ -0,0 +1,145 @@ +--- +title: Regional Tables +summary: Guidance on using Regional Tables in a multi-region deployment. +toc: true +redirect_from: +- topology-geo-partitioned-replicas.html +- topology-geo-partitioned-leaseholders.html +--- + +In a [multi-region deployment](multiregion-overview.html), the [Regional Table Locality Pattern](multiregion-overview.html#table-locality) is a good choice for tables with the following requirements: + +- Read and write latency must be low. +- Rows in the table, and all latency-sensitive queries, can be tied to specific regions. + +{{site.data.alerts.callout_info}} +Tables with the Regional Table Locality Pattern can survive zone or region failures, depending on the database-level [survival goal](multiregion-overview.html#survival-goals) setting. +{{site.data.alerts.end}} + +{{site.data.alerts.callout_info}} +Regional tables (and the other [multi-region capabilities](multiregion-overview.html)) require an [Enterprise license](https://www.cockroachlabs.com/get-cockroachdb). +{{site.data.alerts.end}} + +## Prerequisites + +### Fundamentals + +{% include {{ page.version.version }}/topology-patterns/multiregion-fundamentals.md %} + +### Cluster setup + +{% include {{ page.version.version }}/topology-patterns/multi-region-cluster-setup.md %} + +## Configuration + +### Summary + +To use this pattern, you tell CockroachDB to set the [table locality](multiregion-overview.html#table-locality) to either [`REGIONAL BY TABLE`](#regional-tables) or [`REGIONAL BY ROW`](#regional-by-row-tables). + +#### Regional tables + +{% include {{page.version.version}}/sql/regional-table-description.md %} + +#### Regional by row tables + +{% include {{page.version.version}}/sql/regional-by-row-table-description.md %} + +### Steps + +{{site.data.alerts.callout_info}} +By default, all tables in a multi-region database are [Regional tables](#regional-tables). Therefore, the steps below show how to set up [Regional by row tables](#regional-by-row-tables). +{{site.data.alerts.end}} + +{% include {{page.version.version}}/topology-patterns/multiregion-db-setup.md %} + +Next, create a `users` table: + +{% include copy-clipboard.html %} +~~~ sql +CREATE TABLE users ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + city STRING NOT NULL, + first_name STRING NOT NULL, + last_name STRING NOT NULL, + address STRING NOT NULL +); +~~~ + +By default, all tables in a multi-region cluster default to the [`REGIONAL BY TABLE`](#regional-tables) locality setting. To verify this, issue a [`SHOW CREATE`](show-create.html) on the `users` table you just created: + +{% include copy-clipboard.html %} +~~~ sql +SHOW CREATE TABLE users; +~~~ + +~~~ + table_name | create_statement +-------------+------------------------------------------------------------------ + users | CREATE TABLE public.users ( + | id UUID NOT NULL DEFAULT gen_random_uuid(), + | city STRING NOT NULL, + | first_name STRING NOT NULL, + | last_name STRING NOT NULL, + | address STRING NOT NULL, + | CONSTRAINT "primary" PRIMARY KEY (id ASC), + | FAMILY "primary" (id, city, first_name, last_name, address) + | ) LOCALITY REGIONAL BY TABLE IN PRIMARY REGION +~~~ + +Next, set the table's locality to [`REGIONAL BY ROW`](#regional-by-row-tables) using the [`ALTER TABLE ... SET LOCALITY`](set-locality.html) statement: + +{% include copy-clipboard.html %} +~~~ sql +ALTER TABLE users SET LOCALITY REGIONAL BY ROW; +~~~ + +~~~ +NOTICE: LOCALITY changes will be finalized asynchronously; further schema changes on this table may be restricted until the job completes +ALTER TABLE SET LOCALITY +~~~ + +Now that the table is regional by row, we need to tell CockroachDB which rows need to be optimized for access from which regions. We do this by issuing [`UPDATE`](update.html) statements that modify the automatically created [`crdb_region`](set-locality.html#crdb_region) column. + +Issue the statements below to associate each row with a home region that depends on its `city` column: + +{% include copy-clipboard.html %} +~~~ sql +UPDATE users SET crdb_region = 'us-central' WHERE city IN ('chicago', 'milwaukee', 'dallas'); +UPDATE users SET crdb_region = 'us-east' WHERE city IN ('washington dc', 'boston', 'new york'); +UPDATE users SET crdb_region = 'us-west' WHERE city IN ('los angeles', 'san francisco', 'seattle'); +~~~ + +By default, the region column will get auto-assigned on insert; this is also known as "auto-homing". For more information about how the `crdb_region` column works, see [`ALTER TABLE ... SET LOCALITY REGIONAL BY ROW`](set-locality.html#regional-by-row). + +{% include {{page.version.version}}/sql/locality-optimized-search.md %} + +{{site.data.alerts.callout_success}} +A good way to check that your [table locality settings](multiregion-overview.html#table-locality) are having the expected effect is by monitoring how the performance metrics of a workload change as the settings are applied to a running cluster. For a tutorial showing how table localities can improve performance metrics across a multi-region cluster, see [Low Latency Reads and Writes in a Multi-Region Cluster](demo-low-latency-multi-region-deployment.html). +{{site.data.alerts.end}} + +## Characteristics + +### Latency + +For [`REGIONAL BY TABLE` tables](#regional-tables), you get low latency for single-region writes and reads, as well as multi-region stale reads. + +For [`REGIONAL BY ROW` tables](#regional-by-row-tables), you get low-latency consistent multi-region reads & writes for rows which are homed in specific regions, and low-latency multi-region stale reads from all other regions. + +### Resiliency + +Because the `test` database does not specify a [survival goal](multiregion-overview.html#survival-goals), it uses the default [`ZONE` survival goal](multiregion-overview.html#surviving-zone-failures). With the default settings, an entire availability zone (AZ) can fail without interrupting access to the database. + +For more information about how to choose a database survival goal, see [When to use `ZONE` vs. `REGION` survival goals](when-to-use-zone-vs-region-survival-goals.html). + +## Alternatives + +- If rows in the table **cannot** be tied to specific geographies, reads must be up-to-date for business reasons or because the table is referenced by [foreign keys](foreign-key.html), and the table is rarely modified, consider the [`GLOBAL` Table Locality Pattern](global-tables.html). +- If your application can tolerate historical reads in some cases, consider the [Follower Reads](topology-follower-reads.html) pattern. + +## Tutorial + +For a step-by-step demonstration showing how CockroachDB's multi-region capabilities (including [`REGIONAL BY ROW` tables](#regional-by-row-tables)) give you low-latency reads in a distributed cluster, see the tutorial on [Low Latency Reads and Writes in a Multi-Region Cluster](demo-low-latency-multi-region-deployment.html). + +## See also + +{% include {{ page.version.version }}/topology-patterns/see-also.md %} diff --git a/v21.1/secure-a-cluster.md b/v21.1/secure-a-cluster.md index 99bbf28abdc..0e467d80c86 100644 --- a/v21.1/secure-a-cluster.md +++ b/v21.1/secure-a-cluster.md @@ -446,7 +446,7 @@ Adding capacity is as simple as starting more nodes with `cockroach start`. - [Install the client driver](install-client-drivers.html) for your preferred language - Learn more about [CockroachDB SQL](learn-cockroachdb-sql.html) and the [built-in SQL client](cockroach-sql.html) -- Further explore CockroachDB capabilities like [fault tolerance and automated repair](demo-fault-tolerance-and-recovery.html), [geo-partitioning](demo-low-latency-multi-region-deployment.html), [serializable transactions](demo-serializable.html), and [JSON support](demo-json-support.html) +- Further explore CockroachDB capabilities like [fault tolerance and automated repair](demo-fault-tolerance-and-recovery.html), [multi-region performance](demo-low-latency-multi-region-deployment.html), [serializable transactions](demo-serializable.html), and [JSON support](demo-json-support.html) You might also be interested in the following pages: diff --git a/v21.1/set-locality.md b/v21.1/set-locality.md index b8229db299a..6366a241331 100644 --- a/v21.1/set-locality.md +++ b/v21.1/set-locality.md @@ -135,6 +135,8 @@ ALTER TABLE rides ADD COLUMN region crdb_internal_region AS ( ) STORED ~~~ +{% include {{page.version.version}}/sql/locality-optimized-search.md %} + ### Set the table locality to `GLOBAL` diff --git a/v21.1/show-locality.md b/v21.1/show-locality.md index cdf635c7b87..a3643c3f273 100644 --- a/v21.1/show-locality.md +++ b/v21.1/show-locality.md @@ -78,7 +78,7 @@ For a more extensive example, see [Create a table with node locality information ## See also -- [Geo-Partitioning](demo-low-latency-multi-region-deployment.html) +- [Multi-Region Performance](demo-low-latency-multi-region-deployment.html) - [Locality](cockroach-start.html#locality) - [Orchestrated Deployment](orchestration.html) - [Manual Deployment](manual-deployment.html) diff --git a/v21.1/show-partitions.md b/v21.1/show-partitions.md index f519a2bef88..180cbb4e01f 100644 --- a/v21.1/show-partitions.md +++ b/v21.1/show-partitions.md @@ -6,6 +6,8 @@ toc: true Use the `SHOW PARTITIONS` [statement](sql-statements.html) to view details about existing [partitions](partitioning.html). +{% include {{page.version.version}}/sql/use-multiregion-instead-of-partitioning.md %} + {% include enterprise-feature.md %} {% include {{page.version.version}}/sql/crdb-internal-partitions.md %} @@ -234,4 +236,4 @@ If a partitioned table has no zones configured, the `SHOW CREATE TABLE` output i - [Define Table Partitions](partitioning.html) - [SQL Statements](sql-statements.html) -- [Geo-Partitioning](demo-low-latency-multi-region-deployment.html) +- [Multi-Region Performance](demo-low-latency-multi-region-deployment.html) diff --git a/v21.1/show-zone-configurations.md b/v21.1/show-zone-configurations.md index 9d2d8f4c4f7..4c68e6bae6f 100644 --- a/v21.1/show-zone-configurations.md +++ b/v21.1/show-zone-configurations.md @@ -81,6 +81,8 @@ CONFIGURE ZONE 1 ### View the replication zone for a partition +{% include {{page.version.version}}/sql/use-multiregion-instead-of-partitioning.md %} + {% include {{ page.version.version }}/zone-configs/create-a-replication-zone-for-a-table-partition.md %} ## See also diff --git a/v21.1/start-a-local-cluster-in-docker-linux.md b/v21.1/start-a-local-cluster-in-docker-linux.md index 0b30fd6dbda..289b8ffb17f 100644 --- a/v21.1/start-a-local-cluster-in-docker-linux.md +++ b/v21.1/start-a-local-cluster-in-docker-linux.md @@ -36,4 +36,4 @@ Also, feel free to watch this process in action before going through the steps y - Learn more about [CockroachDB SQL](learn-cockroachdb-sql.html) and the [built-in SQL client](cockroach-sql.html) - [Install the client driver](install-client-drivers.html) for your preferred language - [Build an app with CockroachDB](build-an-app-with-cockroachdb.html) -- Further explore CockroachDB capabilities like [fault tolerance and automated repair](demo-fault-tolerance-and-recovery.html), [geo-partitioning](demo-low-latency-multi-region-deployment.html), [serializable transactions](demo-serializable.html), and [JSON support](demo-json-support.html) +- Further explore CockroachDB capabilities like [fault tolerance and automated repair](demo-fault-tolerance-and-recovery.html), [multi-region performance](demo-low-latency-multi-region-deployment.html), [serializable transactions](demo-serializable.html), and [JSON support](demo-json-support.html) diff --git a/v21.1/start-a-local-cluster-in-docker-mac.md b/v21.1/start-a-local-cluster-in-docker-mac.md index b8afd3fce7e..333a5ed2ab6 100644 --- a/v21.1/start-a-local-cluster-in-docker-mac.md +++ b/v21.1/start-a-local-cluster-in-docker-mac.md @@ -35,4 +35,4 @@ Also, feel free to watch this process in action before going through the steps y - Learn more about [CockroachDB SQL](learn-cockroachdb-sql.html) and the [built-in SQL client](cockroach-sql.html) - [Install the client driver](install-client-drivers.html) for your preferred language - [Build an app with CockroachDB](build-an-app-with-cockroachdb.html) -- Further explore CockroachDB capabilities like [fault tolerance and automated repair](demo-fault-tolerance-and-recovery.html), [geo-partitioning](demo-low-latency-multi-region-deployment.html), [serializable transactions](demo-serializable.html), and [JSON support](demo-json-support.html) +- Further explore CockroachDB capabilities like [fault tolerance and automated repair](demo-fault-tolerance-and-recovery.html), [multi-region performance](demo-low-latency-multi-region-deployment.html), [serializable transactions](demo-serializable.html), and [JSON support](demo-json-support.html) diff --git a/v21.1/start-a-local-cluster-in-docker-windows.md b/v21.1/start-a-local-cluster-in-docker-windows.md index 4ad2b8b1395..a260aa19910 100644 --- a/v21.1/start-a-local-cluster-in-docker-windows.md +++ b/v21.1/start-a-local-cluster-in-docker-windows.md @@ -236,4 +236,4 @@ PS C:\Users\username> Remove-Item cockroach-data -recurse - Learn more about [CockroachDB SQL](learn-cockroachdb-sql.html) and the [built-in SQL client](cockroach-sql.html) - [Install the client driver](install-client-drivers.html) for your preferred language - [Build an app with CockroachDB](build-an-app-with-cockroachdb.html) -- Further explore CockroachDB capabilities like [fault tolerance and automated repair](demo-fault-tolerance-and-recovery.html), [geo-partitioning](demo-low-latency-multi-region-deployment.html), [serializable transactions](demo-serializable.html), and [JSON support](demo-json-support.html) +- Further explore CockroachDB capabilities like [fault tolerance and automated repair](demo-fault-tolerance-and-recovery.html), [multi-region performance](demo-low-latency-multi-region-deployment.html), [serializable transactions](demo-serializable.html), and [JSON support](demo-json-support.html) diff --git a/v21.1/start-a-local-cluster.md b/v21.1/start-a-local-cluster.md index 3772e45ad2e..6f0070c5640 100644 --- a/v21.1/start-a-local-cluster.md +++ b/v21.1/start-a-local-cluster.md @@ -376,4 +376,4 @@ Adding capacity is as simple as starting more nodes with `cockroach start`. - [Install the client driver](install-client-drivers.html) for your preferred language - Learn more about [CockroachDB SQL](learn-cockroachdb-sql.html) and the [built-in SQL client](cockroach-sql.html) - [Build an app with CockroachDB](build-an-app-with-cockroachdb.html) -- Further explore CockroachDB capabilities like [fault tolerance and automated repair](demo-fault-tolerance-and-recovery.html), [geo-partitioning](demo-low-latency-multi-region-deployment.html), [serializable transactions](demo-serializable.html), and [JSON support](demo-json-support.html) +- Further explore CockroachDB capabilities like [fault tolerance and automated repair](demo-fault-tolerance-and-recovery.html), [multi-region performance](demo-low-latency-multi-region-deployment.html), [serializable transactions](demo-serializable.html), and [JSON support](demo-json-support.html) diff --git a/v21.1/topology-duplicate-indexes.md b/v21.1/topology-duplicate-indexes.md deleted file mode 100644 index 2b7df847900..00000000000 --- a/v21.1/topology-duplicate-indexes.md +++ /dev/null @@ -1,191 +0,0 @@ ---- -title: Duplicate Indexes Topology -summary: Guidance on using the duplicate indexes topology in a multi-region deployment. -toc: true ---- - -In a multi-region deployment, the duplicate indexes pattern is a good choice for tables with the following requirements: - -- Read latency must be low, but write latency can be much higher. -- Reads must be up-to-date for business reasons or because the table is referenced by [foreign keys](foreign-key.html). -- Rows in the table, and all latency-sensitive queries, **cannot** be tied to specific geographies. -- Table data must remain available during a region failure. - -In general, this pattern is suited well for immutable/reference tables that are rarely or never updated. - - - -{{site.data.alerts.callout_success}} -**See It In Action** - Read about how a [financial software company](https://www.cockroachlabs.com/case-studies/top-u-s-financial-software-company-turns-to-cockroachdb-to-improve-its-application-login-experience/) is using the Duplicate Indexes topology for low latency reads in their identity access management layer. -{{site.data.alerts.end}} - -## Prerequisites - -### Fundamentals - -{% include {{ page.version.version }}/topology-patterns/fundamentals.md %} - -### Cluster setup - -{% include {{ page.version.version }}/topology-patterns/multi-region-cluster-setup.md %} - -## Configuration - -{{site.data.alerts.callout_info}} -Pinning secondary indexes requires an [Enterprise license](https://www.cockroachlabs.com/get-cockroachdb). -{{site.data.alerts.end}} - -### Summary - -Using this pattern, you tell CockroachDB to put the leaseholder for the table itself (also called the primary index) in one region, create 2 secondary indexes on the table, and tell CockroachDB to put the leaseholder for each secondary index in one of the other regions. This means that reads will access the local leaseholder (either for the table itself or for one of the secondary indexes). Writes, however, will still leave the region to get consensus for the table and its secondary indexes. - -
-
-### Steps
-
-Assuming you have a [cluster deployed across three regions](#cluster-setup) and a table like the following:
-
-{% include copy-clipboard.html %}
-~~~ sql
-> CREATE TABLE postal_codes (
- id INT PRIMARY KEY,
- code STRING
-);
-~~~
-
-1. If you do not already have one, [request a trial Enterprise license](https://www.cockroachlabs.com/get-cockroachdb).
-
-2. [Create a replication zone](configure-zone.html) for the table and set a leaseholder preference telling CockroachDB to put the leaseholder for the table in one of the regions, for example `us-west`:
-
- {% include copy-clipboard.html %}
- ~~~ sql
- > ALTER TABLE postal_codes
- CONFIGURE ZONE USING
- num_replicas = 3,
- constraints = '{"+region=us-west":1}',
- lease_preferences = '[[+region=us-west]]';
- ~~~
-
-3. [Create secondary indexes](create-index.html) on the table for each of your other regions, including all of the columns you wish to read either in the key or in the key and a [`STORING`](create-index.html#store-columns) clause:
-
- {% include copy-clipboard.html %}
- ~~~ sql
- > CREATE INDEX idx_central ON postal_codes (id)
- STORING (code);
- ~~~
-
- {% include copy-clipboard.html %}
- ~~~ sql
- > CREATE INDEX idx_east ON postal_codes (id)
- STORING (code);
- ~~~
-
-4. [Create a replication zone](configure-zone.html) for each secondary index, in each case setting a leaseholder preference telling CockroachDB to put the leaseholder for the index in a distinct region:
-
- {% include copy-clipboard.html %}
- ~~~ sql
- > ALTER INDEX postal_codes@idx_central
- CONFIGURE ZONE USING
- num_replicas = 3,
- constraints = '{"+region=us-central":1}',
- lease_preferences = '[[+region=us-central]]';
- ~~~
-
- {% include copy-clipboard.html %}
- ~~~ sql
- > ALTER INDEX postal_codes@idx_east
- CONFIGURE ZONE USING
- num_replicas = 3,
- constraints = '{"+region=us-east":1}',
- lease_preferences = '[[+region=us-east]]';
- ~~~
-
-5. To confirm that replication zones are in effect, you can use the [`SHOW CREATE TABLE`](show-create.html):
-
- {% include copy-clipboard.html %}
- ~~~ sql
- > SHOW CREATE TABLE postal_codes;
- ~~~
-
- ~~~
- table_name | create_statement
- +--------------+----------------------------------------------------------------------------+
- postal_codes | CREATE TABLE postal_codes (
- | id INT8 NOT NULL,
- | code STRING NULL,
- | CONSTRAINT "primary" PRIMARY KEY (id ASC),
- | INDEX idx_central (id ASC) STORING (code),
- | INDEX idx_east (id ASC) STORING (code),
- | FAMILY "primary" (id, code)
- | );
- | ALTER TABLE defaultdb.public.postal_codes CONFIGURE ZONE USING
- | num_replicas = 3,
- | constraints = '{+region=us-west: 1}',
- | lease_preferences = '[[+region=us-west]]';
- | ALTER INDEX defaultdb.public.postal_codes@idx_central CONFIGURE ZONE USING
- | num_replicas = 3,
- | constraints = '{+region=us-central: 1}',
- | lease_preferences = '[[+region=us-central]]';
- | ALTER INDEX defaultdb.public.postal_codes@idx_east CONFIGURE ZONE USING
- | num_replicas = 3,
- | constraints = '{+region=us-east: 1}',
- | lease_preferences = '[[+region=us-east]]'
- (1 row)
- ~~~
-
-## Characteristics
-
-### Latency
-
-#### Reads
-
-Reads access the local leaseholder and, therefore, never leave the region. This makes read latency very low.
-
-For example, in the animation below:
-
-1. The read request in `us-central` reaches the regional load balancer.
-2. The load balancer routes the request to a gateway node.
-3. The gateway node routes the request to the relevant leaseholder. In `us-west`, the leaseholder is for the table itself. In the other regions, the leaseholder is for the relevant index, which the [cost-based optimizer](cost-based-optimizer.html) uses due to the leaseholder preferences.
-4. The leaseholder retrieves the results and returns to the gateway node.
-5. The gateway node returns the results to the client.
-
-
-
-#### Writes
-
-The replicas for the table and its secondary indexes are spread across all 3 regions, so writes involve multiple network hops across regions to achieve consensus. This increases write latency significantly. It's also important to understand that the replication of extra indexes can reduce throughput and increase storage cost.
-
-For example, in the animation below:
-
-1. The write request in `us-central` reaches the regional load balancer.
-2. The load balancer routes the request to a gateway node.
-3. The gateway node routes the request to the leaseholder replicas for the table and its secondary indexes.
-4. While each leaseholder appends the write to its Raft log, it notifies its follower replicas.
-5. In each case, as soon as one follower has appended the write to its Raft log (and thus a majority of replicas agree based on identical Raft logs), it notifies the leaseholder and the write is committed on the agreeing replicas.
-6. The leaseholders then return acknowledgement of the commit to the gateway node.
-7. The gateway node returns the acknowledgement to the client.
-
-
-
-### Resiliency
-
-Because this pattern balances the replicas for the table and its secondary indexes across regions, one entire region can fail without interrupting access to the table:
-
-
-
-
-
-## Alternatives
-
-- If reads from a table can be historical (4.8 seconds or more in the past), consider the [Follower Reads](topology-follower-reads.html) pattern.
-- If rows in the table, and all latency-sensitive queries, can be tied to specific geographies, consider the [Geo-Partitioned Leaseholders](topology-geo-partitioned-leaseholders.html) pattern. Both patterns avoid extra secondary indexes, which increase data replication and, therefore, higher throughput and less storage.
-
-## Tutorial
-
-For a step-by-step demonstration of how this pattern gets you low-latency reads in a broadly distributed cluster, see the [Low Latency Multi-Region Deployment](demo-low-latency-multi-region-deployment.html) tutorial.
-
-## See also
-
-{% include {{ page.version.version }}/topology-patterns/see-also.md %}
diff --git a/v21.1/topology-follow-the-workload.md b/v21.1/topology-follow-the-workload.md
index 193b95ce296..ca8451a9fa5 100644
--- a/v21.1/topology-follow-the-workload.md
+++ b/v21.1/topology-follow-the-workload.md
@@ -4,7 +4,7 @@ summary: Guidance on using the follow-the-workload topology in a multi-region de
toc: true
---
-In a multi-region deployment, follow-the-workload is the default pattern for tables that use no other pattern. In general, this default pattern is a good choice only for tables with the following requirements:
+In a multi-region deployment, follow-the-workload is the default behavior for tables that do not have a [table locality](multiregion-overview.html#table-locality). In general, this is a good choice only for tables with the following requirements:
- The table is active mostly in one region at a time, e.g., following the sun.
- In the active region, read latency must be low, but write latency can be higher.
@@ -12,7 +12,7 @@ In a multi-region deployment, follow-the-workload is the default pattern for tab
- Table data must remain available during a region failure.
{{site.data.alerts.callout_success}}
-If read performance is your main focus for a table, but you want low-latency reads everywhere instead of just in the most active region, consider the [Duplicate Indexes](topology-duplicate-indexes.html) or [Follower Reads](topology-follower-reads.html) pattern.
+If read performance is your main focus for a table, but you want low-latency reads everywhere instead of just in the most active region, consider [Global Tables](global-tables.html) or [Follower Reads](topology-follower-reads.html).
{{site.data.alerts.end}}
## Prerequisites
@@ -27,12 +27,12 @@ If read performance is your main focus for a table, but you want low-latency rea
## Configuration
-Aside from [deploying a cluster across three regions](#cluster-setup) properly, with each node started with the [`--locality`](cockroach-start.html#locality) flag specifying its region and AZ combination, this pattern requires no extra configuration. CockroachDB will balance the replicas for a table across the three regions and will assign the range lease to the replica in the region with the greatest demand at any given time (the follow-the-workload feature). This means that read latency in the active region will be low while read latency in other regions will be higher due to having to leave the region to reach the leaseholder. Write latency will be higher as well due to always involving replicas in multiple regions.
+Aside from [deploying a cluster across three regions](#cluster-setup) properly, with each node started with the [`--locality`](cockroach-start.html#locality) flag specifying its region and zone combination, this behavior requires no extra configuration. CockroachDB will balance the replicas for a table across the three regions and will assign the range lease to the replica in the region with the greatest demand at any given time (the follow-the-workload feature). This means that read latency in the active region will be low while read latency in other regions will be higher due to having to leave the region to reach the leaseholder. Write latency will be higher as well due to always involving replicas in multiple regions.
{{site.data.alerts.callout_info}}
-This pattern is also used by [system ranges containing important internal data](configure-replication-zones.html#create-a-replication-zone-for-a-system-range).
+Follow-the-workload is also used by [system ranges containing important internal data](configure-replication-zones.html#create-a-replication-zone-for-a-system-range).
{{site.data.alerts.end}}
## Characteristics
diff --git a/v21.1/topology-follower-reads.md b/v21.1/topology-follower-reads.md
index de9445311fe..7b32093ea60 100644
--- a/v21.1/topology-follower-reads.md
+++ b/v21.1/topology-follower-reads.md
@@ -4,7 +4,7 @@ summary: Guidance on using the follower reads topology in a multi-region deploym
toc: true
---
-In a multi-region deployment, the follower reads pattern is a good choice for tables with the following requirements:
+In a multi-region deployment, [Follower Reads](follower-reads.html) are a good choice for tables with the following requirements:
- Read latency must be low, but write latency can be higher.
- Reads can be historical (4.8 seconds or more in the past).
@@ -12,7 +12,7 @@ In a multi-region deployment, the follower reads pattern is a good choice for ta
- Table data must remain available during a region failure.
{{site.data.alerts.callout_success}}
-This pattern is compatible with all of the other multi-region patterns except [Geo-Partitioned Replicas](topology-geo-partitioned-replicas.html). However, if reads from a table must be exactly up-to-date, use the [Duplicate Indexes](topology-duplicate-indexes.html) or [Geo-Partitioned Leaseholders](topology-geo-partitioned-leaseholders.html) pattern instead. Up-to-date reads are required by tables referenced by [foreign keys](foreign-key.html), for example.
+If reads from a table must be exactly up-to-date, use [Global Tables](global-tables.html) or [Regional Tables](regional-tables.html) instead. Up-to-date reads are required by tables referenced by [foreign keys](foreign-key.html), for example.
{{site.data.alerts.end}}
## Prerequisites
@@ -33,7 +33,7 @@ Follower reads requires an [Enterprise license](https://www.cockroachlabs.com/ge
### Summary
-Using this pattern, you configure your application to use the [follower reads](follower-reads.html) feature by adding an `AS OF SYSTEM TIME` clause when reading from the table. This tells CockroachDB to read slightly historical data from the closest replica so as to avoid being routed to the leaseholder, which may be in an entirely different region. Writes, however, will still leave the region to get consensus for the table.
+You configure your application to use [follower reads](follower-reads.html) by adding an [`AS OF SYSTEM TIME`](as-of-system-time.html) clause when reading from the table. This tells CockroachDB to read slightly historical data from the closest replica so as to avoid being routed to the leaseholder, which may be in an entirely different region. Writes, however, will still leave the region to get consensus for the table.
### Steps
diff --git a/v21.1/topology-geo-partitioned-leaseholders.md b/v21.1/topology-geo-partitioned-leaseholders.md
deleted file mode 100644
index ce813ca9032..00000000000
--- a/v21.1/topology-geo-partitioned-leaseholders.md
+++ /dev/null
@@ -1,251 +0,0 @@
----
-title: Geo-Partitioned Leaseholders Topology
-summary: Common cluster topology patterns with setup examples and performance considerations.
-toc: true
----
-
-In a multi-region deployment, the geo-partitioned [leaseholders](architecture/replication-layer.html#leases) topology is a good choice for tables with the following requirements:
-
-- Read latency must be low, but write latency can be higher.
-- Reads must be up-to-date for business reasons or because the table is referenced by [foreign keys](foreign-key.html).
-- Rows in the table, and all latency-sensitive queries, can be tied to specific geographies, e.g., city, state, region.
-- Table data must remain available during a region failure.
-
-{{site.data.alerts.callout_success}}
-**See It In Action** - Read about how a [large telecom provider](https://www.cockroachlabs.com/case-studies/telecom-provider-replaces-amazon-aurora-with-cockroachdb-to-attain-analways-on-customer-experience/) with millions of customers across the United States is using the Geo-Partitioned Leaseholders topology in production for strong resiliency and performance.
-{{site.data.alerts.end}}
-
-## Prerequisites
-
-### Fundamentals
-
-{% include {{ page.version.version }}/topology-patterns/fundamentals.md %}
-
-### Cluster setup
-
-{% include {{ page.version.version }}/topology-patterns/multi-region-cluster-setup.md %}
-
-## Configuration
-
-{{site.data.alerts.callout_info}}
-Geo-partitioning requires an [Enterprise license](https://www.cockroachlabs.com/get-cockroachdb).
-{{site.data.alerts.end}}
-
-### Summary
-
-Using this pattern, you design your table schema to allow for [partitioning](partitioning.html#table-creation), with a column identifying geography as the first column in the table's compound primary key (e.g., city/id). You tell CockroachDB to partition the table and all of its secondary indexes by that geography column, each partition becoming its own range of 3 replicas. You then tell CockroachDB to put the leaseholder for each partition in the relevant region (e.g., LA partitions in `us-west`, NY partitions in `us-east`). The other replicas of a partition remain balanced across the other regions. This means that reads in each region will access local leaseholders and, therefore, will have low, intra-region latencies. Writes, however, will leave the region to get consensus and, therefore, will have higher, cross-region latencies.
-
-
-
-### Steps
-
-Assuming you have a [cluster deployed across three regions](#cluster-setup) and a table and secondary index like the following:
-
-{% include copy-clipboard.html %}
-~~~ sql
-> CREATE TABLE users (
- id UUID NOT NULL DEFAULT gen_random_uuid(),
- city STRING NOT NULL,
- first_name STRING NOT NULL,
- last_name STRING NOT NULL,
- address STRING NOT NULL,
- PRIMARY KEY (city ASC, id ASC)
-);
-~~~
-
-{% include copy-clipboard.html %}
-~~~ sql
-> CREATE INDEX users_last_name_index ON users (city, last_name);
-~~~
-
-1. If you do not already have one, [request a trial Enterprise license](https://www.cockroachlabs.com/get-cockroachdb).
-
-2. Partition the table by `city`. For example, assuming there are three possible `city` values, `los angeles`, `chicago`, and `new york`:
-
- {% include copy-clipboard.html %}
- ~~~ sql
- > ALTER TABLE users PARTITION BY LIST (city) (
- PARTITION la VALUES IN ('los angeles'),
- PARTITION chicago VALUES IN ('chicago'),
- PARTITION ny VALUES IN ('new york')
- );
- ~~~
-
- This creates distinct ranges for each partition of the table.
-
-3. Partition the secondary index by `city` as well:
-
- {% include copy-clipboard.html %}
- ~~~ sql
- > ALTER INDEX users_last_name_index PARTITION BY LIST (city) (
- PARTITION la VALUES IN ('los angeles'),
- PARTITION chicago VALUES IN ('chicago'),
- PARTITION ny VALUES IN ('new york')
- );
- ~~~
-
- This creates distinct ranges for each partition of the secondary index.
-
-4. For each partition of the table and its secondary index, [create a replication zone](configure-zone.html) that tells CockroachDB to put the partition's leaseholder in the relevant region:
-
- {% include copy-clipboard.html %}
- ~~~ sql
- > ALTER PARTITION la OF INDEX users@*
- CONFIGURE ZONE USING
- num_replicas = 3,
- constraints = '{"+region=us-west":1}',
- lease_preferences = '[[+region=us-west]]';
- ALTER PARTITION chicago OF INDEX users@*
- CONFIGURE ZONE USING
- num_replicas = 3,
- constraints = '{"+region=us-central":1}',
- lease_preferences = '[[+region=us-central]]';
- ALTER PARTITION ny OF INDEX users@*
- CONFIGURE ZONE USING
- num_replicas = 3,
- constraints = '{"+region=us-east":1}',
- lease_preferences = '[[+region=us-east]]';
- ~~~
-
-5. To confirm that partitions are in effect, you can use the [`SHOW CREATE TABLE`](show-create.html) or [`SHOW PARTITIONS`](show-partitions.html) statement:
-
- {% include copy-clipboard.html %}
- ~~~ sql
- > SHOW CREATE TABLE users;
- ~~~
-
- ~~~
- table_name | create_statement
- +------------+----------------------------------------------------------------------------------------------------+
- users | CREATE TABLE users (
- | id UUID NOT NULL DEFAULT gen_random_uuid(),
- | city STRING NOT NULL,
- | first_name STRING NOT NULL,
- | last_name STRING NOT NULL,
- | address STRING NOT NULL,
- | CONSTRAINT "primary" PRIMARY KEY (city ASC, id ASC),
- | INDEX users_last_name_index (city ASC, last_name ASC) PARTITION BY LIST (city) (
- | PARTITION la VALUES IN (('los angeles')),
- | PARTITION chicago VALUES IN (('chicago')),
- | PARTITION ny VALUES IN (('new york'))
- | ),
- | FAMILY "primary" (id, city, first_name, last_name, address)
- | ) PARTITION BY LIST (city) (
- | PARTITION la VALUES IN (('los angeles')),
- | PARTITION chicago VALUES IN (('chicago')),
- | PARTITION ny VALUES IN (('new york'))
- | );
- | ALTER PARTITION chicago OF INDEX defaultdb.public.users@primary CONFIGURE ZONE USING
- | num_replicas = 3,
- | constraints = '{+region=us-central: 1}',
- | lease_preferences = '[[+region=us-central]]';
- | ALTER PARTITION la OF INDEX defaultdb.public.users@primary CONFIGURE ZONE USING
- | num_replicas = 3,
- | constraints = '{+region=us-west: 1}',
- | lease_preferences = '[[+region=us-west]]';
- | ALTER PARTITION ny OF INDEX defaultdb.public.users@primary CONFIGURE ZONE USING
- | num_replicas = 3,
- | constraints = '{+region=us-east: 1}',
- | lease_preferences = '[[+region=us-east]]';
- | ALTER PARTITION chicago OF INDEX defaultdb.public.users@users_last_name_index CONFIGURE ZONE USING
- | num_replicas = 3,
- | constraints = '{+region=us-central1: 1}',
- | lease_preferences = '[[+region=us-central1]]';
- | ALTER PARTITION la OF INDEX defaultdb.public.users@users_last_name_index CONFIGURE ZONE USING
- | num_replicas = 3,
- | constraints = '{+region=us-west: 1}',
- | lease_preferences = '[[+region=us-west]]';
- | ALTER PARTITION ny OF INDEX defaultdb.public.users@users_last_name_index CONFIGURE ZONE USING
- | num_replicas = 3,
- | constraints = '{+region=us-east: 1}',
- | lease_preferences = '[[+region=us-east]]'
- (1 row)
- ~~~
-
- {% include copy-clipboard.html %}
- ~~~ sql
- > SHOW PARTITIONS FROM TABLE users;
- ~~~
-
- ~~~
- database_name | table_name | partition_name | parent_partition | column_names | index_name | partition_value | zone_config
- +---------------+------------+----------------+------------------+--------------+-----------------------------+-----------------+-----------------------------------------------+
- defaultdb | users | la | NULL | city | users@primary | ('los angeles') | num_replicas = 3,
- | | | | | | | constraints = '{+region=us-west1: 1}',
- | | | | | | | lease_preferences = '[[+region=us-west1]]'
- defaultdb | users | chicago | NULL | city | users@primary | ('chicago') | num_replicas = 3,
- | | | | | | | constraints = '{+region=us-central1: 1}',
- | | | | | | | lease_preferences = '[[+region=us-central1]]'
- defaultdb | users | ny | NULL | city | users@primary | ('new york') | num_replicas = 3,
- | | | | | | | constraints = '{+region=us-east1: 1}',
- | | | | | | | lease_preferences = '[[+region=us-east1]]'
- defaultdb | users | la | NULL | city | users@users_last_name_index | ('los angeles') | num_replicas = 3,
- | | | | | | | constraints = '{+region=us-west1: 1}',
- | | | | | | | lease_preferences = '[[+region=us-west1]]'
- defaultdb | users | chicago | NULL | city | users@users_last_name_index | ('chicago') | num_replicas = 3,
- | | | | | | | constraints = '{+region=us-central1: 1}',
- | | | | | | | lease_preferences = '[[+region=us-central1]]'
- defaultdb | users | ny | NULL | city | users@users_last_name_index | ('new york') | num_replicas = 3,
- | | | | | | | constraints = '{+region=us-east1: 1}',
- | | | | | | | lease_preferences = '[[+region=us-east1]]'
- (6 rows)
- ~~~
-
-{{site.data.alerts.callout_success}}
-As you scale and add more cities, you can repeat steps 2 and 3 with the new complete list of cities to re-partition the table and its secondary indexes, and then repeat step 4 to create replication zones for the new partitions.
-{{site.data.alerts.end}}
-
-{% include {{page.version.version}}/sql/crdb-internal-partitions.md %}
-
-## Characteristics
-
-### Latency
-
-#### Reads
-
-Because each partition's leaseholder is constrained to the relevant region (e.g., the `la` partitions' leaseholders are located in the `us-west` region), reads that specify the local region key access the relevant leaseholder locally. This makes read latency very low, with the exception of reads that do not specify a region key or that refer to a partition in another region.
-
-For example, in the animation below:
-
-1. The read request in `us-west` reaches the regional load balancer.
-2. The load balancer routes the request to a gateway node.
-3. The gateway node routes the request to the leaseholder for the relevant partition.
-4. The leaseholder retrieves the results and returns to the gateway node.
-5. The gateway node returns the results to the client.
-
-
-
-#### Writes
-
-Just like for reads, because each partition's leaseholder is constrained to the relevant region (e.g., the `la` partitions' leaseholders are located in the `us-west` region), writes that specify the local region key access the relevant leaseholder replicas locally. However, a partition's other replicas are spread across the other regions, so writes involve multiple network hops across regions to achieve consensus. This increases write latency significantly.
-
-For example, in the animation below:
-
-1. The write request in `us-west` reaches the regional load balancer.
-2. The load balancer routes the request to a gateway node.
-3. The gateway node routes the request to the leaseholder replicas for the relevant table and secondary index partitions.
-4. While each leaseholder appends the write to its Raft log, it notifies its follower replicas, which are in the other regions.
-5. In each case, as soon as one follower has appended the write to its Raft log (and thus a majority of replicas agree based on identical Raft logs), it notifies the leaseholder and the write is committed on the agreeing replicas.
-6. The leaseholders then return acknowledgement of the commit to the gateway node.
-7. The gateway node returns the acknowledgement to the client.
-
-
-
-### Resiliency
-
-Because this pattern balances the replicas for each partition across regions, one entire region can fail without interrupting access to any partitions. In this case, if any range loses its leaseholder in the region-wide outage, CockroachDB makes one of the range's other replicas the leaseholder:
-
-
-
-
-
-## Alternatives
-
-- If reads from a table can be historical (4.8 seconds or more in the past), consider the [Follower Reads](topology-follower-reads.html) pattern.
-- If rows in the table, and all latency-sensitive queries, **cannot** be tied to specific geographies, consider the [Duplicate Indexes](topology-duplicate-indexes.html) pattern.
-
-## See also
-
-{% include {{ page.version.version }}/topology-patterns/see-also.md %}
diff --git a/v21.1/topology-geo-partitioned-replicas.md b/v21.1/topology-geo-partitioned-replicas.md
deleted file mode 100644
index 2749650cc0b..00000000000
--- a/v21.1/topology-geo-partitioned-replicas.md
+++ /dev/null
@@ -1,224 +0,0 @@
----
-title: Geo-Partitioned Replicas Topology
-summary: Guidance on using the geo-partitioned replicas topology in a multi-region deployment.
-toc: true
----
-
-In a multi-region deployment, the geo-partitioned replicas topology is a good choice for tables with the following requirements:
-
-- Read and write latency must be low.
-- Rows in the table, and all latency-sensitive queries, can be tied to specific geographies, e.g., city, state, region.
-- Regional data must remain available during an AZ failure, but it's OK for regional data to become unavailable during a region-wide failure.
-
-{{site.data.alerts.callout_success}}
-**See It In Action** - Read about how an [electronic lock manufacturer](https://www.cockroachlabs.com/case-studies/european-electronic-lock-manufacturer-modernizes-iam-system-with-managed-cockroachdb/) and [multi-national bank](https://www.cockroachlabs.com/case-studies/top-five-multinational-bank-modernizes-its-european-core-banking-services-migrating-from-oracle-to-cockroachdb/) are using the Geo-Partitioned Replicas topology in production for improved performance and regulatory compliance.
-{{site.data.alerts.end}}
-
-## Prerequisites
-
-### Fundamentals
-
-{% include {{ page.version.version }}/topology-patterns/fundamentals.md %}
-
-### Cluster setup
-
-{% include {{ page.version.version }}/topology-patterns/multi-region-cluster-setup.md %}
-
-## Configuration
-
-{{site.data.alerts.callout_info}}
-Geo-partitioning requires an [Enterprise license](https://www.cockroachlabs.com/get-cockroachdb).
-{{site.data.alerts.end}}
-
-### Summary
-
-Using this pattern, you design your table schema to allow for [partitioning](partitioning.html#table-creation), with a column identifying geography as the first column in the table's compound primary key (e.g., city/id). You tell CockroachDB to partition the table and all of its secondary indexes by that geography column, each partition becoming its own range of 3 replicas. You then tell CockroachDB to pin each partition (all of its replicas) to the relevant region (e.g., LA partitions in `us-west`, NY partitions in `us-east`). This means that reads and writes in each region will always have access to the relevant replicas and, therefore, will have low, intra-region latencies.
-
-
-
-### Steps
-
-Assuming you have a [cluster deployed across three regions](#cluster-setup) and a table and secondary index like the following:
-
-{% include copy-clipboard.html %}
-~~~ sql
-> CREATE TABLE users (
- id UUID NOT NULL DEFAULT gen_random_uuid(),
- city STRING NOT NULL,
- first_name STRING NOT NULL,
- last_name STRING NOT NULL,
- address STRING NOT NULL,
- PRIMARY KEY (city ASC, id ASC)
-);
-~~~
-
-{% include copy-clipboard.html %}
-~~~ sql
-> CREATE INDEX users_last_name_index ON users (city, last_name);
-~~~
-
-{{site.data.alerts.callout_info}}
-A geo-partitioned table does not require a secondary index. However, if the table does have one or more secondary indexes, each index must be partitioned as well. This means that the indexes must start with the column identifying geography, like the table itself, which impacts the queries they'll be useful for. If you can't partition all secondary indexes on a table you want to geo-partition, consider the [Geo-Partitioned Leaseholders](topology-geo-partitioned-leaseholders.html) pattern instead.
-{{site.data.alerts.end}}
-
-1. If you do not already have one, [request a trial Enterprise license](https://www.cockroachlabs.com/get-cockroachdb).
-
-2. Partition the table by `city`. For example, assuming there are three possible `city` values, `los angeles`, `chicago`, and `new york`:
-
- {% include copy-clipboard.html %}
- ~~~ sql
- > ALTER TABLE users PARTITION BY LIST (city) (
- PARTITION la VALUES IN ('los angeles'),
- PARTITION chicago VALUES IN ('chicago'),
- PARTITION ny VALUES IN ('new york')
- );
- ~~~
-
- This creates distinct ranges for each partition of the table.
-
-3. Partition the secondary index by `city` as well:
-
- {% include copy-clipboard.html %}
- ~~~ sql
- > ALTER INDEX users_last_name_index PARTITION BY LIST (city) (
- PARTITION la VALUES IN ('los angeles'),
- PARTITION chicago VALUES IN ('chicago'),
- PARTITION ny VALUES IN ('new york')
- );
- ~~~
-
- This creates distinct ranges for each partition of the secondary index.
-
-4. For each partition of the table and its secondary index, [create a replication zone](configure-zone.html) that constrains the partition's replicas to nodes in the relevant region:
-
- {{site.data.alerts.callout_success}}
- The `
-
-#### Writes
-
-Just like for reads, because each partition is constrained to the relevant region (e.g., the `la` partitions are located in the `us-west` region), writes that specify the local region key access the relevant replicas without leaving the region. This makes write latency very low, with the exception of writes that do not specify a region key or that refer to a partition in another region; such writes will be transactionally consistent but won't have local latencies.
-
-For example, in the animation below:
-
-1. The write request in `us-central` reaches the regional load balancer.
-2. The load balancer routes the request to a gateway node.
-3. The gateway node routes the request to the leaseholder replicas for the relevant table and secondary index partitions.
-4. While each leaseholder appends the write to its Raft log, it notifies its follower replicas, which are in the same region.
-5. In each case, as soon as one follower has appended the write to its Raft log (and thus a majority of replicas agree based on identical Raft logs), it notifies the leaseholder and the write is committed on the agreeing replicas.
-6. The leaseholders then return acknowledgement of the commit to the gateway node.
-7. The gateway node returns the acknowledgement to the client.
-
-
-
-### Resiliency
-
-Because each partition is constrained to the relevant region and balanced across the 3 AZs in the region, one AZ can fail per region without interrupting access to the partitions in that region:
-
-
-
-However, if an entire region fails, the partitions in that region become unavailable for reads and writes, even if your load balancer can redirect requests to a different region:
-
-
-
-## Tutorial
-
-For a step-by-step demonstration of how this pattern gets you low-latency reads and writes in a broadly distributed cluster, see the [Low Latency Multi-Region Deployment](demo-low-latency-multi-region-deployment.html) tutorial.
-
-## See also
-
-{% include {{ page.version.version }}/topology-patterns/see-also.md %}
diff --git a/v21.1/topology-patterns.md b/v21.1/topology-patterns.md
index 7e9b4489fd7..0fac5fa131e 100644
--- a/v21.1/topology-patterns.md
+++ b/v21.1/topology-patterns.md
@@ -1,46 +1,53 @@
---
-title: Topology Patterns
-summary: Recommended topology patterns for running CockroachDB in a cloud environment.
+title: Topology Patterns Overview
+summary: Recommended patterns for running CockroachDB in a cloud environment.
toc: true
redirect_from: cluster-topology-patterns.html
key: cluster-topology-patterns.html
---
-This section provides recommended topology patterns for running CockroachDB in a cloud environment, each with required configurations and latency and resiliency characteristics.
+This section provides recommended patterns for running CockroachDB in a cloud environment.
{{site.data.alerts.callout_info}}
-You can observe latency patterns for your cluster on the [Network Latency page](ui-network-latency-page.html) of the DB Console.
+You can observe latency for your cluster on the [Network Latency page](ui-network-latency-page.html) of the DB Console.
{{site.data.alerts.end}}
-## Single-region patterns
+## Single-region
-When your clients are in a single geographic region, choosing a topology is straightforward.
+When your clients are in a single geographic region, choosing a pattern is straightforward.
-Pattern | Latency | Resiliency | Configuration
+Deployment Type | Latency | Resiliency | Configuration
--------|---------|------------|--------------
[Development](topology-development.html) |