Document when to use GLOBAL tables vs. REGIONAL plus Follower Reads

[rmloveland]

Richard Loveland (rmloveland) commented:

We should document when to use GLOBAL tables vs. when to use REGIONAL tables plus Follower Reads.

Jira Issue: DOC-1001

[rmloveland]

@andreimatei or @nvanbenschoten - would one of you mind adding more detail re: when it might be better to use REGIONAL tables plus Follower Reads vs. GLOBAL tables?

(Some things were just said about this in a meeting but I didn't fully catch what was said)

[nvanbenschoten]

This is a very important topic, but it's also quite subtle, so we'll need to be careful about how we present this. The way I would talk about this is to start by saying that users can perform low-latency, stale reads using AS OF SYSTEM TIME follower_read_timestamp() across all of the tables in their database in any region that is part of the database, regardless of each table's locality setting. So people should default to using stale follower reads on REGIONAL tables (the default locality) if they want low-latency read-only access to their data in multiple regions.

There are two reasons why a user might want to upgrade a table to GLOBAL.

The first is that they want low-latency consistent (non-stale) read access to the table in multiple regions from read-write transactions. So they want to be able to read the same data in the table (i.e. it can't be partitioned) consistently while performing writes. One very interesting case where this is important is if the table is referenced by a foreign key from a REGIONAL BY ROW table. In these cases, the foreign key check that accompanies a write cannot use a stale read because it needs to be transactionally consistent with the write. So to keep this foreign-key check fast, we can slow down writes to the reference table and make it GLOBAL.

The second reason is less strong, but may be a practical reality. We've tried to make follower reads as accessible as possible and even have new ways of using them, like #10242. But there are bound to be cases where an ORM or application-level tool gets in the way and makes follower reads too hard to use. In these cases, GLOBAL tables can allow users to achieve low-latency reads through a schema-level setting.

So to summarize, I think the way we should frame this is "use follower reads whenever you can, only use GLOBAL tables when you can't".

[exalate-issue-sync]

Vy Ton (vy-ton) commented:
Richard Loveland to work with Stephanie Bodoff when this is prioritized

[exalate-issue-sync]

Stephanie Bodoff (stbof) commented:
I just added content in lines 318-361 here for this. Is it sufficient?

[exalate-issue-sync]

Richard Loveland (rmloveland) commented:
Stephanie Bodoff that PR mentions that both global tables and follower reads will do some load balancing, but AFAICT it does not explain the tradeoffs of when to use global tables vs. when to use regional tables plus follower reads , as described above. I don’t think the nuances described in Nathan’s comment above are covered, unless I missed it

[exalate-issue-sync]

Stephanie Bodoff (stbof) commented:
OK. I’ll dig into it further.