Skip to content

Commit

Permalink
Add note about skipscan not working on compressed chunks (#2596)
Browse files Browse the repository at this point in the history
* Add note about skipscan not working on compressed chunks

* Edit note and move

---------

Co-authored-by: Rajakavitha Kodhandapani <krajakavitha@gmail.com>
  • Loading branch information
Loquacity and Rajakavitha1 authored Aug 14, 2023
1 parent 0a6d9bc commit a97e5d6
Show file tree
Hide file tree
Showing 3 changed files with 43 additions and 34 deletions.
1 change: 1 addition & 0 deletions .github/styles/Vocab/Timescale/accept.txt
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ Unix
[Rr]ollups?
[Ss]erverless
[Ss]hard(?:s|ing)?
SkipScan
(?i)timestamptz
URLs?
URIs?
Expand Down
59 changes: 32 additions & 27 deletions api/compression.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,50 +7,55 @@ tags: [hypertables]

# Compression <Tag type="community">Community</Tag>

We highly recommend reading the [blog post][blog-compression] and
[tutorial][using-compression] about compression before trying to set it up
for the first time.
Before you set up compression, you need to
[configure the hypertable for compression][configure-compression] and then
[set up a compression policy][add_compression_policy].

Setting up compression on TimescaleDB requires users to first [configure the
hypertable for compression][configure-compression] and then [set up a
policy][add_compression_policy] for when to compress chunks.
<Highlight type="note">
Before you set up compression for the first time, read the compression
[blog post][blog-compression] and
[documentation][using-compression].
</Highlight>

Advanced usage of compression allows users to [compress chunks
manually][compress_chunk], instead of automatically as they age.
You can also [compress chunks manually][compress_chunk], instead of using an
automated compression policy to compress chunks as they age.

Compressed chunks have the following limitations:

* `ROW LEVEL SECURITY` is not supported.
* Creation of unique constraints on compressed chunks is not supported
* You can add them by disabling compression on the hypertable and re-enabling after constraint creation.

### Restrictions
* `ROW LEVEL SECURITY` is not supported on compressed chunks.
* Creation of unique constraints on compressed chunks is not supported. You
can add them by disabling compression on the hypertable and re-enabling
after constraint creation.
* [Timescale SkipScan][skipscan] does not currently work on compressed chunks.

In general, compressing a hypertable does impose some limitations on the types
of data modifications that can occur on data inside a compressed chunk.
The table below provides a brief outline of allowed modifications
based on the version of TimescaleDB you are currently running.
## Restrictions

|TimescaleDB Version|Supported data modifications on compressed chunks|
|---|---|
In general, compressing a hypertable imposes some limitations on the types
of data modifications that you can perform on data inside a compressed chunk.

This table shows changes to the compression feature, added in different versions
of TimescaleDB:

|TimescaleDB version|Supported data modifications on compressed chunks|
|-|-|
|1.5 - 2.0|Data and schema modifications are not supported.|
|2.1 - 2.2|Schema may be modified on compressed hypertables. Data modification not supported.|
|2.3|Schema modifications and basic insert of new data are allowed. Deleting, updating and some advanced insert statements are not supported.|
|2.3|Schema modifications and basic insert of new data is allowed. Deleting, updating and some advanced insert statements are not supported.|
|2.11|Deleting, updating and advanced insert statements are supported.|

Starting with TimescaleDB 2.1, users have the ability to modify the schema
of hypertables that have compressed chunks.
Specifically, you can add columns to and rename existing columns of
such compressed hypertables.
In TimescaleDB 2.1&nbsp;and later, you can modify the schema of hypertables that
have compressed chunks. Specifically, you can add columns to and rename existing
columns of compressed hypertables.

Starting with TimescaleDB 2.3, users have the ability to insert data into compressed chunks
In TimescaleDB&nbsp;2.3 and later, you can insert data into compressed chunks
and to enable compression policies on distributed hypertables.

Starting with TimescaleDB 2.11, you can update and delete compressed data.
You can also use advanced insert statements like `ON CONFLICT` and `RETURNING` is also supported.
In TimescaleDB&nbsp;2.11 and later, you can update and delete compressed data.
You can also use advanced insert statements like `ON CONFLICT` and `RETURNING`.

[add_compression_policy]: /api/:currentVersion:/compression/add_compression_policy/
[blog-compression]: https://blog.timescale.com/blog/building-columnar-compression-in-a-row-oriented-database/
[compress_chunk]: /api/:currentVersion:/compression/compress_chunk/
[configure-compression]: /api/:currentVersion:/compression/alter_table_compression/
[using-compression]: /use-timescale/:currentVersion:/compression/
[skipscan]: /use-timescale/:currentVersion:/query-data/skipscan/
17 changes: 10 additions & 7 deletions use-timescale/query-data/skipscan.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@ SkipScan improves query times for `DISTINCT` queries. It works on PostgreSQL
tables, Timescale hypertables, and Timescale distributed hypertables.
SkipScan is included in TimescaleDB&nbsp;2.2.1 and later.

<Highlight type="note">
This page discusses the Timescale Skipscan feature. SkipScan is not currently
available in standard PostgreSQL.
</Highlight>

## Speed up `DISTINCT` queries

To query your database and find the most recent value of an item, you
Expand All @@ -26,17 +31,15 @@ the exact order and columns for these kinds of queries, PostgreSQL scans the
entire index to find all unique values. As a table grows, this operation keeps
getting slower.

<Highlight type="note">
Timescale SkipScan does not currently work on compressed chunks.
</Highlight>

SkipScan allows queries to incrementally jump from one ordered value to the next
without reading all of the rows in between. Without support for this feature,
the database engine has to scan the entire ordered index and then de-duplicate
at the end, which is a much slower process.

<Highlight type="note">
PostgreSQL has plans to implement a native feature like SkipScan, but it is
unlikely to be included until at least PostgreSQL&nbsp;15. This section
documents Timescale SkipScan, which is not a native PostgreSQL feature.
</Highlight>

SkipScan is an optimization for queries of the form `SELECT DISTINCT ON
column_name`. Conceptually, SkipScan is a regular IndexScan that skips across an
index looking for the next value that is greater than the current value.
Expand All @@ -50,7 +53,7 @@ the next item. This is a much more efficient way of finding distinct items in an
ordered index.

For benchmarking information on how SkipScan compares to regular `DISTINCT`
queries, see our [SkipScan blog post][blog-skipscan].
queries, see the [SkipScan blog post][blog-skipscan].

## Use SkipScan queries

Expand Down

0 comments on commit a97e5d6

Please sign in to comment.