feat(uptime): add initial table migration

[JoshFerge]

broken out from #6686

context: https://github.com/getsentry/team-uptime/issues/23

[github-actions]

This PR has a migration; here is the generated SQL for ./snuba/migrations/groups.py ()

-- start migrations

-- forward migration uptime_monitor_checks : 0001_uptime_monitor_checks
Local op: CREATE TABLE IF NOT EXISTS uptime_monitor_checks_local (organization_id UInt64, project_id UInt64, environment LowCardinality(Nullable(String)), uptime_subscription_id UUID, uptime_check_id UUID, scheduled_check_time DateTime64(3), timestamp DateTime64(3), duration_ms UInt64, region_slug LowCardinality(String), check_status LowCardinality(String), check_status_reason LowCardinality(Nullable(String)), http_status_code UInt16, trace_id UUID, retention_days UInt16) ENGINE ReplicatedReplacingMergeTree('/clickhouse/tables/uptime_monitor_checks/{shard}/default/uptime_monitor_checks_local', '{replica}') PRIMARY KEY (organization_id, project_id, toDateTime(timestamp), uptime_check_id, trace_id) ORDER BY (organization_id, project_id, toDateTime(timestamp), uptime_check_id, trace_id) PARTITION BY (retention_days, toMonday(timestamp)) TTL toDateTime(timestamp) + toIntervalDay(retention_days) SETTINGS index_granularity=8192;
Distributed op: CREATE TABLE IF NOT EXISTS uptime_monitor_checks_dist (organization_id UInt64, project_id UInt64, environment LowCardinality(Nullable(String)), uptime_subscription_id UUID, uptime_check_id UUID, scheduled_check_time DateTime64(3), timestamp DateTime64(3), duration_ms UInt64, region_slug LowCardinality(String), check_status LowCardinality(String), check_status_reason LowCardinality(Nullable(String)), http_status_code UInt16, trace_id UUID, retention_days UInt16) ENGINE Distributed(`cluster_one_sh`, default, uptime_monitor_checks_local, cityHash64(reinterpretAsUInt128(trace_id)));
-- end forward migration uptime_monitor_checks : 0001_uptime_monitor_checks




-- backward migration uptime_monitor_checks : 0001_uptime_monitor_checks
Distributed op: DROP TABLE IF EXISTS uptime_monitor_checks_dist;
Local op: DROP TABLE IF EXISTS uptime_monitor_checks_local;
-- end backward migration uptime_monitor_checks : 0001_uptime_monitor_checks

[JoshFerge]

TODO: make non-nullable / a low cardinality string

[phacops]

Indicate the unit in the field name.

[phacops]

I wouldn't start with adding indices already. If you don't want to put the trace_id in the sort key, fine, but this might not be necessary.

[JoshFerge]

i've added the trace_id to the sort key / primary key

[JoshFerge]

NOTE: this should be a UUID.

[phacops]

Is there only 1 check per trace? You might something more specific here otherwise the ReplacingMergeTree will merge 2 rows with the same sort key.

The engine differs from MergeTree in that it removes duplicate entries with the same sorting key value (ORDER BY table section, not PRIMARY KEY).

https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/replacingmergetree

If you're not going to update values or if you don't care about duplicates, you could just use a MergeTree.

[JoshFerge]

only one check per trace. duplicates would not be ideal

[codecov]

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
2619	1	2618	5

View the top 1 failed tests by shortest run time

tests.utils.streams.test_topics::test_valid_topics

Stack Traces | 0.124s run time

Traceback (most recent call last):
  File ".../local/lib/python3.11....../site-packages/sentry_kafka_schemas/sentry_kafka_schemas.py", line 80, in get_topic
    with open(topic_path) as f:
         ^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '.../local/lib/python3.11.../sentry_kafka_schemas/topics/snuba-dead-letter-uptime-monitor-checks.yaml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../utils/streams/test_topics.py", line 10, in test_valid_topics
    sentry_kafka_schemas.get_topic(
  File ".../local/lib/python3.11....../site-packages/sentry_kafka_schemas/sentry_kafka_schemas.py", line 83, in get_topic
    raise SchemaNotFound
sentry_kafka_schemas.sentry_kafka_schemas.SchemaNotFound

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

[JoshFerge]

closing as the migration needs to happen first.