Skip to content

sql: index creation on 1.3 billion row table taking 5 days #36339

@vivekmenezes

Description

@vivekmenezes
select count(1) from t;
   count
+------------+
 1393445658
(1 row)

Time: 7m25.905025076s

on the above table split across 14K ranges an index backfill takes around 5 days on the 19.1 branch

with 14K ranges this is approximately on an average 100K rows per range or 100K index entries per range.
We currently make index SSTs by range. I'm expecting #36192 to help but we need to verify that it indeed solves this problem

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-schema-changesC-performancePerf of queries or internals. Solution not expected to change functional behavior.O-communityOriginated from the community

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions