This is an Epic issue for initial Cube Store implementation

Motivation

Over the last year, we accumulated feedback among various use cases with pre-aggregations and how to store them.
We figured out that there's a set of problems where relational databases as a storage layer have significant performance and functionality issues.

These problems include high cardinality rollup (1B and more) performance issues, lack of HyperLogLog support, degraded performance for big UNION ALL queries, poor join performance across rolled up tables, and more.

Over time we realized that if we try to fix these issues with existing database engines, we end up modifying these databases' code one way or another.

We decided to go another way and write our own materialized OLAP cache store designed solely to store and serve rollup tables at scale.

To optimize performance as much as possible, we decided to go native and use Rust to develop Cube Store utilizing a set of technologies like RocksDB, Apache Parquet, and Arrow that proved its effectiveness in solving data access problems.

Cube Store will be fully open source and released under the Apache 2.0 license.

The initial release plan is:

• Support most of basic queries available in examples.
• Native builds for popular OS.
• Make Cube Store default external pre-aggregation database.
• Deployment instructions and documentation.