GreptimeDB Roadmap 2024

[tisonkun]

You may find the roadmap blog at https://greptime.com/blogs/2024-02-29-greptimedb-2024-roadmap.

Overview

This roadmap outlines the objectives our team is pursuing and the visions we harbor for our collective community.

The evolution of GreptimeDB in 2024 is marked by a suite of main feature updates. These enhancements are a testament to our ongoing commitment to excellence, driven by feedback from our community and the latest requirements in real-world scenarios.

Our roadmap for the year includes significant advancements that promise to elevate the capabilities of GreptimeDB and enrich the user experience.

Join Force

For tasks with tracking issues available, directly diving into the issue and commenting would be desirable.

For open-ended tasks or if you have any other ideas to improve GreptimeDB, don't hesitate to comment below, ask on GitHub Discussion or join the GreptimeDB Community on Slack.

Details

Metrics Engine

• RFC
• Tracking issue for the MetricEngine #2070

A new engine designed for observability scenarios. Its primary aim is to handle a large number of small tables, making it particularly suitable for using Prometheus metrics. By utilizing synthetic wide tables, this new Engine offers the capability to store metric data and reuse metadata, rendering "tables" atop it more lightweight and overcoming some of the limitations of the existing Mito engine, which is considered too heavy for such tasks.

Inverted Index

• RFC
• Tracking issue for introducing inverted index #2705

DataFlow (GreptimeFlow)

• RFC
• Tracking issue for dataflow framework #3187

A lightweight stream computing component capable of performing continuous aggregation on GreptimeDB data streams. It can be embedded into the GreptimeDB Frontend or deployed as a separate service within the GreptimeDB cluster.

Here is a flow job can be submitted in the form of SQL:

CREATE TASK avg_over_5m WINDOW_SIZE = "5m" AS SELECT avg(value) FROM table WHERE time > now() - 5m GROUP BY time(1m);

Full-text Index

• Tracking Issue: support full-text index #4246

In SQL statements, you can use the MATCHES function to perform full-text searches, which is especially useful for log analysis. The MATCHES function supports full-text searches on String type columns. Here’s an example of how it can be used:

SELECT * FROM logs WHERE MATCHES(message, 'Barack OR Obama');

Region Migration

• RFC
• Tracking issue for Region Migration #2700

It allows migrating regions between Datanodes, facilitating the relocation of hot data and the horizontal scaling of load balancing.

Logs Engine

A storage engine designed specifically for the characteristics of log data, sharing most of GreptimeDB's architecture and capabilities, such as the SQL query layer, data sharding, distributed routing, querying indexing, and compression ability. This enables GreptimeDB to become a unified system offering optimized storage and a consistent access experience for Metrics and Logs data based on a multi-engine architecture.

[tisonkun]

@fengjiachun I remember that "Region Migration" is under development, do we have some issues to linked here?

[fengjiachun]

@fengjiachun I remember that "Region Migration" is under development, do we have some issues to linked here?

I added RFC and tracking issue link.

[tisonkun]

@fengjiachun Cool! So it even has been implemented and would be delivered at 0.7?

[fengjiachun]

@tisonkun Yes

[tisonkun]

No longer valid. Need an update.

[killme2008]

Yes. We need to update it but keep it open.

@fengjiachun Please update it next week

[fengjiachun]

I'll update it in the next few days.