roadmap-1.0.0.md

DataVines v1.0.0 RoadMap

Architecture Design

• Decentralized distributed cluster architecture design
• Modular design
• Plug-In design

Features

• Easy-to-use Client
• Plug-in design of various core components
• Server node online and offline notification
• DataSource management
• Metric management
• Job management
• Notification management
• Task execution
• Task retry
• Task timeout alarm
• Task fault tolerance after the server node down

Plug-in Component

• Register
  • MySQL
  • PostgreSQL
  • ZooKeeper
• Engine
  • JDBC
    • Connector
      • PostgreSQL
      • MySQL
      • Hive
      • Impala
      • ClickHouse
      • Presto
  • Spark
    • Connector
      • JDBC
• Metric
  • column_null
  • column_not_null
  • column_value_between
  • column_match_regex
  • column_match_not_regex
  • column_length
  • column_not_in_enums
  • column_in_enums
  • column_unique
  • column_duplicate
  • custom_sql
  • table_row_count
  • table_freshness
• Connector
  • JDBC
    • PostgreSQL
    • MySQL
    • Hive
    • Impala
    • ClickHouse
    • Presto
  • File
• ExpectedValue
  • fix_value
  • table_total_rows
  • target_table_total_rows
  • daily_avg
  • weekly_avg
  • monthly_avg
  • last_7_day_avg
  • last_30_day_avg
• ResultFormula
  • count
  • percentage
  • diff
  • diff_percentage
• Notification Channel
  • email