Skip to content

Commit

Permalink
docs(stonedb): update the latest docs(#391) (#392)
Browse files Browse the repository at this point in the history
* docs(stonedb): update the latest docs(#391)

* docs(stonedb): update the docs of Architechure and Limits
  • Loading branch information
Nliver authored Aug 11, 2022
1 parent 8f60d0e commit a11644b
Show file tree
Hide file tree
Showing 28 changed files with 580 additions and 499 deletions.
Binary file added Docs/00-about-stonedb/KnowledgeGrid-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Docs/00-about-stonedb/KnowledgeGrid-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Docs/00-about-stonedb/KnowledgeGrid-3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
84 changes: 44 additions & 40 deletions Docs/00-about-stonedb/architecture.md

Large diffs are not rendered by default.

24 changes: 12 additions & 12 deletions Docs/00-about-stonedb/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,26 +6,26 @@ sidebar_position: 1.1

StoneDB is an open-source hybrid transaction/analytical processing (HTAP) database designed and developed by StoneAtom based on the MySQL kernel. It is the first database of this type launched in China. StoneDB can be seamlessly switched from MySQL. It provides features such as optimal performance and real-time analytics, offering you a one-stop solution to process online transaction processing (OLTP), online analytical processing (OLAP), and HTAP workloads.

StoneDB is fully compatible with the MySQL 5.6 and 5.7 protocols, the MySQL ecosystem, and common MySQL features and syntaxes. You can use tools and clients in the MySQL ecosystem on StoneDB, such as Navicat, Workbench, mysqldump, and mydumper. In addition, all workloads on StoneDB can be run on MySQL.
StoneDB is fully compatible with the MySQL 5.6 and 5.7 protocols, the MySQL ecosystem, and common MySQL features and syntaxes. Tools and clients in the MySQL ecosystem, such as Navicat, Workbench, mysqldump, and mydumper, can be directly used on StoneDB. In addition, all workloads on StoneDB can be run on MySQL.

StoneDB is optimized for OLAP applications. StoneDB that runs on a common server can process complex queries on tens of billions of data records, while ensuring high performance. Compared to databases that use MySQL Community Edition, StoneDB is at least 10 times faster in processing queries.

StoneDB uses the knowledge grid technology and a column-based storage engine. This storage engine is designed for OLAP applications and uses techniques such as column-based storage, knowledge grid-based filtering, and high-efficiency data compression. With such storage engine, StoneDB provides application systems with high-performance and reduces the total cost of ownership (TCO).
StoneDB uses the Knowledge Grid technology and a column-based storage engine. The column-based storage engine is designed for OLAP applications and uses techniques such as column-based storage, Knowledge Grid-based filtering, and high-efficiency data compression. With such storage engine, StoneDB ensures the high performance of application systems and reduces the total cost of ownership (TCO).
## Advantages
### Integration of MySQL
StoneDB is an HTAP database built on MySQL. To enhance analytics capabilities, it integrates a self-developed engine also named StoneDB. (In this topic, StoneDB refers to the database, if not otherwise specified.) For this reason, StoneDB is fully compatible with MySQL. You can use standard interfaces, such as Open Database Connectivity (ODBC) and Java Database Connectivity (JDBC) to connect to StoneDB. In addition, you can create local connections to connect to StoneDB. StoneDB supports APIs written in various programming languages, such as C, C++, C#, Java, PHP, and Perl. StoneDB is fully compatible with views and stored procedures that comply with the ANSI SQL-92 standard and the SQL-99 standard. In this way, your application systems that can run on MySQL can directly run on StoneDB, without the need to modify the code. This allows you to seamlessly switch MySQL to StoneDB.
### Real-time HTAP
StoneDB provides two engines: row-based storage engine InnoDB and column-based storage engine StoneDB. StoneDB uses binlogs to replicate data from the row-based storage engine to the column-based storage engine in real time. This ensures strong data consistency between the two storage engines.
### Full compatibility with MySQL
StoneDB is an HTAP database built on MySQL. You can use standard interfaces, such as Open Database Connectivity (ODBC) and Java Database Connectivity (JDBC) to connect to StoneDB. In addition, you can create local connections to connect to StoneDB. StoneDB supports APIs written in various programming languages, such as C, C++, C#, Java, PHP, and Perl. StoneDB is fully compatible with views and stored procedures that comply with the ANSI SQL-92 standard and the SQL-99 standard. In this way, your application systems that can run on MySQL can directly run on StoneDB, without the need to modify the code. This allows you to seamlessly switch MySQL to StoneDB.
### High query performance
When processing complex queries on tens of or even billions of data records, StoneDB reduces the processing time to one tenth or even shorter, compared to MySQL.
When processing complex queries on tens of or even hundreds of billions of data records, StoneDB reduces the processing time to one tenth or even shorter, compared to MySQL or other row-oriented databases.
### Minimal storage cost
StoneDB supports high compression ratio which can be up to 40:1. This greatly reduces the disk space required for storing data, cutting down the TCO.
## Key techniques
### Column-based storage engine
A column-based storage engine stores data to disks column by column. When you query data, only the required fields are retrieved, which greatly reduces memory bandwidth traffic and disk I/O. In addition, in a column-based storage engine, columns do not need to be indexed, freeing the database from maintaining such indexes.
Tables created on StoneDB are stored to disks column by column. Because data in the same column is of the same data type, the data can be densely compressed. This allows StoneDB to achieve much higher compression ratio than row-oriented databases. When processing a query that requires data in certain fields, StoneDB retrieves only the required fields, while a row-oriented database retrieves all rows that contain values of these fields. Compared to the row-oriented database, StoneDB reduces memory bandwidth traffic and disk I/O. In addition, StoneDB does not require indexing of columns, freeing from maintaining such indexes.
### High-efficiency data compression
In a relational database, values in the same column are of the same data type. More duplicate values stored in a column indicate a higher data compression ratio and a smaller data volume. By virtue of this, less data is retrieved for queries, and thus memory bandwidth traffic and disk I/O are reduced. <br />StoneDB saves storage space by using column-based storage. The data compression ratio of a column-oriented database is at least 10 times higher than that of a row-oriented database.
### Knowledge grid
A knowledge grid can filter data packs based on metadata, and then decompress the data packs to obtain the data that meets the query conditions. This greatly reduces I/O, and improves response speed and network utilization.
StoneDB supports various compression algorithms, such as PPM, LZ4, B2, and Delta, and it uses different compression algorithms to compress data of different data types. After the data is compressed, the volume of the data becomes smaller, and thus less network bandwidth and disk I/O resources are required to retrieve the data. StoneDB saves storage space by using column-based storage. The data compression ratio of a column-oriented database is at least 10 times higher than that of a row-oriented database.
### Knowledge Grid
In StoneDB, Data Packs are classified into relevant Data Packs, irrelevant Data Packs, and suspect Data Packs. This classification helps filter out irrelevant Data Packs. StoneDB needs only to read metadata of relevant Data Packs, and decompress suspect Data Packs and then examine the data records to filter relevant data records. If the result set of the relevant Data Packs can be directly obtained through their Data Pack Nodes (also known as metadata nodes), relevant Data Packs will not be decompressed. The process of handling relevant Data Packs does not consume I/O, since no data is decompressed.
### High-performance import
StoneDB provides an independent client to import data from various sources, written in different programming languages. Before data is imported, it is preprocessed, such as data compression and construction of Knowledge Nodes. In this way, operations such as parsing, verification, and transaction processing are eliminated when the data is being processed by the storage engine.
### Push-based vectorized query execution
When processing a query, StoneDB pushes column-based data packs from one operator to another based on the execution plan. Compared to the execution model used by row-oriented databases, push-based execution prevents in-depth calls of stacks and saves resources.
When processing a query, StoneDB pushes column-based Data Packs from one operator to another based on the execution plan. Compared to the execution model used by row-oriented databases, push-based execution prevents in-depth calls of stacks and saves resources.
45 changes: 19 additions & 26 deletions Docs/00-about-stonedb/limits.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ sidebar_position: 1.3
---

# Limits
As a conlumn-based storage engine, StoneDB is built on MySQL. Therefore, StoneDB is highly compatible with the MySQL 5.6 and 5.7 protocols, and the ecosystem, common features, and common syntaxes of MySQL. However, due to characteristics of column-based storage, StoneDB is incompatible with certain MySQL operations and features.
### Unsupported DDL operations
Developed based on MySQL, StoneDB is compatible with the MySQL 5.6 and 5.7 protocols, and the ecosystem, common features, and common syntaxes of MySQL. However, due to characteristics of column-based storage, StoneDB is incompatible with certain MySQL operations and features.
# Unsupported DDL operations
StoneDB does not support the following DDL operations:

- Modify the data type of a field.
Expand All @@ -16,7 +16,7 @@ StoneDB does not support the following DDL operations:
- Analyze a table.
- Lock a table.
- Repair a table.
- Execute a CREATE TABLE… AS SELECT statement.
- Execute a `CREATE TABLE … AS SELECT` statement.
- Reorganize a table.
- Rename a field.
- Configure the default value for a field.
Expand All @@ -28,17 +28,17 @@ StoneDB does not support the following DDL operations:
- Remove an index.
- Modify a table comment.

Data stored in StoneDB is highly compressed. For this reason, table attributes and column attributes are difficult to modify. The character sets, data types, constraints, and indexes must be properly defined when tables are being created.
### Unsupported DML operations
The table attributes and column attributes are difficult to modify. The character sets, data types, constraints, and indexes must be properly defined when tables are being created.
# Unsupported DML operations
StoneDB does not support the following DML operations:

- Execute a DELETE statement.
- Use subqueries in an UPDATE statement.
- Execute an UPDATE… JOIN statement to update multiple tables.
- Execute a REPLACE… INTO statement.
- Execute a `DELETE` statement.
- Use subqueries in an `UPDATE` statement.
- Execute an `UPDATE … JOIN` statement to update multiple tables.
- Execute a `REPLACE … INTO` statement.

StoneDB is not suitable for applications that are frequently updated. It supports only single-table update and insert operations. This is because a column-oriented database needs to find each corresponding column and update the value in the row when processing an update operation. However, a row-oriented database stores data by row. When processing an update operation, the row-oriented database only needs to find the corresponding page or block and update the data directly in the row.
### Unsupported objects
# Unsupported objects
StoneDB does not support the following objects:

- Global indexes
Expand All @@ -47,34 +47,27 @@ StoneDB does not support the following objects:
- Temporary tables
- Stored procedures containing dynamic SQL statements
- User-defined functions containing nested SQL statements

If you want to use user-defined functions that contain nested SQL statements, set the **stonedb_ini_allowmysqlquerypath** parameter to **1** in the **my.cnf** configuration file.
### Unsupported data types

# Unsupported data types
StoneDB does not support the following data types:

- bit
- enum
- set
- json
- decimal whose precision is higher than 18, for example, decimal(19,x)
- Data types that contain keyword **unsigned** or **zerofill**
### Unsupported binary log formats

# Unsupported binary log formats
StoneDB does not support the following binary log formats:

- row
- mixed

Column-based storage engines support only statement-based binary logs. Row-based binary logs and mixed binary logs are not supported.
### Join queries across storage engines not supported
By default, StoneDB does not support join queries across storage engines. If a join query involves tables in both InnoDB and StoneDB, an error will be reported. You can set the **stonedb_ini_allowmysqlquerypath** parameter to **1** in the **my.cnf** configuration file to remove this limit.

### Transactions not supported

# Join queries across storage engines not supported
By default, StoneDB does not support join queries across storage engines. If a join query involves tables in both other storage engines and StoneDB, an error will be reported. You can set the **tianmu_ini_allowmysqlquerypath** parameter to **1 **in the **my.cnf** configuration file to remove this limit.
# Transactions not supported
Transactions must strictly comply with the ACID attributes. However, StoneDB does not support redo and undo logs and thus does not support transactions.
### Partitions not supported

# Partitions not supported
Column-based storage engines do not support partitioning.
### Column locking and table locking not supported

Column-based storage engines do not support column locking or table locking.
# Row locks and table locks not supported
Column-based storage engines do not support row locks or table locks.
Binary file modified Docs/00-about-stonedb/stonedb-architecture-V1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 3 additions & 1 deletion Docs/00-about-stonedb/terms.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,10 @@ sidebar_position: 1.4

# Terms

The following table provides descriptions of common terms.

| **Term** | **Description** |
| :-- | :-- |
| --- | --- |
| **row** | A series of data that makes up a record. |
| **column** | Also referred to as field. In a relational database, a field must be associated with a data type when the field is being created. |
| **table** | Consists of rows and columns. Databases use tables to store data. Tables are essential objects in databases. |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,32 +5,24 @@ sidebar_position: 2.2
# Server Configuration Requirements

This topic describes the configuration requirements for a development or test environment and a production environment.


# Configuration requirements for a development or test environment

## Configuration requirements for a development or test environment
The following table describes the configuration requirements for a development or test environment.

| **CPU** | **Memory** | **Storage** | **Network** |
| --- | --- | --- | --- |
| 2 cores+ | 2 GB+ | 10 GB+ | Megabit network card |

:::caution

:::info
If the development or test environment is deployed on a virtual machine, the AVX instruction set must be enabled. Otherwise, StoneDB cannot be installed.

:::

# Configuration requirements for a production environment

## Configuration requirements for a production environment
The following table describes the configuration requirements for a production environment.

| **CPU** | **Memory** | **Storage** | **Network** |
| --- | --- | --- | --- |
| 8 cores+ | 8 GB+ | 100 GB+ | Gigabit network card |

:::tip

:::info
We recommend you use higher configuration in your production environment.

:::

9 changes: 3 additions & 6 deletions Docs/01-environment-requirements/supported-servers-and-OSs.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,20 @@ id: supported-servers-and-OSs
sidebar_position: 2.1
---
# Support Servers and OSs

StoneDB is an open-source hybrid transaction/analytical processing (HTAP) database designed and developed by StoneAtom based on the MySQL kernel. It can be deployed and run on 64-bit x86 servers and supports most mainstream network hardware and Linux OSs.

# Supported servers
## Supported servers
The following table lists the servers on which StoneDB can run.

| **Architecture** | **Supported server** |
| --- | --- |
| x86_64 architecture | Common x86_64 servers with AVX instruction sets enabled |
| x86_64 | Common x86_64 servers with AVX instruction sets enabled |

:::info

Support for the ARM64 or Power architecture is under testing.

:::

# Supported OSs
## Supported OSs
The following table lists the OSs supported by StoneDB.

| **OS** | **Version** |
Expand Down
31 changes: 15 additions & 16 deletions Docs/02-getting-started/basic-operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,16 @@ sidebar_position: 3.4
---

# Basic Operations

Structured Query Language (SQL) is a programming language for communicating with databases. You can use it to manage relational databases by performing insert, query, update, and other operations.

StoneDB is compatible with MySQL. You can use clients supported by MySQL to connect to StoneDB. In addition, StoneDB supports most SQL syntaxes. This section describes the basic SQL operations supported by StoneDB.
StoneDB is fully compatible with MySQL. You can use clients supported by MySQL to connect to StoneDB. In addition, StoneDB supports most SQL syntaxes. This section describes the basic SQL operations supported by StoneDB.

SQL can be classified into the following four parts by usage:

- Data Definition Language (DDL): is used to manage database objects, such as CREATE, ALTER, and DROP statements.
- Data Manipulation Language (DML): is used to manage data in tables, such as INSERT, DELETE, and UPDATE statements.
- Data Query Language (DQL): is used to query objects, such as SELECT statements.
- Data Control Language (DCL): is used to control access to data, such as GRANT and REVOKE statements.


## Operations on databases
This section provides examples of performing basic operations on databases.
### Create a database
Expand Down Expand Up @@ -45,20 +42,21 @@ This section provides examples of performing basic operations on tables.
Execute the following SQL statement to create a table which is named **student** and consists of the **id**, **name**, **age**, and **birthday** fields:
```sql
create table student(
id int(11) primary key,
name varchar(255),
age smallint,
birthday DATE
) engine=stonedb;
id int(11) primary key,
name varchar(20),
age smallint,
birthday DATE
) engine=stonedb;
```
:::info
If you do not specify **engine=stonedb** in the SQL statement, the storage engine on which the table is created is determined by the value of parameter **default_storage_engine**. For more information, see [Configure parameters](https://stoneatom.yuque.com/staff-ft8n1u/dghuxr/xg9czr).
:::

The row-based storage engine is named StoneDB in StoneDB-5.6, and is renamed to Tianmu in StoneDB-5.7 to distinguish from the database StoneDB.<br />If you do not specify **engine=stonedb** in the SQL statement, the storage engine on which the table is created is determined by the value of parameter **default_storage_engine**. For more information, see [Configure parameters](../04-developer-guide/05-appendix/configuration-parameters.md).

:::
### Query the schema of a table
Execute the following SQL statement to query the schema of table **student**:
```sql
show create table student\G
show create table student;
```
### Drop a table
Execute the following SQL statement to drop table **student**:
Expand All @@ -83,8 +81,9 @@ Execute the following TRUNCATE statement to clear data in table **student**:
```sql
truncate table student ;
```
#### Remove specific data from a table
:::info
As a column-based storage engine, StoneDB does not support DELETE operations.
:::
### Query data from a table
Execute a SELECT statement to query data from a table.

Expand All @@ -104,11 +103,11 @@ Execute the following SQL statement to create a user named **tiger** and set the
```sql
create user 'tiger'@'%' identified by '123456';
```

:::info

The username together with the hostname uniquely identify a user in the format of '_username_'@'_host_'. In this way, 'tiger'@'%' and 'tiger'@'localhost' are two users.
:::

:::
### Grant a user permissions
Execute the following SQL statement to grant user **tiger** the permissions to query all tables in database **test_db**:
```sql
Expand All @@ -123,4 +122,4 @@ show grants for 'tiger'@'%';
Execute the following SQL statement to drop user '**tiger'@'%'**:
```sql
drop user 'tiger'@'%';
```
```
Loading

0 comments on commit a11644b

Please sign in to comment.