Skip to content

Commit

Permalink
[docs](typo) Version mgt and fix typo (#1362)
Browse files Browse the repository at this point in the history
#1360
#1359
#1358
#1356
#1355
#1354
#1353
#1270
# Versions 

- [x] dev
- [x] 3.0
- [x] 2.1
- [x] 2.0

# Languages

- [x] Chinese
- [x] English
  • Loading branch information
KassieZ authored Nov 19, 2024
1 parent 2b574c6 commit 768e146
Show file tree
Hide file tree
Showing 50 changed files with 179 additions and 396 deletions.
2 changes: 0 additions & 2 deletions common_docs_zh/gettingStarted/what-is-new.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -31,5 +31,3 @@ import Latest from './demo-block/latest.tsx'





44 changes: 6 additions & 38 deletions community/join-community.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,38 +31,6 @@ We have graduated from Apache incubator successfully and become an Top-Level Pro

<hr />

## 🙌 More Developers Join Us
[![Monthly Active Contributors](https://contributor-overtime-api.apiseven.com/contributors-svg?chart=contributorMonthlyActivity&repo=apache/doris)](https://www.apiseven.com/en/contributor-graph?chart=contributorMonthlyActivity&repo=apache/doris)













[![Contributor over time](https://contributor-overtime-api.apiseven.com/contributors-svg?chart=contributorOverTime&repo=apache/doris)](https://www.apiseven.com/en/contributor-graph?chart=contributorOverTime&repo=apache/doris)

















## 🌟 More Stars on Github
<a href="https://star-history.com/#apache/doris&Date">
Expand All @@ -87,25 +55,25 @@ We have graduated from Apache incubator successfully and become an Top-Level Pro



##### We deeply appreciate 🔗[community contributors](https://github.com/apache/doris/graphs/contributors) for their contribution to Apache Doris.
**We deeply appreciate 🔗[community contributors](https://github.com/apache/doris/graphs/contributors) for their contribution to Apache Doris.**




<hr />

# Don't Miss Out the Latest News and Events
## Don't Miss Out the Latest News and Events

Learn our latest techniques, get inspirations from our rich use cases, and see what the community has been up to !


- ### Join our heated discussions - 💬 [Slack](https://join.slack.com/t/apachedoriscommunity/shared_invite/zt-2kl08hzc0-SPJe4VWmL_qzrFd2u2XYQA) 📇 [Github](https://github.com/apache/doris)
- Join our heated discussions - 💬 [Slack](https://join.slack.com/t/apachedoriscommunity/shared_invite/zt-2kl08hzc0-SPJe4VWmL_qzrFd2u2XYQA) 📇 [Github](https://github.com/apache/doris)

- ### Use cases and tech insight - 📭 [Twitter](https://twitter.com/doris_apache)
- Use cases and tech insight - 📭 [Twitter](https://twitter.com/doris_apache)

- ### Come and connect with us - 🌐 [LinkedIn](https://www.linkedin.com/company/doris-apache/)
- Come and connect with us - 🌐 [LinkedIn](https://www.linkedin.com/company/doris-apache/)

- ### Events Videos - ▶️ [YouTube](https://www.youtube.com/@Select_DB) 📺 [Bilibili](https://space.bilibili.com/362350065)
- Events Videos - ▶️ [YouTube](https://www.youtube.com/@Select_DB) 📺 [Bilibili](https://space.bilibili.com/362350065)



Expand Down
4 changes: 2 additions & 2 deletions docs/install/cluster-deployment/k8s-deploy/install-env.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,6 @@ sysctl -w vm.max_map_count=2000000
When deploying Doris, it is recommended to turn off transparent huge pages.

```shell
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo madvise > /sys/kernel/mm/transparent_hugepage/enabled
echo madvise > /sys/kernel/mm/transparent_hugepage/defrag
```
Original file line number Diff line number Diff line change
Expand Up @@ -28,17 +28,19 @@ under the License.

Doris supports the use of hints to adjust the type of data shuffle in Join operations, thereby optimizing query performance. This section will introduce in detail how to use hints in Doris to specify the type of Join shuffle.

Currently, Doris is limited to specifying the Distribute Type for the right table in a Join, offering two types to choose from: `[shuffle]` and `[broadcast]`. The Distribute Type should be placed before the right table in the Join, and can be represented using either square brackets `[]` or double slashes `//`.
Currently, Doris is limited to specifying the Distribute Type for the right table in a Join and provides only two types to choose from: `[shuffle]` and `[broadcast]`. The Distribute Type needs to be placed before the right table in the Join, using square brackets `[]`.


Examples are as follows:

```sql
SELECT COUNT(*) FROM t2 JOIN [broadcast] t1 ON t1.c1 = t2.c2;
SELECT COUNT(*) FROM t2 JOIN /*+broadcast*/ t1 ON t1.c1 = t2.c2;
SELECT COUNT(*) FROM t2 JOIN [shuffle] t1 ON t1.c1 = t2.c2;
SELECT /*+ ordered */ COUNT(*) FROM t2 JOIN [broadcast] t1 ON t1.c1 = t2.c2;
SELECT /*+ ordered */ COUNT(*) FROM t2 JOIN /+broadcast/ t1 ON t1.c1 = t2.c2;
SELECT /*+ ordered */ COUNT(*) FROM t2 JOIN [shuffle] t1 ON t1.c1 = t2.c2;
```


When using hints, please note the following:

1. If Doris encounters a DistributeHint that cannot correctly generate an execution plan, it will not display the hint but will make it effective according to the "best effort" principle. Ultimately, the distribute method displayed by EXPLAIN shall be deemed as final.
Expand Down Expand Up @@ -118,4 +120,8 @@ In the EXPLAIN results, you can see the relevant information for the distribute

## Summary

By properly using DistributeHint, you can optimize the shuffle method of Join operations and improve query performance. In practice, it is recommended to first analyze the query execution plan through EXPLAIN and then choose the appropriate shuffle type based on the actual situation.
By reasonably using DistributeHint, you can optimize the Shuffle method for Join operations and enhance query performance. In practice, it is recommended to first analyze the query execution plan using EXPLAIN and then select the appropriate Shuffle type based on the actual situation. When using it, please pay attention to the following points:

1. If a DistributeHint fails to correctly generate an execution plan, Doris will not display the Hint but will make it effective based on the principle of "best effort". Ultimately, the Distribute method displayed by EXPLAIN prevails.
2. In the current version, DistributeHint temporarily does not support mixing with LEADING, and the Hint will only take effect when the table specified by Distribute is located on the right side of the Join.
3. It is recommended to mix DistributeHint with ORDERED. First, use ORDERED to fix the Join order, and then specify the expected Distribute method for the corresponding Join.
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,14 @@ specific language governing permissions and limitations
under the License.
-->


Doris currently supports two types of indexes:

1. Built-in Indexes. These include prefix indexes, ZoneMap indexes, etc.
1. Built-in Indexes: These include prefix indexes, ZoneMap indexes, etc.

2. Secondary Indexes. These include inverted indexes, Bloom filter indexes, N-Gram Bloom filter indexes, and Bitmap indexes, etc.
2. Secondary Indexes: These include inverted indexes, Bloom filter indexes, N-Gram Bloom filter indexes, and Bitmap indexes, etc.

In the process of business optimization, fully analyzing business characteristics and effectively utilizing indexes can greatly enhance the effectiveness of queries and analyses, thereby achieving the purpose of performance tuning.
In the process of business optimization, fully analyzing business characteristics and make effective use of indexes can greatly enhance the effectiveness of queries and analyses, thereby achieving the purpose of performance tuning.

For a detailed introduction to various indexes, please refer to the [Table Index](../../../table-design/index/index-overview) section. This chapter will demonstrate index usage techniques in several typical scenarios from the perspective of actual cases and summarize optimization suggestions for reference in business tuning.

Expand Down Expand Up @@ -85,7 +86,7 @@ When defining the schema column order, reference the high-frequency and high-pri

Doris supports inverted indexes as secondary indexes to accelerate business scenarios such as equal value, range, and full-text search of text types. The creation and management of inverted indexes are independent, allowing for convenient business performance optimization without affecting the original table schema and without the need to re-import table data.

For typical usage scenarios, syntax, and cases, please refer to the Table Index - Inverted Index section for a detailed introduction, so this chapter will not repeat the explanation.
For typical usage scenarios, syntax, and cases, please refer to the [Table Index - Inverted Index](../../../table-design/index/inverted-index) section for a detailed introduction, so this chapter will not repeat the explanation.

:::tip

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ When estimating the filter rate, the optimizer typically relies on the assumptio
Considering the following SQL query:

```sql
select count()
select count(1)
from orders, customer
where o_custkey = c_custkey and o_orderdate < '1920-01-02'
```
Expand All @@ -67,7 +67,7 @@ To optimize the query, we need to adjust the SQL statement based on the actual s
Revising the SQL as follows:

```sql
select /* leading(orders customer) */ count()
select /* +leading(orders customer) */ count(1)
from orders, customer
where o_custkey = c_custkey and o_orderdate < '1920-01-02'
```
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,11 @@ Next, through an example, we will demonstrate in detail how to use aync-material

### 1 Creation of Base Tables and Data Insertion

First, create two tables, `orders` and `lineitem`, in the TPC-H database, and insert the corresponding data.
First, create two tables, `orders` and `lineitem`, in the tpch database, and insert the corresponding data.

```sql
CREATE DATABASE IF NOT EXISTS tpch;

USE tpch;

-- Create the orders table
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,44 +40,6 @@ Materialized views are suitable for the following scenarios:

For queries that frequently reuse the same subquery results, a sync-materialized view can significantly enhance performance. Doris automatically maintains the data in the materialized view, ensuring data consistency between the base table and the materialized view without additional manual maintenance costs. During queries, the system automatically matches the optimal materialized view and reads data directly from it.

When using materialized views, please note the following points:

1. In Doris version 2.0, materialized views have enhanced features. It is recommended that users confirm in a test environment whether the expected queries can hit the desired materialized views before using them in a formal production environment.

2. It is not advisable to create multiple similar materialized views on the same table, as this may lead to conflicts between them, resulting in query misses.

## Usage Process

The usage process for materialized views is as follows:

### 1 Create a Materialized View

1. Determine the type of materialized view to create based on the characteristics of the query statement.

2. Extract the common grouping and aggregation methods from multiple queries as the basis for defining the materialized view.

3. It is not necessary to create materialized views for all dimension combinations; only create them for commonly used dimension combinations.

4. Creating a materialized view is an asynchronous operation. After submitting the creation task, Doris will compute the existing data in the background until the creation is successful.

### 2 Automatic Query Matching

1. After the materialized view is successfully created, when a user queries the base table, Doris will automatically select an optimal materialized view and read data from it for computation.

2. Users can use the EXPLAIN command to check whether the current query is using a materialized view.

### 3 Update Strategy

To ensure data consistency between the materialized view and the base table, Doris synchronizes operations on the base table to the materialized view, using incremental updates to improve update efficiency and ensuring the atomicity of operations through transactions.

### 4 Supported Aggregation Functions

1. SUM, MIN, MAX (applicable to Version 0.12)

2. COUNT, BITMAP_UNION, HLL_UNION (applicable to Version 0.13)

3. General aggregation functions (applicable to Version 2.0)

## Tuning Usage Case

The following is a specific example to illustrate the use of single-table materialized views:
Expand Down
8 changes: 4 additions & 4 deletions gettingStarted/demo-block/latest.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ export default function Latest() {
</div>
</div> */}
<div className="home-page-hero-right">
<a className="latest-button" href="https://www.meetup.com/apache-doris-meetup/events/303712793/?utm_medium=referral&utm_campaign=share-btn_savedevents_share_modal&utm_source=link">
<a className="latest-button" href="https://www.velodb.io/events/meetup/singapore">
<div className="home-page-hero-button-label"><div>Event</div></div>
<div className="latest-button-title">
{/* <div className="home-page-hero-button-icon">
Expand All @@ -32,10 +32,10 @@ export default function Latest() {
<path fill="none" d="M0 0h24v24H0Z"></path>
</svg>
</div> */}
<div style={{ marginBottom: 10 }}>Apache Doris Meetup @ Singapore on Oct 24</div>
<div style={{ marginBottom: 10 }}>Apache Doris Meetup @ Singapore</div>
</div>
<div style={{ fontSize: 12, marginBottom: 20 }}>This meetup brings together our main developers, users to discuss various topics, including technology introductions and user experiences.</div>
<div style={{ fontSize: 14, marginBottom: 10 }}>Join us</div>
<div style={{ fontSize: 12, marginBottom: 20 }}>The Apache Doris community came together for an engaging meetup in Singapore to share insights, and explore the latest developments in the world of big data and analytics.</div>
<div style={{ fontSize: 14, marginBottom: 10 }}>Watch Video & Slides</div>
</a>
<a className="latest-button" href={`/docs${currentVersion === '' ? '' : `/${currentVersion}`}/releasenotes/v3.0/release-3.0.2`}>
<div className="home-page-hero-button-label"><div>Release</div></div>
Expand Down
6 changes: 3 additions & 3 deletions gettingStarted/what-is-apache-doris.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,15 @@ specific language governing permissions and limitations
under the License.
-->

# What's Apache Doris


Apache Doris is an MPP-based real-time data warehouse known for its high query speed. For queries on large datasets, it returns results in sub-seconds. It supports both high-concurrent point queries and high-throughput complex analysis. It can be used for report analysis, ad-hoc queries, unified data warehouse, and data lake query acceleration. Based on Apache Doris, users can build applications for user behavior analysis, A/B testing platform, log analysis, user profile analysis, and e-commerce order analysis.

Apache Doris, formerly known as Palo, was initially created to support Baidu's ad reporting business. It was officially open-sourced in 2017 and donated by Baidu to the Apache Software Foundation in July 2018, where it was operated by members of the incubator project management committee under the guidance of Apache mentors. In June 2022, Apache Doris graduated from the Apache incubator as a Top-Level Project. By 2024, the Apache Doris community has gathered more than 600 contributors from hundreds of companies in different industries, with over 120 monthly active contributors.

Apache Doris has a wide user base. It has been used in production environments of over 4000 companies worldwide, including giants such as TikTok, Baidu, Cisco, Tencent, and NetEase. It is also widely used across industries from finance, retailing, and telecommunications to energy, manufacturing, medical care, etc.

# Usage Scenarios
## Usage Scenarios

The figure below shows what Apache Doris can do in a data pipeline. Data sources, after integration and processing, are ingested into the Apache Doris real-time data warehouse and offline data lakehouses such as Hive, Iceberg, and Hudi. Apache Doris can be used for the following purposes:

Expand Down Expand Up @@ -88,7 +88,7 @@ The query engine of Apache Doris is fully vectorized, with all memory structures

![Query engine](/images/apache-doris-query-engine-2.png)

Apache Doris uses **adaptive query execution** technology to dynamically adjust the execution plan based on runtime statistics. For example, it can generate a runtime filter and push it to the probe side. Specifically, it pushes the filters to the lowest-level scan node on the probe side, which largely reduces the data amount to be processed and increases join performance. The runtime filter of Apache Doriz supports In/Min/Max/Bloom Filter.
Apache Doris uses **adaptive query execution** technology to dynamically adjust the execution plan based on runtime statistics. For example, it can generate a runtime filter and push it to the probe side. Specifically, it pushes the filters to the lowest-level scan node on the probe side, which largely reduces the data amount to be processed and increases join performance. The runtime filter of Apache Doris supports In/Min/Max/Bloom Filter.

The query **optimizer** of Apache Doris is a combination of CBO and RBO. RBO supports constant folding, subquery rewriting, and predicate pushdown while CBO supports join reorder. The Apache Doris CBO is under continuous optimization for more accurate statistics collection and inference as well as a more accurate cost model.

Original file line number Diff line number Diff line change
Expand Up @@ -31,39 +31,6 @@ under the License.




## 🙌 更多开发者加入我们

[![Monthly Active Contributors](https://contributor-overtime-api.apiseven.com/contributors-svg?chart=contributorMonthlyActivity&repo=apache/doris)](https://www.apiseven.com/en/contributor-graph?chart=contributorMonthlyActivity&repo=apache/doris)















[![Contributor over time](https://contributor-overtime-api.apiseven.com/contributors-svg?chart=contributorOverTime&repo=apache/doris)](https://www.apiseven.com/en/contributor-graph?chart=contributorOverTime&repo=apache/doris)













## 🌟 更多用户认可我们

<a href="https://star-history.com/#apache/doris&Date">
Expand All @@ -81,14 +48,6 @@ under the License.











**我们非常感谢 🔗[社区贡献者](https://github.com/apache/doris/graphs/contributors) 对 Apache Doris 的大力支持!**


Expand Down
Loading

0 comments on commit 768e146

Please sign in to comment.