Skip to content

Commit

Permalink
update
Browse files Browse the repository at this point in the history
  • Loading branch information
Yulei-Yang committed Oct 22, 2024
1 parent c9da20f commit 3d2709b
Show file tree
Hide file tree
Showing 20 changed files with 12 additions and 4,535 deletions.
11 changes: 6 additions & 5 deletions docs/table-design/temporary-table.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,14 @@ specific language governing permissions and limitations
under the License.
-->

:::note
Supported since Doris version 2.1.8 / 3.0.3
:::
When performing complex data processing tasks, breaking down large SQL queries into multiple steps and temporarily saving the results of each step as physical tables is an effective strategy. This method can significantly reduce the complexity of SQL queries and enhance data debuggability. However, it is important to note that these physical tables must be manually cleaned up after they have served their purpose. If non-physical temporary tables are preferred, Doris currently only supports defining them via the `WITH` clause.

To address the above issues, Doris introduces the temporary table feature. Temporary tables are temporarily existing materialized internal tables with the following key characteristics:
1. **Session Binding**: Temporary tables exist only within the session in which they were created. Their lifecycle is tightly bound to the current session, meaning that when the session ends, the temporary tables created within that session are automatically deleted.

When handling complex data processing, saving intermediate calculation results as physical tables is a great method to significantly reduce SQL complexity and improve data debuggability. However, such tables need to be manually cleaned up after use. Currently, Doris only supports defining non-physical temporary tables through the WITH clause.
2. **Session-specific Visibility**: The visibility of temporary tables is strictly confined to the session in which they were created. Even another session started by the same user at the same time cannot access these temporary tables.

To address this issue, Doris introduces the concept of temporary tables. A temporary table is a materialized internal table that exists temporarily and is primarily different from internal tables in that it only exists in the session that created it. Its lifecycle is bound to the current session, and the temporary table will be automatically deleted when the session ends. Moreover, the visibility of a temporary table is restricted to the session in which it was created, meaning it is not visible to another session of the same user at the same time.
By introducing the temporary table feature, Doris not only simplifies the temporary data storage and management in complex data processing but also further enhances the flexibility and security of data processing.


:::note
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,14 @@ specific language governing permissions and limitations
under the License.
-->

:::info 备注
自 2.1.8 / 3.0.3 版本开始支持
:::
在进行复杂的数据处理任务时,将大型 SQL 查询拆分为多个步骤,并将每个步骤的计算结果临时保存为实体表,是一种有效的策略。这种方法能够显著降低 SQL 查询的复杂度,并提升数据的可调试性。然而,需要注意的是,实体表在完成其使用目的后,需要手动进行清理。若选择使用非实体临时表,当前 Doris 仅支持通过 `WITH` 子句进行定义。

为了解决上述问题,Doris 引入了临时表功能。临时表是一种临时存在的物化内表,具备以下关键特性:
1. **会话绑定**:临时表仅存在于创建它的会话(Session)中。其生命周期与当前会话紧密绑定,即当会话结束时,该会话中创建的临时表会自动被删除。

在进行复杂数据处理时,将中间计算结果保存为实体表是一种可显著降低 SQL 复杂度、提高数据可调试性的好方法。但这种表需要在使用完毕之后手动清理。目前 Doris 仅支持通过 with 从句定义非实体临时表
2. **会话内可见性**:临时表的可见性严格限制在创建它的会话范围内。即使在同一时间由同一用户启动的另一个会话,也无法访问这些临时表

为了解决上述问题,Doris 引入临时表类型。临时表是一种临时存在的物化内表,和内表的主要区别是其仅存在于创建它的 Session中,它的生命周期和当前 Session 绑定。当会话结束时,在其中创建的临时表会被自动删除。从另一方面讲,临时表的可见性也仅在创建它的会话中,即使同一时间同一个用户的另一个会话也不可见
通过引入临时表功能,Doris 不仅简化了复杂数据处理过程中的临时数据存储与管理,还进一步增强了数据处理的灵活性和安全性


:::info 备注
Expand Down

This file was deleted.

This file was deleted.

Loading

0 comments on commit 3d2709b

Please sign in to comment.