diff --git a/src/UserGuide/Master/Table/User-Manual/Data-Sync_apache.md b/src/UserGuide/Master/Table/User-Manual/Data-Sync_apache.md index 6c04f4d1c..1ca2b9944 100644 --- a/src/UserGuide/Master/Table/User-Manual/Data-Sync_apache.md +++ b/src/UserGuide/Master/Table/User-Manual/Data-Sync_apache.md @@ -292,41 +292,7 @@ WITH SINK ( 'node-urls' = '127.0.0.1:6668' -- The URL of the DataNode's data service port in the target IoTDB instance. ) ``` -### 3.3 Bidirectional Data Transmission - -This example demonstrates a scenario where two IoTDB instances act as dual-active systems. The data pipeline is shown below: - -![](/img/e3.png) - -To avoid infinite data loops, the `source.mode.double-living` parameter must be set to `true` on both IoTDB A and B, indicating that data forwarded from another pipe will not be retransmitted. - -SQL Example: On IoTDB A: - -```SQL -CREATE PIPE AB -WITH SOURCE ( - 'source.mode.double-living' = 'true' -- Do not forward data from other pipes -) -WITH SINK ( - 'sink' = 'iotdb-thrift-sink', - 'node-urls' = '127.0.0.1:6668' -- URL of the DataNode service port on the target IoTDB -) -``` - -On IoTDB B: - -```SQL -CREATE PIPE BA -WITH SOURCE ( - 'source.mode.double-living' = 'true' -- Do not forward data from other pipes -) -WITH SINK ( - 'sink' = 'iotdb-thrift-sink', - 'node-urls' = '127.0.0.1:6667' -- URL of the DataNode service port on the target IoTDB -) -``` - -### 3.4 Edge-to-Cloud Data Transmission +### 3.3 Edge-to-Cloud Data Transmission This example demonstrates synchronizing data from multiple IoTDB clusters (B, C, D) to a central IoTDB cluster (A). The data pipeline is shown below: @@ -376,14 +342,13 @@ WITH SINK ( ) ``` -### 3.5 Cascaded Data Transmission +### 3.4 Cascaded Data Transmission This example demonstrates cascading data transmission from IoTDB A to IoTDB B and then to IoTDB C. The data pipeline is shown below: ![](/img/sync_en_04.png) -To synchronize data from cluster A to cluster C, the `source.mode.double-living` parameter is set to `true` in the pipe between B and C. - + SQL Example: On IoTDB A: ```SQL @@ -407,7 +372,7 @@ WITH SINK ( ``` -### 3.6 Compressed Synchronization +### 3.5 Compressed Synchronization IoTDB supports specifying data compression methods during synchronization. The `compressor` parameter can be configured to enable real-time data compression and transmission. Supported algorithms include `snappy`, `gzip`, `lz4`, `zstd`, and `lzma2`. Multiple algorithms can be combined and applied in the configured order. The `rate-limit-bytes-per-second` parameter (supported in V1.3.3 and later) limits the maximum number of bytes transmitted per second (calculated after compression). If set to a value less than 0, there is no limit. @@ -423,7 +388,7 @@ WITH SINK ( ``` -### 3.7 Encrypted Synchronization +### 3.6 Encrypted Synchronization IoTDB supports SSL encryption during synchronization to securely transmit data between IoTDB instances. By configuring SSL-related parameters such as the certificate path (`ssl.trust-store-path`) and password (`ssl.trust-store-pwd`), data can be protected by SSL encryption during synchronization. @@ -519,7 +484,6 @@ pipe_all_sinks_rate_limit_bytes_per_second=-1 | table-name | When the user connects with `sql_dialect` set to `table`, this parameter can be specified. Determines the scope of data capture, affecting the `data` in `inclusion`. Specifies the table name to filter. It can be a specific table name or a Java-style regular expression to match multiple tables. By default, all tables are matched. | String: Data table name or data table regular expression pattern string, which can be uncreated or non - existent tables. | No | ".*" | | start-time | Determines the scope of data capture, affecting the `data` in `inclusion`. Data with an event time **greater than or equal to** this parameter will be selected for stream processing in the pipe. | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp)orString: ISO format timestamp supported by IoTDB | No | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp) | | end-time | Determines the scope of data capture, affecting the `data` in `inclusion`. Data with an event time **less than or equal to** this parameter will be selected for stream processing in the pipe. | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp)orString: ISO format timestamp supported by IoTDB | No | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp) | -| mode.double-living | Whether to enable full dual-active mode. When enabled, the system will ignore the `-sql_dialect` connection method to capture all tree-table model data and not forward data synced from another pipe (to avoid circular synchronization). | Boolean: true / false | No | false | > 💎 **Note:** The difference between the values of true and false for the data extraction mode `mode.streaming` > diff --git a/src/UserGuide/Master/Table/User-Manual/Data-Sync_timecho.md b/src/UserGuide/Master/Table/User-Manual/Data-Sync_timecho.md index 1e07c55f0..f522c00f1 100644 --- a/src/UserGuide/Master/Table/User-Manual/Data-Sync_timecho.md +++ b/src/UserGuide/Master/Table/User-Manual/Data-Sync_timecho.md @@ -393,8 +393,7 @@ This example demonstrates cascading data transmission from IoTDB A to IoTDB B an ![](/img/sync_en_04.png) -To synchronize data from cluster A to cluster C, the `source.mode.double-living` parameter is set to `true` in the pipe between B and C. - + SQL Example: On IoTDB A: ```SQL diff --git a/src/UserGuide/latest-Table/User-Manual/Data-Sync_apache.md b/src/UserGuide/latest-Table/User-Manual/Data-Sync_apache.md index 6c04f4d1c..2dafb162c 100644 --- a/src/UserGuide/latest-Table/User-Manual/Data-Sync_apache.md +++ b/src/UserGuide/latest-Table/User-Manual/Data-Sync_apache.md @@ -292,41 +292,7 @@ WITH SINK ( 'node-urls' = '127.0.0.1:6668' -- The URL of the DataNode's data service port in the target IoTDB instance. ) ``` -### 3.3 Bidirectional Data Transmission - -This example demonstrates a scenario where two IoTDB instances act as dual-active systems. The data pipeline is shown below: - -![](/img/e3.png) - -To avoid infinite data loops, the `source.mode.double-living` parameter must be set to `true` on both IoTDB A and B, indicating that data forwarded from another pipe will not be retransmitted. - -SQL Example: On IoTDB A: - -```SQL -CREATE PIPE AB -WITH SOURCE ( - 'source.mode.double-living' = 'true' -- Do not forward data from other pipes -) -WITH SINK ( - 'sink' = 'iotdb-thrift-sink', - 'node-urls' = '127.0.0.1:6668' -- URL of the DataNode service port on the target IoTDB -) -``` - -On IoTDB B: - -```SQL -CREATE PIPE BA -WITH SOURCE ( - 'source.mode.double-living' = 'true' -- Do not forward data from other pipes -) -WITH SINK ( - 'sink' = 'iotdb-thrift-sink', - 'node-urls' = '127.0.0.1:6667' -- URL of the DataNode service port on the target IoTDB -) -``` - -### 3.4 Edge-to-Cloud Data Transmission +### 3.3 Edge-to-Cloud Data Transmission This example demonstrates synchronizing data from multiple IoTDB clusters (B, C, D) to a central IoTDB cluster (A). The data pipeline is shown below: @@ -376,13 +342,12 @@ WITH SINK ( ) ``` -### 3.5 Cascaded Data Transmission +### 3.4 Cascaded Data Transmission This example demonstrates cascading data transmission from IoTDB A to IoTDB B and then to IoTDB C. The data pipeline is shown below: ![](/img/sync_en_04.png) -To synchronize data from cluster A to cluster C, the `source.mode.double-living` parameter is set to `true` in the pipe between B and C. SQL Example: On IoTDB A: @@ -407,7 +372,7 @@ WITH SINK ( ``` -### 3.6 Compressed Synchronization +### 3.5 Compressed Synchronization IoTDB supports specifying data compression methods during synchronization. The `compressor` parameter can be configured to enable real-time data compression and transmission. Supported algorithms include `snappy`, `gzip`, `lz4`, `zstd`, and `lzma2`. Multiple algorithms can be combined and applied in the configured order. The `rate-limit-bytes-per-second` parameter (supported in V1.3.3 and later) limits the maximum number of bytes transmitted per second (calculated after compression). If set to a value less than 0, there is no limit. @@ -423,7 +388,7 @@ WITH SINK ( ``` -### 3.7 Encrypted Synchronization +### 3.6 Encrypted Synchronization IoTDB supports SSL encryption during synchronization to securely transmit data between IoTDB instances. By configuring SSL-related parameters such as the certificate path (`ssl.trust-store-path`) and password (`ssl.trust-store-pwd`), data can be protected by SSL encryption during synchronization. @@ -519,7 +484,6 @@ pipe_all_sinks_rate_limit_bytes_per_second=-1 | table-name | When the user connects with `sql_dialect` set to `table`, this parameter can be specified. Determines the scope of data capture, affecting the `data` in `inclusion`. Specifies the table name to filter. It can be a specific table name or a Java-style regular expression to match multiple tables. By default, all tables are matched. | String: Data table name or data table regular expression pattern string, which can be uncreated or non - existent tables. | No | ".*" | | start-time | Determines the scope of data capture, affecting the `data` in `inclusion`. Data with an event time **greater than or equal to** this parameter will be selected for stream processing in the pipe. | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp)orString: ISO format timestamp supported by IoTDB | No | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp) | | end-time | Determines the scope of data capture, affecting the `data` in `inclusion`. Data with an event time **less than or equal to** this parameter will be selected for stream processing in the pipe. | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp)orString: ISO format timestamp supported by IoTDB | No | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp) | -| mode.double-living | Whether to enable full dual-active mode. When enabled, the system will ignore the `-sql_dialect` connection method to capture all tree-table model data and not forward data synced from another pipe (to avoid circular synchronization). | Boolean: true / false | No | false | > 💎 **Note:** The difference between the values of true and false for the data extraction mode `mode.streaming` > diff --git a/src/UserGuide/latest-Table/User-Manual/Data-Sync_timecho.md b/src/UserGuide/latest-Table/User-Manual/Data-Sync_timecho.md index 1e07c55f0..8ff1773bb 100644 --- a/src/UserGuide/latest-Table/User-Manual/Data-Sync_timecho.md +++ b/src/UserGuide/latest-Table/User-Manual/Data-Sync_timecho.md @@ -393,7 +393,6 @@ This example demonstrates cascading data transmission from IoTDB A to IoTDB B an ![](/img/sync_en_04.png) -To synchronize data from cluster A to cluster C, the `source.mode.double-living` parameter is set to `true` in the pipe between B and C. SQL Example: On IoTDB A: diff --git a/src/zh/UserGuide/Master/Table/User-Manual/Data-Sync_apache.md b/src/zh/UserGuide/Master/Table/User-Manual/Data-Sync_apache.md index 34fbaa961..599a10aa0 100644 --- a/src/zh/UserGuide/Master/Table/User-Manual/Data-Sync_apache.md +++ b/src/zh/UserGuide/Master/Table/User-Manual/Data-Sync_apache.md @@ -286,42 +286,7 @@ with SINK ( ) ``` -### 3.3 双向数据传输 - -本例子用来演示两个 IoTDB 之间互为双活的场景，数据链路如下图所示： - -![](/img/1706698592139.jpg) - -在这个例子中，为了避免数据无限循环，需要将 A 和 B 上的参数`source.mode.double-living` 均设置为 `true`，表示不转发从另一 pipe 传输而来的数据。 - -详细语句如下： - -在 A IoTDB 上执行下列语句： - -```SQL -create pipe AB -with source ( - 'source.mode.double-living' ='true' --不转发由其他 Pipe 写入的数据 -) -with sink ( - 'sink'='iotdb-thrift-sink', - 'node-urls' = '127.0.0.1:6668', -- 目标端 IoTDB 中 DataNode 节点的数据服务端口的 url -) -``` - -在 B IoTDB 上执行下列语句： - -```SQL -create pipe BA -with source ( - 'source.mode.double-living' ='true' --不转发由其他 Pipe 写入的数据 -) -with sink ( - 'sink'='iotdb-thrift-sink', - 'node-urls' = '127.0.0.1:6667', -- 目标端 IoTDB 中 DataNode 节点的数据服务端口的 url -) -``` -### 3.4 边云数据传输 +### 3.3 边云数据传输 本例子用来演示多个 IoTDB 之间边云传输数据的场景，数据由 B 、C、D 集群分别都同步至 A 集群，数据链路如下图所示： @@ -371,13 +336,12 @@ with sink ( ) ``` -### 3.5 级联数据传输 +### 3.4 级联数据传输 本例子用来演示多个 IoTDB 之间级联传输数据的场景，数据由 A 集群同步至 B 集群，再同步至 C 集群，数据链路如下图所示： ![](/img/1706698610134.jpg) -在这个例子中，为了将 A 集群的数据同步至 C，在 BC 之间的 pipe 需要将 `source.mode.double-living` 配置为`true`，详细语句如下： 在 A IoTDB 上执行下列语句，将 A 中数据同步至 B： @@ -401,7 +365,7 @@ with sink ( ) ``` -### 3.6 压缩同步 +### 3.5 压缩同步 IoTDB 支持在同步过程中指定数据压缩方式。可通过配置 `compressor` 参数，实现数据的实时压缩和传输。`compressor`目前支持 snappy / gzip / lz4 / zstd / lzma2 5 种可选算法，且可以选择多种压缩算法组合，按配置的顺序进行压缩。`rate-limit-bytes-per-second`（V1.3.3 及以后版本支持）每秒最大允许传输的byte数,计算压缩后的byte，若小于0则不限制。 @@ -417,7 +381,7 @@ with sink ( ``` -### 3.7 加密同步 +### 3.6 加密同步 IoTDB 支持在同步过程中使用 SSL 加密，从而在不同的 IoTDB 实例之间安全地传输数据。通过配置 SSL 相关的参数，如证书地址和密码（`ssl.trust-store-path`）、（`ssl.trust-store-pwd`）可以确保数据在同步过程中被 SSL 加密所保护。 @@ -511,7 +475,6 @@ pipe_all_sinks_rate_limit_bytes_per_second=-1 | table-name | 当用户连接指定的 sql_dialect 为 table 时可以指定。此参数决定时序数据的捕获范围，影响`inclusion`中的`data`数据。表示要过滤的表的名称。它可以是具体的表名，也可以是 Java 风格正则表达式来匹配多个表。默认情况下，匹配所有的表。 | String：数据表名或数据表正则模式串，可以是未创建的、不存在的表 | 否 | ".*" | | start-time | 此参数决定时序数据的捕获范围，影响`inclusion`中的`data`数据。当数据的 event time 大于等于该参数时，数据会被筛选出来进入流处理 pipe。 | Long: [Long.MIN_VALUE， Long.MAX_VALUE] （unix 裸时间戳）或 String：IoTDB 支持的 ISO 格式时间戳 | 否 | Long.MIN_VALUE（unix 裸时间戳） | | end-time | 此参数决定时序数据的捕获范围，影响`inclusion`中的`data`数据。当数据的 event time 小于等于该参数时，数据会被筛选出来进入流处理 pipe。 | Long: [Long.MIN_VALUE, Long.MAX_VALUE]（unix 裸时间戳）或String：IoTDB 支持的 ISO 格式时间戳 | 否 | Long.MAX_VALUE（unix 裸时间戳） | -| mode.double-living | 是否开启全量双活模式，开启后将忽略`-sql_dialect`连接方式，树表模型数据均会被捕获，且不会转发由另一pipe同步而来的数据。 | Boolean: true / false | 否 | false | > 💎 **说明：数据抽取模式 mode.streaming 取值 true 和 false 的差异** > - **true（推荐）**：该取值下，任务将对数据进行实时处理、发送，其特点是高时效、低吞吐 diff --git a/src/zh/UserGuide/Master/Table/User-Manual/Data-Sync_timecho.md b/src/zh/UserGuide/Master/Table/User-Manual/Data-Sync_timecho.md index f814c76b1..fbc378d08 100644 --- a/src/zh/UserGuide/Master/Table/User-Manual/Data-Sync_timecho.md +++ b/src/zh/UserGuide/Master/Table/User-Manual/Data-Sync_timecho.md @@ -391,8 +391,6 @@ with sink ( ![](/img/1706698610134.jpg) -在这个例子中，为了将 A 集群的数据同步至 C，在 BC 之间的 pipe 需要将 `source.mode.double-living` 配置为`true`，详细语句如下： - 在 A IoTDB 上执行下列语句，将 A 中数据同步至 B： ```SQL diff --git a/src/zh/UserGuide/latest-Table/User-Manual/Data-Sync_apache.md b/src/zh/UserGuide/latest-Table/User-Manual/Data-Sync_apache.md index 34fbaa961..599a10aa0 100644 --- a/src/zh/UserGuide/latest-Table/User-Manual/Data-Sync_apache.md +++ b/src/zh/UserGuide/latest-Table/User-Manual/Data-Sync_apache.md @@ -286,42 +286,7 @@ with SINK ( ) ``` -### 3.3 双向数据传输 - -本例子用来演示两个 IoTDB 之间互为双活的场景，数据链路如下图所示： - -![](/img/1706698592139.jpg) - -在这个例子中，为了避免数据无限循环，需要将 A 和 B 上的参数`source.mode.double-living` 均设置为 `true`，表示不转发从另一 pipe 传输而来的数据。 - -详细语句如下： - -在 A IoTDB 上执行下列语句： - -```SQL -create pipe AB -with source ( - 'source.mode.double-living' ='true' --不转发由其他 Pipe 写入的数据 -) -with sink ( - 'sink'='iotdb-thrift-sink', - 'node-urls' = '127.0.0.1:6668', -- 目标端 IoTDB 中 DataNode 节点的数据服务端口的 url -) -``` - -在 B IoTDB 上执行下列语句： - -```SQL -create pipe BA -with source ( - 'source.mode.double-living' ='true' --不转发由其他 Pipe 写入的数据 -) -with sink ( - 'sink'='iotdb-thrift-sink', - 'node-urls' = '127.0.0.1:6667', -- 目标端 IoTDB 中 DataNode 节点的数据服务端口的 url -) -``` -### 3.4 边云数据传输 +### 3.3 边云数据传输 本例子用来演示多个 IoTDB 之间边云传输数据的场景，数据由 B 、C、D 集群分别都同步至 A 集群，数据链路如下图所示： @@ -371,13 +336,12 @@ with sink ( ) ``` -### 3.5 级联数据传输 +### 3.4 级联数据传输 本例子用来演示多个 IoTDB 之间级联传输数据的场景，数据由 A 集群同步至 B 集群，再同步至 C 集群，数据链路如下图所示： ![](/img/1706698610134.jpg) -在这个例子中，为了将 A 集群的数据同步至 C，在 BC 之间的 pipe 需要将 `source.mode.double-living` 配置为`true`，详细语句如下： 在 A IoTDB 上执行下列语句，将 A 中数据同步至 B： @@ -401,7 +365,7 @@ with sink ( ) ``` -### 3.6 压缩同步 +### 3.5 压缩同步 IoTDB 支持在同步过程中指定数据压缩方式。可通过配置 `compressor` 参数，实现数据的实时压缩和传输。`compressor`目前支持 snappy / gzip / lz4 / zstd / lzma2 5 种可选算法，且可以选择多种压缩算法组合，按配置的顺序进行压缩。`rate-limit-bytes-per-second`（V1.3.3 及以后版本支持）每秒最大允许传输的byte数,计算压缩后的byte，若小于0则不限制。 @@ -417,7 +381,7 @@ with sink ( ``` -### 3.7 加密同步 +### 3.6 加密同步 IoTDB 支持在同步过程中使用 SSL 加密，从而在不同的 IoTDB 实例之间安全地传输数据。通过配置 SSL 相关的参数，如证书地址和密码（`ssl.trust-store-path`）、（`ssl.trust-store-pwd`）可以确保数据在同步过程中被 SSL 加密所保护。 @@ -511,7 +475,6 @@ pipe_all_sinks_rate_limit_bytes_per_second=-1 | table-name | 当用户连接指定的 sql_dialect 为 table 时可以指定。此参数决定时序数据的捕获范围，影响`inclusion`中的`data`数据。表示要过滤的表的名称。它可以是具体的表名，也可以是 Java 风格正则表达式来匹配多个表。默认情况下，匹配所有的表。 | String：数据表名或数据表正则模式串，可以是未创建的、不存在的表 | 否 | ".*" | | start-time | 此参数决定时序数据的捕获范围，影响`inclusion`中的`data`数据。当数据的 event time 大于等于该参数时，数据会被筛选出来进入流处理 pipe。 | Long: [Long.MIN_VALUE， Long.MAX_VALUE] （unix 裸时间戳）或 String：IoTDB 支持的 ISO 格式时间戳 | 否 | Long.MIN_VALUE（unix 裸时间戳） | | end-time | 此参数决定时序数据的捕获范围，影响`inclusion`中的`data`数据。当数据的 event time 小于等于该参数时，数据会被筛选出来进入流处理 pipe。 | Long: [Long.MIN_VALUE, Long.MAX_VALUE]（unix 裸时间戳）或String：IoTDB 支持的 ISO 格式时间戳 | 否 | Long.MAX_VALUE（unix 裸时间戳） | -| mode.double-living | 是否开启全量双活模式，开启后将忽略`-sql_dialect`连接方式，树表模型数据均会被捕获，且不会转发由另一pipe同步而来的数据。 | Boolean: true / false | 否 | false | > 💎 **说明：数据抽取模式 mode.streaming 取值 true 和 false 的差异** > - **true（推荐）**：该取值下，任务将对数据进行实时处理、发送，其特点是高时效、低吞吐 diff --git a/src/zh/UserGuide/latest-Table/User-Manual/Data-Sync_timecho.md b/src/zh/UserGuide/latest-Table/User-Manual/Data-Sync_timecho.md index f814c76b1..926303015 100644 --- a/src/zh/UserGuide/latest-Table/User-Manual/Data-Sync_timecho.md +++ b/src/zh/UserGuide/latest-Table/User-Manual/Data-Sync_timecho.md @@ -391,7 +391,6 @@ with sink ( ![](/img/1706698610134.jpg) -在这个例子中，为了将 A 集群的数据同步至 C，在 BC 之间的 pipe 需要将 `source.mode.double-living` 配置为`true`，详细语句如下： 在 A IoTDB 上执行下列语句，将 A 中数据同步至 B：