Skip to content

remove bidirectional examples in table model apache #852

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

remove bidirectional examples in table model apache #852

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
adfe0c9
remove bidirectional examples in table model apache
leto-b Aug 15, 2025
3513271
remove double-living from apache
leto-b Oct 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 5 additions & 41 deletions src/UserGuide/Master/Table/User-Manual/Data-Sync_apache.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -292,41 +292,7 @@ WITH SINK (
'node-urls' = '127.0.0.1:6668' -- The URL of the DataNode's data service port in the target IoTDB instance.
)
```
### 3.3 Bidirectional Data Transmission

This example demonstrates a scenario where two IoTDB instances act as dual-active systems. The data pipeline is shown below:

![](/img/e3.png)

To avoid infinite data loops, the `source.mode.double-living` parameter must be set to `true` on both IoTDB A and B, indicating that data forwarded from another pipe will not be retransmitted.

SQL Example: On IoTDB A:

```SQL
CREATE PIPE AB
WITH SOURCE (
'source.mode.double-living' = 'true' -- Do not forward data from other pipes
)
WITH SINK (
'sink' = 'iotdb-thrift-sink',
'node-urls' = '127.0.0.1:6668' -- URL of the DataNode service port on the target IoTDB
)
```

On IoTDB B:

```SQL
CREATE PIPE BA
WITH SOURCE (
'source.mode.double-living' = 'true' -- Do not forward data from other pipes
)
WITH SINK (
'sink' = 'iotdb-thrift-sink',
'node-urls' = '127.0.0.1:6667' -- URL of the DataNode service port on the target IoTDB
)
```

### 3.4 Edge-to-Cloud Data Transmission
### 3.3 Edge-to-Cloud Data Transmission

This example demonstrates synchronizing data from multiple IoTDB clusters (B, C, D) to a central IoTDB cluster (A). The data pipeline is shown below:

Expand Down Expand Up @@ -376,14 +342,13 @@ WITH SINK (
)
```

### 3.5 Cascaded Data Transmission
### 3.4 Cascaded Data Transmission

This example demonstrates cascading data transmission from IoTDB A to IoTDB B and then to IoTDB C. The data pipeline is shown below:

![](/img/sync_en_04.png)

To synchronize data from cluster A to cluster C, the `source.mode.double-living` parameter is set to `true` in the pipe between B and C.


SQL Example: On IoTDB A:

```SQL
Expand All @@ -407,7 +372,7 @@ WITH SINK (
```


### 3.6 Compressed Synchronization
### 3.5 Compressed Synchronization

IoTDB supports specifying data compression methods during synchronization. The `compressor` parameter can be configured to enable real-time data compression and transmission. Supported algorithms include `snappy`, `gzip`, `lz4`, `zstd`, and `lzma2`. Multiple algorithms can be combined and applied in the configured order. The `rate-limit-bytes-per-second` parameter (supported in V1.3.3 and later) limits the maximum number of bytes transmitted per second (calculated after compression). If set to a value less than 0, there is no limit.

Expand All @@ -423,7 +388,7 @@ WITH SINK (
```


### 3.7 Encrypted Synchronization
### 3.6 Encrypted Synchronization

IoTDB supports SSL encryption during synchronization to securely transmit data between IoTDB instances. By configuring SSL-related parameters such as the certificate path (`ssl.trust-store-path`) and password (`ssl.trust-store-pwd`), data can be protected by SSL encryption during synchronization.

Expand Down Expand Up @@ -519,7 +484,6 @@ pipe_all_sinks_rate_limit_bytes_per_second=-1
| table-name | When the user connects with `sql_dialect` set to `table`, this parameter can be specified. Determines the scope of data capture, affecting the `data` in `inclusion`. Specifies the table name to filter. It can be a specific table name or a Java-style regular expression to match multiple tables. By default, all tables are matched. | String: Data table name or data table regular expression pattern string, which can be uncreated or non - existent tables. | No | ".*" |
| start-time | Determines the scope of data capture, affecting the `data` in `inclusion`. Data with an event time **greater than or equal to** this parameter will be selected for stream processing in the pipe. | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp)orString: ISO format timestamp supported by IoTDB | No | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp) |
| end-time | Determines the scope of data capture, affecting the `data` in `inclusion`. Data with an event time **less than or equal to** this parameter will be selected for stream processing in the pipe. | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp)orString: ISO format timestamp supported by IoTDB | No | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp) |
| mode.double-living | Whether to enable full dual-active mode. When enabled, the system will ignore the `-sql_dialect` connection method to capture all tree-table model data and not forward data synced from another pipe (to avoid circular synchronization). | Boolean: true / false | No | false |

> 💎 **Note:** The difference between the values of true and false for the data extraction mode `mode.streaming`
>
Expand Down
3 changes: 1 addition & 2 deletions src/UserGuide/Master/Table/User-Manual/Data-Sync_timecho.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -393,8 +393,7 @@ This example demonstrates cascading data transmission from IoTDB A to IoTDB B an

![](/img/sync_en_04.png)

To synchronize data from cluster A to cluster C, the `source.mode.double-living` parameter is set to `true` in the pipe between B and C.


SQL Example: On IoTDB A:

```SQL
Expand Down
44 changes: 4 additions & 40 deletions src/UserGuide/latest-Table/User-Manual/Data-Sync_apache.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -292,41 +292,7 @@ WITH SINK (
'node-urls' = '127.0.0.1:6668' -- The URL of the DataNode's data service port in the target IoTDB instance.
)
```
### 3.3 Bidirectional Data Transmission

This example demonstrates a scenario where two IoTDB instances act as dual-active systems. The data pipeline is shown below:

![](/img/e3.png)

To avoid infinite data loops, the `source.mode.double-living` parameter must be set to `true` on both IoTDB A and B, indicating that data forwarded from another pipe will not be retransmitted.

SQL Example: On IoTDB A:

```SQL
CREATE PIPE AB
WITH SOURCE (
'source.mode.double-living' = 'true' -- Do not forward data from other pipes
)
WITH SINK (
'sink' = 'iotdb-thrift-sink',
'node-urls' = '127.0.0.1:6668' -- URL of the DataNode service port on the target IoTDB
)
```

On IoTDB B:

```SQL
CREATE PIPE BA
WITH SOURCE (
'source.mode.double-living' = 'true' -- Do not forward data from other pipes
)
WITH SINK (
'sink' = 'iotdb-thrift-sink',
'node-urls' = '127.0.0.1:6667' -- URL of the DataNode service port on the target IoTDB
)
```

### 3.4 Edge-to-Cloud Data Transmission
### 3.3 Edge-to-Cloud Data Transmission

This example demonstrates synchronizing data from multiple IoTDB clusters (B, C, D) to a central IoTDB cluster (A). The data pipeline is shown below:

Expand Down Expand Up @@ -376,13 +342,12 @@ WITH SINK (
)
```

### 3.5 Cascaded Data Transmission
### 3.4 Cascaded Data Transmission

This example demonstrates cascading data transmission from IoTDB A to IoTDB B and then to IoTDB C. The data pipeline is shown below:

![](/img/sync_en_04.png)

To synchronize data from cluster A to cluster C, the `source.mode.double-living` parameter is set to `true` in the pipe between B and C.

SQL Example: On IoTDB A:

Expand All @@ -407,7 +372,7 @@ WITH SINK (
```


### 3.6 Compressed Synchronization
### 3.5 Compressed Synchronization

IoTDB supports specifying data compression methods during synchronization. The `compressor` parameter can be configured to enable real-time data compression and transmission. Supported algorithms include `snappy`, `gzip`, `lz4`, `zstd`, and `lzma2`. Multiple algorithms can be combined and applied in the configured order. The `rate-limit-bytes-per-second` parameter (supported in V1.3.3 and later) limits the maximum number of bytes transmitted per second (calculated after compression). If set to a value less than 0, there is no limit.

Expand All @@ -423,7 +388,7 @@ WITH SINK (
```


### 3.7 Encrypted Synchronization
### 3.6 Encrypted Synchronization

IoTDB supports SSL encryption during synchronization to securely transmit data between IoTDB instances. By configuring SSL-related parameters such as the certificate path (`ssl.trust-store-path`) and password (`ssl.trust-store-pwd`), data can be protected by SSL encryption during synchronization.

Expand Down Expand Up @@ -519,7 +484,6 @@ pipe_all_sinks_rate_limit_bytes_per_second=-1
| table-name | When the user connects with `sql_dialect` set to `table`, this parameter can be specified. Determines the scope of data capture, affecting the `data` in `inclusion`. Specifies the table name to filter. It can be a specific table name or a Java-style regular expression to match multiple tables. By default, all tables are matched. | String: Data table name or data table regular expression pattern string, which can be uncreated or non - existent tables. | No | ".*" |
| start-time | Determines the scope of data capture, affecting the `data` in `inclusion`. Data with an event time **greater than or equal to** this parameter will be selected for stream processing in the pipe. | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp)orString: ISO format timestamp supported by IoTDB | No | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp) |
| end-time | Determines the scope of data capture, affecting the `data` in `inclusion`. Data with an event time **less than or equal to** this parameter will be selected for stream processing in the pipe. | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp)orString: ISO format timestamp supported by IoTDB | No | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp) |
| mode.double-living | Whether to enable full dual-active mode. When enabled, the system will ignore the `-sql_dialect` connection method to capture all tree-table model data and not forward data synced from another pipe (to avoid circular synchronization). | Boolean: true / false | No | false |

> 💎 **Note:** The difference between the values of true and false for the data extraction mode `mode.streaming`
>
Expand Down
1 change: 0 additions & 1 deletion src/UserGuide/latest-Table/User-Manual/Data-Sync_timecho.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -393,7 +393,6 @@ This example demonstrates cascading data transmission from IoTDB A to IoTDB B an

![](/img/sync_en_04.png)

To synchronize data from cluster A to cluster C, the `source.mode.double-living` parameter is set to `true` in the pipe between B and C.

SQL Example: On IoTDB A:

Expand Down
Loading