Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

backup: change the location for s3-multi-part-size (#10786) #10791

Merged
merged 3 commits into from
Oct 12, 2022
Merged

backup: change the location for s3-multi-part-size (#10786) #10791

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d29408e
This is an automated cherry-pick of #10786
shichun-0415 Oct 12, 2022
97b7a7e
backup: change the location for s3-multi-part-size
shichun-0415 Oct 12, 2022
b54979f
Update tikv-configuration-file.md
shichun-0415 Oct 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions tikv-configuration-file.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1417,6 +1417,16 @@ Configuration items related to BR backup.
+ Default value: `MIN(CPU * 0.75, 32)`.
+ Minimum value: `1`

### `s3-multi-part-size` <span class="version-mark">New in v5.3.2</span>

> **Note:**
>
> This configuration is introduced to address backup failures caused by S3 rate limiting. This problem has been fixed by [refining the backup data storage structure](/br/backup-and-restore-design.md#backup-file-structure). Therefore, this configuration is deprecated from v6.1.1 and is no longer recommended.
shichun-0415 marked this conversation as resolved.
Show resolved Hide resolved

+ The part size used when you perform multipart upload to S3 during backup. You can adjust the value of this configuration to control the number of requests sent to S3.
+ If data is backed up to S3 and the backup file is larger than the value of this configuration item, [multipart upload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPart.html) is automatically enabled. Based on the compression ratio, the backup file generated by a 96-MiB Region is approximately 10 MiB to 30 MiB.
+ Default value: 5MiB

## cdc

Configuration items related to TiCDC.
Expand Down Expand Up @@ -1490,13 +1500,3 @@ For pessimistic transaction usage, refer to [TiDB Pessimistic Transaction Mode](

- This configuration item enables the pipelined process of adding the pessimistic lock. With this feature enabled, after detecting that data can be locked, TiKV immediately notifies TiDB to execute the subsequent requests and write the pessimistic lock asynchronously, which reduces most of the latency and significantly improves the performance of pessimistic transactions. But there is a still low probability that the asynchronous write of the pessimistic lock fails, which might cause the failure of pessimistic transaction commits.
- Default value: `true`

### `s3-multi-part-size` <span class="version-mark">New in v5.3.2</span>

> **Note:**
>
> This configuration is introduced to address backup failures caused by S3 rate limiting. This problem has been fixed by [refining the backup data storage structure](https://docs.pingcap.com/tidb/stable/backup-and-restore-design#backup-file-structure). Therefore, this configuration is deprecated from v6.1.1 and is no longer recommended.

+ The part size used when you perform multipart upload to S3 during backup. You can adjust the value of this configuration to control the number of requests sent to S3.
+ If data is backed up to S3 and the backup file is larger than the value of this configuration item, [multipart upload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPart.html) is automatically enabled. Based on the compression ratio, the backup file generated by a 96-MiB Region is approximately 10 MiB to 30 MiB.
+ Default value: 5MiB