Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

backup: change the location for s3-multi-part-size (#10786) #10787

Merged
merged 3 commits into from
Oct 12, 2022
Merged

backup: change the location for s3-multi-part-size (#10786) #10787

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4dfa983
This is an automated cherry-pick of #10786
shichun-0415 Oct 12, 2022
8da09ae
resolve conflict
shichun-0415 Oct 12, 2022
1b68511
Update tikv-configuration-file.md
shichun-0415 Oct 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions tikv-configuration-file.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1507,6 +1507,16 @@ Configuration items related to BR backup.
+ Controls whether to limit the resources used by backup tasks to reduce the impact on the cluster when the cluster resource utilization is high. For more information, refer to [BR Auto-Tune](/br/br-auto-tune.md).
+ Default value: `true`

### `s3-multi-part-size` <span class="version-mark">New in v5.3.2</span>

> **Note:**
>
> This configuration is introduced to address backup failures caused by S3 rate limiting. This problem has been fixed by [refining the backup data storage structure](https://docs.pingcap.com/tidb/stable/backup-and-restore-design#backup-file-structure). Therefore, this configuration is deprecated from v6.1.1 and is no longer recommended.

+ The part size used when you perform multipart upload to S3 during backup. You can adjust the value of this configuration to control the number of requests sent to S3.
+ If data is backed up to S3 and the backup file is larger than the value of this configuration item, [multipart upload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPart.html) is automatically enabled. Based on the compression ratio, the backup file generated by a 96-MiB Region is approximately 10 MiB to 30 MiB.
+ Default value: 5MiB

## cdc

Configuration items related to TiCDC.
Expand Down Expand Up @@ -1618,13 +1628,3 @@ Suppose that your machine on which TiKV is deployed has limited resources, for e

+ The maximum time that a single read or write request is forced to wait before it is processed in the foreground.
+ Default value: `500ms`

### `s3-multi-part-size` <span class="version-mark">New in v5.3.2</span>

> **Note:**
>
> This configuration is introduced to address backup failures caused by S3 rate limiting. This problem has been fixed by [refining the backup data storage structure](https://docs.pingcap.com/tidb/stable/backup-and-restore-design#backup-file-structure). Therefore, this configuration is deprecated from v6.1.1 and is no longer recommended.

+ The part size used when you perform multipart upload to S3 during backup. You can adjust the value of this configuration to control the number of requests sent to S3.
+ If data is backed up to S3 and the backup file is larger than the value of this configuration item, [multipart upload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPart.html) is automatically enabled. Based on the compression ratio, the backup file generated by a 96-MiB Region is approximately 10 MiB to 30 MiB.
+ Default value: 5MiB