Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch API: Fixes the size limit to reduce timeouts #1757

Merged
merged 8 commits into from
Aug 7, 2020

Conversation

rakkuma
Copy link
Contributor

@rakkuma rakkuma commented Aug 6, 2020

Description

This updates the batch size to performance tuned value 200KB of bulk v2. This is also to solve an issue where too many index terms leads to frequent timeouts. With less size, we will have less number of batch operations and less number of index terms.

Later we want to read these configs from database account config.

Type of change

  • [] Bug fix (non-breaking change which fixes an issue)

Signed-off-by: Rakesh Kumar <rakkuma@microsoft.com>
Signed-off-by: Rakesh Kumar <rakkuma@microsoft.com>
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow the required format: "[Internal] Category: (Adds|Fixes|Refactors) Description"

Examples:
Diagnostics: Adds GetElapsedClientLatency to CosmosDiagnostics
PartitionKey: Fixes null reference when using default(PartitionKey)
[v4] Client Encryption: Refactors code to external project
[Internal] Query: Adds code generator for CosmosNumbers for easy additions in the future.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow the required format: "[Internal] Category: (Adds|Fixes|Refactors) Description"

Examples:
Diagnostics: Adds GetElapsedClientLatency to CosmosDiagnostics
PartitionKey: Fixes null reference when using default(PartitionKey)
[v4] Client Encryption: Refactors code to external project
[Internal] Query: Adds code generator for CosmosNumbers for easy additions in the future.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow the required format: "[Internal] Category: (Adds|Fixes|Refactors) Description"

Examples:
Diagnostics: Adds GetElapsedClientLatency to CosmosDiagnostics
PartitionKey: Fixes null reference when using default(PartitionKey)
[v4] Client Encryption: Refactors code to external project
[Internal] Query: Adds code generator for CosmosNumbers for easy additions in the future.

@rakkuma rakkuma changed the title BugFix Batch - Updating batch size to match with bulk v2. Internal Batch - Fixes batch size to match with bulk v2. Aug 6, 2020
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow the required format: "[Internal] Category: (Adds|Fixes|Refactors) Description"

Examples:
Diagnostics: Adds GetElapsedClientLatency to CosmosDiagnostics
PartitionKey: Fixes null reference when using default(PartitionKey)
[v4] Client Encryption: Refactors code to external project
[Internal] Query: Adds code generator for CosmosNumbers for easy additions in the future.

@rakkuma rakkuma requested a review from abhijitpai August 6, 2020 06:49
@rakkuma rakkuma changed the title Internal Batch - Fixes batch size to match with bulk v2. [Internal] Batch - Fixes batch size to match with bulk v2. Aug 6, 2020
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow the required format: "[Internal] Category: (Adds|Fixes|Refactors) Description"

Examples:
Diagnostics: Adds GetElapsedClientLatency to CosmosDiagnostics
PartitionKey: Fixes null reference when using default(PartitionKey)
[v4] Client Encryption: Refactors code to external project
[Internal] Query: Adds code generator for CosmosNumbers for easy additions in the future.

@rakkuma rakkuma changed the title [Internal] Batch - Fixes batch size to match with bulk v2. [Internal] Batch: Fixes batch size to match with bulk v2. Aug 6, 2020
@github-actions github-actions bot dismissed stale reviews from themself August 6, 2020 06:52

All good!

abhijitpai
abhijitpai previously approved these changes Aug 6, 2020
ealsur
ealsur previously approved these changes Aug 6, 2020
Copy link
Member

@ealsur ealsur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please adjust the title, this is not an internal change as it affects the behavior of Bulk with large documents (it increases the # of network trips). Since it's a public change PR, please try to put in the description as much detail and explanation as possible in case users refer to it to understand.

Signed-off-by: Rakesh Kumar <rakkuma@microsoft.com>
@rakkuma rakkuma changed the title [Internal] Batch: Fixes batch size to match with bulk v2. Bulk API: Fixes the size limit to match with bulk v2. Aug 6, 2020
@rakkuma rakkuma changed the title Bulk API: Fixes the size limit to match with bulk v2. Batch API: Fixes the size limit to match with bulk v2. Aug 6, 2020
Signed-off-by: Rakesh Kumar <rakkuma@microsoft.com>
@ealsur ealsur merged commit 4411655 into master Aug 7, 2020
@ealsur ealsur deleted the users/rakkuma/batch-config-update branch August 7, 2020 23:16
@j82w j82w changed the title Batch API: Fixes the size limit to match with bulk v2. Batch API: Fixes the size limit to reduce timeouts Aug 12, 2020
@ghost
Copy link

ghost commented Dec 15, 2021

Closing due to in-activity, pease feel free to re-open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants