Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: recluster disable sort spill #15490

Merged
merged 1 commit into from
May 14, 2024
Merged

Conversation

zhyass
Copy link
Member

@zhyass zhyass commented May 12, 2024

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Briefly describe what this PR aims to solve. Include background context that will help reviewers understand the purpose of the PR.

  • Fixes #[Link the issue here]

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-chore this PR only has small changes that no need to record, like coding styles. label May 12, 2024
@BohuTANG
Copy link
Member

BohuTANG commented May 12, 2024

I think sort spill not need this setting, but recluster need a setting to disable spill. Because sort spill not only for recluster.

Another issue is if we disable sort spill, the recluster will be OOM? If so the root cause may not to disable spill, but how to avoid spill when recluster?

@zhyass
Copy link
Member Author

zhyass commented May 12, 2024

f we disable sort spill, the recluster will be OOM? If so the root cause may not to disable spill, but how to avoid spill when recluster?

Yes, sort spill will disabled when recluster, others are not affected.

We scaled down the default value of recluster_block_size to minimize the impact of oom.

@zhyass zhyass added the ci-cloud Build docker image for cloud test label May 12, 2024
Copy link
Contributor

Docker Image for PR

  • tag: pr-15490-116ff0f

note: this image tag is only available for internal use,
please check the internal doc for more details.

@zhyass zhyass requested review from sundy-li and dantengsky May 13, 2024 07:54
@BohuTANG BohuTANG merged commit 5c408b8 into databendlabs:main May 14, 2024
83 of 86 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-cloud Build docker image for cloud test pr-chore this PR only has small changes that no need to record, like coding styles.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants