Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Change default compaction threads from 1 to 8 #169

Closed
bosmatt opened this issue Sep 21, 2022 · 2 comments · Fixed by #194
Closed

Change default compaction threads from 1 to 8 #169

bosmatt opened this issue Sep 21, 2022 · 2 comments · Fixed by #194
Assignees
@isaac-io @assaf-speedb
Labels
compaction
Milestone
v2.1.0

Comments

@bosmatt
Copy link
Contributor

bosmatt commented Sep 21, 2022 •
edited by isaac-io
Loading

Changing the default compaction threads shows performance improvement as can be seen below.
We believe that even there are not too many cores in the system, the performance is still better with more threads than with a single thread.

Image
@Guyme Guyme assigned bosmatt and unassigned erez-speedb Sep 21, 2022
@bosmatt bosmatt assigned isaac-io and unassigned bosmatt Sep 21, 2022
@isaac-io isaac-io added this to the v2.1.0 milestone Sep 29, 2022
isaac-io added a commit that referenced this issue Oct 19, 2022
@isaac-io
options: change the default value for compaction threads to 8 (#169)
4d57e71 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
isaac-io added a commit that referenced this issue Oct 19, 2022
@isaac-io
options: change the default value for compaction threads to 8 (#169)
b18e0ea 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
@isaac-io isaac-io linked a pull request Oct 19, 2022 that will close this issue
options: change the default value for compaction threads to 8 (#169) #194
Merged
isaac-io added a commit that referenced this issue Oct 19, 2022
@isaac-io
options: change the default value for compaction threads to 8 (#169)
09f9928 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
isaac-io added a commit that referenced this issue Oct 20, 2022
@isaac-io
options: change the default value for compaction threads to 8 (#169)
49ce560 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
@erez-speedb
Copy link

@isaac-io testing is done, please merge

@erez-speedb erez-speedb removed their assignment Oct 20, 2022
@isaac-io
Copy link
Contributor

Thanks! Waiting for Assaf to approve that QA tests passed as well.

Yuval-Ariel pushed a commit that referenced this issue Oct 20, 2022
@isaac-io @Yuval-Ariel
options: change the default value for compaction threads to 8 (#169)
0010a87 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
Yuval-Ariel pushed a commit that referenced this issue Oct 20, 2022
@isaac-io @Yuval-Ariel
options: change the default value for compaction threads to 8 (#169)
2868d10 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
isaac-io added a commit that referenced this issue Oct 20, 2022
@isaac-io
options: change the default value for compaction threads to 8 (#169)
d4affd7 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
isaac-io added a commit that referenced this issue Oct 24, 2022
@isaac-io
options: change the default value for compaction threads to 8 (#169)
62c6129 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
isaac-io added a commit that referenced this issue Oct 24, 2022
@isaac-io
options: change the default value for compaction threads to 8 (#169)
4a1dbb1 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
isaac-io added a commit that referenced this issue Oct 26, 2022
@isaac-io
options: change the default value for compaction threads to 8 (#169)
0c9d468 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
@bosmatt bosmatt moved this to ✅ Shipped in Speedb Roadmap Nov 3, 2022
Yuval-Ariel pushed a commit that referenced this issue Nov 23, 2022
@isaac-io @Yuval-Ariel
options: change the default value for compaction threads to 8 (#169)
1268313 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
Yuval-Ariel pushed a commit that referenced this issue Nov 25, 2022
@isaac-io @Yuval-Ariel
options: change the default value for compaction threads to 8 (#169)
05055a6 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
Yuval-Ariel pushed a commit that referenced this issue Apr 30, 2023
@isaac-io @Yuval-Ariel
options: change the default value for compaction threads to 8 (#169)
1ed8ad0 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
Yuval-Ariel pushed a commit that referenced this issue May 4, 2023
@isaac-io @Yuval-Ariel
options: change the default value for compaction threads to 8 (#169)
04b536b 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
udi-speedb pushed a commit that referenced this issue Nov 13, 2023
@isaac-io @udi-speedb
options: change the default value for compaction threads to 8 (#169)
f0779ba 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
udi-speedb pushed a commit that referenced this issue Dec 3, 2023
@isaac-io @udi-speedb
options: change the default value for compaction threads to 8 (#169)
ec94260 
This is done through the deprecated `max_background_compactions` option,
which overrides `max_background_jobs`, so in order to avoid having an
adverse effect on users who set `max_background_jobs` to a high value
we try to detect that case and shift back to the user's choice in case
there are enough background jobs to be enough for both flushes and
compactions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@isaac-io isaac-io

@assaf-speedb assaf-speedb

Labels
compaction
Projects
Speedb Roadmap
Status: ✅ Shipped
Milestone
v2.1.0
Development

Successfully merging a pull request may close this issue.

options: change the default value for compaction threads to 8 (#169) speedb-io/speedb
5 participants
@isaac-io @Yuval-Ariel @erez-speedb @bosmatt @assaf-speedb