Avoid extra taskq_dispatch() calls by DMU. #8909
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kithrup
reviewed
Jun 15, 2019
Codecov Report
@@ Coverage Diff @@
## master #8909 +/- ##
==========================================
- Coverage 78.61% 78.59% -0.02%
==========================================
Files 388 388
Lines 120071 120087 +16
==========================================
- Hits 94388 94383 -5
- Misses 25683 25704 +21
Continue to review full report at Codecov.
|
behlendorf
reviewed
Jun 19, 2019
DMU sync code calls taskq_dispatch() for each sublist of os_dirty_dnodes and os_synced_dnodes. Since the number of sublists by default is equal to number of CPUs, it will dispatch equal, potentially large, number of tasks, waking up many CPUs to handle them, even if only one or few of sublists actually have any work to do. This change adds check for empty sublists to avoid this. Signed-off-by: Alexander Motin <mav@FreeBSD.org>
|
OK. Considering the objection I've remade the patch, keeping full locking in place. It probably does not make much sense to fight too much, safety is better. |
ahrens
approved these changes
Jun 24, 2019
| @@ -89,6 +89,8 @@ void multilist_sublist_insert_head(multilist_sublist_t *, void *); | |||
| void multilist_sublist_insert_tail(multilist_sublist_t *, void *); | |||
| void multilist_sublist_move_forward(multilist_sublist_t *mls, void *obj); | |||
| void multilist_sublist_remove(multilist_sublist_t *, void *); | |||
| int multilist_sublist_is_empty(multilist_sublist_t *); | |||
| int multilist_sublist_is_empty_idx(multilist_t *, unsigned int); | |||
There was a problem hiding this comment.
I don't see a difference. I just copied multilist_is_empty().
behlendorf
approved these changes
Jun 24, 2019
behlendorf
added
Status: Accepted
tonyhutter
pushed a commit
to tonyhutter/zfs
that referenced
this pull request
Aug 13, 2019
DMU sync code calls taskq_dispatch() for each sublist of os_dirty_dnodes and os_synced_dnodes. Since the number of sublists by default is equal to number of CPUs, it will dispatch equal, potentially large, number of tasks, waking up many CPUs to handle them, even if only one or few of sublists actually have any work to do. This change adds check for empty sublists to avoid this. Reviewed by: Sean Eric Fagan <sef@ixsystems.com> Reviewed by: Matt Ahrens <matt@delphix.com> Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Alexander Motin <mav@FreeBSD.org> Closes openzfs#8909
tonyhutter
pushed a commit
to tonyhutter/zfs
that referenced
this pull request
Aug 22, 2019
DMU sync code calls taskq_dispatch() for each sublist of os_dirty_dnodes and os_synced_dnodes. Since the number of sublists by default is equal to number of CPUs, it will dispatch equal, potentially large, number of tasks, waking up many CPUs to handle them, even if only one or few of sublists actually have any work to do. This change adds check for empty sublists to avoid this. Reviewed by: Sean Eric Fagan <sef@ixsystems.com> Reviewed by: Matt Ahrens <matt@delphix.com> Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Alexander Motin <mav@FreeBSD.org> Closes openzfs#8909
tonyhutter
pushed a commit
to tonyhutter/zfs
that referenced
this pull request
Aug 23, 2019
DMU sync code calls taskq_dispatch() for each sublist of os_dirty_dnodes and os_synced_dnodes. Since the number of sublists by default is equal to number of CPUs, it will dispatch equal, potentially large, number of tasks, waking up many CPUs to handle them, even if only one or few of sublists actually have any work to do. This change adds check for empty sublists to avoid this. Reviewed by: Sean Eric Fagan <sef@ixsystems.com> Reviewed by: Matt Ahrens <matt@delphix.com> Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Alexander Motin <mav@FreeBSD.org> Closes openzfs#8909
tonyhutter
pushed a commit
to tonyhutter/zfs
that referenced
this pull request
Sep 17, 2019
DMU sync code calls taskq_dispatch() for each sublist of os_dirty_dnodes and os_synced_dnodes. Since the number of sublists by default is equal to number of CPUs, it will dispatch equal, potentially large, number of tasks, waking up many CPUs to handle them, even if only one or few of sublists actually have any work to do. This change adds check for empty sublists to avoid this. Reviewed by: Sean Eric Fagan <sef@ixsystems.com> Reviewed by: Matt Ahrens <matt@delphix.com> Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Alexander Motin <mav@FreeBSD.org> Closes openzfs#8909
tonyhutter
pushed a commit
to tonyhutter/zfs
that referenced
this pull request
Sep 18, 2019
DMU sync code calls taskq_dispatch() for each sublist of os_dirty_dnodes and os_synced_dnodes. Since the number of sublists by default is equal to number of CPUs, it will dispatch equal, potentially large, number of tasks, waking up many CPUs to handle them, even if only one or few of sublists actually have any work to do. This change adds check for empty sublists to avoid this. Reviewed by: Sean Eric Fagan <sef@ixsystems.com> Reviewed by: Matt Ahrens <matt@delphix.com> Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Alexander Motin <mav@FreeBSD.org> Closes openzfs#8909
tonyhutter
pushed a commit
that referenced
this pull request
Sep 26, 2019
DMU sync code calls taskq_dispatch() for each sublist of os_dirty_dnodes and os_synced_dnodes. Since the number of sublists by default is equal to number of CPUs, it will dispatch equal, potentially large, number of tasks, waking up many CPUs to handle them, even if only one or few of sublists actually have any work to do. This change adds check for empty sublists to avoid this. Reviewed by: Sean Eric Fagan <sef@ixsystems.com> Reviewed by: Matt Ahrens <matt@delphix.com> Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Alexander Motin <mav@FreeBSD.org> Closes #8909
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
DMU sync code calls taskq_dispatch() for each sublist of os_dirty_dnodes
and os_synced_dnodes. Since the number of sublists by default is equal
to number of CPUs, it will dispatch equal, potentially large, number of
tasks, waking up many CPUs to handle them, even if only one or few of
sublists actually list any dnodes.
Description
This change adds very quick lockless check for empty sublists to avoid
this. Locking is not needed there, since the multilists are stable at
that time.
I also removed existing locking from multilist_is_empty(), since it any
way provides no any guaranties, unless multilist is stable, in which
case locking is not needed. Even if the multilist is not stable, it
is not critical for the list implementation, since list_is_empty() only
compares one variable with constant, and the worst that can happen is
it read some stale value, that may happen any way, unless caller locked
all the sublists explicitly first.
How Has This Been Tested?
Without the patch simple DTrace script on FreeBSD head on 72-core system shown tons of the tasks being dispatched on each dmu_objset_sync(). With the patch, if only small set of dnodes is modified (ZVOL or single file), it shows only few.
Types of changes
Checklist:
Signed-off-by.