Implement parallel dbuf eviction #16487

allanjude · 2024-08-29T18:16:57Z

Sponsored-by: Expensify, Inc.
Sponsored-by: Klara, Inc.

Motivation and Context

Replace dbuf_evict_one() with dbuf_evict_many() to more efficiently evict dbuf objects without looping over the same locked objects over and over.

Description

In the previous code, dbuf_evict_thread() would called dbuf_evict_one() in a look while dbuf_cache_above_lowater().

dbuf_evict_one() would select a random sublist from the dbuf cache, then walk it from the tail forward, attempting to acquire the lock on each object until it succeeded, then evict that object and return.

As the name suggests, it would evict only a single object from the cache. However, evicting one object is not likely to bring us below the desired low water mark, so dbuf_evict_one() will be called again, where it will loop over all of the same busy objects again, until it founds one it can evict.

This has been replaced with dbuf_evict_many() which takes a specific sublist as a parameter, as well as a desired amount of data to evict. It then walks the sublist from the tail forward, evicting what it can until the number of bytes evicted satisfies the input parameter or the head of the sublist is reached.

The dbuf_evict_thread now runs is parallel as well, allowing it to keep up with demand more easily. For the dbuf cache, if the single thread was not able to keep up, ZFS would shift the work of evicting some items to each incoming I/O thread. While that is still the case it should be seen much less often now that dbuf_evict is more efficient and no longer bottlenecked to a single thread.

How Has This Been Tested?

Performance benchmarks while the dbuf cache is full

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Performance enhancement (non-breaking change which improves efficiency)
Code cleanup (non-breaking change which makes code smaller or more readable)
Breaking change (fix or feature that would cause existing functionality to change)
Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
Documentation (a change to man pages or other documentation)

Checklist:

My code follows the OpenZFS code style requirements.
I have updated the documentation accordingly.
I have read the contributing document.
I have added tests to cover my changes.
I have run the ZFS Test Suite with this change applied.
All commit messages are properly formatted and contain Signed-off-by.

amotin · 2024-09-05T17:52:27Z

module/zfs/dbuf.c

+			if (skip == 0)
+				skip = 1;


Why to check before assign?

amotin · 2024-09-05T17:55:28Z

module/zfs/dbuf.c

 */
 static void
-dbuf_evict_one(void)
+dbuf_evict_many(uint64_t bytes, unsigned int idx)


I would change the arguments order.

amotin · 2024-09-05T18:00:08Z

module/zfs/dbuf.c

+	uint64_t nchunks = ((bytes - 1) >> DBUF_MIN_EVICT_PERTASK_SHIFT) + 1;
+	unsigned n = nchunks < num_sublists ? nchunks : num_sublists;
+	uint64_t fullrows = nchunks / n;
+	unsigned lastrowcols = nchunks % n;
+	unsigned k = (lastrowcols ? lastrowcols : n);
+
+	uint64_t bytes_pertask_low = fullrows << DBUF_MIN_EVICT_PERTASK_SHIFT;
+	uint64_t bytes_pertask = bytes_pertask_low + (lastrowcols ?
+	    (1 << DBUF_MIN_EVICT_PERTASK_SHIFT) : 0);


Same as for ARC seems over-engineered. No need for tasks to be multiple of 1 << DBUF_MIN_EVICT_PERTASK_SHIFT.

amotin · 2024-09-05T18:01:09Z

module/zfs/dbuf.c

+		evarg[i].idx = idx;
+		evarg[i].bytes = evict;
+
+		taskq_dispatch_ent(dbuf_evict_taskq, dbuf_evict_task,
+		    &evarg[i], 0, &evarg[i].tqe);


Somewhere here should be taskq_init_ent(), probably.

amotin · 2024-09-05T18:39:23Z

module/zfs/dbuf.c

 	if (size > dbuf_cache_target_bytes()) {
 		if (size > dbuf_cache_hiwater_bytes())
-			dbuf_evict_one();
+			dbuf_evict();


Lets assume we have 10 user threads calling this. I suppose each of them will try to create own task sets to evict the same full amount of extra dbuf caches using all the same CPUs. In best case it may end up with empty dbuf cache. I am not sure I greatly like the design of one main eviction thread calling bunch of other taskqs, but each client thread doing that definitely looks weird. I think if user threads has to do evictions, they should do it directly, just doing more than one buffer at a time to be more efficient, as you have said.

0mp · 2024-09-12T12:00:25Z

I have updated the patch with a different logic for picking the default maximum number of dbuf eviction threads. The new logic aims to pick the number that is one-eighth of the available CPUs, with a minimum of 2 and a maximum of 16.

tonyhutter · 2024-10-10T22:25:40Z

man/man4/zfs.4

+.It Sy dbuf_evict_parallel Ns = Ns Sy 0 Pq uint
+When set to 1, ZFS will use up to
+.Sy dbuf_evict_threads
+threads to evict dbuf data in parallel, improving the responsiveness
+of ZFS to memory pressure.
+.
+.It Sy dbuf_evict_threads Ns = Ns Sy 0 Pq uint
+Sets the maximum number of dbuf eviction threads to be used.
+When set to 0, ZFS uses one-eighth of the available CPUs,
+with a minimum of 2 and a maximum of 16.


Is there a time you would want both dbuf_evict_threads >= 2 and dbuf_evict_parallel=0? Just wondering if you can simplify this to dbuf_evict_threads only, and imply "parallel" if it's set to 2 threads or more.

In the previous code, dbuf_evict_thread() would called dbuf_evict_one() in a look while dbuf_cache_above_lowater(). dbuf_evict_one() would select a random sublist from the dbuf cache, then walk it from the tail forward, attempting to acquire the lock on each object until it succeeded, then evict that object and return. As the name suggests, it would evict only a single object from the cache. However, evicting one object is not likely to bring us below the desired low water mark, so dbuf_evict_one() will be called again, where it will loop over all of the same busy objects again, until it founds one it can evict. This has been replaced with dbuf_evict_many() which takes a specific sublist as a parameter, as well as a desired amount of data to evict. It then walks the sublist from the tail forward, evicting what it can until the number of bytes evicted satisfies the input parameter or the head of the sublist is reached. The dbuf_evict_thread now runs is parallel as well, allowing it to keep up with demand more easily. For the dbuf cache, if the single thread was not able to keep up, ZFS would shift the work of evicting some items to each incoming I/O thread. While that is still the case it should be seen much less often now that dbuf_evict is more efficient and no longer bottlenecked to a single thread. Sponsored-by: Expensify, Inc. Sponsored-by: Klara, Inc. Co-authored-by: Allan Jude <allan@klarasystems.com> Co-authored-by: Mateusz Piotrowski <mateusz.piotrowski@klarasystems.com> Signed-off-by: Alexander Stetsenko <alex.stetsenko@gmail.com> Signed-off-by: Allan Jude <allan@klarasystems.com> Signed-off-by: Mateusz Piotrowski <mateusz.piotrowski@klarasystems.com>

0mp · 2024-11-13T14:58:19Z

I've rebased onto latest master. I'll address the feedback soon.

amotin · 2024-11-13T15:02:02Z

man/man4/zfs.4

+    This controls how quickly the transaction delay approaches infinity.
+    Larger values cause longer delays for a given amount of dirty data.
+    .Pp
+    For the smoothest delay, this value should be about 1 billion divided


Have you added some tabs here?

allanjude force-pushed the parallel_dbuf_evict branch from d3aad0f to 3570e89 Compare August 29, 2024 18:36

rincebrain mentioned this pull request Aug 30, 2024

dmu_objset: replace dnode_hash impl with cityhash4 #16483

Closed

13 tasks

allanjude force-pushed the parallel_dbuf_evict branch from 3570e89 to 5b070d1 Compare September 4, 2024 23:17

amotin reviewed Sep 5, 2024

View reviewed changes

0mp force-pushed the parallel_dbuf_evict branch 3 times, most recently from 54888d8 to 883cd30 Compare September 12, 2024 11:59

behlendorf added the Status: Code Review Needed Ready for review and testing label Sep 13, 2024

tonyhutter reviewed Oct 10, 2024

View reviewed changes

0mp mentioned this pull request Oct 23, 2024

Implement parallel ARC eviction #16486

Open

13 tasks

amotin added the Status: Revision Needed Changes are required for the PR to be accepted label Oct 29, 2024

0mp force-pushed the parallel_dbuf_evict branch from 883cd30 to 47c490b Compare November 13, 2024 14:57

github-actions bot removed the Status: Revision Needed Changes are required for the PR to be accepted label Nov 13, 2024

amotin reviewed Nov 13, 2024

View reviewed changes

amotin added the Status: Revision Needed Changes are required for the PR to be accepted label Nov 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement parallel dbuf eviction #16487

Implement parallel dbuf eviction #16487

allanjude commented Aug 29, 2024

amotin Sep 5, 2024

amotin Sep 5, 2024

amotin Sep 5, 2024

amotin Sep 5, 2024

amotin Sep 5, 2024

0mp commented Sep 12, 2024

tonyhutter Oct 10, 2024

0mp commented Nov 13, 2024

amotin Nov 13, 2024

Implement parallel dbuf eviction #16487

Are you sure you want to change the base?

Implement parallel dbuf eviction #16487

Conversation

allanjude commented Aug 29, 2024

Motivation and Context

Description

How Has This Been Tested?

Types of changes

Checklist:

amotin Sep 5, 2024

Choose a reason for hiding this comment

amotin Sep 5, 2024

Choose a reason for hiding this comment

amotin Sep 5, 2024

Choose a reason for hiding this comment

amotin Sep 5, 2024

Choose a reason for hiding this comment

amotin Sep 5, 2024

Choose a reason for hiding this comment

0mp commented Sep 12, 2024

tonyhutter Oct 10, 2024

Choose a reason for hiding this comment

0mp commented Nov 13, 2024

amotin Nov 13, 2024

Choose a reason for hiding this comment