Skip to content

Commit

Permalink
mm, slab: call kvfree_rcu_barrier() from kmem_cache_destroy()
Browse files Browse the repository at this point in the history
We would like to replace call_rcu() users with kfree_rcu() where the
existing callback is just a kmem_cache_free(). However this causes
issues when the cache can be destroyed (such as due to module unload).

Currently such modules should be issuing rcu_barrier() before
kmem_cache_destroy() to have their call_rcu() callbacks processed first.
This barrier is however not sufficient for kfree_rcu() in flight due
to the batching introduced by a35d169 ("rcu: Add basic support for
kfree_rcu() batching").

This is not a problem for kmalloc caches which are never destroyed, but
since removing SLOB, kfree_rcu() is allowed also for any other cache,
that might be destroyed.

In order not to complicate the API, put the responsibility for handling
outstanding kfree_rcu() in kmem_cache_destroy() itself. Use the newly
introduced kvfree_rcu_barrier() to wait before destroying the cache.
This is similar to how we issue rcu_barrier() for SLAB_TYPESAFE_BY_RCU
caches, but has to be done earlier, as the latter only needs to wait for
the empty slab pages to finish freeing, and not objects from the slab.

Users of call_rcu() with arbitrary callbacks should still issue
rcu_barrier() before destroying the cache and unloading the module, as
kvfree_rcu_barrier() is not a superset of rcu_barrier() and the
callbacks may be invoking module code or performing other actions that
are necessary for a successful unload.

Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
  • Loading branch information
tehcaster committed Aug 27, 2024
1 parent 2b55d6a commit 6c6c47b
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions mm/slab_common.c
Original file line number Diff line number Diff line change
Expand Up @@ -508,6 +508,9 @@ void kmem_cache_destroy(struct kmem_cache *s)
if (unlikely(!s) || !kasan_check_byte(s))
return;

/* in-flight kfree_rcu()'s may include objects from our cache */
kvfree_rcu_barrier();

cpus_read_lock();
mutex_lock(&slab_mutex);

Expand Down

0 comments on commit 6c6c47b

Please sign in to comment.