Skip to content

Commit

Permalink
Move gethrtime() calls out of vdev queue lock
Browse files Browse the repository at this point in the history
This dramatically reduces the lock contention on systems with slower
(non-TSC) timecounters.  With TSC the difference is minimal, but since
this lock is pretty congested, any improvement counts.  Plus I don't
see any reason to do it under the lock other than the latency of the
lock itself, which this change actually reduces.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Closes openzfs#12281
  • Loading branch information
amotin authored and tonyhutter committed Sep 14, 2021
1 parent dab147d commit 41b33dc
Showing 1 changed file with 5 additions and 6 deletions.
11 changes: 5 additions & 6 deletions module/zfs/vdev_queue.c
Original file line number Diff line number Diff line change
Expand Up @@ -912,9 +912,9 @@ vdev_queue_io(zio_t *zio)
}

zio->io_flags |= ZIO_FLAG_DONT_CACHE | ZIO_FLAG_DONT_QUEUE;
zio->io_timestamp = gethrtime();

mutex_enter(&vq->vq_lock);
zio->io_timestamp = gethrtime();
vdev_queue_io_add(vq, zio);
nio = vdev_queue_io_to_issue(vq);
mutex_exit(&vq->vq_lock);
Expand All @@ -936,14 +936,13 @@ vdev_queue_io_done(zio_t *zio)
vdev_queue_t *vq = &zio->io_vd->vdev_queue;
zio_t *nio;

mutex_enter(&vq->vq_lock);
hrtime_t now = gethrtime();
vq->vq_io_complete_ts = now;
vq->vq_io_delta_ts = zio->io_delta = now - zio->io_timestamp;

mutex_enter(&vq->vq_lock);
vdev_queue_pending_remove(vq, zio);

zio->io_delta = gethrtime() - zio->io_timestamp;
vq->vq_io_complete_ts = gethrtime();
vq->vq_io_delta_ts = vq->vq_io_complete_ts - zio->io_timestamp;

while ((nio = vdev_queue_io_to_issue(vq)) != NULL) {
mutex_exit(&vq->vq_lock);
if (nio->io_done == vdev_queue_agg_io_done) {
Expand Down

0 comments on commit 41b33dc

Please sign in to comment.