Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

0.6.0.39 does not build on kernel 3.2 #463

Closed
tisoft opened this issue Nov 24, 2011 · 3 comments
Closed

0.6.0.39 does not build on kernel 3.2 #463

tisoft opened this issue Nov 24, 2011 · 3 comments
Labels
Type: Building Indicates an issue related to building binaries
Milestone
0.6.0

Comments

@tisoft
Copy link

tisoft commented Nov 24, 2011

I have tried to install the oneiric daily ppa packages on precise. spl builds fine, but ifs fails. This might not be a high priority since it is some time until precise comes out, but here is the log anyway:

hoth:~# cat /var/lib/dkms/zfs/0.6.0.39/build/make.log
DKMS make.log for zfs-0.6.0.39 for kernel 3.2.0-1-generic (x86_64)
Thu Nov 24 15:13:23 CET 2011
make all-recursive
make[1]: Entering directory /var/lib/dkms/zfs/0.6.0.39/build' Making all in module make[2]: Entering directory/var/lib/dkms/zfs/0.6.0.39/build/module'
make -C /lib/modules/3.2.0-1-generic/build SUBDIRS=pwd modules
make[3]: Entering directory `/usr/src/linux-headers-3.2.0-1-generic'
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/avl/../../module/avl/avl.o
LD [M] /var/lib/dkms/zfs/0.6.0.39/build/module/avl/zavl.o
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/nvpair/../../module/nvpair/nvpair.o
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/nvpair/../../module/nvpair/nvpair_alloc_spl.o
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/nvpair/../../module/nvpair/nvpair_alloc_fixed.o
LD [M] /var/lib/dkms/zfs/0.6.0.39/build/module/nvpair/znvpair.o
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/unicode/../../module/unicode/u8_textprep.o
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/unicode/../../module/unicode/uconv.o
LD [M] /var/lib/dkms/zfs/0.6.0.39/build/module/unicode/zunicode.o
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zcommon/../../module/zcommon/zfs_deleg.o
In file included from /usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/t_lock.h:30:0,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zfs_context.h:33,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/spa.h:30,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/dsl_pool.h:28,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/dsl_deleg.h:29,
from /var/lib/dkms/zfs/0.6.0.39/build/module/zcommon/../../module/zcommon/zfs_deleg.c:38:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_set_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:55:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:57:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_clear_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:65:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:67:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘rw_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:76:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:78:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_READ_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_WRITE_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_LOCK_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zcommon/../../module/zcommon/zfs_prop.o
In file included from /usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/t_lock.h:30:0,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zfs_context.h:33,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zio.h:29,
from /var/lib/dkms/zfs/0.6.0.39/build/module/zcommon/../../module/zcommon/zfs_prop.c:28:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_set_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:55:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:57:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_clear_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:65:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:67:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘rw_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:76:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:78:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_READ_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_WRITE_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_LOCK_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zcommon/../../module/zcommon/zprop_common.o
In file included from /usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/t_lock.h:30:0,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zfs_context.h:33,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zio.h:29,
from /var/lib/dkms/zfs/0.6.0.39/build/module/zcommon/../../module/zcommon/zprop_common.c:30:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_set_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:55:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:57:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_clear_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:65:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:67:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘rw_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:76:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:78:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_READ_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_WRITE_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_LOCK_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zcommon/../../module/zcommon/zfs_namecheck.o
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zcommon/../../module/zcommon/zfs_comutil.o
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zcommon/../../module/zcommon/zfs_fletcher.o
In file included from /usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/t_lock.h:30:0,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zfs_context.h:33,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zio.h:29,
from /var/lib/dkms/zfs/0.6.0.39/build/module/zcommon/../../module/zcommon/zfs_fletcher.c:131:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_set_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:55:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:57:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_clear_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:65:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:67:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘rw_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:76:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:78:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_READ_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_WRITE_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_LOCK_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zcommon/../../module/zcommon/zfs_uio.o
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zcommon/../../module/zcommon/zpool_prop.o
In file included from /usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/t_lock.h:30:0,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zfs_context.h:33,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zio.h:29,
from /var/lib/dkms/zfs/0.6.0.39/build/module/zcommon/../../module/zcommon/zpool_prop.c:25:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_set_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:55:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:57:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_clear_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:65:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:67:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘rw_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:76:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:78:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_READ_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_WRITE_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_LOCK_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
LD [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zcommon/zcommon.o
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/arc.o
In file included from /usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/t_lock.h:30:0,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zfs_context.h:33,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/spa.h:30,
from /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/arc.c:119:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_set_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:55:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:57:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_clear_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:65:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:67:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘rw_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:76:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:78:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_READ_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_WRITE_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_LOCK_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/bplist.o
In file included from /usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/t_lock.h:30:0,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zfs_context.h:33,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/bplist.h:28,
from /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/bplist.c:25:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_set_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:55:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:57:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_clear_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:65:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:67:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘rw_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:76:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:78:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_READ_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_WRITE_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_LOCK_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/bpobj.o
In file included from /usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/t_lock.h:30:0,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zfs_context.h:33,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/spa.h:30,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/bpobj.h:29,
from /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/bpobj.c:25:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_set_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:55:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:57:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_clear_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:65:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:67:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘rw_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:76:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:78:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_READ_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_WRITE_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_LOCK_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/dbuf.o
In file included from /usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/t_lock.h:30:0,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zfs_context.h:33,
from /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/dbuf.c:26:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_set_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:55:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:57:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_clear_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:65:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:67:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘rw_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:76:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:78:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_READ_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_WRITE_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_LOCK_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/ddt.o
In file included from /usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/t_lock.h:30:0,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zfs_context.h:33,
from /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/ddt.c:26:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_set_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:55:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:57:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_clear_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:65:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:67:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘rw_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:76:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:78:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_READ_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_WRITE_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_LOCK_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/ddt_zap.o
In file included from /usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/t_lock.h:30:0,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zfs_context.h:33,
from /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/ddt_zap.c:26:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_set_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:55:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:57:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_clear_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:65:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:67:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘rw_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:76:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:78:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_READ_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_WRITE_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_LOCK_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/dmu.o
In file included from /usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/t_lock.h:30:0,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zfs_context.h:33,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/spa.h:30,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/txg_impl.h:29,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/dmu_impl.h:29,
from /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/dmu.c:26:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_set_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:55:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:57:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_clear_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:65:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:67:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘rw_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:76:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:78:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_READ_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_WRITE_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_LOCK_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/dmu_diff.o
In file included from /usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/t_lock.h:30:0,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zfs_context.h:33,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/spa.h:30,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/txg_impl.h:29,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/dmu_impl.h:29,
from /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/dmu_diff.c:26:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_set_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:55:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:57:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_clear_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:65:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:67:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘rw_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:76:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:78:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_READ_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_WRITE_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_LOCK_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/dmu_object.o
In file included from /usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/t_lock.h:30:0,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zfs_context.h:33,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/spa.h:30,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/dmu_objset.h:30,
from /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/dmu_object.c:26:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_set_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:55:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:57:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘spl_rw_clear_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:65:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:67:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘rw_owner’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:76:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:78:33: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_READ_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:86:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_WRITE_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:93:10: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h: In function ‘RW_LOCK_HELD’:
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spinlock_check’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:272:31: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
/usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/rwlock.h:100:9: warning: passing argument 1 of ‘spin_unlock_irqrestore’ from incompatible pointer type [enabled by default]
include/linux/spinlock.h:338:20: note: expected ‘struct spinlock_t *’ but argument is of type ‘struct raw_spinlock_t *’
CC [M] /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/dmu_objset.o
In file included from /usr/src/spl-0.6.0.39/3.2.0-1-generic/sys/t_lock.h:30:0,
from /var/lib/dkms/zfs/0.6.0.39/build/include/sys/zfs_context.h:33,
from /var/lib/dkms/zfs/0.6.0.39/build/module/zfs/../../module/zfs/dmu_objset.c:28:
/usr/src/spl-0.6.0.39/3.2.0-1-gen
@behlendorf
Copy link
Contributor

We'll certainly take a look once the upstream 3.2 kernel is tagged at kernel.org. If someone in the community wants to work on this sooner that's OK with me!

@dajhorn
Copy link
Contributor

dajhorn commented Dec 4, 2011

Also note that even if zfs-dkms compiles properly, the installation on Pangolin will fail on the udev dependency. It looks like a few non-trivial things must be updated for Ubuntu 12.04 LTS, so don't force the Oneiric packages.

@dajhorn dajhorn mentioned this issue Dec 17, 2011
Closed
@dajhorn dajhorn mentioned this issue Jan 11, 2012
Closed
behlendorf pushed a commit to openzfs/spl that referenced this issue Jan 12, 2012
@dajhorn @behlendorf
Linux 3.2 compat: rw_semaphore.wait_lock is raw
588d900 
The wait_lock member of the rw_semaphore struct became a raw_spinlock_t
in Linux 3.2 at torvalds/linux@ddb6c9b.

Wrap spin_lock_* function calls in a new spl_rwsem_* interface to
ensure type safety if raw_spinlock_t becomes architecture specific,
and to satisfy these compiler warnings:

  warning: passing argument 1 of ‘spinlock_check’
    from incompatible pointer type [enabled by default]
  note: expected ‘struct spinlock_t *’
    but argument is of type ‘struct raw_spinlock_t *’

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes: #76
Closes: openzfs/zfs#463
@behlendorf
Copy link
Contributor

Closed by spl commit openzfs/spl@588d900

ryao added a commit to ryao/zfs that referenced this issue Oct 9, 2014
@ryao
vdev_raidz_io_start: Ignore empty writes
0489068 
The below excerpt of a backtrace is from a ztest failure when running
ZoL's ztest.

/openzfs#453 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#454 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350003de0) at ../../module/zfs/vdev_queue.c:747
/openzfs#455 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350003de0) at ../../module/zfs/zio.c:2659
/openzfs#456 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1399
/openzfs#457 zio_nowait (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1456
/openzfs#458 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350003a10) at ../../module/zfs/vdev_mirror.c:374
/openzfs#459 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1399
/openzfs#460 zio_nowait (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1456
/openzfs#461 0x00007f03c806464c in vdev_raidz_io_start (zio=0x7f0350003380) at ../../module/zfs/vdev_raidz.c:1607
/openzfs#462 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003380) at ../../module/zfs/zio.c:1399
/openzfs#463 zio_nowait (zio=0x7f0350003380) at ../../module/zfs/zio.c:1456
/openzfs#464 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002fb0) at ../../module/zfs/vdev_mirror.c:374
/openzfs#465 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1399
/openzfs#466 zio_nowait (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1456
/openzfs#467 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033957ebf0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#468 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:2707
/openzfs#469 0x00007f03c808285b in __zio_execute (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:1399
/openzfs#470 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f0390001330, pio=0x7f033957ebf0) at ../../module/zfs/zio.c:547
/openzfs#471 zio_done (zio=0x7f0390001330) at ../../module/zfs/zio.c:3278
/openzfs#472 0x00007f03c808285b in __zio_execute (zio=0x7f0390001330) at ../../module/zfs/zio.c:1399
/openzfs#473 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4013a00, pio=0x7f0390001330) at ../../module/zfs/zio.c:547
/openzfs#474 zio_done (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:3278
/openzfs#475 0x00007f03c808285b in __zio_execute (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:1399
/openzfs#476 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014210, pio=0x7f03b4013a00) at ../../module/zfs/zio.c:547
/openzfs#477 zio_done (zio=0x7f03b4014210) at ../../module/zfs/zio.c:3278
/openzfs#478 0x00007f03c808285b in __zio_execute (zio=0x7f03b4014210) at ../../module/zfs/zio.c:1399
/openzfs#479 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014620, pio=0x7f03b4014210) at ../../module/zfs/zio.c:547
/openzfs#480 zio_done (zio=0x7f03b4014620) at ../../module/zfs/zio.c:3278
/openzfs#481 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03b4014620) at ../../module/zfs/zio.c:1399
/openzfs#482 zio_execute (zio=zio@entry=0x7f03b4014620) at ../../module/zfs/zio.c:1337
/openzfs#483 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#484 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350002be0) at ../../module/zfs/vdev_queue.c:747
/openzfs#485 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350002be0) at ../../module/zfs/zio.c:2659
/openzfs#486 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1399
/openzfs#487 zio_nowait (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1456
/openzfs#488 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002810) at ../../module/zfs/vdev_mirror.c:374
/openzfs#489 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002810) at ../../module/zfs/zio.c:1399
/openzfs#490 zio_nowait (zio=0x7f0350002810) at ../../module/zfs/zio.c:1456
/openzfs#491 0x00007f03c8064593 in vdev_raidz_io_start (zio=0x7f0350001270) at ../../module/zfs/vdev_raidz.c:1591
/openzfs#492 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001270) at ../../module/zfs/zio.c:1399
/openzfs#493 zio_nowait (zio=0x7f0350001270) at ../../module/zfs/zio.c:1456
/openzfs#494 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350001e60) at ../../module/zfs/vdev_mirror.c:374
/openzfs#495 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1399
/openzfs#496 zio_nowait (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1456
/openzfs#497 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#498 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:2707
/openzfs#499 0x00007f03c808285b in __zio_execute (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:1399
/openzfs#500 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03a8003c00, pio=0x7f033a0c39c0) at ../../module/zfs/zio.c:547
/openzfs#501 zio_done (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:3278
/openzfs#502 0x00007f03c808285b in __zio_execute (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:1399
/openzfs#503 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800c400, pio=0x7f03a8003c00) at ../../module/zfs/zio.c:547
/openzfs#504 zio_done (zio=0x7f038800c400) at ../../module/zfs/zio.c:3278
/openzfs#505 0x00007f03c808285b in __zio_execute (zio=0x7f038800c400) at ../../module/zfs/zio.c:1399
/openzfs#506 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800da00, pio=0x7f038800c400) at ../../module/zfs/zio.c:547
/openzfs#507 zio_done (zio=0x7f038800da00) at ../../module/zfs/zio.c:3278
/openzfs#508 0x00007f03c808285b in __zio_execute (zio=0x7f038800da00) at ../../module/zfs/zio.c:1399
/openzfs#509 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800fd80, pio=0x7f038800da00) at ../../module/zfs/zio.c:547
/openzfs#510 zio_done (zio=0x7f038800fd80) at ../../module/zfs/zio.c:3278
/openzfs#511 0x00007f03c807a6d3 in __zio_execute (zio=0x7f038800fd80) at ../../module/zfs/zio.c:1399
/openzfs#512 zio_execute (zio=zio@entry=0x7f038800fd80) at ../../module/zfs/zio.c:1337
/openzfs#513 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#514 0x00007f03c806119d in vdev_queue_io_done (zio=zio@entry=0x7f03a0010950) at ../../module/zfs/vdev_queue.c:775
/openzfs#515 0x00007f03c807a0e8 in zio_vdev_io_done (zio=0x7f03a0010950) at ../../module/zfs/zio.c:2686
/openzfs#516 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1399
/openzfs#517 zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1337
/openzfs#518 0x00007f03c7fcd0c4 in taskq_thread (arg=0x966d50) at ../../lib/libzpool/taskq.c:215
/openzfs#519 0x00007f03c7fc7937 in zk_thread_helper (arg=0x967e90) at ../../lib/libzpool/kernel.c:135
/openzfs#520 0x00007f03c78890a3 in start_thread (arg=0x7f03c2703700) at pthread_create.c:309
/openzfs#521 0x00007f03c75c50fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

The backtrace was an infinite loop of `vdev_queue_io_to_issue()` invoking
`zio_execute()` until it overran the stack. vdev_queue_io_to_issue() will ony
invoke `zio_execute()` on raidz vdevs when aggregation I/Os are generated to
improve aggregation continuity. These I/Os do not trigger any writes. However,
it appears that they can be generated in such a way that they recurse
infinitely upon return to `vdev_queue_io_to_issue()`. As a consequence, we see
the number of parents by 1 each time the recursion returns to
`vdev_raidz_io_start()`.

Signed-off-by: Richard Yao <ryao@gentoo.org>
@ryao ryao mentioned this issue Oct 9, 2014
Closed
ryao added a commit to ryao/zfs that referenced this issue Oct 9, 2014
@ryao
vdev_raidz_io_start: Ignore empty writes
0bd246f 
The below excerpt of a backtrace is from a ztest failure when running
ZoL's ztest.

/openzfs#453 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#454 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350003de0) at ../../module/zfs/vdev_queue.c:747
/openzfs#455 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350003de0) at ../../module/zfs/zio.c:2659
/openzfs#456 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1399
/openzfs#457 zio_nowait (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1456
/openzfs#458 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350003a10) at ../../module/zfs/vdev_mirror.c:374
/openzfs#459 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1399
/openzfs#460 zio_nowait (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1456
/openzfs#461 0x00007f03c806464c in vdev_raidz_io_start (zio=0x7f0350003380) at ../../module/zfs/vdev_raidz.c:1607
/openzfs#462 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003380) at ../../module/zfs/zio.c:1399
/openzfs#463 zio_nowait (zio=0x7f0350003380) at ../../module/zfs/zio.c:1456
/openzfs#464 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002fb0) at ../../module/zfs/vdev_mirror.c:374
/openzfs#465 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1399
/openzfs#466 zio_nowait (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1456
/openzfs#467 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033957ebf0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#468 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:2707
/openzfs#469 0x00007f03c808285b in __zio_execute (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:1399
/openzfs#470 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f0390001330, pio=0x7f033957ebf0) at ../../module/zfs/zio.c:547
/openzfs#471 zio_done (zio=0x7f0390001330) at ../../module/zfs/zio.c:3278
/openzfs#472 0x00007f03c808285b in __zio_execute (zio=0x7f0390001330) at ../../module/zfs/zio.c:1399
/openzfs#473 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4013a00, pio=0x7f0390001330) at ../../module/zfs/zio.c:547
/openzfs#474 zio_done (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:3278
/openzfs#475 0x00007f03c808285b in __zio_execute (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:1399
/openzfs#476 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014210, pio=0x7f03b4013a00) at ../../module/zfs/zio.c:547
/openzfs#477 zio_done (zio=0x7f03b4014210) at ../../module/zfs/zio.c:3278
/openzfs#478 0x00007f03c808285b in __zio_execute (zio=0x7f03b4014210) at ../../module/zfs/zio.c:1399
/openzfs#479 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014620, pio=0x7f03b4014210) at ../../module/zfs/zio.c:547
/openzfs#480 zio_done (zio=0x7f03b4014620) at ../../module/zfs/zio.c:3278
/openzfs#481 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03b4014620) at ../../module/zfs/zio.c:1399
/openzfs#482 zio_execute (zio=zio@entry=0x7f03b4014620) at ../../module/zfs/zio.c:1337
/openzfs#483 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#484 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350002be0) at ../../module/zfs/vdev_queue.c:747
/openzfs#485 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350002be0) at ../../module/zfs/zio.c:2659
/openzfs#486 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1399
/openzfs#487 zio_nowait (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1456
/openzfs#488 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002810) at ../../module/zfs/vdev_mirror.c:374
/openzfs#489 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002810) at ../../module/zfs/zio.c:1399
/openzfs#490 zio_nowait (zio=0x7f0350002810) at ../../module/zfs/zio.c:1456
/openzfs#491 0x00007f03c8064593 in vdev_raidz_io_start (zio=0x7f0350001270) at ../../module/zfs/vdev_raidz.c:1591
/openzfs#492 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001270) at ../../module/zfs/zio.c:1399
/openzfs#493 zio_nowait (zio=0x7f0350001270) at ../../module/zfs/zio.c:1456
/openzfs#494 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350001e60) at ../../module/zfs/vdev_mirror.c:374
/openzfs#495 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1399
/openzfs#496 zio_nowait (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1456
/openzfs#497 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#498 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:2707
/openzfs#499 0x00007f03c808285b in __zio_execute (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:1399
/openzfs#500 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03a8003c00, pio=0x7f033a0c39c0) at ../../module/zfs/zio.c:547
/openzfs#501 zio_done (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:3278
/openzfs#502 0x00007f03c808285b in __zio_execute (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:1399
/openzfs#503 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800c400, pio=0x7f03a8003c00) at ../../module/zfs/zio.c:547
/openzfs#504 zio_done (zio=0x7f038800c400) at ../../module/zfs/zio.c:3278
/openzfs#505 0x00007f03c808285b in __zio_execute (zio=0x7f038800c400) at ../../module/zfs/zio.c:1399
/openzfs#506 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800da00, pio=0x7f038800c400) at ../../module/zfs/zio.c:547
/openzfs#507 zio_done (zio=0x7f038800da00) at ../../module/zfs/zio.c:3278
/openzfs#508 0x00007f03c808285b in __zio_execute (zio=0x7f038800da00) at ../../module/zfs/zio.c:1399
/openzfs#509 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800fd80, pio=0x7f038800da00) at ../../module/zfs/zio.c:547
/openzfs#510 zio_done (zio=0x7f038800fd80) at ../../module/zfs/zio.c:3278
/openzfs#511 0x00007f03c807a6d3 in __zio_execute (zio=0x7f038800fd80) at ../../module/zfs/zio.c:1399
/openzfs#512 zio_execute (zio=zio@entry=0x7f038800fd80) at ../../module/zfs/zio.c:1337
/openzfs#513 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#514 0x00007f03c806119d in vdev_queue_io_done (zio=zio@entry=0x7f03a0010950) at ../../module/zfs/vdev_queue.c:775
/openzfs#515 0x00007f03c807a0e8 in zio_vdev_io_done (zio=0x7f03a0010950) at ../../module/zfs/zio.c:2686
/openzfs#516 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1399
/openzfs#517 zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1337
/openzfs#518 0x00007f03c7fcd0c4 in taskq_thread (arg=0x966d50) at ../../lib/libzpool/taskq.c:215
/openzfs#519 0x00007f03c7fc7937 in zk_thread_helper (arg=0x967e90) at ../../lib/libzpool/kernel.c:135
/openzfs#520 0x00007f03c78890a3 in start_thread (arg=0x7f03c2703700) at pthread_create.c:309
/openzfs#521 0x00007f03c75c50fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

The backtrace was an infinite loop of `vdev_queue_io_to_issue()` invoking
`zio_execute()` until it overran the stack. vdev_queue_io_to_issue() will ony
invoke `zio_execute()` on raidz vdevs when aggregation I/Os are generated to
improve aggregation continuity. These I/Os do not trigger any writes. However,
it appears that they can be generated in such a way that they recurse
infinitely upon return to `vdev_queue_io_to_issue()`. As a consequence, we see
the number of parents by 1 each time the recursion returns to
`vdev_raidz_io_start()`.

Signed-off-by: Richard Yao <richard.yao@clusterhq.com>
ryao added a commit to ryao/zfs that referenced this issue Oct 9, 2014
@ryao
vdev_raidz_io_start: Ignore empty writes
7fbb0cb 
The below excerpt of a backtrace is from a ztest failure when running
ZoL's ztest.

/openzfs#453 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#454 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350003de0) at ../../module/zfs/vdev_queue.c:747
/openzfs#455 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350003de0) at ../../module/zfs/zio.c:2659
/openzfs#456 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1399
/openzfs#457 zio_nowait (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1456
/openzfs#458 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350003a10) at ../../module/zfs/vdev_mirror.c:374
/openzfs#459 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1399
/openzfs#460 zio_nowait (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1456
/openzfs#461 0x00007f03c806464c in vdev_raidz_io_start (zio=0x7f0350003380) at ../../module/zfs/vdev_raidz.c:1607
/openzfs#462 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003380) at ../../module/zfs/zio.c:1399
/openzfs#463 zio_nowait (zio=0x7f0350003380) at ../../module/zfs/zio.c:1456
/openzfs#464 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002fb0) at ../../module/zfs/vdev_mirror.c:374
/openzfs#465 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1399
/openzfs#466 zio_nowait (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1456
/openzfs#467 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033957ebf0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#468 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:2707
/openzfs#469 0x00007f03c808285b in __zio_execute (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:1399
/openzfs#470 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f0390001330, pio=0x7f033957ebf0) at ../../module/zfs/zio.c:547
/openzfs#471 zio_done (zio=0x7f0390001330) at ../../module/zfs/zio.c:3278
/openzfs#472 0x00007f03c808285b in __zio_execute (zio=0x7f0390001330) at ../../module/zfs/zio.c:1399
/openzfs#473 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4013a00, pio=0x7f0390001330) at ../../module/zfs/zio.c:547
/openzfs#474 zio_done (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:3278
/openzfs#475 0x00007f03c808285b in __zio_execute (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:1399
/openzfs#476 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014210, pio=0x7f03b4013a00) at ../../module/zfs/zio.c:547
/openzfs#477 zio_done (zio=0x7f03b4014210) at ../../module/zfs/zio.c:3278
/openzfs#478 0x00007f03c808285b in __zio_execute (zio=0x7f03b4014210) at ../../module/zfs/zio.c:1399
/openzfs#479 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014620, pio=0x7f03b4014210) at ../../module/zfs/zio.c:547
/openzfs#480 zio_done (zio=0x7f03b4014620) at ../../module/zfs/zio.c:3278
/openzfs#481 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03b4014620) at ../../module/zfs/zio.c:1399
/openzfs#482 zio_execute (zio=zio@entry=0x7f03b4014620) at ../../module/zfs/zio.c:1337
/openzfs#483 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#484 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350002be0) at ../../module/zfs/vdev_queue.c:747
/openzfs#485 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350002be0) at ../../module/zfs/zio.c:2659
/openzfs#486 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1399
/openzfs#487 zio_nowait (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1456
/openzfs#488 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002810) at ../../module/zfs/vdev_mirror.c:374
/openzfs#489 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002810) at ../../module/zfs/zio.c:1399
/openzfs#490 zio_nowait (zio=0x7f0350002810) at ../../module/zfs/zio.c:1456
/openzfs#491 0x00007f03c8064593 in vdev_raidz_io_start (zio=0x7f0350001270) at ../../module/zfs/vdev_raidz.c:1591
/openzfs#492 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001270) at ../../module/zfs/zio.c:1399
/openzfs#493 zio_nowait (zio=0x7f0350001270) at ../../module/zfs/zio.c:1456
/openzfs#494 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350001e60) at ../../module/zfs/vdev_mirror.c:374
/openzfs#495 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1399
/openzfs#496 zio_nowait (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1456
/openzfs#497 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#498 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:2707
/openzfs#499 0x00007f03c808285b in __zio_execute (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:1399
/openzfs#500 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03a8003c00, pio=0x7f033a0c39c0) at ../../module/zfs/zio.c:547
/openzfs#501 zio_done (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:3278
/openzfs#502 0x00007f03c808285b in __zio_execute (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:1399
/openzfs#503 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800c400, pio=0x7f03a8003c00) at ../../module/zfs/zio.c:547
/openzfs#504 zio_done (zio=0x7f038800c400) at ../../module/zfs/zio.c:3278
/openzfs#505 0x00007f03c808285b in __zio_execute (zio=0x7f038800c400) at ../../module/zfs/zio.c:1399
/openzfs#506 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800da00, pio=0x7f038800c400) at ../../module/zfs/zio.c:547
/openzfs#507 zio_done (zio=0x7f038800da00) at ../../module/zfs/zio.c:3278
/openzfs#508 0x00007f03c808285b in __zio_execute (zio=0x7f038800da00) at ../../module/zfs/zio.c:1399
/openzfs#509 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800fd80, pio=0x7f038800da00) at ../../module/zfs/zio.c:547
/openzfs#510 zio_done (zio=0x7f038800fd80) at ../../module/zfs/zio.c:3278
/openzfs#511 0x00007f03c807a6d3 in __zio_execute (zio=0x7f038800fd80) at ../../module/zfs/zio.c:1399
/openzfs#512 zio_execute (zio=zio@entry=0x7f038800fd80) at ../../module/zfs/zio.c:1337
/openzfs#513 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#514 0x00007f03c806119d in vdev_queue_io_done (zio=zio@entry=0x7f03a0010950) at ../../module/zfs/vdev_queue.c:775
/openzfs#515 0x00007f03c807a0e8 in zio_vdev_io_done (zio=0x7f03a0010950) at ../../module/zfs/zio.c:2686
/openzfs#516 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1399
/openzfs#517 zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1337
/openzfs#518 0x00007f03c7fcd0c4 in taskq_thread (arg=0x966d50) at ../../lib/libzpool/taskq.c:215
/openzfs#519 0x00007f03c7fc7937 in zk_thread_helper (arg=0x967e90) at ../../lib/libzpool/kernel.c:135
/openzfs#520 0x00007f03c78890a3 in start_thread (arg=0x7f03c2703700) at pthread_create.c:309
/openzfs#521 0x00007f03c75c50fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

The backtrace was an infinite loop of `vdev_queue_io_to_issue()` invoking
`zio_execute()` until it overran the stack. vdev_queue_io_to_issue() will ony
invoke `zio_execute()` on raidz vdevs when aggregation I/Os are generated to
improve aggregation continuity. These I/Os do not trigger any writes. However,
it appears that they can be generated in such a way that they recurse
infinitely upon return to `vdev_queue_io_to_issue()`.

Signed-off-by: Richard Yao <richard.yao@clusterhq.com>
ryao added a commit to ryao/zfs that referenced this issue Oct 10, 2014
@ryao
Redispatch ZIOs in deep notification call graphs
8d523a5 
The below excerpt of a backtrace is from a ztest failure when running ZoL's
ztest:

/openzfs#453 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#454 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350003de0) at ../../module/zfs/vdev_queue.c:747
/openzfs#455 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350003de0) at ../../module/zfs/zio.c:2659
/openzfs#456 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1399
/openzfs#457 zio_nowait (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1456
/openzfs#458 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350003a10) at ../../module/zfs/vdev_mirror.c:374
/openzfs#459 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1399
/openzfs#460 zio_nowait (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1456
/openzfs#461 0x00007f03c806464c in vdev_raidz_io_start (zio=0x7f0350003380) at ../../module/zfs/vdev_raidz.c:1607
/openzfs#462 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003380) at ../../module/zfs/zio.c:1399
/openzfs#463 zio_nowait (zio=0x7f0350003380) at ../../module/zfs/zio.c:1456
/openzfs#464 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002fb0) at ../../module/zfs/vdev_mirror.c:374
/openzfs#465 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1399
/openzfs#466 zio_nowait (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1456
/openzfs#467 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033957ebf0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#468 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:2707
/openzfs#469 0x00007f03c808285b in __zio_execute (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:1399
/openzfs#470 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f0390001330, pio=0x7f033957ebf0) at ../../module/zfs/zio.c:547
/openzfs#471 zio_done (zio=0x7f0390001330) at ../../module/zfs/zio.c:3278
/openzfs#472 0x00007f03c808285b in __zio_execute (zio=0x7f0390001330) at ../../module/zfs/zio.c:1399
/openzfs#473 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4013a00, pio=0x7f0390001330) at ../../module/zfs/zio.c:547
/openzfs#474 zio_done (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:3278
/openzfs#475 0x00007f03c808285b in __zio_execute (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:1399
/openzfs#476 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014210, pio=0x7f03b4013a00) at ../../module/zfs/zio.c:547
/openzfs#477 zio_done (zio=0x7f03b4014210) at ../../module/zfs/zio.c:3278
/openzfs#478 0x00007f03c808285b in __zio_execute (zio=0x7f03b4014210) at ../../module/zfs/zio.c:1399
/openzfs#479 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014620, pio=0x7f03b4014210) at ../../module/zfs/zio.c:547
/openzfs#480 zio_done (zio=0x7f03b4014620) at ../../module/zfs/zio.c:3278
/openzfs#481 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03b4014620) at ../../module/zfs/zio.c:1399
/openzfs#482 zio_execute (zio=zio@entry=0x7f03b4014620) at ../../module/zfs/zio.c:1337
/openzfs#483 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#484 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350002be0) at ../../module/zfs/vdev_queue.c:747
/openzfs#485 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350002be0) at ../../module/zfs/zio.c:2659
/openzfs#486 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1399
/openzfs#487 zio_nowait (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1456
/openzfs#488 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002810) at ../../module/zfs/vdev_mirror.c:374
/openzfs#489 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002810) at ../../module/zfs/zio.c:1399
/openzfs#490 zio_nowait (zio=0x7f0350002810) at ../../module/zfs/zio.c:1456
/openzfs#491 0x00007f03c8064593 in vdev_raidz_io_start (zio=0x7f0350001270) at ../../module/zfs/vdev_raidz.c:1591
/openzfs#492 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001270) at ../../module/zfs/zio.c:1399
/openzfs#493 zio_nowait (zio=0x7f0350001270) at ../../module/zfs/zio.c:1456
/openzfs#494 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350001e60) at ../../module/zfs/vdev_mirror.c:374
/openzfs#495 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1399
/openzfs#496 zio_nowait (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1456
/openzfs#497 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#498 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:2707
/openzfs#499 0x00007f03c808285b in __zio_execute (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:1399
/openzfs#500 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03a8003c00, pio=0x7f033a0c39c0) at ../../module/zfs/zio.c:547
/openzfs#501 zio_done (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:3278
/openzfs#502 0x00007f03c808285b in __zio_execute (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:1399
/openzfs#503 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800c400, pio=0x7f03a8003c00) at ../../module/zfs/zio.c:547
/openzfs#504 zio_done (zio=0x7f038800c400) at ../../module/zfs/zio.c:3278
/openzfs#505 0x00007f03c808285b in __zio_execute (zio=0x7f038800c400) at ../../module/zfs/zio.c:1399
/openzfs#506 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800da00, pio=0x7f038800c400) at ../../module/zfs/zio.c:547
/openzfs#507 zio_done (zio=0x7f038800da00) at ../../module/zfs/zio.c:3278
/openzfs#508 0x00007f03c808285b in __zio_execute (zio=0x7f038800da00) at ../../module/zfs/zio.c:1399
/openzfs#509 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800fd80, pio=0x7f038800da00) at ../../module/zfs/zio.c:547
/openzfs#510 zio_done (zio=0x7f038800fd80) at ../../module/zfs/zio.c:3278
/openzfs#511 0x00007f03c807a6d3 in __zio_execute (zio=0x7f038800fd80) at ../../module/zfs/zio.c:1399
/openzfs#512 zio_execute (zio=zio@entry=0x7f038800fd80) at ../../module/zfs/zio.c:1337
/openzfs#513 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#514 0x00007f03c806119d in vdev_queue_io_done (zio=zio@entry=0x7f03a0010950) at ../../module/zfs/vdev_queue.c:775
/openzfs#515 0x00007f03c807a0e8 in zio_vdev_io_done (zio=0x7f03a0010950) at ../../module/zfs/zio.c:2686
/openzfs#516 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1399
/openzfs#517 zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1337
/openzfs#518 0x00007f03c7fcd0c4 in taskq_thread (arg=0x966d50) at ../../lib/libzpool/taskq.c:215
/openzfs#519 0x00007f03c7fc7937 in zk_thread_helper (arg=0x967e90) at ../../lib/libzpool/kernel.c:135
/openzfs#520 0x00007f03c78890a3 in start_thread (arg=0x7f03c2703700) at pthread_create.c:309
/openzfs#521 0x00007f03c75c50fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

This occurred when ztest was simulating a scrub under heavy I/O load. Under
those circumstances, it was possible for a mix of noop I/Os for aggregation
continuity and the I/O elevator to generate arbitrarily deep recursion.

This patch modifies ZFS to propapage a recursion counter inside the zio_t
objects such that IOs will be redispatched upon reaching a given recursion
depth.  We can detect long call chains and dispatch to another ZIO taskq. We
cut in-line when we do this to minimize the potential for taskq exhaustion that
can prevent a zio from notifying its parent.

Signed-off-by: Richard Yao <ryao@gentoo.org>
@ryao ryao mentioned this issue Oct 10, 2014
Closed
ryao added a commit to ryao/zfs that referenced this issue Oct 10, 2014
@ryao
Redispatch ZIOs in deep call graphs
a726161 
The below excerpt of a backtrace is from a ztest failure when running ZoL's
ztest:

/openzfs#453 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#454 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350003de0) at ../../module/zfs/vdev_queue.c:747
/openzfs#455 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350003de0) at ../../module/zfs/zio.c:2659
/openzfs#456 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1399
/openzfs#457 zio_nowait (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1456
/openzfs#458 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350003a10) at ../../module/zfs/vdev_mirror.c:374
/openzfs#459 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1399
/openzfs#460 zio_nowait (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1456
/openzfs#461 0x00007f03c806464c in vdev_raidz_io_start (zio=0x7f0350003380) at ../../module/zfs/vdev_raidz.c:1607
/openzfs#462 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003380) at ../../module/zfs/zio.c:1399
/openzfs#463 zio_nowait (zio=0x7f0350003380) at ../../module/zfs/zio.c:1456
/openzfs#464 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002fb0) at ../../module/zfs/vdev_mirror.c:374
/openzfs#465 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1399
/openzfs#466 zio_nowait (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1456
/openzfs#467 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033957ebf0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#468 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:2707
/openzfs#469 0x00007f03c808285b in __zio_execute (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:1399
/openzfs#470 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f0390001330, pio=0x7f033957ebf0) at ../../module/zfs/zio.c:547
/openzfs#471 zio_done (zio=0x7f0390001330) at ../../module/zfs/zio.c:3278
/openzfs#472 0x00007f03c808285b in __zio_execute (zio=0x7f0390001330) at ../../module/zfs/zio.c:1399
/openzfs#473 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4013a00, pio=0x7f0390001330) at ../../module/zfs/zio.c:547
/openzfs#474 zio_done (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:3278
/openzfs#475 0x00007f03c808285b in __zio_execute (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:1399
/openzfs#476 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014210, pio=0x7f03b4013a00) at ../../module/zfs/zio.c:547
/openzfs#477 zio_done (zio=0x7f03b4014210) at ../../module/zfs/zio.c:3278
/openzfs#478 0x00007f03c808285b in __zio_execute (zio=0x7f03b4014210) at ../../module/zfs/zio.c:1399
/openzfs#479 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014620, pio=0x7f03b4014210) at ../../module/zfs/zio.c:547
/openzfs#480 zio_done (zio=0x7f03b4014620) at ../../module/zfs/zio.c:3278
/openzfs#481 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03b4014620) at ../../module/zfs/zio.c:1399
/openzfs#482 zio_execute (zio=zio@entry=0x7f03b4014620) at ../../module/zfs/zio.c:1337
/openzfs#483 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#484 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350002be0) at ../../module/zfs/vdev_queue.c:747
/openzfs#485 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350002be0) at ../../module/zfs/zio.c:2659
/openzfs#486 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1399
/openzfs#487 zio_nowait (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1456
/openzfs#488 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002810) at ../../module/zfs/vdev_mirror.c:374
/openzfs#489 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002810) at ../../module/zfs/zio.c:1399
/openzfs#490 zio_nowait (zio=0x7f0350002810) at ../../module/zfs/zio.c:1456
/openzfs#491 0x00007f03c8064593 in vdev_raidz_io_start (zio=0x7f0350001270) at ../../module/zfs/vdev_raidz.c:1591
/openzfs#492 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001270) at ../../module/zfs/zio.c:1399
/openzfs#493 zio_nowait (zio=0x7f0350001270) at ../../module/zfs/zio.c:1456
/openzfs#494 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350001e60) at ../../module/zfs/vdev_mirror.c:374
/openzfs#495 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1399
/openzfs#496 zio_nowait (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1456
/openzfs#497 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#498 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:2707
/openzfs#499 0x00007f03c808285b in __zio_execute (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:1399
/openzfs#500 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03a8003c00, pio=0x7f033a0c39c0) at ../../module/zfs/zio.c:547
/openzfs#501 zio_done (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:3278
/openzfs#502 0x00007f03c808285b in __zio_execute (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:1399
/openzfs#503 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800c400, pio=0x7f03a8003c00) at ../../module/zfs/zio.c:547
/openzfs#504 zio_done (zio=0x7f038800c400) at ../../module/zfs/zio.c:3278
/openzfs#505 0x00007f03c808285b in __zio_execute (zio=0x7f038800c400) at ../../module/zfs/zio.c:1399
/openzfs#506 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800da00, pio=0x7f038800c400) at ../../module/zfs/zio.c:547
/openzfs#507 zio_done (zio=0x7f038800da00) at ../../module/zfs/zio.c:3278
/openzfs#508 0x00007f03c808285b in __zio_execute (zio=0x7f038800da00) at ../../module/zfs/zio.c:1399
/openzfs#509 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800fd80, pio=0x7f038800da00) at ../../module/zfs/zio.c:547
/openzfs#510 zio_done (zio=0x7f038800fd80) at ../../module/zfs/zio.c:3278
/openzfs#511 0x00007f03c807a6d3 in __zio_execute (zio=0x7f038800fd80) at ../../module/zfs/zio.c:1399
/openzfs#512 zio_execute (zio=zio@entry=0x7f038800fd80) at ../../module/zfs/zio.c:1337
/openzfs#513 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#514 0x00007f03c806119d in vdev_queue_io_done (zio=zio@entry=0x7f03a0010950) at ../../module/zfs/vdev_queue.c:775
/openzfs#515 0x00007f03c807a0e8 in zio_vdev_io_done (zio=0x7f03a0010950) at ../../module/zfs/zio.c:2686
/openzfs#516 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1399
/openzfs#517 zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1337
/openzfs#518 0x00007f03c7fcd0c4 in taskq_thread (arg=0x966d50) at ../../lib/libzpool/taskq.c:215
/openzfs#519 0x00007f03c7fc7937 in zk_thread_helper (arg=0x967e90) at ../../lib/libzpool/kernel.c:135
/openzfs#520 0x00007f03c78890a3 in start_thread (arg=0x7f03c2703700) at pthread_create.c:309
/openzfs#521 0x00007f03c75c50fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

This occurred when ztest was simulating a scrub under heavy I/O load. Under
those circumstances, it was possible for a mix of noop I/Os for aggregation
continuity and the I/O elevator to generate arbitrarily deep recursion.

This patch modifies ZFS to propapage a recursion counter inside the zio_t
objects such that IOs will be redispatched upon reaching a given recursion
depth.  We can detect long call chains and dispatch to another ZIO taskq. We
cut in-line when we do this to minimize the potential for taskq exhaustion that
can prevent a zio from notifying its parent.

Signed-off-by: Richard Yao <ryao@gentoo.org>
ryao added a commit to ryao/zfs that referenced this issue Oct 10, 2014
@ryao
Redispatch ZIOs in deep call graphs
efb4636 
The below excerpt of a backtrace is from a ztest failure when running ZoL's
ztest:

/openzfs#453 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#454 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350003de0) at ../../module/zfs/vdev_queue.c:747
/openzfs#455 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350003de0) at ../../module/zfs/zio.c:2659
/openzfs#456 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1399
/openzfs#457 zio_nowait (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1456
/openzfs#458 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350003a10) at ../../module/zfs/vdev_mirror.c:374
/openzfs#459 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1399
/openzfs#460 zio_nowait (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1456
/openzfs#461 0x00007f03c806464c in vdev_raidz_io_start (zio=0x7f0350003380) at ../../module/zfs/vdev_raidz.c:1607
/openzfs#462 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003380) at ../../module/zfs/zio.c:1399
/openzfs#463 zio_nowait (zio=0x7f0350003380) at ../../module/zfs/zio.c:1456
/openzfs#464 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002fb0) at ../../module/zfs/vdev_mirror.c:374
/openzfs#465 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1399
/openzfs#466 zio_nowait (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1456
/openzfs#467 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033957ebf0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#468 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:2707
/openzfs#469 0x00007f03c808285b in __zio_execute (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:1399
/openzfs#470 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f0390001330, pio=0x7f033957ebf0) at ../../module/zfs/zio.c:547
/openzfs#471 zio_done (zio=0x7f0390001330) at ../../module/zfs/zio.c:3278
/openzfs#472 0x00007f03c808285b in __zio_execute (zio=0x7f0390001330) at ../../module/zfs/zio.c:1399
/openzfs#473 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4013a00, pio=0x7f0390001330) at ../../module/zfs/zio.c:547
/openzfs#474 zio_done (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:3278
/openzfs#475 0x00007f03c808285b in __zio_execute (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:1399
/openzfs#476 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014210, pio=0x7f03b4013a00) at ../../module/zfs/zio.c:547
/openzfs#477 zio_done (zio=0x7f03b4014210) at ../../module/zfs/zio.c:3278
/openzfs#478 0x00007f03c808285b in __zio_execute (zio=0x7f03b4014210) at ../../module/zfs/zio.c:1399
/openzfs#479 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014620, pio=0x7f03b4014210) at ../../module/zfs/zio.c:547
/openzfs#480 zio_done (zio=0x7f03b4014620) at ../../module/zfs/zio.c:3278
/openzfs#481 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03b4014620) at ../../module/zfs/zio.c:1399
/openzfs#482 zio_execute (zio=zio@entry=0x7f03b4014620) at ../../module/zfs/zio.c:1337
/openzfs#483 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#484 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350002be0) at ../../module/zfs/vdev_queue.c:747
/openzfs#485 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350002be0) at ../../module/zfs/zio.c:2659
/openzfs#486 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1399
/openzfs#487 zio_nowait (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1456
/openzfs#488 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002810) at ../../module/zfs/vdev_mirror.c:374
/openzfs#489 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002810) at ../../module/zfs/zio.c:1399
/openzfs#490 zio_nowait (zio=0x7f0350002810) at ../../module/zfs/zio.c:1456
/openzfs#491 0x00007f03c8064593 in vdev_raidz_io_start (zio=0x7f0350001270) at ../../module/zfs/vdev_raidz.c:1591
/openzfs#492 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001270) at ../../module/zfs/zio.c:1399
/openzfs#493 zio_nowait (zio=0x7f0350001270) at ../../module/zfs/zio.c:1456
/openzfs#494 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350001e60) at ../../module/zfs/vdev_mirror.c:374
/openzfs#495 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1399
/openzfs#496 zio_nowait (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1456
/openzfs#497 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#498 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:2707
/openzfs#499 0x00007f03c808285b in __zio_execute (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:1399
/openzfs#500 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03a8003c00, pio=0x7f033a0c39c0) at ../../module/zfs/zio.c:547
/openzfs#501 zio_done (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:3278
/openzfs#502 0x00007f03c808285b in __zio_execute (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:1399
/openzfs#503 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800c400, pio=0x7f03a8003c00) at ../../module/zfs/zio.c:547
/openzfs#504 zio_done (zio=0x7f038800c400) at ../../module/zfs/zio.c:3278
/openzfs#505 0x00007f03c808285b in __zio_execute (zio=0x7f038800c400) at ../../module/zfs/zio.c:1399
/openzfs#506 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800da00, pio=0x7f038800c400) at ../../module/zfs/zio.c:547
/openzfs#507 zio_done (zio=0x7f038800da00) at ../../module/zfs/zio.c:3278
/openzfs#508 0x00007f03c808285b in __zio_execute (zio=0x7f038800da00) at ../../module/zfs/zio.c:1399
/openzfs#509 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800fd80, pio=0x7f038800da00) at ../../module/zfs/zio.c:547
/openzfs#510 zio_done (zio=0x7f038800fd80) at ../../module/zfs/zio.c:3278
/openzfs#511 0x00007f03c807a6d3 in __zio_execute (zio=0x7f038800fd80) at ../../module/zfs/zio.c:1399
/openzfs#512 zio_execute (zio=zio@entry=0x7f038800fd80) at ../../module/zfs/zio.c:1337
/openzfs#513 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#514 0x00007f03c806119d in vdev_queue_io_done (zio=zio@entry=0x7f03a0010950) at ../../module/zfs/vdev_queue.c:775
/openzfs#515 0x00007f03c807a0e8 in zio_vdev_io_done (zio=0x7f03a0010950) at ../../module/zfs/zio.c:2686
/openzfs#516 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1399
/openzfs#517 zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1337
/openzfs#518 0x00007f03c7fcd0c4 in taskq_thread (arg=0x966d50) at ../../lib/libzpool/taskq.c:215
/openzfs#519 0x00007f03c7fc7937 in zk_thread_helper (arg=0x967e90) at ../../lib/libzpool/kernel.c:135
/openzfs#520 0x00007f03c78890a3 in start_thread (arg=0x7f03c2703700) at pthread_create.c:309
/openzfs#521 0x00007f03c75c50fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

This occurred when ztest was simulating a scrub under heavy I/O load. Under
those circumstances, it was possible for a mix of noop I/Os for aggregation
continuity and the I/O elevator to generate arbitrarily deep recursion.

This patch modifies ZFS to propapage a recursion counter inside the zio_t
objects such that IOs will be redispatched upon reaching a given recursion
depth.  We can detect long call chains and dispatch to another ZIO taskq. We
cut in-line when we do this to minimize the potential for taskq exhaustion that
can prevent a zio from notifying its parent.

Signed-off-by: Richard Yao <ryao@gentoo.org>
ryao added a commit to ryao/zfs that referenced this issue Oct 10, 2014
@ryao
Redispatch ZIOs in deep call graphs
5300bd4 
The below excerpt of a backtrace is from a ztest failure when running ZoL's
ztest:

/openzfs#453 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#454 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350003de0) at ../../module/zfs/vdev_queue.c:747
/openzfs#455 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350003de0) at ../../module/zfs/zio.c:2659
/openzfs#456 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1399
/openzfs#457 zio_nowait (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1456
/openzfs#458 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350003a10) at ../../module/zfs/vdev_mirror.c:374
/openzfs#459 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1399
/openzfs#460 zio_nowait (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1456
/openzfs#461 0x00007f03c806464c in vdev_raidz_io_start (zio=0x7f0350003380) at ../../module/zfs/vdev_raidz.c:1607
/openzfs#462 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003380) at ../../module/zfs/zio.c:1399
/openzfs#463 zio_nowait (zio=0x7f0350003380) at ../../module/zfs/zio.c:1456
/openzfs#464 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002fb0) at ../../module/zfs/vdev_mirror.c:374
/openzfs#465 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1399
/openzfs#466 zio_nowait (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1456
/openzfs#467 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033957ebf0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#468 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:2707
/openzfs#469 0x00007f03c808285b in __zio_execute (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:1399
/openzfs#470 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f0390001330, pio=0x7f033957ebf0) at ../../module/zfs/zio.c:547
/openzfs#471 zio_done (zio=0x7f0390001330) at ../../module/zfs/zio.c:3278
/openzfs#472 0x00007f03c808285b in __zio_execute (zio=0x7f0390001330) at ../../module/zfs/zio.c:1399
/openzfs#473 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4013a00, pio=0x7f0390001330) at ../../module/zfs/zio.c:547
/openzfs#474 zio_done (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:3278
/openzfs#475 0x00007f03c808285b in __zio_execute (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:1399
/openzfs#476 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014210, pio=0x7f03b4013a00) at ../../module/zfs/zio.c:547
/openzfs#477 zio_done (zio=0x7f03b4014210) at ../../module/zfs/zio.c:3278
/openzfs#478 0x00007f03c808285b in __zio_execute (zio=0x7f03b4014210) at ../../module/zfs/zio.c:1399
/openzfs#479 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014620, pio=0x7f03b4014210) at ../../module/zfs/zio.c:547
/openzfs#480 zio_done (zio=0x7f03b4014620) at ../../module/zfs/zio.c:3278
/openzfs#481 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03b4014620) at ../../module/zfs/zio.c:1399
/openzfs#482 zio_execute (zio=zio@entry=0x7f03b4014620) at ../../module/zfs/zio.c:1337
/openzfs#483 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#484 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350002be0) at ../../module/zfs/vdev_queue.c:747
/openzfs#485 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350002be0) at ../../module/zfs/zio.c:2659
/openzfs#486 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1399
/openzfs#487 zio_nowait (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1456
/openzfs#488 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002810) at ../../module/zfs/vdev_mirror.c:374
/openzfs#489 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002810) at ../../module/zfs/zio.c:1399
/openzfs#490 zio_nowait (zio=0x7f0350002810) at ../../module/zfs/zio.c:1456
/openzfs#491 0x00007f03c8064593 in vdev_raidz_io_start (zio=0x7f0350001270) at ../../module/zfs/vdev_raidz.c:1591
/openzfs#492 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001270) at ../../module/zfs/zio.c:1399
/openzfs#493 zio_nowait (zio=0x7f0350001270) at ../../module/zfs/zio.c:1456
/openzfs#494 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350001e60) at ../../module/zfs/vdev_mirror.c:374
/openzfs#495 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1399
/openzfs#496 zio_nowait (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1456
/openzfs#497 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#498 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:2707
/openzfs#499 0x00007f03c808285b in __zio_execute (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:1399
/openzfs#500 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03a8003c00, pio=0x7f033a0c39c0) at ../../module/zfs/zio.c:547
/openzfs#501 zio_done (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:3278
/openzfs#502 0x00007f03c808285b in __zio_execute (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:1399
/openzfs#503 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800c400, pio=0x7f03a8003c00) at ../../module/zfs/zio.c:547
/openzfs#504 zio_done (zio=0x7f038800c400) at ../../module/zfs/zio.c:3278
/openzfs#505 0x00007f03c808285b in __zio_execute (zio=0x7f038800c400) at ../../module/zfs/zio.c:1399
/openzfs#506 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800da00, pio=0x7f038800c400) at ../../module/zfs/zio.c:547
/openzfs#507 zio_done (zio=0x7f038800da00) at ../../module/zfs/zio.c:3278
/openzfs#508 0x00007f03c808285b in __zio_execute (zio=0x7f038800da00) at ../../module/zfs/zio.c:1399
/openzfs#509 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800fd80, pio=0x7f038800da00) at ../../module/zfs/zio.c:547
/openzfs#510 zio_done (zio=0x7f038800fd80) at ../../module/zfs/zio.c:3278
/openzfs#511 0x00007f03c807a6d3 in __zio_execute (zio=0x7f038800fd80) at ../../module/zfs/zio.c:1399
/openzfs#512 zio_execute (zio=zio@entry=0x7f038800fd80) at ../../module/zfs/zio.c:1337
/openzfs#513 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#514 0x00007f03c806119d in vdev_queue_io_done (zio=zio@entry=0x7f03a0010950) at ../../module/zfs/vdev_queue.c:775
/openzfs#515 0x00007f03c807a0e8 in zio_vdev_io_done (zio=0x7f03a0010950) at ../../module/zfs/zio.c:2686
/openzfs#516 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1399
/openzfs#517 zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1337
/openzfs#518 0x00007f03c7fcd0c4 in taskq_thread (arg=0x966d50) at ../../lib/libzpool/taskq.c:215
/openzfs#519 0x00007f03c7fc7937 in zk_thread_helper (arg=0x967e90) at ../../lib/libzpool/kernel.c:135
/openzfs#520 0x00007f03c78890a3 in start_thread (arg=0x7f03c2703700) at pthread_create.c:309
/openzfs#521 0x00007f03c75c50fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

This occurred when ztest was simulating a scrub under heavy I/O load. Under
those circumstances, it was possible for a mix of noop I/Os for aggregation
continuity and the I/O elevator to generate arbitrarily deep recursion.

This patch modifies ZFS to propapage a recursion counter inside the zio_t
objects such that IOs will be redispatched upon reaching a given recursion
depth.  We can detect long call chains and dispatch to another ZIO taskq. We
cut in-line when we do this to minimize the potential for taskq exhaustion that
can prevent a zio from notifying its parent.

Signed-off-by: Richard Yao <ryao@gentoo.org>
ryao added a commit to ryao/zfs that referenced this issue Oct 10, 2014
@ryao
Redispatch ZIOs in deep call graphs
373534d 
The below excerpt of a backtrace is from a ztest failure when running ZoL's
ztest:

/openzfs#453 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#454 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350003de0) at ../../module/zfs/vdev_queue.c:747
/openzfs#455 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350003de0) at ../../module/zfs/zio.c:2659
/openzfs#456 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1399
/openzfs#457 zio_nowait (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1456
/openzfs#458 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350003a10) at ../../module/zfs/vdev_mirror.c:374
/openzfs#459 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1399
/openzfs#460 zio_nowait (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1456
/openzfs#461 0x00007f03c806464c in vdev_raidz_io_start (zio=0x7f0350003380) at ../../module/zfs/vdev_raidz.c:1607
/openzfs#462 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003380) at ../../module/zfs/zio.c:1399
/openzfs#463 zio_nowait (zio=0x7f0350003380) at ../../module/zfs/zio.c:1456
/openzfs#464 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002fb0) at ../../module/zfs/vdev_mirror.c:374
/openzfs#465 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1399
/openzfs#466 zio_nowait (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1456
/openzfs#467 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033957ebf0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#468 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:2707
/openzfs#469 0x00007f03c808285b in __zio_execute (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:1399
/openzfs#470 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f0390001330, pio=0x7f033957ebf0) at ../../module/zfs/zio.c:547
/openzfs#471 zio_done (zio=0x7f0390001330) at ../../module/zfs/zio.c:3278
/openzfs#472 0x00007f03c808285b in __zio_execute (zio=0x7f0390001330) at ../../module/zfs/zio.c:1399
/openzfs#473 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4013a00, pio=0x7f0390001330) at ../../module/zfs/zio.c:547
/openzfs#474 zio_done (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:3278
/openzfs#475 0x00007f03c808285b in __zio_execute (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:1399
/openzfs#476 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014210, pio=0x7f03b4013a00) at ../../module/zfs/zio.c:547
/openzfs#477 zio_done (zio=0x7f03b4014210) at ../../module/zfs/zio.c:3278
/openzfs#478 0x00007f03c808285b in __zio_execute (zio=0x7f03b4014210) at ../../module/zfs/zio.c:1399
/openzfs#479 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014620, pio=0x7f03b4014210) at ../../module/zfs/zio.c:547
/openzfs#480 zio_done (zio=0x7f03b4014620) at ../../module/zfs/zio.c:3278
/openzfs#481 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03b4014620) at ../../module/zfs/zio.c:1399
/openzfs#482 zio_execute (zio=zio@entry=0x7f03b4014620) at ../../module/zfs/zio.c:1337
/openzfs#483 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#484 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350002be0) at ../../module/zfs/vdev_queue.c:747
/openzfs#485 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350002be0) at ../../module/zfs/zio.c:2659
/openzfs#486 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1399
/openzfs#487 zio_nowait (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1456
/openzfs#488 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002810) at ../../module/zfs/vdev_mirror.c:374
/openzfs#489 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002810) at ../../module/zfs/zio.c:1399
/openzfs#490 zio_nowait (zio=0x7f0350002810) at ../../module/zfs/zio.c:1456
/openzfs#491 0x00007f03c8064593 in vdev_raidz_io_start (zio=0x7f0350001270) at ../../module/zfs/vdev_raidz.c:1591
/openzfs#492 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001270) at ../../module/zfs/zio.c:1399
/openzfs#493 zio_nowait (zio=0x7f0350001270) at ../../module/zfs/zio.c:1456
/openzfs#494 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350001e60) at ../../module/zfs/vdev_mirror.c:374
/openzfs#495 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1399
/openzfs#496 zio_nowait (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1456
/openzfs#497 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#498 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:2707
/openzfs#499 0x00007f03c808285b in __zio_execute (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:1399
/openzfs#500 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03a8003c00, pio=0x7f033a0c39c0) at ../../module/zfs/zio.c:547
/openzfs#501 zio_done (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:3278
/openzfs#502 0x00007f03c808285b in __zio_execute (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:1399
/openzfs#503 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800c400, pio=0x7f03a8003c00) at ../../module/zfs/zio.c:547
/openzfs#504 zio_done (zio=0x7f038800c400) at ../../module/zfs/zio.c:3278
/openzfs#505 0x00007f03c808285b in __zio_execute (zio=0x7f038800c400) at ../../module/zfs/zio.c:1399
/openzfs#506 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800da00, pio=0x7f038800c400) at ../../module/zfs/zio.c:547
/openzfs#507 zio_done (zio=0x7f038800da00) at ../../module/zfs/zio.c:3278
/openzfs#508 0x00007f03c808285b in __zio_execute (zio=0x7f038800da00) at ../../module/zfs/zio.c:1399
/openzfs#509 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800fd80, pio=0x7f038800da00) at ../../module/zfs/zio.c:547
/openzfs#510 zio_done (zio=0x7f038800fd80) at ../../module/zfs/zio.c:3278
/openzfs#511 0x00007f03c807a6d3 in __zio_execute (zio=0x7f038800fd80) at ../../module/zfs/zio.c:1399
/openzfs#512 zio_execute (zio=zio@entry=0x7f038800fd80) at ../../module/zfs/zio.c:1337
/openzfs#513 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#514 0x00007f03c806119d in vdev_queue_io_done (zio=zio@entry=0x7f03a0010950) at ../../module/zfs/vdev_queue.c:775
/openzfs#515 0x00007f03c807a0e8 in zio_vdev_io_done (zio=0x7f03a0010950) at ../../module/zfs/zio.c:2686
/openzfs#516 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1399
/openzfs#517 zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1337
/openzfs#518 0x00007f03c7fcd0c4 in taskq_thread (arg=0x966d50) at ../../lib/libzpool/taskq.c:215
/openzfs#519 0x00007f03c7fc7937 in zk_thread_helper (arg=0x967e90) at ../../lib/libzpool/kernel.c:135
/openzfs#520 0x00007f03c78890a3 in start_thread (arg=0x7f03c2703700) at pthread_create.c:309
/openzfs#521 0x00007f03c75c50fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

This occurred when ztest was simulating a scrub under heavy I/O load. Under
those circumstances, it was possible for a mix of noop I/Os for aggregation
continuity and the I/O elevator to generate arbitrarily deep recursion.

This patch modifies ZFS to propapage a recursion counter inside the zio_t
objects such that IOs will be redispatched upon reaching a given recursion
depth.  We can detect long call chains and dispatch to another ZIO taskq. We
cut in-line when we do this to minimize the potential for taskq exhaustion that
can prevent a zio from notifying its parent.

Signed-off-by: Richard Yao <ryao@gentoo.org>
ryao added a commit to ryao/zfs that referenced this issue Oct 10, 2014
@ryao
Redispatch ZIOs in deep call graphs
ddf4bde 
The below excerpt of a backtrace is from a ztest failure when running ZoL's
ztest:

/openzfs#453 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#454 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350003de0) at ../../module/zfs/vdev_queue.c:747
/openzfs#455 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350003de0) at ../../module/zfs/zio.c:2659
/openzfs#456 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1399
/openzfs#457 zio_nowait (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1456
/openzfs#458 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350003a10) at ../../module/zfs/vdev_mirror.c:374
/openzfs#459 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1399
/openzfs#460 zio_nowait (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1456
/openzfs#461 0x00007f03c806464c in vdev_raidz_io_start (zio=0x7f0350003380) at ../../module/zfs/vdev_raidz.c:1607
/openzfs#462 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003380) at ../../module/zfs/zio.c:1399
/openzfs#463 zio_nowait (zio=0x7f0350003380) at ../../module/zfs/zio.c:1456
/openzfs#464 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002fb0) at ../../module/zfs/vdev_mirror.c:374
/openzfs#465 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1399
/openzfs#466 zio_nowait (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1456
/openzfs#467 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033957ebf0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#468 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:2707
/openzfs#469 0x00007f03c808285b in __zio_execute (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:1399
/openzfs#470 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f0390001330, pio=0x7f033957ebf0) at ../../module/zfs/zio.c:547
/openzfs#471 zio_done (zio=0x7f0390001330) at ../../module/zfs/zio.c:3278
/openzfs#472 0x00007f03c808285b in __zio_execute (zio=0x7f0390001330) at ../../module/zfs/zio.c:1399
/openzfs#473 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4013a00, pio=0x7f0390001330) at ../../module/zfs/zio.c:547
/openzfs#474 zio_done (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:3278
/openzfs#475 0x00007f03c808285b in __zio_execute (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:1399
/openzfs#476 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014210, pio=0x7f03b4013a00) at ../../module/zfs/zio.c:547
/openzfs#477 zio_done (zio=0x7f03b4014210) at ../../module/zfs/zio.c:3278
/openzfs#478 0x00007f03c808285b in __zio_execute (zio=0x7f03b4014210) at ../../module/zfs/zio.c:1399
/openzfs#479 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014620, pio=0x7f03b4014210) at ../../module/zfs/zio.c:547
/openzfs#480 zio_done (zio=0x7f03b4014620) at ../../module/zfs/zio.c:3278
/openzfs#481 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03b4014620) at ../../module/zfs/zio.c:1399
/openzfs#482 zio_execute (zio=zio@entry=0x7f03b4014620) at ../../module/zfs/zio.c:1337
/openzfs#483 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#484 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350002be0) at ../../module/zfs/vdev_queue.c:747
/openzfs#485 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350002be0) at ../../module/zfs/zio.c:2659
/openzfs#486 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1399
/openzfs#487 zio_nowait (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1456
/openzfs#488 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002810) at ../../module/zfs/vdev_mirror.c:374
/openzfs#489 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002810) at ../../module/zfs/zio.c:1399
/openzfs#490 zio_nowait (zio=0x7f0350002810) at ../../module/zfs/zio.c:1456
/openzfs#491 0x00007f03c8064593 in vdev_raidz_io_start (zio=0x7f0350001270) at ../../module/zfs/vdev_raidz.c:1591
/openzfs#492 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001270) at ../../module/zfs/zio.c:1399
/openzfs#493 zio_nowait (zio=0x7f0350001270) at ../../module/zfs/zio.c:1456
/openzfs#494 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350001e60) at ../../module/zfs/vdev_mirror.c:374
/openzfs#495 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1399
/openzfs#496 zio_nowait (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1456
/openzfs#497 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#498 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:2707
/openzfs#499 0x00007f03c808285b in __zio_execute (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:1399
/openzfs#500 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03a8003c00, pio=0x7f033a0c39c0) at ../../module/zfs/zio.c:547
/openzfs#501 zio_done (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:3278
/openzfs#502 0x00007f03c808285b in __zio_execute (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:1399
/openzfs#503 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800c400, pio=0x7f03a8003c00) at ../../module/zfs/zio.c:547
/openzfs#504 zio_done (zio=0x7f038800c400) at ../../module/zfs/zio.c:3278
/openzfs#505 0x00007f03c808285b in __zio_execute (zio=0x7f038800c400) at ../../module/zfs/zio.c:1399
/openzfs#506 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800da00, pio=0x7f038800c400) at ../../module/zfs/zio.c:547
/openzfs#507 zio_done (zio=0x7f038800da00) at ../../module/zfs/zio.c:3278
/openzfs#508 0x00007f03c808285b in __zio_execute (zio=0x7f038800da00) at ../../module/zfs/zio.c:1399
/openzfs#509 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800fd80, pio=0x7f038800da00) at ../../module/zfs/zio.c:547
/openzfs#510 zio_done (zio=0x7f038800fd80) at ../../module/zfs/zio.c:3278
/openzfs#511 0x00007f03c807a6d3 in __zio_execute (zio=0x7f038800fd80) at ../../module/zfs/zio.c:1399
/openzfs#512 zio_execute (zio=zio@entry=0x7f038800fd80) at ../../module/zfs/zio.c:1337
/openzfs#513 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#514 0x00007f03c806119d in vdev_queue_io_done (zio=zio@entry=0x7f03a0010950) at ../../module/zfs/vdev_queue.c:775
/openzfs#515 0x00007f03c807a0e8 in zio_vdev_io_done (zio=0x7f03a0010950) at ../../module/zfs/zio.c:2686
/openzfs#516 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1399
/openzfs#517 zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1337
/openzfs#518 0x00007f03c7fcd0c4 in taskq_thread (arg=0x966d50) at ../../lib/libzpool/taskq.c:215
/openzfs#519 0x00007f03c7fc7937 in zk_thread_helper (arg=0x967e90) at ../../lib/libzpool/kernel.c:135
/openzfs#520 0x00007f03c78890a3 in start_thread (arg=0x7f03c2703700) at pthread_create.c:309
/openzfs#521 0x00007f03c75c50fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

This occurred when ztest was simulating a scrub under heavy I/O load. Under
those circumstances, it was possible for a mix of noop I/Os for aggregation
continuity and the I/O elevator to generate arbitrarily deep recursion.

This patch modifies ZFS to propapage a recursion counter inside the zio_t
objects such that IOs will be redispatched upon reaching a given recursion
depth.  We can detect long call chains and dispatch to another ZIO taskq. We
cut in-line when we do this to minimize the potential for taskq exhaustion that
can prevent a zio from notifying its parent.

Signed-off-by: Richard Yao <ryao@gentoo.org>
ryao added a commit to ryao/zfs that referenced this issue Oct 11, 2014
@ryao
Redispatch ZIOs in deep call graphs
a7d6be6 
The below excerpt of a backtrace is from a ztest failure when running ZoL's
ztest:

/openzfs#453 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#454 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350003de0) at ../../module/zfs/vdev_queue.c:747
/openzfs#455 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350003de0) at ../../module/zfs/zio.c:2659
/openzfs#456 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1399
/openzfs#457 zio_nowait (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1456
/openzfs#458 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350003a10) at ../../module/zfs/vdev_mirror.c:374
/openzfs#459 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1399
/openzfs#460 zio_nowait (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1456
/openzfs#461 0x00007f03c806464c in vdev_raidz_io_start (zio=0x7f0350003380) at ../../module/zfs/vdev_raidz.c:1607
/openzfs#462 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003380) at ../../module/zfs/zio.c:1399
/openzfs#463 zio_nowait (zio=0x7f0350003380) at ../../module/zfs/zio.c:1456
/openzfs#464 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002fb0) at ../../module/zfs/vdev_mirror.c:374
/openzfs#465 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1399
/openzfs#466 zio_nowait (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1456
/openzfs#467 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033957ebf0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#468 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:2707
/openzfs#469 0x00007f03c808285b in __zio_execute (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:1399
/openzfs#470 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f0390001330, pio=0x7f033957ebf0) at ../../module/zfs/zio.c:547
/openzfs#471 zio_done (zio=0x7f0390001330) at ../../module/zfs/zio.c:3278
/openzfs#472 0x00007f03c808285b in __zio_execute (zio=0x7f0390001330) at ../../module/zfs/zio.c:1399
/openzfs#473 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4013a00, pio=0x7f0390001330) at ../../module/zfs/zio.c:547
/openzfs#474 zio_done (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:3278
/openzfs#475 0x00007f03c808285b in __zio_execute (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:1399
/openzfs#476 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014210, pio=0x7f03b4013a00) at ../../module/zfs/zio.c:547
/openzfs#477 zio_done (zio=0x7f03b4014210) at ../../module/zfs/zio.c:3278
/openzfs#478 0x00007f03c808285b in __zio_execute (zio=0x7f03b4014210) at ../../module/zfs/zio.c:1399
/openzfs#479 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014620, pio=0x7f03b4014210) at ../../module/zfs/zio.c:547
/openzfs#480 zio_done (zio=0x7f03b4014620) at ../../module/zfs/zio.c:3278
/openzfs#481 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03b4014620) at ../../module/zfs/zio.c:1399
/openzfs#482 zio_execute (zio=zio@entry=0x7f03b4014620) at ../../module/zfs/zio.c:1337
/openzfs#483 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#484 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350002be0) at ../../module/zfs/vdev_queue.c:747
/openzfs#485 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350002be0) at ../../module/zfs/zio.c:2659
/openzfs#486 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1399
/openzfs#487 zio_nowait (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1456
/openzfs#488 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002810) at ../../module/zfs/vdev_mirror.c:374
/openzfs#489 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002810) at ../../module/zfs/zio.c:1399
/openzfs#490 zio_nowait (zio=0x7f0350002810) at ../../module/zfs/zio.c:1456
/openzfs#491 0x00007f03c8064593 in vdev_raidz_io_start (zio=0x7f0350001270) at ../../module/zfs/vdev_raidz.c:1591
/openzfs#492 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001270) at ../../module/zfs/zio.c:1399
/openzfs#493 zio_nowait (zio=0x7f0350001270) at ../../module/zfs/zio.c:1456
/openzfs#494 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350001e60) at ../../module/zfs/vdev_mirror.c:374
/openzfs#495 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1399
/openzfs#496 zio_nowait (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1456
/openzfs#497 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#498 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:2707
/openzfs#499 0x00007f03c808285b in __zio_execute (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:1399
/openzfs#500 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03a8003c00, pio=0x7f033a0c39c0) at ../../module/zfs/zio.c:547
/openzfs#501 zio_done (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:3278
/openzfs#502 0x00007f03c808285b in __zio_execute (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:1399
/openzfs#503 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800c400, pio=0x7f03a8003c00) at ../../module/zfs/zio.c:547
/openzfs#504 zio_done (zio=0x7f038800c400) at ../../module/zfs/zio.c:3278
/openzfs#505 0x00007f03c808285b in __zio_execute (zio=0x7f038800c400) at ../../module/zfs/zio.c:1399
/openzfs#506 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800da00, pio=0x7f038800c400) at ../../module/zfs/zio.c:547
/openzfs#507 zio_done (zio=0x7f038800da00) at ../../module/zfs/zio.c:3278
/openzfs#508 0x00007f03c808285b in __zio_execute (zio=0x7f038800da00) at ../../module/zfs/zio.c:1399
/openzfs#509 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800fd80, pio=0x7f038800da00) at ../../module/zfs/zio.c:547
/openzfs#510 zio_done (zio=0x7f038800fd80) at ../../module/zfs/zio.c:3278
/openzfs#511 0x00007f03c807a6d3 in __zio_execute (zio=0x7f038800fd80) at ../../module/zfs/zio.c:1399
/openzfs#512 zio_execute (zio=zio@entry=0x7f038800fd80) at ../../module/zfs/zio.c:1337
/openzfs#513 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#514 0x00007f03c806119d in vdev_queue_io_done (zio=zio@entry=0x7f03a0010950) at ../../module/zfs/vdev_queue.c:775
/openzfs#515 0x00007f03c807a0e8 in zio_vdev_io_done (zio=0x7f03a0010950) at ../../module/zfs/zio.c:2686
/openzfs#516 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1399
/openzfs#517 zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1337
/openzfs#518 0x00007f03c7fcd0c4 in taskq_thread (arg=0x966d50) at ../../lib/libzpool/taskq.c:215
/openzfs#519 0x00007f03c7fc7937 in zk_thread_helper (arg=0x967e90) at ../../lib/libzpool/kernel.c:135
/openzfs#520 0x00007f03c78890a3 in start_thread (arg=0x7f03c2703700) at pthread_create.c:309
/openzfs#521 0x00007f03c75c50fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

This occurred when ztest was simulating a scrub under heavy I/O load. Under
those circumstances, it was possible for a mix of noop I/Os for aggregation
continuity and the I/O elevator to generate arbitrarily deep recursion.

This patch modifies ZFS to propapage a recursion counter inside the zio_t
objects such that IOs will be redispatched upon reaching a given recursion
depth.  We can detect long call chains and dispatch to another ZIO taskq. We
cut in-line when we do this to minimize the potential for taskq exhaustion that
can prevent a zio from notifying its parent.

Signed-off-by: Richard Yao <ryao@gentoo.org>
ryao added a commit to ryao/zfs that referenced this issue Oct 11, 2014
@ryao
Redispatch ZIOs in deep call graphs
2d33ca8 
The below excerpt of a backtrace is from a ztest failure when running ZoL's
ztest:

/openzfs#453 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#454 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350003de0) at ../../module/zfs/vdev_queue.c:747
/openzfs#455 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350003de0) at ../../module/zfs/zio.c:2659
/openzfs#456 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1399
/openzfs#457 zio_nowait (zio=0x7f0350003de0) at ../../module/zfs/zio.c:1456
/openzfs#458 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350003a10) at ../../module/zfs/vdev_mirror.c:374
/openzfs#459 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1399
/openzfs#460 zio_nowait (zio=0x7f0350003a10) at ../../module/zfs/zio.c:1456
/openzfs#461 0x00007f03c806464c in vdev_raidz_io_start (zio=0x7f0350003380) at ../../module/zfs/vdev_raidz.c:1607
/openzfs#462 0x00007f03c807f243 in __zio_execute (zio=0x7f0350003380) at ../../module/zfs/zio.c:1399
/openzfs#463 zio_nowait (zio=0x7f0350003380) at ../../module/zfs/zio.c:1456
/openzfs#464 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002fb0) at ../../module/zfs/vdev_mirror.c:374
/openzfs#465 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1399
/openzfs#466 zio_nowait (zio=0x7f0350002fb0) at ../../module/zfs/zio.c:1456
/openzfs#467 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033957ebf0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#468 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:2707
/openzfs#469 0x00007f03c808285b in __zio_execute (zio=0x7f033957ebf0) at ../../module/zfs/zio.c:1399
/openzfs#470 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f0390001330, pio=0x7f033957ebf0) at ../../module/zfs/zio.c:547
/openzfs#471 zio_done (zio=0x7f0390001330) at ../../module/zfs/zio.c:3278
/openzfs#472 0x00007f03c808285b in __zio_execute (zio=0x7f0390001330) at ../../module/zfs/zio.c:1399
/openzfs#473 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4013a00, pio=0x7f0390001330) at ../../module/zfs/zio.c:547
/openzfs#474 zio_done (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:3278
/openzfs#475 0x00007f03c808285b in __zio_execute (zio=0x7f03b4013a00) at ../../module/zfs/zio.c:1399
/openzfs#476 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014210, pio=0x7f03b4013a00) at ../../module/zfs/zio.c:547
/openzfs#477 zio_done (zio=0x7f03b4014210) at ../../module/zfs/zio.c:3278
/openzfs#478 0x00007f03c808285b in __zio_execute (zio=0x7f03b4014210) at ../../module/zfs/zio.c:1399
/openzfs#479 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03b4014620, pio=0x7f03b4014210) at ../../module/zfs/zio.c:547
/openzfs#480 zio_done (zio=0x7f03b4014620) at ../../module/zfs/zio.c:3278
/openzfs#481 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03b4014620) at ../../module/zfs/zio.c:1399
/openzfs#482 zio_execute (zio=zio@entry=0x7f03b4014620) at ../../module/zfs/zio.c:1337
/openzfs#483 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#484 0x00007f03c806106e in vdev_queue_io (zio=zio@entry=0x7f0350002be0) at ../../module/zfs/vdev_queue.c:747
/openzfs#485 0x00007f03c80818c1 in zio_vdev_io_start (zio=0x7f0350002be0) at ../../module/zfs/zio.c:2659
/openzfs#486 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1399
/openzfs#487 zio_nowait (zio=0x7f0350002be0) at ../../module/zfs/zio.c:1456
/openzfs#488 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350002810) at ../../module/zfs/vdev_mirror.c:374
/openzfs#489 0x00007f03c807f243 in __zio_execute (zio=0x7f0350002810) at ../../module/zfs/zio.c:1399
/openzfs#490 zio_nowait (zio=0x7f0350002810) at ../../module/zfs/zio.c:1456
/openzfs#491 0x00007f03c8064593 in vdev_raidz_io_start (zio=0x7f0350001270) at ../../module/zfs/vdev_raidz.c:1591
/openzfs#492 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001270) at ../../module/zfs/zio.c:1399
/openzfs#493 zio_nowait (zio=0x7f0350001270) at ../../module/zfs/zio.c:1456
/openzfs#494 0x00007f03c805f71b in vdev_mirror_io_start (zio=0x7f0350001e60) at ../../module/zfs/vdev_mirror.c:374
/openzfs#495 0x00007f03c807f243 in __zio_execute (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1399
/openzfs#496 zio_nowait (zio=0x7f0350001e60) at ../../module/zfs/zio.c:1456
/openzfs#497 0x00007f03c805ed43 in vdev_mirror_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/vdev_mirror.c:499
/openzfs#498 0x00007f03c807a0c0 in zio_vdev_io_done (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:2707
/openzfs#499 0x00007f03c808285b in __zio_execute (zio=0x7f033a0c39c0) at ../../module/zfs/zio.c:1399
/openzfs#500 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f03a8003c00, pio=0x7f033a0c39c0) at ../../module/zfs/zio.c:547
/openzfs#501 zio_done (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:3278
/openzfs#502 0x00007f03c808285b in __zio_execute (zio=0x7f03a8003c00) at ../../module/zfs/zio.c:1399
/openzfs#503 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800c400, pio=0x7f03a8003c00) at ../../module/zfs/zio.c:547
/openzfs#504 zio_done (zio=0x7f038800c400) at ../../module/zfs/zio.c:3278
/openzfs#505 0x00007f03c808285b in __zio_execute (zio=0x7f038800c400) at ../../module/zfs/zio.c:1399
/openzfs#506 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800da00, pio=0x7f038800c400) at ../../module/zfs/zio.c:547
/openzfs#507 zio_done (zio=0x7f038800da00) at ../../module/zfs/zio.c:3278
/openzfs#508 0x00007f03c808285b in __zio_execute (zio=0x7f038800da00) at ../../module/zfs/zio.c:1399
/openzfs#509 zio_notify_parent (wait=ZIO_WAIT_DONE, zio=0x7f038800fd80, pio=0x7f038800da00) at ../../module/zfs/zio.c:547
/openzfs#510 zio_done (zio=0x7f038800fd80) at ../../module/zfs/zio.c:3278
/openzfs#511 0x00007f03c807a6d3 in __zio_execute (zio=0x7f038800fd80) at ../../module/zfs/zio.c:1399
/openzfs#512 zio_execute (zio=zio@entry=0x7f038800fd80) at ../../module/zfs/zio.c:1337
/openzfs#513 0x00007f03c8060b35 in vdev_queue_io_to_issue (vq=vq@entry=0x99f8a8) at ../../module/zfs/vdev_queue.c:706
/openzfs#514 0x00007f03c806119d in vdev_queue_io_done (zio=zio@entry=0x7f03a0010950) at ../../module/zfs/vdev_queue.c:775
/openzfs#515 0x00007f03c807a0e8 in zio_vdev_io_done (zio=0x7f03a0010950) at ../../module/zfs/zio.c:2686
/openzfs#516 0x00007f03c807a6d3 in __zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1399
/openzfs#517 zio_execute (zio=0x7f03a0010950) at ../../module/zfs/zio.c:1337
/openzfs#518 0x00007f03c7fcd0c4 in taskq_thread (arg=0x966d50) at ../../lib/libzpool/taskq.c:215
/openzfs#519 0x00007f03c7fc7937 in zk_thread_helper (arg=0x967e90) at ../../lib/libzpool/kernel.c:135
/openzfs#520 0x00007f03c78890a3 in start_thread (arg=0x7f03c2703700) at pthread_create.c:309
/openzfs#521 0x00007f03c75c50fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

This occurred when ztest was simulating a scrub under heavy I/O load. Under
those circumstances, it was possible for a mix of noop I/Os for aggregation
continuity and the I/O elevator to generate arbitrarily deep recursion.

This patch modifies ZFS to propapage a recursion counter inside the zio_t
objects such that IOs will be redispatched upon reaching a given recursion
depth.  We can detect long call chains and dispatch to another ZIO taskq. We
cut in-line when we do this to minimize the potential for taskq exhaustion that
can prevent a zio from notifying its parent.

Signed-off-by: Richard Yao <ryao@gentoo.org>
behlendorf added a commit to behlendorf/zfs that referenced this issue May 21, 2018
@behlendorf
Default to --disable-debug-kmem
9eb361a 
The default kmem debugging (--enable-debug-kmem) can severely impact
performance on large-scale NUMA systems due to the atomic operations
used in the memory accounting. A 32-thread fio test running on a
40-core 80-thread system and performing 100% cached reads with kmem
debugging is:

Enabled:
READ: io=177071MB, aggrb=2951.2MB/s, minb=2951.2MB/s, maxb=2951.2MB/s,

Disabled:
READ: io=271454MB, aggrb=4524.4MB/s, minb=4524.4MB/s, maxb=4524.4MB/s,

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tim Chase <tim@chase2k.com>
Issues openzfs#463
behlendorf added a commit to behlendorf/zfs that referenced this issue May 21, 2018
@behlendorf
Remove skc_ref from alloc/free paths
4699d76 
As described in spl_kmem_cache_destroy() the ->skc_ref count was
added to address the case of a cache reap or grow racing with a
destroy.  They are not strictly needed in the alloc/free paths
because consumers of the cache are responsible for not using it
while it's being destroyed.

Removing this code is desirable because there is some evidence that
contention on this atomic negative impacts performance on large-scale
NUMA systems.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tim Chase <tim@chase2k.com>
Issue openzfs#463
pcd1193182 pushed a commit to pcd1193182/zfs that referenced this issue Jun 27, 2022
@ahrens
include frame pointers (openzfs#463)
5996df3 
Include frame pointers so that we get a little bit of stack trace for
flame graphs (without full dwarf unwinding).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Type: Building Indicates an issue related to building binaries
Projects
None yet
Milestone
0.6.0
Development

Successfully merging a pull request may close this issue.

Linux 3.2 compat: rw_semaphore.wait_lock is raw zfsonlinux/pkg-spl
3 participants
@tisoft @behlendorf @dajhorn