Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KASAN is sad about Lua #12230

Open
rincebrain opened this issue Jun 12, 2021 · 2 comments
Open

KASAN is sad about Lua #12230

rincebrain opened this issue Jun 12, 2021 · 2 comments
Labels
Bot: Not Stale Override for the stale bot Type: Defect Incorrect behavior (e.g. crash, hang)

Comments

@rincebrain
Copy link
Contributor

System information

Type Version/Name
Distribution Name Debian
Distribution Version 10
Linux Kernel 4.19.194 w/KASAN
Architecture x86_64
ZFS Version ffdf019 plus the patch for KASAN to compile

Describe the problem you're observing

As it says on the tin, scripts/zfs-tests.sh -T functional with KASAN. (Though not every time, of course, that would be too easy...and I don't think this is the same one I hit the last boot...)

Describe how to reproduce the problem

GOTO 10 (hi @aerusso )

Include any warning/errors/backtraces from the system logs

test-runner output (I patched it to log the second-since-boot from /proc/uptime when it logs, to ensure we can match log messages to which test was responsible):

Test (1000.00): /home/rich/zfs_vanilla/tests/zfs-tests/tests/functional/casenorm/mixed_formd_lookup (run as root) [00:00] [FAIL]
Test (1000.91): /home/rich/zfs_vanilla/tests/zfs-tests/tests/functional/casenorm/mixed_formd_lookup_ci (run as root) [00:00] [FAIL]
Test (1001.87): /home/rich/zfs_vanilla/tests/zfs-tests/tests/functional/casenorm/mixed_formd_delete (run as root) [00:00] [FAIL]
Test (1002.61): /home/rich/zfs_vanilla/tests/zfs-tests/tests/functional/casenorm/cleanup (run as root) [00:00] [PASS]
Test (1004.04): /home/rich/zfs_vanilla/tests/zfs-tests/tests/functional/channel_program/lua_core/setup (run as root) [00:01] [PASS]
Test (1004.22): /home/rich/zfs_vanilla/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.args_to_lua (run as root) [00:00] [PASS]

Message from syslogd@debianbuster at Jun 11 22:03:38 ...
 kernel:[ 1004.384318] page:ffffea00063547c0 count:0 mapcount:0 mapping:0000000000000000 index:0x0

Message from syslogd@debianbuster at Jun 11 22:03:38 ...
 kernel:[ 1004.384356] flags: 0x17fffc000000000()
Test (1004.53): /home/rich/zfs_vanilla/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.divide_by_zero (run as root) [00:00] [PASS]
Test (1005.25): /home/rich/zfs_vanilla/tests/zfs-tests/tests/functional/channel_program/lua_core/tst.exists (run as root) [00:00] [PASS]

[ 1004.382853] ==================================================================
[ 1004.382892] BUG: KASAN: stack-out-of-bounds in __unwind_start+0x8e/0x9f0
[ 1004.382916] Write of size 96 at addr ffff88818d51f120 by task txg_sync/23534

[ 1004.382950] CPU: 3 PID: 23534 Comm: txg_sync Kdump: loaded Tainted: P           OE     4.19.194ric1 #1
[ 1004.382951] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
[ 1004.382952] Call Trace:
[ 1004.382955]  dump_stack+0x11d/0x1a7
[ 1004.382957]  ? dump_stack_print_info.cold.0+0x1b/0x1b
[ 1004.382959]  ? __unwind_start+0x8e/0x9f0
[ 1004.382961]  ? printk+0x9f/0xc5
[ 1004.382962]  ? log_store.cold.28+0x11/0x11
[ 1004.382964]  ? __unwind_start+0x8e/0x9f0
[ 1004.382966]  print_address_description+0x65/0x22e
[ 1004.382968]  ? __unwind_start+0x8e/0x9f0
[ 1004.382970]  kasan_report.cold.6+0x241/0x2fd
[ 1004.382972]  memset+0x1f/0x40
[ 1004.382974]  __unwind_start+0x8e/0x9f0
[ 1004.382979]  ? thread_generic_wrapper+0x1c8/0x270 [spl]
[ 1004.382981]  ? kthread+0x2e2/0x3a0
[ 1004.382982]  ? ret_from_fork+0x35/0x40
[ 1004.382984]  ? unwind_next_frame+0x18d0/0x18d0
[ 1004.382987]  ? rcu_sched_qs.part.60+0x90/0x90
[ 1004.382991]  ? luaD_precall+0x913/0x17b0 [zlua]
[ 1004.382995]  ? luaD_precall+0xcf7/0x17b0 [zlua]
[ 1004.382998]  __save_stack_trace+0x61/0x110
[ 1004.383001]  ? luaD_call+0x1a8/0x3b0 [zlua]
[ 1004.383003]  ? rcu_sched_qs.part.60+0x90/0x90
[ 1004.383006]  ? resume_cb+0x360/0x360 [zlua]
[ 1004.383010]  ? luaD_call+0x2d1/0x3b0 [zlua]
[ 1004.383011]  ? mutex_unlock+0x1d/0x40
[ 1004.383013]  save_stack+0x32/0xb0
[ 1004.383017]  ? cv_destroy_wakeup+0x160/0x330 [spl]
[ 1004.383021]  ? cv_timedwait_idle_hires+0x150/0x150 [spl]
[ 1004.383022]  ? mutex_unlock+0x1d/0x40
[ 1004.383080]  ? zio_done+0x144d/0x5a30 [zfs]
[ 1004.383085]  ? __cv_destroy+0x219/0x590 [spl]
[ 1004.383089]  ? cv_destroy_wakeup+0x330/0x330 [spl]
[ 1004.383091]  ? mutex_unlock+0x1d/0x40
[ 1004.383136]  ? metaslab_trace_fini+0x1c7/0x480 [zfs]
[ 1004.383185]  ? zio_pop_transforms+0x2c0/0x2c0 [zfs]
[ 1004.383228]  ? metaslab_trace_init+0x1a0/0x1a0 [zfs]
[ 1004.383230]  __kasan_slab_free+0x125/0x170
[ 1004.383234]  ? spl_kmem_cache_free+0x331/0x790 [spl]
[ 1004.383235]  kmem_cache_free+0x78/0x210
[ 1004.383239]  spl_kmem_cache_free+0x331/0x790 [spl]
[ 1004.383288]  zio_destroy+0x382/0x4f0 [zfs]
[ 1004.383336]  ? zio_execute+0x680/0x680 [zfs]
[ 1004.383339]  ? kvm_clock_get_cycles+0xd/0x10
[ 1004.383341]  ? ktime_get+0x1e0/0x1e0
[ 1004.383353]  ? mutex_lock+0xaf/0x140
[ 1004.383404]  zio_wait+0x8fd/0xc50 [zfs]
[ 1004.383452]  ? zio_deadman+0x190/0x190 [zfs]
[ 1004.383488]  ? bplist_append+0x4e0/0x4e0 [zfs]
[ 1004.383539]  ? zio_add_child+0xaa0/0xaa0 [zfs]
[ 1004.383590]  ? zap_lookup_norm+0x14b/0x180 [zfs]
[ 1004.383638]  ? vdev_indirect_should_condense+0x15c/0x560 [zfs]
[ 1004.383686]  ? zio_null+0x2b/0x30 [zfs]
[ 1004.383731]  spa_sync+0x1fc0/0x38d0 [zfs]
[ 1004.383777]  ? spa_load_verify_cb+0x560/0x560 [zfs]
[ 1004.383823]  ? vdev_get_stats_ex_impl+0x5a0/0xb30 [zfs]
[ 1004.383825]  ? __mutex_lock_slowpath+0x10/0x10
[ 1004.383828]  ? rcu_sched_qs.part.60+0x90/0x90
[ 1004.383830]  ? kvm_clock_get_cycles+0xd/0x10
[ 1004.383832]  ? mutex_lock+0xaf/0x140
[ 1004.383834]  ? __mutex_lock_slowpath+0x10/0x10
[ 1004.383837]  ? __cv_broadcast+0x14b/0x210 [spl]
[ 1004.383839]  ? mutex_unlock+0x1d/0x40
[ 1004.383885]  ? spa_txg_history_set.part.6+0x2d9/0x520 [zfs]
[ 1004.383931]  ? spa_mmp_history_clear+0x210/0x210 [zfs]
[ 1004.383976]  ? spa_config_exit+0x19b/0x2e0 [zfs]
[ 1004.384021]  ? spa_remove+0x1ce0/0x1ce0 [zfs]
[ 1004.384025]  ? spl_kmem_alloc+0x166/0x1c0 [spl]
[ 1004.384071]  ? spa_txg_history_init_io+0x1f7/0x270 [zfs]
[ 1004.384117]  ? spa_txg_history_set+0x20/0x20 [zfs]
[ 1004.384120]  ? __cv_signal+0x210/0x210 [spl]
[ 1004.384167]  txg_sync_thread+0x7cc/0x1410 [zfs]
[ 1004.384216]  ? txg_thread_exit.isra.8+0x310/0x310 [zfs]
[ 1004.384221]  ? __kasan_slab_free+0x13a/0x170
[ 1004.384266]  ? txg_thread_exit.isra.8+0x310/0x310 [zfs]
[ 1004.384271]  thread_generic_wrapper+0x1c8/0x270 [spl]
[ 1004.384275]  ? __thread_exit+0x20/0x20 [spl]
[ 1004.384277]  ? print_dl_stats+0x70/0x70
[ 1004.384278]  ? _raw_spin_lock_irqsave+0x25/0x50
[ 1004.384280]  ? __kthread_parkme+0x82/0xf0
[ 1004.384284]  ? __thread_exit+0x20/0x20 [spl]
[ 1004.384285]  kthread+0x2e2/0x3a0
[ 1004.384287]  ? kthread_bind+0x30/0x30
[ 1004.384289]  ret_from_fork+0x35/0x40

[ 1004.384300] The buggy address belongs to the page:
[ 1004.384318] page:ffffea00063547c0 count:0 mapcount:0 mapping:0000000000000000 index:0x0
[ 1004.384356] flags: 0x17fffc000000000()
[ 1004.384372] raw: 017fffc000000000 0000000000000000 dead000000000200 0000000000000000
[ 1004.384399] raw: 0000000000000000 0000000000000000 00000000ffffffff 0000000000000000
[ 1004.384426] page dumped because: kasan: bad access detected

[ 1004.384452] Memory state around the buggy address:
[ 1004.384471]  ffff88818d51f000: 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 f2 f2
[ 1004.384496]  ffff88818d51f080: f2 f2 f2 f2 f2 00 f2 f2 f2 00 00 00 00 00 00 00
[ 1004.384521] >ffff88818d51f100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f1 f8
[ 1004.384546]                                                              ^
[ 1004.384571]  ffff88818d51f180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[ 1004.384596]  ffff88818d51f200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[ 1004.384620] ==================================================================
[ 2651.078942]  zd16: p1 p9
@rincebrain rincebrain added Status: Triage Needed New issue which needs to be triaged Type: Defect Incorrect behavior (e.g. crash, hang) labels Jun 12, 2021
This was referenced Jan 6, 2022
Merged
Open
@rincebrain rincebrain mentioned this issue May 11, 2022
Merged
13 tasks
@stale
Copy link

stale bot commented Jun 12, 2022

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the Status: Stale No recent activity for issue label Jun 12, 2022
@rincebrain
Copy link
Contributor Author

rincebrain commented Jun 12, 2022 via email

@stale stale bot removed the Status: Stale No recent activity for issue label Jun 12, 2022
@behlendorf behlendorf added Bot: Not Stale Override for the stale bot and removed Status: Triage Needed New issue which needs to be triaged labels Jun 14, 2022
@rincebrain rincebrain mentioned this issue Dec 14, 2023
Closed
@rincebrain rincebrain mentioned this issue Sep 16, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bot: Not Stale Override for the stale bot Type: Defect Incorrect behavior (e.g. crash, hang)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@behlendorf @rincebrain