Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unaligned writes and other errors during trim #8552

Closed
lnicola opened this issue Mar 31, 2019 · 4 comments
Closed

Unaligned writes and other errors during trim #8552

lnicola opened this issue Mar 31, 2019 · 4 comments
Labels
Type: Defect Incorrect behavior (e.g. crash, hang)

Comments

@lnicola
Copy link
Contributor

lnicola commented Mar 31, 2019 •
edited
Loading

System information

Distribution Name | Arch Linux
Linux Kernel | 5.0.5-arch1-1-ARCH
Architecture | x64
ZFS Version | 0.8.0-rc3_131_ge03b25a56
SPL Version | 0.8.0-rc3_131_ge03b25a56

Describe the problem you're observing

I tried to do a manual trim and the whole system hung for a couple minutes, but then came back. Running a scrub afterwards reported no errors.

Include any warning/errors/backtraces from the system logs

[  307.971709] ata4.00: exception Emask 0x0 SAct 0x180008e0 SErr 0x40000 action 0x6 frozen
[  307.971857] ata4: SError: { CommWake }
[  307.971924] ata4.00: failed command: SEND FPDMA QUEUED
[  307.972013] ata4.00: cmd 64/01:28:00:00:00/00:00:00:00:00/a0 tag 5 ncq dma 512 out
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  307.972231] ata4.00: status: { DRDY }
[  307.972292] ata4.00: failed command: WRITE FPDMA QUEUED
[  307.972382] ata4.00: cmd 61/01:30:bd:97:5c/00:00:1c:00:00/40 tag 6 ncq dma 512 out
                        res 40/00:01:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  307.972599] ata4.00: status: { DRDY }
[  307.972660] ata4.00: failed command: WRITE FPDMA QUEUED
[  307.972750] ata4.00: cmd 61/08:38:be:97:5c/00:00:1c:00:00/40 tag 7 ncq dma 4096 out
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  307.972968] ata4.00: status: { DRDY }
[  307.973030] ata4.00: failed command: READ FPDMA QUEUED
[  307.973118] ata4.00: cmd 60/10:58:00:0d:1c/00:00:13:00:00/40 tag 11 ncq dma 8192 in
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  307.973336] ata4.00: status: { DRDY }
[  307.973398] ata4.00: failed command: WRITE FPDMA QUEUED
[  307.973488] ata4.00: cmd 61/10:d8:00:f2:50/00:00:14:00:00/40 tag 27 ncq dma 8192 out
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  307.973707] ata4.00: status: { DRDY }
[  307.973768] ata4.00: failed command: WRITE FPDMA QUEUED
[  307.973858] ata4.00: cmd 61/10:e0:60:f3:50/00:00:14:00:00/40 tag 28 ncq dma 8192 out
                        res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  307.974077] ata4.00: status: { DRDY }
[  307.974141] ata4: hard resetting link
[  308.448112] ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[  308.448668] ata4.00: supports DRM functions and may not be fully accessible
[  308.454602] ata4.00: supports DRM functions and may not be fully accessible
[  308.459393] ata4.00: configured for UDMA/133
[  308.459576] ata4.00: device reported invalid CHS sector 0
[  308.459582] ata4.00: device reported invalid CHS sector 0
[  308.459586] ata4.00: device reported invalid CHS sector 0
[  308.459591] ata4.00: device reported invalid CHS sector 0
[  308.459636] sd 3:0:0:0: [sdc] tag#6 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  308.459643] sd 3:0:0:0: [sdc] tag#6 Sense Key : Illegal Request [current] 
[  308.459649] sd 3:0:0:0: [sdc] tag#6 Add. Sense: Unaligned write command
[  308.459657] sd 3:0:0:0: [sdc] tag#6 CDB: Write(10) 2a 00 1c 5c 97 bd 00 00 01 00
[  308.459662] print_req_error: I/O error, dev sdc, sector 475830205 flags 701
[  308.459798] zio pool=tank vdev=/dev/sdc3 error=5 type=2 offset=7305728 size=512 flags=808c0
[  308.459865] sd 3:0:0:0: [sdc] tag#7 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  308.459872] sd 3:0:0:0: [sdc] tag#7 Sense Key : Illegal Request [current] 
[  308.459878] sd 3:0:0:0: [sdc] tag#7 Add. Sense: Unaligned write command
[  308.459884] sd 3:0:0:0: [sdc] tag#7 CDB: Write(10) 2a 00 1c 5c 97 be 00 00 08 00
[  308.459888] print_req_error: I/O error, dev sdc, sector 475830206 flags 701
[  308.460008] zio pool=tank vdev=/dev/sdc3 error=5 type=2 offset=7306240 size=4096 flags=808c0
[  308.460063] sd 3:0:0:0: [sdc] tag#11 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  308.460069] sd 3:0:0:0: [sdc] tag#11 Sense Key : Illegal Request [current] 
[  308.460075] sd 3:0:0:0: [sdc] tag#11 Add. Sense: Unaligned write command
[  308.460082] sd 3:0:0:0: [sdc] tag#11 CDB: Read(10) 28 00 13 1c 0d 00 00 00 10 00
[  308.460086] print_req_error: I/O error, dev sdc, sector 320605440 flags 700
[  308.460209] zio pool=smart vdev=/dev/disk/by-id/ata-Samsung_SSD_860_EVO_250GB_S3YJNB0K339667A-part2 error=5 type=1 offset=163880501248 size=8192 flags=180880
[  308.460273] sd 3:0:0:0: [sdc] tag#27 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  308.460280] sd 3:0:0:0: [sdc] tag#27 Sense Key : Illegal Request [current] 
[  308.460285] sd 3:0:0:0: [sdc] tag#27 Add. Sense: Unaligned write command
[  308.460292] sd 3:0:0:0: [sdc] tag#27 CDB: Write(10) 2a 00 14 50 f2 00 00 00 10 00
[  308.460296] print_req_error: I/O error, dev sdc, sector 340849152 flags 701
[  308.460417] zio pool=smart vdev=/dev/disk/by-id/ata-Samsung_SSD_860_EVO_250GB_S3YJNB0K339667A-part2 error=5 type=2 offset=174245281792 size=8192 flags=180880
[  308.460479] sd 3:0:0:0: [sdc] tag#28 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  308.460485] sd 3:0:0:0: [sdc] tag#28 Sense Key : Illegal Request [current] 
[  308.460491] sd 3:0:0:0: [sdc] tag#28 Add. Sense: Unaligned write command
[  308.460497] sd 3:0:0:0: [sdc] tag#28 CDB: Write(10) 2a 00 14 50 f3 60 00 00 10 00
[  308.460502] print_req_error: I/O error, dev sdc, sector 340849504 flags 701
[  308.460621] zio pool=smart vdev=/dev/disk/by-id/ata-Samsung_SSD_860_EVO_250GB_S3YJNB0K339667A-part2 error=5 type=2 offset=174245462016 size=8192 flags=180880
[  308.460680] ata4: EH complete
[  308.460908] ata4.00: Enabling discard_zeroes_data
[  315.184292] do_IRQ: 3.33 No irq handler for vector
[  348.332634] ata4.00: exception Emask 0x0 SAct 0x3c0f401f SErr 0x0 action 0x6 frozen
[  348.332780] ata4.00: failed command: READ FPDMA QUEUED
[  348.332871] ata4.00: cmd 60/40:00:30:a5:cb/00:00:13:00:00/40 tag 0 ncq dma 32768 in
                        res 40/00:01:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  348.333091] ata4.00: status: { DRDY }
[  348.333153] ata4.00: failed command: WRITE FPDMA QUEUED
[  348.333243] ata4.00: cmd 61/00:08:d0:f5:50/01:00:14:00:00/40 tag 1 ncq dma 131072 out
                        res 40/00:01:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  348.333463] ata4.00: status: { DRDY }
[  348.333524] ata4.00: failed command: WRITE FPDMA QUEUED
[  348.333614] ata4.00: cmd 61/10:10:d0:f6:50/00:00:14:00:00/40 tag 2 ncq dma 8192 out
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  348.333832] ata4.00: status: { DRDY }
[  348.333894] ata4.00: failed command: WRITE FPDMA QUEUED
[  348.333983] ata4.00: cmd 61/30:18:d0:f7:50/00:00:14:00:00/40 tag 3 ncq dma 24576 out
                        res 40/00:01:06:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  348.334203] ata4.00: status: { DRDY }
[  348.334264] ata4.00: failed command: WRITE FPDMA QUEUED
[  348.334354] ata4.00: cmd 61/35:20:c6:97:5c/00:00:1c:00:00/40 tag 4 ncq dma 27136 out
                        res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  348.334573] ata4.00: status: { DRDY }
[  348.334635] ata4.00: failed command: READ FPDMA QUEUED
[  348.334724] ata4.00: cmd 60/10:70:10:56:5c/00:00:1c:00:00/40 tag 14 ncq dma 8192 in
                        res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  348.334941] ata4.00: status: { DRDY }
[  348.335003] ata4.00: failed command: READ FPDMA QUEUED
[  348.335091] ata4.00: cmd 60/10:80:10:54:5c/00:00:1c:00:00/40 tag 16 ncq dma 8192 in
                        res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  348.335309] ata4.00: status: { DRDY }
[  348.335370] ata4.00: failed command: SEND FPDMA QUEUED
[  348.335505] ata4.00: cmd 64/01:88:00:00:00/00:00:00:00:00/a0 tag 17 ncq dma 512 out
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  348.335729] ata4.00: status: { DRDY }
[  348.340548] ata4.00: failed command: WRITE FPDMA QUEUED
[  348.345410] ata4.00: cmd 61/80:90:70:ee:50/00:00:14:00:00/40 tag 18 ncq dma 65536 out
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  348.355218] ata4.00: status: { DRDY }
[  348.358052] ata4.00: failed command: WRITE FPDMA QUEUED
[  348.360833] ata4.00: cmd 61/10:98:10:0a:08/00:00:00:00:00/40 tag 19 ncq dma 8192 out
                        res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  348.366853] ata4.00: status: { DRDY }
[  348.372130] ata4.00: failed command: SEND FPDMA QUEUED
[  348.377329] ata4.00: cmd 64/01:d0:00:00:00/00:00:00:00:00/a0 tag 26 ncq dma 512 out
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  348.387641] ata4.00: status: { DRDY }
[  348.390624] ata4.00: failed command: READ FPDMA QUEUED
[  348.395036] ata4.00: cmd 60/10:d8:10:62:5c/00:00:1c:00:00/40 tag 27 ncq dma 8192 in
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  348.404798] ata4.00: status: { DRDY }
[  348.409659] ata4.00: failed command: READ FPDMA QUEUED
[  348.412640] ata4.00: cmd 60/10:e0:10:54:dc/00:00:1c:00:00/40 tag 28 ncq dma 8192 in
                        res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  348.418455] ata4.00: status: { DRDY }
[  348.421383] ata4.00: failed command: READ FPDMA QUEUED
[  348.424281] ata4.00: cmd 60/10:e8:10:56:dc/00:00:1c:00:00/40 tag 29 ncq dma 8192 in
                        res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  348.430294] ata4.00: status: { DRDY }
[  348.435239] ata4: hard resetting link
[  348.908953] ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[  348.909288] ata4.00: supports DRM functions and may not be fully accessible
[  348.912924] ata4.00: supports DRM functions and may not be fully accessible
[  348.916243] ata4.00: configured for UDMA/133
[  348.916333] ata4.00: device reported invalid CHS sector 0
[  348.916342] ata4.00: device reported invalid CHS sector 0
[  348.916346] ata4.00: device reported invalid CHS sector 0
[  348.916351] ata4.00: device reported invalid CHS sector 0
[  348.916355] ata4.00: device reported invalid CHS sector 0
[  348.916380] sd 3:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  348.916387] sd 3:0:0:0: [sdc] tag#0 Sense Key : Illegal Request [current] 
[  348.916393] sd 3:0:0:0: [sdc] tag#0 Add. Sense: Unaligned write command
[  348.916401] sd 3:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 13 cb a5 30 00 00 40 00
[  348.916406] print_req_error: I/O error, dev sdc, sector 332113200 flags 700
[  348.921640] zio pool=smart vdev=/dev/disk/by-id/ata-Samsung_SSD_860_EVO_250GB_S3YJNB0K339667A-part2 error=5 type=1 offset=169772474368 size=32768 flags=180880
[  348.921715] sd 3:0:0:0: [sdc] tag#1 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  348.921722] sd 3:0:0:0: [sdc] tag#1 Sense Key : Illegal Request [current] 
[  348.921728] sd 3:0:0:0: [sdc] tag#1 Add. Sense: Unaligned write command
[  348.921734] sd 3:0:0:0: [sdc] tag#1 CDB: Write(10) 2a 00 14 50 f5 d0 00 01 00 00
[  348.921738] print_req_error: I/O error, dev sdc, sector 340850128 flags 701
[  348.926974] zio pool=smart vdev=/dev/disk/by-id/ata-Samsung_SSD_860_EVO_250GB_S3YJNB0K339667A-part2 error=5 type=2 offset=174245781504 size=131072 flags=180880
[  348.927029] sd 3:0:0:0: [sdc] tag#2 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  348.927036] sd 3:0:0:0: [sdc] tag#2 Sense Key : Illegal Request [current] 
[  348.927042] sd 3:0:0:0: [sdc] tag#2 Add. Sense: Unaligned write command
[  348.927048] sd 3:0:0:0: [sdc] tag#2 CDB: Write(10) 2a 00 14 50 f6 d0 00 00 10 00
[  348.927052] print_req_error: I/O error, dev sdc, sector 340850384 flags 701
[  348.932240] zio pool=smart vdev=/dev/disk/by-id/ata-Samsung_SSD_860_EVO_250GB_S3YJNB0K339667A-part2 error=5 type=2 offset=174245912576 size=8192 flags=180880
[  348.932316] sd 3:0:0:0: [sdc] tag#3 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  348.932323] sd 3:0:0:0: [sdc] tag#3 Sense Key : Illegal Request [current] 
[  348.932328] sd 3:0:0:0: [sdc] tag#3 Add. Sense: Unaligned write command
[  348.932335] sd 3:0:0:0: [sdc] tag#3 CDB: Write(10) 2a 00 14 50 f7 d0 00 00 30 00
[  348.932339] print_req_error: I/O error, dev sdc, sector 340850640 flags 701
[  348.935253] zio pool=smart vdev=/dev/disk/by-id/ata-Samsung_SSD_860_EVO_250GB_S3YJNB0K339667A-part2 error=5 type=2 offset=174246043648 size=24576 flags=180880
[  348.935283] sd 3:0:0:0: [sdc] tag#4 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  348.935287] sd 3:0:0:0: [sdc] tag#4 Sense Key : Illegal Request [current] 
[  348.935290] sd 3:0:0:0: [sdc] tag#4 Add. Sense: Unaligned write command
[  348.935294] sd 3:0:0:0: [sdc] tag#4 CDB: Write(10) 2a 00 1c 5c 97 c6 00 00 35 00
[  348.935296] print_req_error: I/O error, dev sdc, sector 475830214 flags 701
[  348.938115] zio pool=tank vdev=/dev/sdc3 error=5 type=2 offset=7310336 size=27136 flags=40080cc0
[  348.938175] sd 3:0:0:0: [sdc] tag#18 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  348.938179] sd 3:0:0:0: [sdc] tag#18 Sense Key : Illegal Request [current] 
[  348.938182] sd 3:0:0:0: [sdc] tag#18 Add. Sense: Unaligned write command
[  348.938186] sd 3:0:0:0: [sdc] tag#18 CDB: Write(10) 2a 00 14 50 ee 70 00 00 80 00
[  348.938188] print_req_error: I/O error, dev sdc, sector 340848240 flags 701
[  348.941011] zio pool=smart vdev=/dev/disk/by-id/ata-Samsung_SSD_860_EVO_250GB_S3YJNB0K339667A-part2 error=5 type=2 offset=174244814848 size=65536 flags=180880
[  348.941072] ata4: EH complete
[  348.941290] ata4.00: Enabling discard_zeroes_data
[  351.766790] do_IRQ: 3.33 No irq handler for vector
[  379.280917] ata4.00: exception Emask 0x0 SAct 0x3c019 SErr 0x0 action 0x6 frozen
[  379.285927] ata4.00: failed command: WRITE FPDMA QUEUED
[  379.290931] ata4.00: cmd 61/10:00:10:62:5c/00:00:1c:00:00/40 tag 0 ncq dma 8192 out
                        res 40/00:01:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  379.296593] ata4.00: status: { DRDY }
[  379.299372] ata4.00: failed command: WRITE FPDMA QUEUED
[  379.302190] ata4.00: cmd 61/10:18:10:56:5c/00:00:1c:00:00/40 tag 3 ncq dma 8192 out
                        res 40/00:01:06:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  379.308378] ata4.00: status: { DRDY }
[  379.313607] ata4.00: failed command: WRITE FPDMA QUEUED
[  379.318752] ata4.00: cmd 61/10:20:10:54:5c/00:00:1c:00:00/40 tag 4 ncq dma 8192 out
                        res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  379.329059] ata4.00: status: { DRDY }
[  379.332024] ata4.00: failed command: READ FPDMA QUEUED
[  379.334891] ata4.00: cmd 60/10:70:10:54:dc/00:00:1c:00:00/40 tag 14 ncq dma 8192 in
                        res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  379.340637] ata4.00: status: { DRDY }
[  379.343497] ata4.00: failed command: READ FPDMA QUEUED
[  379.346369] ata4.00: cmd 60/10:78:10:56:dc/00:00:1c:00:00/40 tag 15 ncq dma 8192 in
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  379.352116] ata4.00: status: { DRDY }
[  379.354983] ata4.00: failed command: SEND FPDMA QUEUED
[  379.358227] ata4.00: cmd 64/01:80:00:00:00/00:00:00:00:00/a0 tag 16 ncq dma 512 out
                        res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  379.365913] ata4.00: status: { DRDY }
[  379.370496] ata4.00: failed command: WRITE FPDMA QUEUED
[  379.375080] ata4.00: cmd 61/50:88:10:fb:50/00:00:14:00:00/40 tag 17 ncq dma 40960 out
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  379.384282] ata4.00: status: { DRDY }
[  379.387816] ata4: hard resetting link
[  379.857427] ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[  379.858099] ata4.00: supports DRM functions and may not be fully accessible
[  379.865412] ata4.00: supports DRM functions and may not be fully accessible
[  379.870215] ata4.00: configured for UDMA/133
[  379.870396] ata4.00: device reported invalid CHS sector 0
[  379.870402] ata4.00: device reported invalid CHS sector 0
[  379.870514] sd 3:0:0:0: [sdc] tag#17 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  379.870521] sd 3:0:0:0: [sdc] tag#17 Sense Key : Illegal Request [current] 
[  379.870528] sd 3:0:0:0: [sdc] tag#17 Add. Sense: Unaligned write command
[  379.870535] sd 3:0:0:0: [sdc] tag#17 CDB: Write(10) 2a 00 14 50 fb 10 00 00 50 00
[  379.870541] print_req_error: I/O error, dev sdc, sector 340851472 flags 701
[  379.875748] zio pool=smart vdev=/dev/disk/by-id/ata-Samsung_SSD_860_EVO_250GB_S3YJNB0K339667A-part2 error=5 type=2 offset=174246469632 size=40960 flags=180880
[  379.875830] ata4: EH complete
[  379.876047] ata4.00: Enabling discard_zeroes_data
[  388.366795] do_IRQ: 3.33 No irq handler for vector
[  424.984982] do_IRQ: 3.33 No irq handler for vector
[  439.031311] ata4.00: NCQ disabled due to excessive errors
[  439.031320] ata4.00: exception Emask 0x0 SAct 0x80b83c1e SErr 0x0 action 0x6 frozen
[  439.036562] ata4.00: failed command: WRITE FPDMA QUEUED
[  439.041724] ata4.00: cmd 61/50:08:10:fb:50/00:00:14:00:00/40 tag 1 ncq dma 40960 out
                        res 40/00:01:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  439.052101] ata4.00: status: { DRDY }
[  439.055076] ata4.00: failed command: WRITE FPDMA QUEUED
[  439.057978] ata4.00: cmd 61/30:10:d0:f7:50/00:00:14:00:00/40 tag 2 ncq dma 24576 out
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  439.064199] ata4.00: status: { DRDY }
[  439.069502] ata4.00: failed command: WRITE FPDMA QUEUED
[  439.074704] ata4.00: cmd 61/10:18:d0:f6:50/00:00:14:00:00/40 tag 3 ncq dma 8192 out
                        res 40/00:01:06:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  439.080618] ata4.00: status: { DRDY }
[  439.083512] ata4.00: failed command: WRITE FPDMA QUEUED
[  439.086438] ata4.00: cmd 61/00:20:d0:f5:50/01:00:14:00:00/40 tag 4 ncq dma 131072 out
                        res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  439.092273] ata4.00: status: { DRDY }
[  439.095708] ata4.00: failed command: READ FPDMA QUEUED
[  439.100526] ata4.00: cmd 60/40:50:d0:6b:4b/00:00:12:00:00/40 tag 10 ncq dma 32768 in
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  439.110363] ata4.00: status: { DRDY }
[  439.115721] ata4.00: failed command: READ FPDMA QUEUED
[  439.119664] ata4.00: cmd 60/10:58:30:c4:68/00:00:01:00:00/40 tag 11 ncq dma 8192 in
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  439.127717] ata4.00: status: { DRDY }
[  439.131017] ata4.00: failed command: READ FPDMA QUEUED
[  439.134391] ata4.00: cmd 60/10:60:a0:53:68/00:00:12:00:00/40 tag 12 ncq dma 8192 in
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  439.142878] ata4.00: status: { DRDY }
[  439.146293] ata4.00: failed command: READ FPDMA QUEUED
[  439.149689] ata4.00: cmd 60/10:68:10:f2:25/00:00:1b:00:00/40 tag 13 ncq dma 8192 in
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  439.156697] ata4.00: status: { DRDY }
[  439.160844] ata4.00: failed command: READ FPDMA QUEUED
[  439.165533] ata4.00: cmd 60/10:98:10:54:dc/00:00:1c:00:00/40 tag 19 ncq dma 8192 in
                        res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  439.174805] ata4.00: status: { DRDY }
[  439.179671] ata4.00: failed command: READ FPDMA QUEUED
[  439.184636] ata4.00: cmd 60/10:a0:10:56:dc/00:00:1c:00:00/40 tag 20 ncq dma 8192 in
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  439.194771] ata4.00: status: { DRDY }
[  439.197983] ata4.00: failed command: SEND FPDMA QUEUED
[  439.203533] ata4.00: cmd 64/01:a8:00:00:00/00:00:00:00:00/a0 tag 21 ncq dma 512 out
                        res 40/00:01:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[  439.214395] ata4.00: status: { DRDY }
[  439.217562] ata4.00: failed command: WRITE FPDMA QUEUED
[  439.220600] ata4.00: cmd 61/80:b8:70:ee:50/00:00:14:00:00/40 tag 23 ncq dma 65536 out
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  439.226706] ata4.00: status: { DRDY }
[  439.229771] ata4.00: failed command: WRITE FPDMA QUEUED
[  439.232833] ata4.00: cmd 61/10:f8:60:f3:50/00:00:14:00:00/40 tag 31 ncq dma 8192 out
                        res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[  439.239074] ata4.00: status: { DRDY }
[  439.242139] ata4: hard resetting link
[  439.714292] ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[  439.714631] ata4.00: supports DRM functions and may not be fully accessible
[  439.718171] ata4.00: supports DRM functions and may not be fully accessible
[  439.721079] ata4.00: configured for UDMA/133
[  439.721144] ata4.00: device reported invalid CHS sector 0
[  439.721151] ata4.00: device reported invalid CHS sector 0
[  439.721155] ata4.00: device reported invalid CHS sector 0
[  439.721159] ata4.00: device reported invalid CHS sector 0
[  439.721163] ata4.00: device reported invalid CHS sector 0
[  439.721168] ata4.00: device reported invalid CHS sector 0
[  439.721172] ata4.00: device reported invalid CHS sector 0
[  439.721176] ata4.00: device reported invalid CHS sector 0
[  439.721266] sd 3:0:0:0: [sdc] tag#10 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  439.721273] sd 3:0:0:0: [sdc] tag#10 Sense Key : Illegal Request [current] 
[  439.721279] sd 3:0:0:0: [sdc] tag#10 Add. Sense: Unaligned write command
[  439.721287] sd 3:0:0:0: [sdc] tag#10 CDB: Read(10) 28 00 12 4b 6b d0 00 00 40 00
[  439.721293] print_req_error: I/O error, dev sdc, sector 306932688 flags 700
[  439.726785] zio pool=smart vdev=/dev/disk/by-id/ata-Samsung_SSD_860_EVO_250GB_S3YJNB0K339667A-part2 error=5 type=1 offset=156880052224 size=32768 flags=180880
[  439.726844] sd 3:0:0:0: [sdc] tag#11 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  439.726850] sd 3:0:0:0: [sdc] tag#11 Sense Key : Illegal Request [current] 
[  439.726856] sd 3:0:0:0: [sdc] tag#11 Add. Sense: Unaligned write command
[  439.726862] sd 3:0:0:0: [sdc] tag#11 CDB: Read(10) 28 00 01 68 c4 30 00 00 10 00
[  439.726867] print_req_error: I/O error, dev sdc, sector 23643184 flags 700
[  439.732283] zio pool=smart vdev=/dev/disk/by-id/ata-Samsung_SSD_860_EVO_250GB_S3YJNB0K339667A-part2 error=5 type=1 offset=11835826176 size=8192 flags=180880
[  439.732336] sd 3:0:0:0: [sdc] tag#12 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  439.732343] sd 3:0:0:0: [sdc] tag#12 Sense Key : Illegal Request [current] 
[  439.732348] sd 3:0:0:0: [sdc] tag#12 Add. Sense: Unaligned write command
[  439.732355] sd 3:0:0:0: [sdc] tag#12 CDB: Read(10) 28 00 12 68 53 a0 00 00 10 00
[  439.732359] print_req_error: I/O error, dev sdc, sector 308827040 flags 700
[  439.735568] zio pool=smart vdev=/dev/disk/by-id/ata-Samsung_SSD_860_EVO_250GB_S3YJNB0K339667A-part2 error=5 type=1 offset=157849960448 size=8192 flags=180980
[  439.735608] sd 3:0:0:0: [sdc] tag#13 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  439.735612] sd 3:0:0:0: [sdc] tag#13 Sense Key : Illegal Request [current] 
[  439.735615] sd 3:0:0:0: [sdc] tag#13 Add. Sense: Unaligned write command
[  439.735619] sd 3:0:0:0: [sdc] tag#13 CDB: Read(10) 28 00 1b 25 f2 10 00 00 10 00
[  439.735622] print_req_error: I/O error, dev sdc, sector 455471632 flags 700
[  439.738729] zio pool=smart vdev=/dev/disk/by-id/ata-Samsung_SSD_860_EVO_250GB_S3YJNB0K339667A-part2 error=5 type=1 offset=232931991552 size=8192 flags=180980
[  439.738791] ata4: EH complete
[  439.745032] ata4.00: Enabling discard_zeroes_data

  pool: smart
 state: ONLINE
  scan: scrub repaired 0B in 0 days 00:03:39 with 0 errors on Sun Mar 31 10:12:15 2019
config:

	NAME                                                   STATE     READ WRITE CKSUM
	smart                                                  ONLINE       0     0     0
	  ata-Samsung_SSD_860_EVO_250GB_S3YJNB0K339667A-part2  ONLINE       0     0     0

errors: No known data errors

EDIT: smart is a pool with a single SSD. The errors also mention a different pool because one of the SSD partitions was used as a cache device for it.
@behlendorf
Copy link
Contributor

@lnicola thanks for opening an issue for this. It does look like it could be an alignment issue. Would you mind checking the discard setting for you ssd and adding them to the issue. For example:

# grep -HR [0-9] /sys/block/sdc/discard_* /sys/block/sdc/queue/discard_*
/sys/block/sdc/discard_alignment:0
/sys/block/sdc/queue/discard_granularity:512
/sys/block/sdc/queue/discard_max_bytes:4294966784
/sys/block/sdc/queue/discard_zeroes_data:1

@behlendorf behlendorf added the Type: Defect Incorrect behavior (e.g. crash, hang) label Apr 1, 2019
@richardelling
Copy link
Contributor

More likely this is a drive firmware bug. The misaligned errors are reported against read(10) and write (10) commands, not unmap. Also, there is a track record of Samsung SSDs exhibiting similar behaviour.

@lnicola
Copy link
Contributor Author

lnicola commented Apr 2, 2019 

/sys/block/sdc/discard_alignment:0
/sys/block/sdc/queue/discard_granularity:512
/sys/block/sdc/queue/discard_max_bytes:2147450880
/sys/block/sdc/queue/discard_max_hw_bytes:2147450880
/sys/block/sdc/queue/discard_zeroes_data:0

Not sure abut the firmware version, I can't even see my drive under https://www.samsung.com/semiconductor/minisite/ssd/download/tools/.

@lnicola
Copy link
Contributor Author

lnicola commented Nov 8, 2019

Unrelated to ZFS. Samsung firmware lies about supporting queued TRIM, and the drives stop responding while being trimmed. As a workaround, boot with libata.force=noncq, trim, then remove the kernel option.

@lnicola lnicola closed this as completed Nov 8, 2019
@rincebrain rincebrain mentioned this issue Jun 30, 2021
Closed
@stuckj stuckj mentioned this issue Sep 30, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Type: Defect Incorrect behavior (e.g. crash, hang)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@behlendorf @lnicola @richardelling