System hang / unresponsive

[rayzilt]

Hello!

I've experienced a complete system hang with the configuration explained below. The machine was still responding to ping, but wasn't able to connect through ssh. The console was also not responding (ILO Remote control). Please find attached the latest error messages I could find in the syslog.

This happened before with version zfs version 0.6.4.2-1, random crashes. Now running with the latest code I was hoping the problem was solved.

I just want to inform you guys about this moment and maybe you have some suggestions for me to prevent this in the future.

syslog.txt

Software versions;
OS: Debian Wheezy 7.10
Kernel: Linux storage 3.2.0-4-amd64 #1 SMP Debian 3.2.78-1 x86_64 GNU/Linux
ZFS: 0.6.5.7-8-wheezy
Uptime: 60 Days

Hardware:
Server: HP DL360 G8
Memory: 198G
CPU: 2x E5-2650
HBA: LSI SAS SAS2308
Enclosure: 60 HGST 4T SAS Disks
Nic: 10Gbit

ZFS Configuration:
9 x RAIDZ2 with 6 Disks
2x Zeusram 8G SLOG
2x Samsung SSD 512G L2ARC
spl_kmem_cache_slab_limit=16384

Workload:
~30 VM's running there disks though NFS.
Couple of databases servers running there data files through NFS.
Snapshot are created every 3 hours and 6 are total kept.
Was running a zfs send to a machine (30T). The stream was at 20T when the crash happened.

[D4rk4]

"~30 VM's running there disks though NFS.
Couple of databases servers running there data files through NFS."

Why? I think it's not ZFS problem...

[rayzilt]

@D4rk4 Could you elaborate what you mean? This server runs only zfs and nfs. It's not running the vm's itself, only storage for the virtual disks.

In the past we had similar problems with the older version, see this issue created by an colleague then -> #3445 (same system)

[D4rk4]

@rayzilt I think it's bad idea use NFS for VM images or as database storage. Try iSCSI.

[rayzilt]

I'm closing this issue because the problem did not occur again.