Disk error on NetBSD

[fish4terrisa-MSDSM]

I have just got this error:

WARN: Possible deadlock at src/devices/nvme.c@346                                                 
WARN: The lock was previously held at src/devices/nvme.c@330
WARN: Version: RVVM v0.6-unknown                                                                  
WARN: Attempting to recover execution...                                                           
* * * * * * *                                                                                                                                                                                      WARN: Possible deadlock at src/devices/nvme.c@346                                                 
* WARN: The lock was previously held at src/devices/nvme.c@330
WARN: Version: RVVM v0.6-unknown
WARN: Attempting to recover execution...
 * * * * * * *

Is that a bug of RVVM or SDF?
P.S. The time of the files on SDF is inaccurate.That might be the problem(?)
Is it possible to disable the lock by the user?

[fish4terrisa-MSDSM]

I have just got this error:

WARN: Possible deadlock at src/devices/nvme.c@346                                                 
WARN: The lock was previously held at src/devices/nvme.c@330
WARN: Version: RVVM v0.6-unknown                                                                  
WARN: Attempting to recover execution...                                                           
* * * * * * *                                                                                                                                                                                      WARN: Possible deadlock at src/devices/nvme.c@346                                                 
* WARN: The lock was previously held at src/devices/nvme.c@330
WARN: Version: RVVM v0.6-unknown
WARN: Attempting to recover execution...
 * * * * * * *

Is that a bug of RVVM or SDF? P.S. The time of the files on SDF is inaccurate.That might be the problem(?) Is it possible to disable the lock by the user?

Maybe it just waited a bit long , or the time is wrong .So is it possible to keep wait even through a deadlock is possible?

[fish4terrisa-MSDSM]

I have just got this error:

WARN: Possible deadlock at src/devices/nvme.c@346                                                 
WARN: The lock was previously held at src/devices/nvme.c@330
WARN: Version: RVVM v0.6-unknown                                                                  
WARN: Attempting to recover execution...                                                           
* * * * * * *                                                                                                                                                                                      WARN: Possible deadlock at src/devices/nvme.c@346                                                 
* WARN: The lock was previously held at src/devices/nvme.c@330
WARN: Version: RVVM v0.6-unknown
WARN: Attempting to recover execution...
 * * * * * * *

Is that a bug of RVVM or SDF? P.S. The time of the files on SDF is inaccurate.That might be the problem(?) Is it possible to disable the lock by the user?

Maybe it just waited a bit long , or the time is wrong .So is it possible to keep wait even through a deadlock is possible?

Now I just linked spin_lock() to spin_lock_slow_real(),it seems worked fine , but I will wait for more testing,but the file reads and writes seems faster

[fish4terrisa-MSDSM]

And , I got the disk read and write slow when installing perl,that might give you some clues.

[LekKit]

It seems your host storage is executing IO syscalls for very long and the device is impatient, and thinks that something locked up somewhere (Yeah that's how I debug those things).
I'll see what can be done to improve locking efficiency, thanks for the report.

Is that a bug of RVVM or SDF?

Inefficiency of SDF disk access coupled with RVVM inefficient locking in nvme emulation

Is it possible to disable the lock by the user?

No, the lock is there to protect from dataraces. Without it, chaos and corruption ensues.
It is unrelated from on-disk file locks that protect the image from being used by >1 VM at once.

[fish4terrisa-MSDSM]

So maybe the user can choose to use _slow_real to replace the _real lock?

[LekKit]

Maybe. But I think this needs a more elaborate fix that excludes the lock from being held in IO path

[LekKit]

Working on re-queuing IO commands from NVMe ring to the threadpool queue. This could mean more IO parallelism, running IO without any locks and less stuttering when many VMs are involved.

So far it is working good enough but a slight performance drop is observed in terms of maximum IOPS. Hope to get this completed till v0.6 release.

[LekKit]

Commit 5a6466f should fix this. The locks are never held under IO, and I purposely inserted long (30 second) sleep into IO commands and can't reproduce lockup warnings anymore.
I think this issue can be closed.