Worlds in RAM can lose data in 1 contrived circumstance

[marcuswhybrow]

I experienced a problem (not a bug) using Ahtenus' script that will also occur in MSM. The ultimate effect is the loss of world changes since the in RAM world was last synced back to disk.

The problem is syncing from RAM to disk only happens in two places: when msm <server> worlds to disk or msm <server> stop is called. In the event that the server is stopped in any other way the world will not be synced back to disk, and will be overridden by the disk version when starting the server again.

RAM-to-disk syncing needs to be more aggressive to protect against these edge cases. I suggest invoking to-disk syncing whenever the java process stops lilke:

java ... -jar server.jar ; rsync world_ram_path world_disk_path

This would always run, if someone entered the /sop in-game command–as I discovered the problem–or if the server crashed. It would not run however if the screen process was killed, but that would involve busy waiting for the process to die.

---

Ultimately we need to know on start up which version of a world is the latest. This can be inferred to some degree, but a "latest" flag in the RAM or disk version would make things certain. Or it could be deduced from modified times if rsync effects those.

Note: the latest flag does not avoid the need to sync to disk when a server stops, since a host crash would cause data loss the to RAM version.

[jmarler]

This is happening to me on a frequent basis, and it is extremely frustrating. It looks like I lost several days worth of changes to the server when the backup ran. I have the standard cron.d file, but the backup does not run every morning. I have a new backup file today and also lost all changes from several days. I am running the world for the server in RAM and also using the worldstorage directory. I believe I have everything setup correctly, but I have lost changes three times now and it is extremely frustrating to my users and myself as well. If this issue is entirely based around RAM disk usage, I guess I will just have to stop using RAM until we can get it sorted out. I am happy to do testing if needed.

[marcuswhybrow]

Yes that's right. You can avoid this danger by not putting your world into RAM, and storing it only on disk.

I laid out a probable solution to the problem in the above issue, so I just need to implement it when I get the chance.

[jmarler]

I found a project that may be perfect for use with Minecraft servers: EPRD (See http://sourceforge.net/projects/eprd/) It is the "Eventually Persistent RamDisk" kernel module. It creates a filesystem in a file. You can adjust how much of the filesystem is cached in RAM at a time (including the full filesystem), how often it is flushed to disk, and even supports flushing to disk when a sync() call is made. It basically gives us the ability to use a ramdisk, adjust the parameters, but takes the management of that ramdisk out of msm. The filesystems are easy to create, mount, unmount, etc. I am going to try using it on my minecraft server with msm in "disk" mode. I will let you know how it turns out, but I welcome you to check it out. It looks very promising and could make the ramdisk functionality a bit more robust.

[marcuswhybrow]

An excellent suggestion, would it be easy to install as a requirement of MSM? It doesn't appear to be in the ubuntu repository for example.

[jmarler]

It was simple for me, but I'm a bit biased as I am a Debian package maintainer. It includes a kernel driver and a single executable to manage the ramdisk devices. It compiles extremely quickly and without any issues if you have the correct kernel source headers, compiler, linker, etc. Once I realized that I needed to delete the /dev/shm/msm folder AND the "inram" file it started cooperating with msm very well. I created a 128MB ext3 ramdisk (double the size of my map) and set it to flush to disk every 5 minutes. I have the image file stored in the worldstorage directory and mounted as [server]/worldstorage/world and let msm manage the symlink to [server]/world so that backups and such will continue to work. The server I run is virtualized under ESXi 5 w/ R10 SATA local storage, so it can get a bit laggy when running from disk. So far results seem very positive and the server isn't too laggy. I am going to try exploring some sections of the map that are uncharted in creative mode to see if a large volume of updates has any kind of effect.

I believe the maintainer of the package is a DM as well, but I will have to see if he has filed an ITP for eprd. It didn't pop up in google, but I'll check the maillist archives. It shouldn't be too hard to package up, but would take two separate packages I believe. One for the CLI tool, and one for the kernel module.

[kjkazinski]

With all the different versions of linux people are running this might not work as you intended. Also, I do not use a RAM disk at all because I do not have that much RAM in the computer.