Rawpart #7774
Rawpart #7774
Conversation
|
These checks must be disabled in order to use the PR: OK never mind, I've removed those checks in the following 2 commits. |
What do you think about an option to avoid this manual operation ? stg like |
|
I think anybody who's smart enough to use this without accidentally destroying their live filesystem partitions is smart enough to set a symlink. And I feel there's less chance of accidentally typo'ing the wrong device name on the commandline, by relying on a manually created symlink that you can doublecheck a few times before starting up... |
| @@ -1293,26 +1286,6 @@ BlockchainLMDB::BlockchainLMDB(bool batch_transactions): BlockchainDB() | |||
| m_hardfork = nullptr; | |||
| } | |||
|
|
|||
| void BlockchainLMDB::check_mmap_support() | |||
There was a problem hiding this comment.
Should this check be done after LMDB initializes (and therefore the flags could be checked for !rawpart) ?
| @@ -4148,6 +4179,18 @@ mdb_env_init_meta(MDB_env *env, MDB_meta *meta) | |||
|
|
|||
| psize = env->me_psize; | |||
|
|
|||
| if ((env->me_flags & (MDB_RAWPART|MDB_WRITEMAP)) == (MDB_RAWPART|MDB_WRITEMAP)) { | |||
There was a problem hiding this comment.
Is it possible to have RAWPART == true but WRITEMAP == false? Not questioning the logic or asking for a code change, I'm asking for my understanding of how LMDB works.
There was a problem hiding this comment.
Yes, and WRITEMAP is usually false on most deployments.
There was a problem hiding this comment.
Yeah sorry, brainfart. Everything in *Nix is a file, so this is just opening the rawpart like any other file (albeit one with a fixed size and location on disk) that gets written to like normal.
| @@ -4054,6 +4058,31 @@ mdb_env_read_header(MDB_env *env, int prev, MDB_meta *meta) | |||
| int i, rc, off; | |||
| enum { Size = sizeof(pbuf) }; | |||
|
|
|||
| if (env->me_flags & MDB_RAWPART) { | |||
| #define VM_ALIGN 0x200000 | |||
There was a problem hiding this comment.
This has me completely lost - I have a rough idea of what the code is doing, I just don't understand how this constant was chosen.
Shouldn't this somehow probe the size of the "rawpart" ? Or was this chosen as a sufficiently large value; the OS handles "trapping" writes outside of the RAWPART through interrupts/errors.
There was a problem hiding this comment.
The 2MB size is an optimal alignment for Flash memory erase blocks. Smaller may not be properly aligned for all devices, larger is not necessary.
There was a problem hiding this comment.
Ahh ok I did in fact misunderstand this slightly, thanks for the clarification.
Autodetects that a block device is being used.
Use mmap to read and initialize the meta pages, raw device may not support read/write syscalls.
This reverts commit bd96536. The check interferes with raw device/partition support.
The check interferes with raw device/partition support.
|
Rebased onto current master |
|
The only holdup is the lack of warning if mmap fails ... can/should that be left in? It's a somewhat useful warning to users, but maybe its ignored most of the time? |
|
If mmap fails, LMDB will fail, and the daemon is dead in the water. It's not ignorable. |
Add support for storing database on a raw device/partition. Can improve write performance.
Also may be useful for storing DB on removable storage (USB flash drives, etc.) for transporting
between online and offline daemons, and precludes the possibility of viruses hiding in a filesystem
(since there is no filesystem to hide in).
For use with monerod, would need to manually symlink <data-dir>/lmdb to the target device name.
This is only supported for POSIX systems, not WIN32. While there may be a way to access raw
devices in Windows, that usually requires Administrator access.
Note: currently monerod can't use this feature. Its startup checks for whether the target path is
a directory and if it supports mmap are interfering, they need to be skipped.