A cross-platform library for fast and safe memory-mapped IO and boundary-free ring buffer.
This library defines a convenient API for reading and writing to files
using the hosts virtual memory system, as well as allocating memory and
creating circular memory regions. The design of the API strives to
both minimize the frequency of mapping system calls while still retaining
safe access. Critically, it never attempts the own the File object used
for mapping. That is, it never clones it or in any way retains it. While
this has some implications for the API (i.e. .flush()), it cannot cause
bugs outside of this library through File's leaky abstraction when cloned
and then closed.
The Map and MapMut types are primary means for allocating virtual
memory regions, both for a file and anonymously. Generally, the
Map::with_options() and MapMut::with_options() are used to specify
the mapping requirements. See Options for more information.
The MapMut type maintains interior mutability for the mapped memory,
while the Map is read-only. However, it is possible to convert between
these types (.into_map_mut() and .into_map()) assuming the proper
Options are specified.
Additionally, a variety of buffer implementations are provided in the
vmap::io module. The Ring and InfiniteRing use cross-platform
optimzed circular memory mapping to remove the typical boundary problem
with most circular buffers. This ensures all ranges of the underlying byte
buffer can be viewed as a single byte slice, event when the value wraps
back around to the beginning of the buffer. The BufReader and BufWriter
implement buffered I/O using a Ring as a backing layer.
Take a look at the Documentation for details!
use vmap::Map;
use std::{fs, str};
let path = "example";
// Write some test data
fs::write(&path, b"this is a test")?;
// Map the first 4 bytes
let (map, file) = Map::with_options().len(4).open(&path)?;
assert_eq!(Ok("this"), str::from_utf8(&map[..]));
// Reuse the file to map a different region
let map = Map::with_options().offset(10).len(4).map(&file)?;
assert_eq!(Ok("test"), str::from_utf8(&map[..]));If opened properly, the Map can be moved into a MapMut and modifications
to the underlying file can be performed:
use vmap::Map;
use std::{fs, str};
let path = "example";
// Write some test data
fs::write(&path, b"this is a test")?;
// Open with write permissions so the Map can be converted into a MapMut
let (map, file) = Map::with_options().write().len(14).open(&path)?;
assert_eq!(Ok("this is a test"), str::from_utf8(&map[..]));
// Move the Map into a MapMut
// ... we could have started with MapMut::with_options()
let mut map = map.into_map_mut()?;
map[..4].clone_from_slice(b"that");
// Flush the changes to disk synchronously
map.flush(&file, Flush::Sync)?;
// Move the MapMut back into a Map
let map = map.into_map()?;
assert_eq!(Ok("that is a test"), str::from_utf8(&map[..]));The vmap library contains a Ring that constructs a circular memory
allocation where values can wrap from around from the end of the buffer back
to the beginning with sequential memory addresses. The InfiniteRing is
similar, however it allows writes to overwrite reads.
use vmap::io::{Ring, SeqWrite};
use std::io::{BufRead, Read, Write};
let mut buf = Ring::new(4000).unwrap();
let mut i = 1;
// Fill up the buffer with lines.
while buf.write_len() > 20 {
write!(&mut buf, "this is test line {}\n", i)?;
i += 1;
}
// No more space is available.
assert!(write!(&mut buf, "this is test line {}\n", i).is_err());
let mut line = String::new();
// Read the first line written.
let len = buf.read_line(&mut line)?;
assert_eq!(line, "this is test line 1\n");
line.clear();
// Read the second line written.
let len = buf.read_line(&mut line)?;
assert_eq!(line, "this is test line 2\n");
// Now there is enough space to write more.
write!(&mut buf, "this is test line {}\n", i)?;