Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mmap with mmap::protection::MODIFY throws exception #92

Closed
BinaryCat17 opened this issue Feb 5, 2021 · 4 comments
Closed

mmap with mmap::protection::MODIFY throws exception #92

BinaryCat17 opened this issue Feb 5, 2021 · 4 comments

Comments

@BinaryCat17
Copy link
Contributor

BinaryCat17 commented Feb 5, 2021

I try to run this simple code and I get exception
'std::runtime_error': what(): open file mode not supported
from file "cista/targets/file.h" line 31
verify(read || write, "open file mode not supported");

I use Windows with msys2(mingw64).

    namespace data = cista::offset;
    constexpr auto const MODE = cista::mode::WITH_VERSION | cista::mode::WITH_INTEGRITY;

    struct pos {
        int x, y;
    };

    using vec = data::vector<pos>;

    {
        vec positions{{1, 2}, {3, 4}, {5, 6}, {7, 8}};
        cista::buf mmap{cista::mmap{"data"}};
        cista::serialize<MODE>(mmap, positions);
    }

    auto b = cista::mmap("data", cista::mmap::protection::MODIFY);
    auto positions = cista::deserialize<vec, MODE>(b);
    positions->push_back({5, 5});

    for(auto pos : *positions) {
        std::cout << pos.x << " " << pos.y << std::endl;
    }

And question. I need load data from disk and modify it, but I don't need to sync data always with disk.
Should I serialize data from disk, copy it to memory and, after changing it, serialize it back?
Or should I just use mmap::modify? I do not fully understand how much mmap costs and what is more efficient"

@felixguendling
Copy link
Owner

Note that using MODIFY mode on an mmap is a really bad idea if you want to read and modify serialized data. Using push_back in your example will allocate memory from the heap and replace the pointer in the pointer of the vector serialized in the memory mapped file with the heap pointer. However, the data the pointer points to (the contents of the vector) won't be written to the file. This way, you get a dangling pointer in your serialized data. This can't be fixed.

The only modifications that are possible are changing scalar values in-place. Allocations will allocate from the heap and will lead to dangling pointers in the data serialized in the file.

Should I serialize data from disk, copy it to memory and, after changing it, serialize it back?

Yes: read data from disk, modify it in-memory and write it back to another file. Then replace the original file (swap new vs old).

@felixguendling
Copy link
Owner

I added support for MODIFY on windows. However, this should not be used for changing serialized data beyond simple scalar modifications.

@BinaryCat17
Copy link
Contributor Author

Thanks a lot, I think it's better to cover this issue in the documentation.

@felixguendling
Copy link
Owner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants