test: Add out-of-memory tests #773
Conversation
Codecov Report
@@ Coverage Diff @@
## master #773 +/- ##
==========================================
+ Coverage 99.19% 99.24% +0.05%
==========================================
Files 77 78 +1
Lines 11936 11962 +26
==========================================
+ Hits 11840 11872 +32
+ Misses 96 90 -6
Flags with carried forward coverage won't be shown. Click here to find out more.
|
test/unittests/oom_test.cpp
Outdated
| bool try_set_memory_limit(size_t size) noexcept | ||
| { | ||
| [[maybe_unused]] int err; | ||
| err = getrlimit(RLIMIT_AS, &orig_limit); |
There was a problem hiding this comment.
We could also have some stack tests using RLIMIT_STACK, i.e. having a known limit.
test/unittests/oom_test.cpp
Outdated
| bool try_set_memory_limit(size_t size) noexcept | ||
| { | ||
| [[maybe_unused]] int err; | ||
| err = getrlimit(RLIMIT_AS, &orig_limit); |
There was a problem hiding this comment.
I wonder how RLIMIT_AS is different from RLIMIT_DATA (the latter being available on macos too):
RLIMIT_AS
This is the maximum size of the process's virtual memory
(address space). The limit is specified in bytes, and is
rounded down to the system page size. This limit affects
calls to brk(2), mmap(2), and mremap(2), which fail with
the error ENOMEM upon exceeding this limit. In addition,
automatic stack expansion fails (and generates a SIGSEGV
that kills the process if no alternate stack has been made
available via sigaltstack(2)). Since the value is a long,
on machines with a 32-bit long either this limit is at
most 2 GiB, or this resource is unlimited.
RLIMIT_DATA
This is the maximum size of the process's data segment
(initialized data, uninitialized data, and heap). The
limit is specified in bytes, and is rounded down to the
system page size. This limit affects calls to brk(2),
sbrk(2), and (since Linux 4.7) mmap(2), which fail with
the error ENOMEM upon encountering the soft limit of this
resource.
There was a problem hiding this comment.
What I have found is that RLIMIT_DATA may look more what we want (at least is more concrete). And it also seems to be working fine. And documentation says it also applies to mmap(). However, before (Linux 4.7) the limit could be bypassed by malloc using mmap for big allocations (as our case).
- https://linuxfollies.blogspot.com/2010/10/more-on-rlimitdata-vs-rlimitas.html
- https://stackoverflow.com/questions/23768601/posix-rlimit-what-exactly-can-we-assume-about-rlimit-data
- http://lkml.iu.edu/hypermail/linux/kernel/0707.1/0675.html
Therefore, the RLIMIT_AS seems to be "safer" or "more portable" option.
BTW, @axic should try this on macos (currently disabled).
There was a problem hiding this comment.
BTW, @axic should try this on macos (currently disabled).
I can once it is switched to RLIMIT_DATA.
There was a problem hiding this comment.
It is RLIMIT_DATA but it does not apply to mmap on macos. Please test it and report back.
There was a problem hiding this comment.
I tested this on macOS. Added __APPLE__ next to __linux__. Everything compiles fine, but it "does not work": the syscalls succeed with 0, but the allocation is not terminated.
- Dumped limits, and the new limits are reflected by
getrlimit. So it definitely "works". - Tried setting not only
cur, butmax. No change. - Apparently
RLIMIT_AS(an alias toRLIMIT_RSS) is also available on macOS, just not part of the man page. (ref). Tried, compiles, but no change. - It seems that
setrlimit/getrlimitis properly implemented in macOS (ref), butRLIMIT_DATAorRLIMIT_ASis actually not used by anything later on.RLIMIT_FSIZE/CPU/NPROC/NOFILE/CORE/STACKMEMLOCKare used in places, so those may be enforced.
See also https://stackoverflow.com/a/6169866 for potential solutions.
Note: launchd is some system process, but looking deeper in the end it still uses setrlimit/getrlimit without any special magic. So I think neither of that does anything.
|
Can we squash some of these commits? |
No description provided.