Pari default stack size results in huge physical memory commit after fork on Cygwin

[embray]

It turns out #21582 causes big problems for fork on Cygwin. This is due to an implementation detail of how Cygwin handles copy-on-write private mappings when forking. A fact that might, unfortunately, be difficult to avoid. The problem with this on Cygwin is that, while Windows does not commit pages to physical memory until they are accessed, they do become committed upon any access (even reads, when they haven't already been written to).

So when a process with a private mmap is forked, it loops over all pages in the mmap'd region and copies them into the child's copy of the mmap'd region. This results in committing physical memory on both the parent and the child, even for pages that haven't been written to yet.

TL;DR, if there's a huge private|anonymous mmap in a process, and that process is forked, it will commit the full size of the mmap to memory in both the parent and child processes.

This is a big problem in Sage since we set a very large default stack size for Pari. This was not a problem prior to #21582, since Pari used the MAP_NORESERVE flag which circumvents this issue (only dirty pages need to be copied). The changes in #21582 make sense for Linux, but for Cygwin the opposite is true. Different but unfortunate implementation details on both platforms.

The best way forward is to use MAP_NORESERVE anyway, which helps Cygwin and won't hurt other platforms.

Upstream bug: https://pari.math.u-bordeaux.fr/cgi-bin/bugreport.cgi?bug=1912

Upstream: Fixed upstream, but not in a stable release.

CC: @jdemeyer

Component: porting: Cygwin

Keywords: days85 windows cygwin pari mmap

Author: Erik Bray, Jeroen Demeyer

Branch/Commit: 343f241

Reviewer: Erik Bray, Jeroen Demeyer

Issue created by migration from https://trac.sagemath.org/ticket/22633

[embray]

comment:1

The additional patch I've attached seems to fix this, but I would like to do more testing to see if any more work is needed.

This definitely fixes the immediate problem of memory being eaten up after fork(), but I don't know if there are any other implications for these changes on Cygwin (I suspect the rest is fine, however).

---

New commits:

48b7621

Add patch to Pari (on top of prot_none_1.patch) that restores use of MAP_NORESERVE on Cygwin

[embray]

Branch: u/embray/cygwin/pari-mmap

[embray]

Author: Erik Bray

[embray]

Commit: 48b7621

[embray]

Upstream: Not yet reported upstream; Will do shortly.

[embray]

comment:2

Per jdemeyer it might be fine to simplify this by including MAP_NORESERVE on all platforms that support it. Now that PROT_NONE is used I think the MAP_NORESERVE should be harmless on Linux (though I'd like to know a way to check that...)

[jdemeyer]

comment:3

Replying to @embray:

Now that PROT_NONE is used I think the MAP_NORESERVE should be harmless on Linux (though I'd like to know a way to check that...)

If you have a patch ready, I can test it.

[jdemeyer]

comment:4

Do you want me to report this upstream?

[sagetrac-git]

Changed commit from 48b7621 to a724b2a

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

a724b2a

Update the patch to always apply MAP_NORESERVE if it's defined (not just on Cygwin)

[embray]

comment:6

I just updated the patch with your suggestion. I'll report it now.

[embray]

Changed upstream from Not yet reported upstream; Will do shortly. to Reported upstream. No feedback yet.

[jdemeyer]

comment:9

Can you check if it suffices to add MAP_NORESERVE only to the PROT_NONE allocations? That would be more in line with the intent of the current code. See also my answer to the PARI bug.

[jdemeyer]

comment:10

Beware of #22675: we might need to synchronize these tickets.

[embray]

comment:11

I can update these after #22675.

[embray]

comment:12

Hm, for some reason, while I got Bill's initial reply to my report I didn't get your response in my e-mail. Maybe I need to explicitly subscribe to updates on the bug...

Yeah, I can try making that change. I don't see what difference it would make--what harm does it have on the other allocations? But if you think it will help I can try it. It should be fine.

[jdemeyer]

comment:13

Replying to @jdemeyer:

Can you check if it suffices to add MAP_NORESERVE only to the PROT_NONE allocations? That would be more in line with the intent of the current code. See also my answer to the PARI bug.

ping

[embray]

comment:14

Right. Working on it.

[embray]

comment:15

A minimal example in C seems to indicate that this would not work. Perhaps it's a limitation of Cygwin and/or Windows, but calling mmap(..., PROT_NONE, MAP_NORESERVE) at the address of a previous mmap(NULL, ..., PROT_READ|PROT_WRITE, ...) without MAP_NORESERVE exhibits the same bug on fork.

[jdemeyer]

comment:16

Thanks for testing.

I will push a new patch here, hang on.

[jdemeyer]

Changed branch from u/embray/cygwin/pari-mmap to u/jdemeyer/cygwin/pari-mmap

[jdemeyer]

comment:18

Does this work for you?

---

New commits:

795e3f1

mmap() PARI stack with MAP_NORESERVE (for Cygwin)

[jdemeyer]

Changed commit from a724b2a to 795e3f1

[jdemeyer]

comment:19

Hang on, I will add a check for the return value of mmap().

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

343f241

mmap() PARI stack with MAP_NORESERVE (for Cygwin)

[sagetrac-git]

Changed commit from 795e3f1 to 343f241

[jdemeyer]

comment:22

Does it work on Cygwin?

[embray]

comment:23

It looks like the explicit munmap before remapping works. I don't exactly see what advantage this has over my original patch but this is fine too.

[jdemeyer]

Changed upstream from Reported upstream. No feedback yet. to Fixed upstream, but not in a stable release.

[jdemeyer]

comment:24

Accepted by upstream.

[vbraun]

comment:25

Reviewer name missing...

[jdemeyer]

Changed author from Erik Bray to Erik Bray, Jeroen Demeyer

[jdemeyer]

Reviewer: Erik Bray, Jeroen Demeyer

[vbraun]

Changed branch from u/jdemeyer/cygwin/pari-mmap to 343f241