Emergency EH buffer is overspecified

[jwakely]

The description of the emergency buffer in 3.3.1 Exception Storage seems rather overspecified.

• 16 threads is not very many, that's not even one per core on many of today's systems. A program with a huge number of threads might need to throw bad_alloc from more than 16 at once, as OOM happens for them all at once.
• In this context, a kilobyte is quite large. It's far more than is needed for std::bad_alloc (typically a single pointer) or any other standard exception (std::filesystem::filesystem_error is six pointers for libstdc++, and that contains an error code, a string, and two filesystem::path objects!). The exception header is <= 15 pointers, but that still means you could allocate eight bad_alloc exceptions using that 1kB. I don't see why the spec should be "optimized" for people throwing unreasonably large classes.
• Why not allow the runtime to divide up the buffer how it sees fit? Libstdc++ uses a fixed-size buffer (at present) but supports any allocation size, and doesn't waste a whole kilobyte if __cxa_allocate_exception only requests a smaller allocation. This allows the buffer to be used for far more than 16 * 4 * 1kB allocations.
• 64kB is quite large, and users want to be able to reduce or disable that buffer entirely. If the buffer supports variable-sized allocations then 64kB on x86_64 allows 512 bad_alloc exceptions (including their headers) to fit in the buffer before it fills up. Double that on 32-bit systems. On 16-bit systems reserving 64kB to be able to deal with two thousand exceptions in parallel is completely unnecessary!
• Any fixed size is a problem for somebody. It's either too much or too little. It certainly doesn't scale fairly for non-IA64 systems, which accounts for the vast majority of systems using this ABI today. 32-bit systems have both smaller footprints for __cxa_exception headers, and (typically) fewer cores and fewer threads, making the 64kB doubly wasteful.
• Blocking is not acceptable for some domains and terminating would be better. Maybe that's out of scope for the ABI spec and can be left as an optional (non-conforming) extension for implementations that want that, but terminating matches existing practice. Libstdc++ terminates if malloc fails and the emergency buffer is already full, I think libcxxabi does the same.

Recommendations:

• Remove the fixed 64kB size. Allow implementations to choose the size of the emergency buffer. I'm adding the ability to set the size at runtime for libstdc++, which means we can pick a default smaller than 64kB, and scale it appropriately for 16/32/64-bit systems. Users who really need 64kB (or more!) will be able to request it per-process via an environment variable.
• Remove the 1kB chunk size. If implementations allocate variable-sized chunks then it's possible to support more than 16 threads throwing concurrently, while using less than 64kB.
• Remove the 16 tasks with 4 nested exceptions each limits. There will be some limit, obviously, but it doesn't need to be standardized. Maybe give 16 x 4 as a recommended minimum, with a caveat that unreasonably large exceptions will exhaust the pool sooner.
• Consider removing the requirement to block on pool exhaustion, making it implementation-defined whether to block or terminate.

[fweimer-rh]

I think the ABI should support a compilation mode in which an exception can always be thrown, translated to std::bad_alloc (or more likely, a subclass) if memory allocation fails. Currently, __cxa_allocate_exception is declared noexcept in our headers.

Maybe we can even use a tagged pointer to preserve some information about the exception that had to be translated (like its type_info). In most cases, it will be a memory allocation failure that is being suppressed, so it's not likely that there is much actionable information lost anyway. As a shared resource, memory allocation failures are not properly encapsulated, so the reported failure may not even point to the right subsystem.

Once we have a non-terminating implementation, I wonder if we still need the emergency pool.

[jwakely]

It would be nice if there was some allowance for non-unique exceptions. If the runtime created a single std::bad_alloc (or subclass, possibly in static storage) on program startup then it could just rethrow that same object every time allocation fails (increasing the ref count on the shared object). But I think that requires a language change, so out of scope for the ABI.

[rjmccall]

I agree that this seems weirdly over-specified. The ABI document should only describe the high-level requirements of the interface, and I don't see why anything about the underlying allocation scheme needs to be part of that.

If we can't come up with a high-level requirement that implementations can reasonably live up to (e.g. that we can throw at least one emergency exception on every thread, which would require us to eagerly reserve that space in every thread allocation, which we probably don't want to do), then we shouldn't say anything normative at all. In that case, we should just make a strong recommendation that implementations have some limited ability to throw exceptions even when allocation fails.

[rjmccall]

It would be nice if there was some allowance for non-unique exceptions. If the runtime created a single std::bad_alloc (or subclass, possibly in static storage) on program startup then it could just rethrow that same object every time allocation fails (increasing the ref count on the shared object). But I think that requires a language change, so out of scope for the ABI.

It seems to be an open question whether this is actually ruled out. You can convincingly argue that distinct evaluations of one-operand throw have to produce different exceptions: [except.throw] says that there's an exception object which is initialized from the operand, and the generic object rules say that different objects have to have different addresses. However, I don't see anything in the library section that requires operator new to throw an exception exactly as if by evaluating a throw expression; it just has to throw something of type std::bad_alloc. And I don't see anything semantic about the std::bad_alloc interface which requires objects to be uniquely generated for each failed allocation, like a mutable member or specific information about the allocation request. And if exception objects aren't guaranteed to be unique (already true because of rethrowing), then programs aren't permitted in general to do corner-case stuff like re-using an exception object's memory temporarily (which I guess would otherwise be allowed as long as you put an object of the original type back in place before the unwind system inspects it again). So if all that holds up, in principle there's no reason not to throw a shared bad_alloc exception.

With that said, I don't think implementations should try this without getting the committee's blessing first.

[jwakely]

However, I don't see anything in the library section that requires operator new to throw an exception exactly as if by evaluating a throw expression;

Ah good point. I think we agree that an arbitrary throw std::bad_alloc(); in user code must throw a unique object, but operator new can do something different.

With that said, I don't think implementations should try this without getting the committee's blessing first.

A few of us on the committee have already been tossing the idea around. Maybe it's time to formalize it.

[rjmccall]

Sounds great.

I think we agree that an arbitrary throw std::bad_alloc(); in user code must throw a unique object,

Yeah, completely agreed there, absent some further permission from the standard.