How to bind huge classes

[ghorn]

I embarrassingly ran into the 1<<24 byte size limit on a class and triggered the static_assert. I couldn't find a direct reference to this limit in the docs, did I miss it?

Is nanobind designed to support objects that must be entirely allocated on the heap? I was actually surprised to see the limit, since I thought everything would be done on the heap anyway. How do I bind this sort of struct?

[ghorn]

I mean the obvious thing to do would be to

struct Wrapper {
   unique_ptr<UnreasonablyBigStruct> ptr;
};

Just wondering if there's a different way to do it.

[ghorn]

The wrapper approach breaks down when I do something like bind_vector.h. I'm actually not sure what to do at this point.

Where does this size limit come from?

[ghorn]

I tried simply patching out the static_assert and things seem to work.... I have a pretty good test suite that is passing with a 1GB class. I found all the accidental copies and stack allocation I had in user code and converted it to not copy / allocate on the heap.

It would be very helpful to get a comment on this from the author/maintainer, to know if there are are any landmines here.

[wjakob]

The static_assert is needed. The information that nanobind stores about the type is fairly compressed, and only 24 bits are available to encode the size. You will have undefined behavior if nanobind ever tries to allocate such a class on its own (instead of allocating one in C++ and then returning it to Python), as it will not acquire enough memory for it.

[ghorn]

Thank you. Sounds like Wrapper struct is the only sound approach. When does nanobind allocate a class on its own? Is it feasible for me to simply avoid such operations?

[wjakob]

When you call the __init__ constructor from Python. If the only way to create the class is by return value, then the Python allocation path will not be used.

[wjakob]

Also, return value policies that copy will have an issue.

[ghorn]

Thanks for the information. I'm not confident I'd get this right to go through with the unsafe version.

One long term idea for nanobind to support large classes would be to move the static_assert to the places where that size is accessed. The error message would be "classes above 16MB can't be returned this way, you must allocate them explicitly with new" or similar.

[ghorn]

I'm reading the code and I see the limitation on number of flags (24 bits). But to me, it looks like max size is a full 32 bits:

nanobind/include/nanobind/nb_class.h

Lines 94 to 98 in bbbd022

	struct type_data {
	uint32_t size;
	uint32_t align : 8;
	uint32_t flags : 24;
	const char *name;

am I misunderstanding? Or is the size limited in a different struct?

[wojdyr]

This static_assert was added when the size was changed to 24 bytes in 43e179c and then hasn't been removed when the size was changed back to 32 bits in
0b3548a#diff-2a6e239c3e33400e1251b4087a19af93b4fd4886009de7e1255968681cf300e6L63-L67

[ghorn]

huzzah!

[ghorn]

Would you accept a PR changing the limit to 1 << 32?

[wjakob]

Fixed in 2aacdcf

[wojdyr]

it won't work with 32-bit size_t, but I don't know if any 32-bit platforms are supported

[wjakob]

Good catch!