heavy use of polymorphism results in long compiles

[cwlbraa]

via metaprogramming, a project I'm working on has a very large number of cereal message types registered (~700) and we've started to notice that each call to CEREAL_REGISTER_TYPE incurs a heavy compile time cost, anywhere from 0.5 - 0.8s per invocation. Are you aware of this? Is there any way to shave that time down?

[AzothAmmo]

This is a bit of a tricky issue to optimize as it is difficult to get actionable information from the compiler about what takes time during compilation.

The way polymorphic support currently works is based on the boost implementation and is designed to simplify the process for users, which may come at a cost for compilation time. polymorphic registration currently does not require knowledge of archives and types simultaneously - cereal handles the binding of types to available archives.

In a nutshell, specific cereal archives create an overloaded definition of a special function (instantiate_polymorphic_binding) that is templated on a type. This overload is designed to never actually have highest precedence when invoked, however, due to some black magic, the return types for the unpicked overloads will all be instantiated. It is these return types that actually create all of the metadata needed to perform polymorphic serialization.

When you go and register your polymorphic type, it tries to invoke this special function (remember it doesn't know about any archives), and must find the best possible overload of the function.

Every archive you have included creates a type templated overload of this function. So when you register one type, it creates NUM_ARCHIVES new overloads that must be checked. This isn't a big deal for a few types, but as you've seen when the number of types is large, every registration is both adding to the number of overloads and then searching through that entire list for the best candidate.

I'm not sure if there is a good way to optimize this using the current interface. However, it would likely be significantly faster if you were willing to individually tie types and archives together. This would remove the entire overload process from registration, meaning you'd do something like:

CEREAL_REGISTER_TYPE_WITH_ARCHIVE(myType, cereal::BinaryInputArchive, cereal::BinaryOutputArchive)

or something similar.

[cwlbraa]

I am very much willing, I'll have to give that a shot!

[AzothAmmo]

I just want to note that this feature is not actually in cereal right now, but could be easily added.

[cwlbraa]

aw man you had me so excited

[BotellaA]

Is there any news on this topic? My compilation time is huge and uses a lot of memory too.

[BotellaA]

@AzothAmmo Is this feature still plan for 1.2.3?