Maximum Output Buffer Size PaRSEC with MADNESS Serialisation

[rohanmclure]

Was using TTG with the PaRSEC backend but with TTG_SERIALIZATION_SUPPORTS_MADNESS enabled.

It appears like ttg/ttg/parsec/ttg.h:728 caps the maximum size of messages to just under 1MB. Would it be possible to have the buffer size determined dynamically?

This behaviour should align with: https://github.com/m-a-d-n-e-s-s/madness/blob/320e2c0d9728312b7d41470dbfc4d20e8dac5120/src/madness/world/buffer_archive.h#L100, that is the nbyte value should be indicative of the real size of the buffer.

[devreal]

Thanks for the report. This is indeed an issue we need to solve (shouldn't be hard to fix).

In the meantime, depending on your data structure, you may want to use the split-metadata interface we introduced recently. It allows you to describe your data as iovecs and have the runtime transfer the blob(s) without serialization. You can find a usage example in the POTRF code: https://github.com/TESSEorg/ttg/blob/master/examples/potrf/potrf_pdc.cc#L338

[rohanmclure]

Thanks Joseph, I might try and adapt to the slit-metadata interface. The code example is very helpful.

[devreal]

Let me know if you have any questions :)