Tensor Arena size reported by Recording Memory APIs is insufficient during Invoke()

[joshih-cad]

We have followed the steps mentioned in tensorflow/lite/micro/docs/memory_management.md to identify tensor arena for a given network and then allocated the reported memory to run model inference.

However, after Invoke(), we run into allocation issues in network level Prepare() / tensor allocation functions.
We suspect that the Recording Memory APIs do not consider the memory needed by temporary_buffer allocations in the Prepare() stage.
The total arena size reported = Head(Scratch) + Tail (Persist) which is required by the network during Eval() stage.
But the actual memory size needed is greater than what’s reported. We also think that Tail memory (Persist) is allocated first
and then the remaining memory may/may-not be sufficient for Head + Temp allocations depending on the model.

We have a sample model for which we run into arena size related errors. Sharing details below:

1. For the sample model, the arena size reported was:
   
2. On allocating and running with 960 bytes arena, we see the following error:
   
3. Repo details: commit e440f0a (HEAD -> main, origin/main, origin/HEAD)
4. Tools/target used: gcc compiler on x86 target.
5. Steps to reproduce:
   a. Extract sample_model.zip attached and copy the sample_model folder in tensorflow/lite/micro/examples/
   b. >> make -f tensorflow/lite/micro/tools/make/Makefile test_sample_model_test
   c. In order to print the arena size report details, un-comment line 28 in sample_model/sample_model_test.cc

CC: @suleshahid @pyu10055 @TFLM-bot