[luci-interpreter] Design and implement static memory planner and manager

[binarman]

This is continuation of #7355 that should use introduced memory manager interface.

Goal

Introduce at least one static memory planner and manager suitable for MCU usage.

High level schema of interpreter setup:

• Static memory manager contains memory pool that it can assign to tensors on request.
• Static memory manager do not owns memory buffer, so it is possible to use static array for this purpose.

High level interpreter algorithm:

• Interpreter seeks for "execution order" metadata in model
  • if it is found - execute kernels in given order
  • if it is not found - execute using standard post order
• According allocation plan interpreter asks memory manager to allocate memory for tensor.
  • static memory manager seeks into metadata stored in tensor and assigns actual memory address to a given tensor.

Rationale

• Baremetal MCU has no OS, so we need to manage memory manually.
• Even if we have OS (for example, MBED OS), it's memory subsystem is not optimal for neural networks domain.
  • Only dynamic allocations available (no way to plan ahead of time)
  • Poor utilization of different memory types (for example TCM on ARM-m7)

[binarman]

+cc @BalyshevArtem

[BalyshevArtem]

Brief explanation about circle-execution-plan tool

This tool takes circle file as input and returns modified circle file.
The output circle file contains "plan" (CircleNodeMemoryPlan) information for every node.

"plan" contains:

• number which determines order in which nodes will be executed
• memory offsets for node output tensors from the beginning of shared memory buffer

In order to record and read this metadata, we use CircleImportMetadata and CircleExportMetadata.
For this purpose we use std::map<uint32_t, std::vector<uint32_t>> _memory_plan_table which for each node with key ID contains encoded CircleNodeMemoryPlan data.

[binarman]

Functionality is merged, now we need to cover following code with tests:

• luci-interpreter parts that uses static memory manager
• circle-execution-plan tool

[seanshpark]

1/ there is no necessities for luci-interpreter parts that uses static memory manager as onert-micro is used for MCU runtime.
2/ circle-execution-plan tool not sure what to do for this

[seanshpark]

created left task and close this