The format of tmfile

The format of tmfile

Model Structure

header -> table -> graph -> vector -> node -> op -> type/parameters/input_tensor/output_tensor
                         -> vector -> tensor -> dimension/layout/buffer_id 
                         -> vector -> buffer -> size/data

Graph

The tmfile models are organized by as the following structure.

  	                   node1
  	                     |
  	                     | tensor0
  	                     |
  	                   node2
  	               |           |
  	       tensor2 |           | tensor1
  	       	       |           |
  	       	     node4      node3
  	       	       |           |
  	       tensor4 |           | tensor3
  	       	       |           |
  	        	   node5
  	        	     |
  	                     | tensor5
  	                     |
  	                   node6

Data Format

Header -- 10 Bytes

Header is on the head of tmfile. It shows the version of this model and the root table address.

Header	size	type	description
main version	2 Bytes	uint16_t	The main version of the model.
sub version	2 Bytes	uint16_t	The sub version of the model (default 0).
compile version	2 Bytes	uint16_t	The compiler version of the model (default 0).
offset to root table	4 Bytes	tm_uoffset_t	The address of the root table.

Tables

①root table -- 16 Bytes

root table	size	type	description
original format	4 Bytes	int32_t	The original format of the model.
sub format	4 Bytes	int32_t	The sub format for DLA model.
offset to the vector of subgraph	4 Bytes	tm_uoffset_t	Only one subgraph is supported.
offset to the string of the model name	4 Bytes	tm_uoffset_t	The name

②subgraph -- 36 Bytes

subgraph	size	type	description
subgraph id	4 Bytes	uint32_t	The subgraph id.
graph layout	4 Bytes	int32_t	The actual data layout.
model layout	4 Bytes	int32_t	The data layout of original model.
offset to the vector of input indices	4 Bytes	tm_uoffset_t	The vector of the inputs index.
offset to the vector of output indices	4 Bytes	tm_uoffset_t	The vector of the outputs index.
offset to the vector of nodes	4 Bytes	tm_uoffset_t	The vector of the nodes.
offset to the vector of tensors	4 Bytes	tm_uoffset_t	The vector of the tensors.
offset to the vector of buffers	4 Bytes	tm_uoffset_t	The vector of the buffers.
offset to the string of subgraph name	4 Bytes	tm_uoffset_t	The string of the subgraph name.

③node -- 28 Bytes

node	size	type	description
node id	4 Bytes	uint32_t	The node id.
offset to the vector of input tensors	4 Bytes	tm_uoffset_t	Input tensors of this node.
offset to the vector of output tensors	4 Bytes	tm_uoffset_t	Output tensors of this node.
offset to the table of operator	4 Bytes	tm_uoffset_t	The offset to operator info.
offset to the string of node name	4 Bytes	tm_uoffset_t	The node name address.
offset to the vector of node attributes	4 Bytes	tm_uoffset_t	The vector of node attributes.
dynamic shape	1 bit	Bool	aligned by 4 Bytes.

④tensor -- 32 Bytes

tensor	size	type	description
tensor id	4 Bytes	uint32_t	The tensor id.
buffer id	4 Bytes	uint32_t	The buffer of this tensor.
offset to the vector of dims	4 Bytes	tm_uoffset_t	The vector of input tensors.
offset to the string of tensor name	4 Bytes	tm_uoffset_t	The vector of output tensors.
offset to the vector of quantparams	4 Bytes	tm_uoffset_t	The offset to operator info.
layout	4 Bytes	int32_t	NCHW/NHWC
type	4 Bytes	int32_t	CONST/INPUT/VAR/DEP/UNKNOWN
data type	4 Bytes	int32_t	int8/uint8/fp32

⑤operator -- 12 Bytes

operator	size	type	description
operator version	4 Bytes	uint32_t	The operator version.
operator type	4 Bytes	uint32_t	The type of this operator.
offset to the table of params	4 Bytes	tm_uoffset_t	The offset to the table of this operator.

⑥quantization parameters -- 12 Bytes

quantization parameters	size	type	description
zero_point	4 Bytes	int32_t
scale	4 Bytes	float
width	4 Bytes	int32_t

⑦buffer -- 8 Bytes

buffer	size	type	description
size	4 Bytes	tm_size_t	The size of this buffer (bytes).
offset data	4 Bytes	tm_uoffset_t	The address of the buffer data.

Vectors

①vector offset -- 4+4N Bytes

vector offset	size	type	description
v_num	4 Bytes	tm_size_t	The number of this vector.
offsets	4N Bytes	tm_uoffset_t	The address array in this vector.

②vector indices -- 4+4N Bytes

vector indices	size	type	description
v_num	4 Bytes	tm_size_t	The number of this vector.
indices	4N Bytes	tm_uoffset_t	The indices array in this vector.

③vector dims -- 4+4N Bytes

vector dims	size	type	description
v_num	4 Bytes	tm_size_t	The number of this vector.
dims	4N Bytes	int32_t	The dimension array.

④vector floats -- 4+4N Bytes

vector floats	size	type	description
v_num	4 Bytes	tm_size_t	The number of this vector.
data	4N Bytes	float

⑤vector anchors -- 4+4N Bytes

vector anchors	size	type	description
v_num	4 Bytes	tm_size_t	The number of this vector.
data[0] [4]	16 Bytes	float	x0, y0, x1, y1

String

vector anchors	size	type	description
size	4 Bytes	tm_size_t	The size of this string.
offset to the string	4 Bytes	tm_uoffset_t	The address of the string.

File Structure

TM_Header
Model name & TM_String(Optional)
-------------------------------Nodes-------------------------------
Node name & TM_String(Optional)
Vector(indices of input tensors)
Vector(indices of output tensors)
Operator param(Optional)
TM_Operator
TM_Node
……
Node name & TM_String(Optional)
Vector(indices of input tensors)
Vector(indices of output tensors)
Operator param(Optional)
TM_Operator
TM_Node
Vector(offsets of all nodes)
-------------------------------Tensors-------------------------------
Tensor name & TM_String(Optional)
Vector(tensor dims)
TM_Tensor
……
Tensor name & TM_String(Optional)
Vector(tensor dims)
TM_Tensor
Vector(offsets of all tensors)
-------------------------------Buffers------------------------------------
Buffer data(Optional)
TM_Buffer
……
Buffer data(Optional)
TM_Buffer
-------------------------------Subgraphs-------------------------------
Vector(offsets of all buffers)
Vector(indices of input nodes)
Vector(indices of output nodes)
TM_Subgraph
Vector(offsets of all subgraphs)
-------------------------------Root Table-------------------------------
TM_Model

Variables

Variables	Variables
int8_t	uint8_t	1 byte
int16_t	uint16_t	2 bytes
int32_t	uint32_t	4 bytes
int64_t	uint64_t	8 bytes
	float	4 bytes
typedef uint32_t	tm_uoffset_t	4 bytes
typedef uint32_t	tm_size_t	4 bytes
typedef uint8_t	tm_bool_t	1 byte

Rules

Alignment

All tables, structures are aligned by the longest element in this table or structure, for example:

struct A
{
	int8_t  d1;	// 1-byte, aligned by 1-byte
	int32_t d3;	// 4-bytes, aligned by 4-bytes
	int16_t d2;	// 2-bytes, aligned by 2-bytes
	int64_t d4;     // 8-bytes, aligned by 8-bytes
}

Endian

Tengine tmfile uses little-endian.

Offset

All offsets and address are from the root address which is the original address of the file.

Operator parameters

All parameters are stored by node-> offset to the table of operator -> operator -> offset to the table of params -> params.