gh-992 add submodule about encoding of non default types and described decimals

doc/dev_guide/internals/box_protocol.rst

@@ -43,6 +43,98 @@ MsgPack data types:
 * **MP_OBJECT** - Any MsgPack object
 * **MP_BIN** - MsgPack binary format
 
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Encoding of Tarantool-specific data types
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Some of the data types used in Tarantool are application-specific in terms of
+the MsgPack standard.
+For these data types, we use the following representation.
+
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+Decimals
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+MsgPack EXT type ``MP_EXT`` together with the extension type
+``MP_DECIMAL`` is used as a record header.
+
+MP_DECIMAL is 1.
+
+`MsgPack spec <https://github.com/msgpack/msgpack/blob/master/spec.md#ext-format-family>`_
+defines two kinds of types:
+
+* ``fixext 1/2/4/8/16`` types have fixed length so the length is not encoded explicitly;
+* ``ext 8/16/32`` types require the data length to be encoded.
+
+``MP_EXP`` + optional ``length`` imply using one of these types.
+
+The decimal MsgPack representation looks like this:
+
+.. code-block:: none
+
+    +--------+-------------------+------------+===============+
+    | MP_EXT | length (optional) | MP_DECIMAL | PackedDecimal |
+    +--------+-------------------+------------+===============+
+
+Here ``length`` is the length of ``PackedDecimal`` field, and it is of type
+``MP_UINT``, when encoded explicitly (i.e. when the type is ``ext 8/16/32``).
+
+``PackedDecimal`` has the following structure:
+
+.. code-block:: none
+
+     <--- length bytes -->
+    +-------+=============+
+    | scale |     BCD     |
+    +-------+=============+
+
+Here ``scale`` is either ``MP_INT`` or ``MP_UINT``. |br|
+``scale`` = -exponent (exponent negated!)
+
+``BCD`` is a sequence of bytes representing decimal digits of the encoded number
+(each byte represents two decimal digits each encoded using 4 bits),
+so ``byte >> 4`` is the first digit and ``byte & 0x0f`` is the second digit.
+The leftmost digit in the array is the most significant.
+The rightmost digit in the array is the least significant.
+
+The first byte of the ``BCD`` array contains the first digit of the number,
+represented as follows:
+
+.. code-block:: none
+
+    |  4 bits           |  4 bits           |
+       = 0x                = the 1st digit
+
+The last byte of the ``BCD`` array contains the last digit of the number and the
+``nibble``, represented as follows:
+
+.. code-block:: none
+
+    |  4 bits           |  4 bits           |
+       = the last digit    = nibble
+
+The ``nibble`` represents the number's sign:
+
+* ``0x0a``, ``0x0c``, ``0x0e``, ``0x0f`` stand for plus,
+* ``0x0b`` and ``0x0d`` stand for minus.
+
+**Examples**
+
+The decimal ``-12.34`` will be encoded as ``0xd6,0x01,0x02,0x01,0x23,0x4d``:
+
+.. code-block:: none
+
+    |MP_EXT (fixext 4) | MP_DECIMAL | scale |  1   |  2,3 |  4 (minus) |
+    |       0xd6       |    0x01    | 0x02  | 0x01 | 0x23 | 0x4d       |
+
+The decimal 0.000000000000000000000000000000000010
+will be encoded as ``0xc7,0x03,0x01,0x24,0x01,0x0c``:
+
+.. code-block:: none
+
+    | MP_EXT (ext 8) | length | MP_DECIMAL | scale |  1   | 0 (plus) |
+    |      0xc7      |  0x03  |    0x01    | 0x24  | 0x01 | 0x0c     |
+
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Greeting packet
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~