Document box.session.push() and IPROTO_CHUNK code

[TarantoolBot]

Remaining scope:

Document IPROTO_CHUNK and overall out of bound responses description in the binary protocol documentation.

---

box.session.push is the API to send out of bound responses. Actually it is API for server -> client streaming.

--
-- Push an object to the client with no request finalisation.
-- @param data_to_push Any Lua object to send to the client.
-- @param sync Sync of the parent request.
-- @retval nil, err Error occurred. The error object is returned in the
-- second value.
-- @retval true Push message is written to the network buffer. Like
-- libc write() to the socket.
--
box.session.push(data_to_push, sync)

To get the second parameter - sync you should at the beginning of the request save result of box.session.sync() into a local variable, and use the saved value as the second push argument.
When the session is binary, the data is sent with the same sync as the request using IPROTO_CHUNK = 128 code in the packet header. When the session is text, YAML tag is used for pushed data: %TAG !push! tag:tarantool.io/push,2018, and on the client side the message is just printed to the stdout.

Netbox supports pushes in two ways.
For any sync request (DML, call, eval - no matter) you can specify two options: on_push and on_push_ctx. When in the scope of the request a push message is received, on_push is called as on_push(on_push_ctx, new_message).

For async request (when is_async option is set and true) you can not specify on_push_[ctx], but can iterate over messages using this syntax:

future = connection:<method_name>(<method_args>, {is_async = true})
for i, message in future:pairs(<per_step_timeout>) do
    -- Process the messages and the final result.
end

Here pairs takes one optional argument - timeout per iteration. If no a new message or the final response during this timeout, the error is returned to the cycle. To check for an error a one can use i - the first loop parameter above. If i == nil (actually it is box.NULL - pure nil can not be returned as a loop key due to Lua language limitations) then message is actually the error object. You can check its type and error code like for any Tarantool error. For details see message:unpack(). After an error the iteration is stopped.
Requested by @Gerold103 in tarantool/tarantool#2677.

[Gerold103]

@pgulutzan Please, document that sync parameter is optional. By default it equals to box.session.sync().

[Totktonada]

The net.box part (using is_async + :pairs()) was not documented.

https://tarantool.io/en/doc/1.10/reference/reference_lua/net_box/#net-box-is-async

[pgulutzan]

I added one sentence:
Alternatively the client could check for "out-of-band" messages from the server by calling pairs() in a loop -- see box.session.push().

[Totktonada]

I don't see IPROTO_CHUNK and overall out of bound responses description in the binary protocol documentation.

[pgulutzan]

@Totktonada: IPROTO_CHUNK and out-of-band responses description is in https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_session/push/ and there are references to that in https://www.tarantool.io/en/doc/latest/reference/reference_lua/net_box/ and in https://www.tarantool.io/en/doc/latest/dev_guide/reference_capi/box/#c.box_session_push . Would it satisfy you if a binary protocol "responses" section contained a reference to that as well?

[Totktonada]

@pgulutzan Sorry, but, it seems, no.

Can we describe the binary protocol in the binary protocol article and Lua API in the box.session.push() API description? I don't see a reason to describe things in irrelevant sections.

Why it worth to describe things in the relevant sections? For example, when you use Lua API it does not matter what is binary protocol code.

[pgulutzan]

@Totktonada: I thought that what I was proposing was the same as what you're requesting, but perhaps I caused misunderstanding by leaving out details. I will propose the same thing again, with details.

1. In iproto section, "Symbols and terms",
   https://www.tarantool.io/en/doc/latest/dev_guide/internals/box_protocol/#symbols-and-terms
   after the words "The IPROTO constants that identify requests that we will mention in this section are: ..."
   add IPROTO_CHUNK 0x80 at the end of the list.
2. In iproto section, "Responses if no error and no SQL",
   https://www.tarantool.io/en/doc/latest/dev_guide/internals/box_protocol/#responses-if-no-error-and-no-sql
   add
   "If the response is out-of-band, due to use of box.session.push(),
   then the header Response-Code-indicator will be IPROTO_CHUNK instead of IPROTO_OK."
   The words box.session.push() will be inside a :ref:... so there is a link to where it is documented.
3. In box.session.push() section,
   https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_session/push/#lua-function.box.session.push
   replace the words
   "with the code IPROTO_CHUNK (0x80)", saying instead
   "with a different header code so the client can distinguish from an ordinary Okay response.

[Totktonada]

@pgulutzan It seems, now we understood each other.

1. Ok.
2. Ok, but I would add a section for out of bound responses. It will be tiny, but this way the document will be more structural and easy to navigate.
3. Ok. If you will follow my suggestion about the separate section in the binary protocol document about out of bound responses, it worth to link it from here.