Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RTMP acknowledge overflow causes the encoder to stop streaming #730

Closed
winlinvip opened this issue Jan 6, 2017 · 6 comments
Closed

RTMP acknowledge overflow causes the encoder to stop streaming #730

winlinvip opened this issue Jan 6, 2017 · 6 comments
Assignees
Labels
Bug It might be a bug. TransByAI Translated by AI/GPT.
Milestone

Comments

@winlinvip
Copy link
Member

winlinvip commented Jan 6, 2017

ACK SIZE refers to the number of bytes that the PEER should send an Acknowledge message after receiving. In other words, it indicates the number of bytes that the PEER has received.
RTMP is 32, which means it is 4 bytes long, but it does not explicitly state how overflow should be handled, whether it is int32 or uint32.

5.3. Acknowledgement (3)
   The client or the server sends the acknowledgment to the peer after
   receiving bytes equal to the window size. The window size is the
   maximum number of bytes that the sender sends without receiving
   acknowledgment from the receiver. The server sends the window size to
   the client after application connects. This message specifies the
   sequence number, which is the number of the bytes received so far.
0         1         2         3
01234567890123456789012345678901 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|    sequence number (4 byte)   | 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4 shows the payload for the protocol message 'Acknowledgement'.
   sequence number: 32 bits
     This field holds the number of bytes received so far.

Previously, the bitrate of videos was very low, such as 100kbps, so there would be no overflow issue with these 4 bytes. Now, the bitrate is 80 to 100 times higher, which easily leads to overflow. Some encoders have different rules for handling this overflow, causing them to receive incorrect values and mistakenly assume that the server has not acknowledged, resulting in re-streaming.

TRANS_BY_GPT3

@winlinvip winlinvip added the Bug It might be a bug. label Jan 6, 2017
@winlinvip winlinvip added this to the srs 2.0 release milestone Jan 6, 2017
@winlinvip
Copy link
Member Author

winlinvip commented Jan 6, 2017

Gao Sheng has encountered something similar, but did not explain it clearly: #598

TRANS_BY_GPT3

@winlinvip
Copy link
Member Author

winlinvip commented Jan 6, 2017

Play a stream with a bitrate of 8Mbps using Flash player, set the ack size to 10*1024*1024, and print out the acknowledge messages sent by the Flash player.

`ack=1` indicates an acknowledge message.
`seq` is the sequence number, which represents the number of bytes received by the client.
`sent` is the number of bytes displayed as transmitted by the server.

[2017-01-06 11:23:30.321][warn][13607][114][35] ack=1, seq=0x500ac2, sent=0x5766fb, diff=482361
[2017-01-06 11:23:35.493][warn][13607][114][35] ack=1, seq=0xa0192a, sent=0xa0f73a, diff=56848
[2017-01-06 11:23:39.153][warn][13607][114][35] ack=1, seq=0xf0229e, sent=0xf4e6bb, diff=312349
[2017-01-06 11:23:43.197][warn][13607][114][35] ack=1, seq=0x140241e, sent=0x140499e, diff=9600

The ack size is set to 10*1024*1024, which means an acknowledge message should be sent after the player has played 10MB of data.
For a flash player, the interval between sending messages, which is the difference in seq, is 0xa0192a-0x500ac2=5246568=5*1024*1024, which means it sends approximately half of the ack size.
Subsequently, it sends acknowledgements after receiving 5.002307891845703MB and 5.0003662109375MB, which is roughly half of the window size.

The following messages are:

[2017-01-06 11:50:16.010][warn][13607][114][35] ack=1, seq=0x68ba6930, sent=0x68c14790, diff=450144
[2017-01-06 11:50:20.063][warn][13607][114][35] ack=1, seq=0x690a7297, sent=0x690d58ae, diff=189975
[2017-01-06 11:50:23.727][warn][13607][114][35] ack=1, seq=0x695a79a2, sent=0x695dafa8, diff=210438
[2017-01-06 11:50:28.507][warn][13607][114][35] ack=1, seq=0x69aa7fe2, sent=0x69b1d0f1, diff=479503

The respective values are: 5.00229549407959, 5.0017194747924805, 5.00152587890625.

Therefore, it can be confirmed that the behavior of the flash player is to send an acknowledge confirmation message to the server when it receives more than 1/2 of the ack size (window size).

TRANS_BY_GPT3

@winlinvip
Copy link
Member Author

winlinvip commented Jan 6, 2017

Regarding the server side, my colleague has tested two of them:

FMS: 0xee77d0a7
NGINX-RTMP: 0xf0000000

Once these two thresholds are exceeded, the server will reset the counter to 0.

NGINX-RTMP code: https://github.com/arut/nginx-rtmp-module/blob/c0bf381d10de05c135f913921c58272838d5e1ee/ngx_rtmp_handler.c#L275

            if (s->in_bytes >= 0xf0000000) {
                ngx_log_debug0(NGX_LOG_DEBUG_RTMP, c->log, 0,
                               "resetting byte counter");
                s->in_bytes = 0;
                s->in_last_ack = 0;
            }

This number has no basis and there is no comment explaining its source.
In nginx-rtmp, the ack is sent when it exceeds the ack window (unlike Flash, which sends it when it exceeds half the window).

            if (s->ack_size && s->in_bytes - s->in_last_ack >= s->ack_size) {

                s->in_last_ack = s->in_bytes;

                ngx_log_debug1(NGX_LOG_DEBUG_RTMP, c->log, 0,
                        "sending RTMP ACK(%uD)", s->in_bytes);

                if (ngx_rtmp_send_ack(s, s->in_bytes)) {
                    ngx_rtmp_finalize_session(s);
                    return;
                }
            }

TRANS_BY_GPT3

@winlinvip
Copy link
Member Author

winlinvip commented Jan 6, 2017

About the handling of flash player overflow, the log is as follows:

[2017-01-06 12:24:16.129][warn][13607][114][35] ack=1, seq=0xed47b9cd, sent=0xed49da95, diff=139464
[2017-01-06 12:24:20.549][warn][13607][114][35] ack=1, seq=0xed97c0e6, sent=0xed992be4, diff=92926
[2017-01-06 12:24:26.044][warn][13607][114][35] ack=1, seq=0xede7c793, sent=0xeded7128, diff=371093
[2017-01-06 12:24:29.706][warn][13607][114][35] ack=1, seq=0xee37cff6, sent=0xee3d82a9, diff=373427
[2017-01-06 12:24:34.113][warn][13607][114][35] ack=1, seq=0xee87dca3, sent=0xee90a823, diff=576384
[2017-01-06 12:24:37.781][warn][13607][114][35] ack=1, seq=0xeed7e36a, sent=0xeedd0cbf, diff=338261
[2017-01-06 12:24:42.201][warn][13607][114][35] ack=1, seq=0xef27ed1a, sent=0xef2e1b6d, diff=405075
[2017-01-06 12:24:46.589][warn][13607][114][35] ack=1, seq=0xef77f9ad, sent=0xef7d5424, diff=350839
[2017-01-06 12:24:51.016][warn][13607][114][35] ack=1, seq=0xefc806b8, sent=0xefc8896a, diff=33458
[2017-01-06 12:24:55.432][warn][13607][114][35] ack=1, seq=0x500eb7, sent=0xf01a1a8a, diff=-271971373
[2017-01-06 12:25:00.178][warn][13607][114][35] ack=1, seq=0xa01113, sent=0xf06da792, diff=-271739265
[2017-01-06 12:25:04.577][warn][13607][114][35] ack=1, seq=0xf0186e, sent=0xf0bc6d49, diff=-271821605
[2017-01-06 12:25:09.373][warn][13607][114][35] ack=1, seq=0x140210b, sent=0xf10fe9e5, diff=-271595302
[2017-01-06 12:25:12.673][warn][13607][114][35] ack=1, seq=0x19021c6, sent=0xf15a736a, diff=-271953500
[2017-01-06 12:25:17.435][warn][13607][114][35] ack=1, seq=0x1e0287d, sent=0xf1aa3d1b, diff=-271969122
[2017-01-06 12:25:22.213][warn][13607][114][35] ack=1, seq=0x2302f34, sent=0xf1f9d01b, diff=-271998745
[2017-01-06 12:25:26.996][warn][13607][114][35] ack=1, seq=0x2803a9b, sent=0xf24c3340, diff=-271845211
[2017-01-06 12:25:31.382][warn][13607][114][35] ack=1, seq=0x2d03ed4, sent=0xf299ecd1, diff=-271995395
[2017-01-06 12:25:36.534][warn][13607][114][35] ack=1, seq=0x3204cee, sent=0xf2f33cf8, diff=-271388662
[2017-01-06 12:25:40.566][warn][13607][114][35] ack=1, seq=0x37053d6, sent=0xf33de12b, diff=-271741611
[2017-01-06 12:25:45.332][warn][13607][114][35] ack=1, seq=0x3c05a44, sent=0xf38b265b, diff=-271922153

The overflowing data is padded with either 32 bits or 31 bits, which is significantly different from what flash sends. If we directly set it to zero, it can match, just like in nginx-rtmp, where it is set to zero when it exceeds 0xf0000000.

f3c262d3-099a-4526-a19e-6167ccb03bbb

TRANS_BY_GPT3

@winlinvip
Copy link
Member Author

winlinvip commented Jan 6, 2017

This problem has only two solutions:

  1. If the accumulation of acknowledge exceeds 0xf0000000, it is set to zero.
  2. Send an acknowledge message if it exceeds half of the ack size.

Supported in version 2.0.225.

TRANS_BY_GPT3

@winlinvip
Copy link
Member Author

winlinvip commented Jan 6, 2017

In addition, some encoders do not send an ack size (usually indicating that they do not need to send an acknowledge confirmation message), but they still want the server to send a confirmation message. Therefore, it is necessary to support configuration that can change the ack size for both input and output.

Supported in 3.0.13.

TRANS_BY_GPT3

winlinvip added a commit that referenced this issue Jan 6, 2017
@winlinvip winlinvip self-assigned this Sep 18, 2021
@winlinvip winlinvip changed the title RTMP acknowledge 溢出导致编码器停止推流 RTMP acknowledge overflow causes the encoder to stop streaming Jul 28, 2023
@winlinvip winlinvip added the TransByAI Translated by AI/GPT. label Jul 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug It might be a bug. TransByAI Translated by AI/GPT.
Projects
None yet
Development

No branches or pull requests

1 participant