HLS: SRS2 processes audio-only streams into HLS format, experiencing lag and static noise issues on Apple systems. #512
Comments
|
Topology is RTMP ----> SRS -----> IOS.
|
|
The RTMP stream is pushed to SRS through FMLE. It is in the pure audio AAC format.
|
|
What is the sampling rate of the SRS logs and audio?
|
|
|
FMLE configuration is as follows: format: AAC, Channels: Stereo, Sample Rate: 44100 Hz, Bit Rate: 128 Kbps.
|
|
Try reducing the bitrate to 64. Also, please send me the slices and m3u8 file so that I can package and transfer them to the cloud storage.
|
|
After adjustment, modify the FMLE configuration to: Format: AAC
|
|
There is still a regular noise 'pupu' when playing pure audio on Apple systems.
|
|
RTMP to TS audio explosion, the reason is because the TBN of RTMP is 1/1000, which is in milliseconds, but TS is 1/90000, which is in 1/90 milliseconds. Therefore, TS has a higher resolution. However, I will fix it and see how to solve it most effectively. You can increase the size of the slice to confirm if this is the issue.
|
|
I listened carefully, and there is a slight pause every 5 seconds, but the static noise from the microphone is constant.
|
|
I found out that there has been too much emphasis on PCR, actually it is not necessary.
|
|
Every PCR causes a timestamp error between RTMP and TS, resulting in crackling sound and significant gaps between segments.
|
|
|
The above was recorded using the rtmpdump tool for rtmp streams.
|
|
I adjusted the hls.conf configuration as follows: Note: The hls_fragment parameter has been changed to 30.
|
|
Here are the current startup logs and environment:
|
|
This is the current 30S slice.
|
|
At present, the sound of puffing is not so obvious, but there are still occasional intermittent situations.
|
|
Unable to use rtmpdump for recording, use srs_rtmp_dump instead, which can be found in the objs directory compiled. Rtmpdump will alter the timestamps.
|
|
Okay.
|
|
[root@ objs]# ./srs_rtmp_dump -r rtmp://127.0.0.1:1935/live/mytest -o output.flv |
|
Take a screenshot of the RTMP stream through srs_rtmp_dump
|
|
Can the issue you just mentioned about the time stamp mismatch between RTMP and TS caused by each PCR, which leads to popping sound and large gaps between clips, be improved by modifying the configuration file parameters?
|
|
It seems that there is no issue with accuracy. I previously figured it out, but unfortunately, I have forgotten now and I don't remember it in my notes either. It is possible that nginx-rtmp, in order to reduce HLS overload, combines multiple AAC frames and writes them into ts files, which requires calculating time differences and such. I will figure out and fix this issue.
|
|
Okay, and I just tried SRS_1.0 and found that it seems to not support hls_acodec and hls_vcodec parameters, right?
|
|
I changed to write each audio package, but this will increase the overhead of HLS. This can be avoided by changing the audio bitrate. The recommended bitrate is: For the example video
Therefore, the overhead is minimized when the audio bitrate is between 48 and 54. If the video bitrate increases, the impact of the audio will be smaller.
|
|
2.0.196 There is no problem with video+audio.
|
|
Okay, I will continue to pay attention to the SRS product. Thank you very much for taking the time to reply to my question.
|
|
I will be downloading version 2.0.196 and testing it in the near future.
|
|
Yes, it was also found during the testing process that as long as VIDEO and AUDIO are enabled, the iOS playback is normal. Now I understand what's going on.
|
|
refer to #547 |
|
The DTS of the audio needs to be consistent. If it is inconsistent, there will be a popping sound (in the case of both audio and video), but I have not tried it with pure audio.
|
|
It is caused by the cumulative error of the audio. For example, for AV streams (which include both audio and video), if multiple RTMP audio packets are combined into one AAC packet (to save the payload of TS), the total length of these packets should be calculated again based on the AAC timestamp (calculated by the number of samples and the sampling rate, with AAC generally fixed at 1024 bytes per sample, which can be obtained after parsing). Suppose there are 10 A packets: A0-A9, converting their original encoder timestamps to 1/1000th of RTMP precision leads to loss of accuracy. Therefore, combining the errors of the 10 packets will display their errors. For example, if the RTMP timestamp for A0 is 100 milliseconds and A9 is 300 milliseconds, the actual combined AAC frame may have a timestamp of 303 milliseconds. The next RTMP timestamp is 300 milliseconds, resulting in some gaps. Therefore, when combining multiple RTMP AAC packets into TS packets, the timestamps must be recalculated. Using 1024 bytes as the sample length directly in NGINX-RTMP is inaccurate. SRS can handle AV streams without combining audio packets, but for pure audio, packet combination is necessary; otherwise, it cannot be played.
|
|
I will find time to fix it completely.
|
|
This morning, I checked and it appears to be like this: An AAC frame is encoded using 1024 sample samples (not 1024 Bytes). The playback time of an audio frame (in seconds) = number of sample samples in an AAC frame / sample rate. If the sample rate is 44100Hz, according to the above, the playback time of this audio frame = 1024/44100 = 0.02232s * 1000 = 22.32ms. The "pts" base clock is calculated based on 1000ms and reused in "ts", where the base clock is 90k, so it should also be multiplied by 90. So, I'm looking at https://github.com/arut/nginx-rtmp-module/blob/e08959247dc840bb42cdf3389b1f5edb5686825f/hls/ngx_rtmp_hls_module.c#L1809.
|
|
Didn't you see that he annotated TODO?
|
|
And it only explains how to calculate, but does not explain why it should be calculated this way (recalculating timestamps based on sampling instead of using rtmp timestamps). This is the biggest problem in my opinion, in fact, it is unnecessary to calculate in HLS, only necessary when merging audio frames. If both audio and video streams are available in HLS, an audio frame can be written without needing to calculate DTS, directly using rtmp is fine.
|
|
1024 or 960 for AAC-LC
|
|
Calculating DTS based on the number of AAC samples is determined by TBN, and this is clear. The TBN for FLV is 1/1000, while the TBN for TS is 1/90000. TS has higher precision, and in reality, the original encoder also loses precision when outputting FLV/RTMP.
|
|
AAC Audio - Encoder Delay and Synchronization It's also unclear here. Perhaps it's necessary to carefully examine the source code of fdkaac in order to understand.
|
|
The issue of explosive sound in HLS pure audio (there is a popping sound between each segment), Apple's reply email states that AAC files should be used instead of TS files, and the timestamps should be recalculated.
|
|
If TBN is 44100, then converting from FLV with TBN=1000 will result in loss, for example, a 34-millisecond FLV audio converted to TBN44100 would be
|
|
About PCR, read #311 |
winlinvip
changed the title
|
The issue of loud noise has been identified and it is caused by the sampling rate. Please refer to: #547 (comment).
|
winlinvip
changed the title
When slicing the pure audio stream with srs_2.0 into HLS, if the configuration option hls_vcodec vn; hls_acodec aac; is selected, there will be no video information in the PMT of ts. However, when playing the audio using Apple iOS system, there will be stuttering and noise, while it plays normally on Android system.
hls.conf configuration file:
TRANS_BY_GPT3The text was updated successfully, but these errors were encountered: