Accessing the data from RTP header extensions

[destructeur]

From what I understand, it's possible to use and define custom RTP Header Extensions to suit our needs.
As long as the extension is not mandatory for properly receiving and decoding the video, there's nothing that prohibits its use.
It's possible to negotiate the use of these extensions in the SDP, but there's no way to access the raw data of the received packets.

Should it be possible with the RTCRtpReceiver/RTCEncodedVideoFrame to access the data of the different header extensions received in the RTP packets?

[fippo]

Header extensions would need to be negotiated in the SDP so are not freeform.
Exposing them in the API is tricky because they tend to be on individual packets (first, last or all of the frame) and it is not known up-front how many packets a frame is going to be split up into.

[destructeur]

Exactly, however, it's possible to add the extensions we want to the SDP and to accept these same extensions during the negotiation.
When we receive a frame, we could simply have a list of all the header extensions present in the frame with their ID and other useful information.
Alternatively, we could allow the reception of individual RTP packets and not just complete frames.
We can receive the data of the encoded frame but not the data from the header extensions. In my opinion, this information is just as useful, if not more, than the data of an frame.