Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stream.hls: fix maps with and without keys #5861

Conversation

bastimeyer
Copy link
Member

Add the key attribute to the Map class, so segment maps can be encrypted independently or left as plain text, depending on the order of the EXT-X-MAP and EXT-X-KEY tags in the playlist.

TODO:
Rewrite the HLSSegment, HLSStreamWriter and HLSStreamWorker classes based on the logic of the DASH implementation, where init segments are queued by the worker separately.


Fixes #5856

Let's fix this bug, even though the current implementation is terrible.

$ streamlink -l debug --hls-duration 20s --player-no-close --http-cookie domand_bid=5e377f548bdfca20defd15f0b178b8b42e538ed14569c97f6338178cde548a49 'https://delivery.domand.nicovideo.jp/hlsbid/65d4695bd76609a88f8bfe85/playlists/media/audio-aac-192kbps.m3u8?session=8b75576e7a29181c14eb064f0359172c7d7896c56598baea0000000065da9fe6f06982f4871bf258&Expires=1708826598&Signature=r0s0nwOBDvaFFKTHFC22EuBanWTXQ0gSXQ2gDAn41bDDtAWsSb1k3yxKAO6-X96as8A2jH4pvoYiw8EKO4p704xTcRGijnmxy2fSBelLhSZ~C0KZNXnWNzgmF15zPKGc1BS8py2RF1UpYRP9y23hPbQR2eWh9c1-XEFnK~drvCltt5O8NgiR79OkZ5WzxR2mhvReS7XUoiNekR1mQsagRsTlZ-Tyktod8imWiMwPGQ6rx0zfohFliGh6Iwb~AJqLS7B4B3FWOxrsA2MTe4SxX97yoLReZ-sNZNrdwc-P5vZ8HSeiGERR4riGyNJpKEalsfG4OKHb-BoIGM2yWjjZeQ__&Key-Pair-Id=K11RB80NFXU134' best
[cli][debug] OS:         Linux-6.7.6-1-git-x86_64-with-glibc2.39
[cli][debug] Python:     3.12.2
[cli][debug] OpenSSL:    OpenSSL 3.2.1 30 Jan 2024
[cli][debug] Streamlink: 6.6.2+3.g88256054
[cli][debug] Dependencies:
[cli][debug]  certifi: 2024.2.2
[cli][debug]  isodate: 0.6.1
[cli][debug]  lxml: 5.1.0
[cli][debug]  pycountry: 23.12.11
[cli][debug]  pycryptodome: 3.20.0
[cli][debug]  PySocks: 1.7.1
[cli][debug]  requests: 2.31.0
[cli][debug]  trio: 0.24.0
[cli][debug]  trio-websocket: 0.11.1
[cli][debug]  typing-extensions: 4.9.0
[cli][debug]  urllib3: 2.2.1
[cli][debug]  websocket-client: 1.7.0
[cli][debug] Arguments:
[cli][debug]  url=https://delivery.domand.nicovideo.jp/hlsbid/65d4695bd76609a88f8bfe85/playlists/media/audio-aac-192kbps.m3u8?session=8b75576e7a29181c14eb064f0359172c7d7896c56598baea0000000065da9fe6f06982f4871bf258&Expires=1708826598&Signature=r0s0nwOBDvaFFKTHFC22EuBanWTXQ0gSXQ2gDAn41bDDtAWsSb1k3yxKAO6-X96as8A2jH4pvoYiw8EKO4p704xTcRGijnmxy2fSBelLhSZ~C0KZNXnWNzgmF15zPKGc1BS8py2RF1UpYRP9y23hPbQR2eWh9c1-XEFnK~drvCltt5O8NgiR79OkZ5WzxR2mhvReS7XUoiNekR1mQsagRsTlZ-Tyktod8imWiMwPGQ6rx0zfohFliGh6Iwb~AJqLS7B4B3FWOxrsA2MTe4SxX97yoLReZ-sNZNrdwc-P5vZ8HSeiGERR4riGyNJpKEalsfG4OKHb-BoIGM2yWjjZeQ__&Key-Pair-Id=K11RB80NFXU134
[cli][debug]  stream=['best']
[cli][debug]  --loglevel=debug
[cli][debug]  --player=/usr/bin/mpv
[cli][debug]  --player-no-close=True
[cli][debug]  --hls-duration=20.0
[cli][debug]  --http-cookie=[('domand_bid', '5e377f548bdfca20defd15f0b178b8b42e538ed14569c97f6338178cde548a49')]
[cli][info] Found matching plugin hls for URL https://delivery.domand.nicovideo.jp/hlsbid/65d4695bd76609a88f8bfe85/playlists/media/audio-aac-192kbps.m3u8?session=8b75576e7a29181c14eb064f0359172c7d7896c56598baea0000000065da9fe6f06982f4871bf258&Expires=1708826598&Signature=r0s0nwOBDvaFFKTHFC22EuBanWTXQ0gSXQ2gDAn41bDDtAWsSb1k3yxKAO6-X96as8A2jH4pvoYiw8EKO4p704xTcRGijnmxy2fSBelLhSZ~C0KZNXnWNzgmF15zPKGc1BS8py2RF1UpYRP9y23hPbQR2eWh9c1-XEFnK~drvCltt5O8NgiR79OkZ5WzxR2mhvReS7XUoiNekR1mQsagRsTlZ-Tyktod8imWiMwPGQ6rx0zfohFliGh6Iwb~AJqLS7B4B3FWOxrsA2MTe4SxX97yoLReZ-sNZNrdwc-P5vZ8HSeiGERR4riGyNJpKEalsfG4OKHb-BoIGM2yWjjZeQ__&Key-Pair-Id=K11RB80NFXU134
[plugins.hls][debug] URL=https://delivery.domand.nicovideo.jp/hlsbid/65d4695bd76609a88f8bfe85/playlists/media/audio-aac-192kbps.m3u8?session=8b75576e7a29181c14eb064f0359172c7d7896c56598baea0000000065da9fe6f06982f4871bf258&Expires=1708826598&Signature=r0s0nwOBDvaFFKTHFC22EuBanWTXQ0gSXQ2gDAn41bDDtAWsSb1k3yxKAO6-X96as8A2jH4pvoYiw8EKO4p704xTcRGijnmxy2fSBelLhSZ~C0KZNXnWNzgmF15zPKGc1BS8py2RF1UpYRP9y23hPbQR2eWh9c1-XEFnK~drvCltt5O8NgiR79OkZ5WzxR2mhvReS7XUoiNekR1mQsagRsTlZ-Tyktod8imWiMwPGQ6rx0zfohFliGh6Iwb~AJqLS7B4B3FWOxrsA2MTe4SxX97yoLReZ-sNZNrdwc-P5vZ8HSeiGERR4riGyNJpKEalsfG4OKHb-BoIGM2yWjjZeQ__&Key-Pair-Id=K11RB80NFXU134; params={}
[utils.l10n][debug] Language code: en_US
[cli][info] Available streams: live (worst, best)
[cli][info] Opening stream: live (hls)
[cli][info] Starting player: /usr/bin/mpv
[stream.hls][debug] Reloading playlist
[cli][debug] Pre-buffering 8192 bytes
[stream.hls][debug] Segments in this playlist are encrypted
[stream.hls][debug] First Sequence: 1; Last Sequence: 29
[stream.hls][debug] Start offset: 0; Duration: 20; Start Sequence: 1; End Sequence: 29
[stream.hls][debug] Adding segment 1 to queue
[stream.hls][debug] Adding segment 2 to queue
[stream.hls][debug] Adding segment 3 to queue
[stream.hls][debug] Adding segment 4 to queue
[stream.hls][info] Stopping stream early after 20
[stream.segmented][debug] Closing worker thread
[stream.hls][debug] Writing segment 1 to output
[stream.hls][debug] Segment initialization 1 complete
[cli.output][debug] Opening subprocess: ['/usr/bin/mpv', '--force-media-title=https://delivery.domand.nicovideo.jp/hlsbid/65d4695bd76609a88f8bfe85/playlists/media/audio-aac-192kbps.m3u8?session=8b75576e7a29181c14eb064f0359172c7d7896c56598baea0000000065da9fe6f06982f4871bf258&Expires=1708826598&Signature=r0s0nwOBDvaFFKTHFC22EuBanWTXQ0gSXQ2gDAn41bDDtAWsSb1k3yxKAO6-X96as8A2jH4pvoYiw8EKO4p704xTcRGijnmxy2fSBelLhSZ~C0KZNXnWNzgmF15zPKGc1BS8py2RF1UpYRP9y23hPbQR2eWh9c1-XEFnK~drvCltt5O8NgiR79OkZ5WzxR2mhvReS7XUoiNekR1mQsagRsTlZ-Tyktod8imWiMwPGQ6rx0zfohFliGh6Iwb~AJqLS7B4B3FWOxrsA2MTe4SxX97yoLReZ-sNZNrdwc-P5vZ8HSeiGERR4riGyNJpKEalsfG4OKHb-BoIGM2yWjjZeQ__&Key-Pair-Id=K11RB80NFXU134', '-']
[stream.hls][debug] Writing segment 1 to output
[cli][debug] Writing stream to output
[stream.hls][debug] Segment 1 complete
[stream.hls][debug] Writing segment 2 to output
[stream.hls][debug] Segment 2 complete
[stream.hls][debug] Writing segment 3 to output
[stream.hls][debug] Segment 3 complete
[stream.hls][debug] Writing segment 4 to output
[stream.hls][debug] Segment 4 complete
[stream.segmented][debug] Closing writer thread
[cli][info] Stream ended
[cli][info] Closing currently open stream...
$ streamlink \
  --quiet \
  --stdout \
  --http-cookie domand_bid=5e377f548bdfca20defd15f0b178b8b42e538ed14569c97f6338178cde548a49 \
  'https://delivery.domand.nicovideo.jp/hlsbid/65d4695bd76609a88f8bfe85/playlists/media/audio-aac-192kbps.m3u8?session=8b75576e7a29181c14eb064f0359172c7d7896c56598baea0000000065da9fe6f06982f4871bf258&Expires=1708826598&Signature=r0s0nwOBDvaFFKTHFC22EuBanWTXQ0gSXQ2gDAn41bDDtAWsSb1k3yxKAO6-X96as8A2jH4pvoYiw8EKO4p704xTcRGijnmxy2fSBelLhSZ~C0KZNXnWNzgmF15zPKGc1BS8py2RF1UpYRP9y23hPbQR2eWh9c1-XEFnK~drvCltt5O8NgiR79OkZ5WzxR2mhvReS7XUoiNekR1mQsagRsTlZ-Tyktod8imWiMwPGQ6rx0zfohFliGh6Iwb~AJqLS7B4B3FWOxrsA2MTe4SxX97yoLReZ-sNZNrdwc-P5vZ8HSeiGERR4riGyNJpKEalsfG4OKHb-BoIGM2yWjjZeQ__&Key-Pair-Id=K11RB80NFXU134' \
  best \
  2>/dev/null \
  | ffprobe -v error -of json -show_streams -
{
    "streams": [
        {
            "index": 0,
            "codec_name": "aac",
            "codec_long_name": "AAC (Advanced Audio Coding)",
            "profile": "LC",
            "codec_type": "audio",
            "codec_tag_string": "mp4a",
            "codec_tag": "0x6134706d",
            "sample_fmt": "fltp",
            "sample_rate": "48000",
            "channels": 2,
            "channel_layout": "stereo",
            "bits_per_sample": 0,
            "initial_padding": 0,
            "id": "0x1",
            "r_frame_rate": "0/0",
            "avg_frame_rate": "0/0",
            "time_base": "1/48000",
            "start_pts": 0,
            "start_time": "0.000000",
            "duration_ts": 289792,
            "duration": "6.037333",
            "bit_rate": "222535",
            "extradata_size": 2,
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0,
                "non_diegetic": 0,
                "captions": 0,
                "descriptions": 0,
                "metadata": 0,
                "dependent": 0,
                "still_image": 0
            },
            "tags": {
                "creation_time": "2024-02-20T08:58:32.000000Z",
                "language": "eng",
                "handler_name": "ETI ISO Audio Media Handler",
                "vendor_id": "[0][0][0][0]"
            }
        }
    ]
}

Add the `key` attribute to the `Map` class, so segment maps can be
encrypted independently or left as plain text, depending on the order
of the `EXT-X-MAP` and `EXT-X-KEY` tags in the playlist.

TODO:
Rewrite the `HLSSegment`, `HLSStreamWriter` and `HLSStreamWorker`
classes based on the logic of the DASH implementation, where
init segments are queued by the worker separately.
@bastimeyer bastimeyer merged commit de981a4 into streamlink:master Feb 25, 2024
22 checks passed
@bastimeyer bastimeyer deleted the stream/hls/fix-plaintext-maps-encrypted-segments branch February 25, 2024 08:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

stream.hls: streamlink incorrectly decrypts media initialization section and gets error
2 participants