Podcast episodes are not completely downloading

[lohningerthomas]

I noticed that for some podcasts the end of the episodes are missing.

For example I downloaded this show: https://open.spotify.com/show/5JYitG4bOM3sVmAQRdX1Na and for every episode the last minute or so is missing in the downloaded file.

After running the debug output I noticed that Librespot stopped downloading before all chunks were completed:

DEBUG:Librespot:Session:Chunk 156/181 completed, cached: False, stream: file_id: 7c9d5d4e37e4fb6fd91b2460c54afb2059c11daa

DEBUG:urllib3.connectionpool:https://audio4-ak-spotify-com.akamaized.net:443 "GET /audio/7c9d5d4e37e4fb6fd91b2460c54afb2059c11daa?__token__=exp=1657893805~hmac=9f418cd6ab2eaa013c4147924038e9eda979bb052984e084697417f1b16659e9 HTTP/1.1" 206 131072

DEBUG:Librespot:Session:Chunk 157/181 completed, cached: False, stream: file_id: 7c9d5d4e37e4fb6fd91b2460c54afb2059c11daa

DEBUG:urllib3.connectionpool:https://audio4-ak-spotify-com.akamaized.net:443 "GET /audio/7c9d5d4e37e4fb6fd91b2460c54afb2059c11daa?__token__=exp=1657893805~hmac=9f418cd6ab2eaa013c4147924038e9eda979bb052984e084697417f1b16659e9 HTTP/1.1" 206 131072

DEBUG:Librespot:Session:Chunk 158/181 completed, cached: False, stream: file_id: 7c9d5d4e37e4fb6fd91b2460c54afb2059c11daa

INFO:spodcast.podcast:transcoding ogg->mp3

This has to be some issues with certain shows, as either all of the episodes of a show are missing the last minute or they are all fine.

An example for a working podcast would be:
https://open.spotify.com/show/7BTOsF2boKmlYr76BelijW

I have no clue what the issue could be and am open for any suggestion!

[diveflo]

same thing happening for e.g. this podcast:
https://open.spotify.com/show/6UUIXmp1V0fK4ZpK7vzAbQ

[hugolinhares]

It is happening to this podcast too: https://open.spotify.com/show/0znjxREK2kgs1iL0i7W46V

[Yetangitu]

I'll have a look at these examples to see what is going on, the problem can found somewhere in librespot-python or within spodcast but since spodcast seems to be one of the few tools using the netcast/podcast endpoints it is not surprising for these problems to show up here.

[lohningerthomas]

A quick fix that seems to work for me is to set the chunk size to 1 - the download gets quite slow but it seems to download all chunks and therefore the whole episode

[Yetangitu]

The file size reported by librespot-python - and I assume Spotify - seems to be unreliable for some shows. I removed the dependency on this reported size, this works without loosing the chunked download functionality.

[Yetangitu]

Give the new release (0.5.2) a try to see if it works for you, it Works for Me™ so I assume it will.

[lohningerthomas]

Can confirm, works for me as well - thy for the fix!

[hugolinhares]

Thank you for the fix!! Can you can update the docker image too? Thy again.

[Yetangitu]

Thank you for the fix!! Can you can update the docker image too? Thy again.

@heywoodlh Your turn...

[heywoodlh]

Done, image tag has been updated: 0.5.2

[Yetangitu]

Thanks

[arwk]

I noticed something weird:

When fetching an entire Podcast (https://open.spotify.com/show/3JgH71EJblvCEXAca1XRsT), most of the episodes are completely downloaded. But a few of them are missing some chunks.
However if I proceed downloading the specific episode (https://open.spotify.com/episode/4988PIBdSWNVgYqe3jGzRL), everything gets downloaded.
Unfortunately I don't know if the issue is reproducible and if always the same episodes are not complete, or if this is some stochastic behaviour.

I used this script to process the log (--log-level debug) generated by spodcast. It outputs all episodes with missing chunks.

with open("nur-verheiratet.log") as f:
    last_fileid = None
    last_line = None
    for line in f.readlines():
        if line.startswith("DEBUG:Librespot:Session:Chunk"):
            file_id = line.split()[-1]
            if file_id != last_fileid:
                if last_line != None:
                    a, b = last_line.split()[1].split("/")
                    if a != b:
                        print(last_line)
                last_fileid = file_id
            last_line = line

On my machine they are

DEBUG:Librespot:Session:Chunk 28/451 completed, cached: False, stream: file_id: 0c9562623a92522548116128fc4bc313dec40768
DEBUG:Librespot:Session:Chunk 194/402 completed, cached: False, stream: file_id: e73bf556faf7bf4a7669d5de9cca36920c15a58c
DEBUG:Librespot:Session:Chunk 105/437 completed, cached: False, stream: file_id: 44745459c28e3070ae2f837c507ca25705c45299
DEBUG:Librespot:Session:Chunk 279/332 completed, cached: False, stream: file_id: b50c057a1ccda851ce21c6ed0db6061884030fb5

$ spodcast --version
spodcast 0.5.2