Playlists stopped working

[Fubukimaru]

I have two computers, one with Ubuntu 18.04 and the other one with Manjaro and on both the playlists are not shown. Playback and search work without problems.

I'm getting the following log with mopidy -vvv

...
DEBUG    2018-05-24 17:41:57,643 [25524:SpotifyEventLoop] spotify.eventloop
  Timeout reached; processing events
DEBUG    2018-05-24 17:41:57,643 [25524:SpotifyEventLoop] spotify.session
  Notify main thread
DEBUG    2018-05-24 17:41:57,644 [25524:SpotifyEventLoop] spotify.eventloop
  Waiting 0.067s for new events
DEBUG    2018-05-24 17:41:57,644 [25524:SpotifyEventLoop] spotify.eventloop
  Notification received; processing events
DEBUG    2018-05-24 17:41:57,644 [25524:SpotifyEventLoop] spotify.eventloop
  Waiting 0.066s for new events
DEBUG    2018-05-24 17:41:57,711 [25524:SpotifyEventLoop] spotify.eventloop
  Timeout reached; processing events
DEBUG    2018-05-24 17:41:57,711 [25524:SpotifyEventLoop] spotify.eventloop
  Waiting 0.877s for new events
DEBUG    2018-05-24 17:41:57,742 [25524:Dummy-9] spotify.session
  Notify main thread
DEBUG    2018-05-24 17:41:57,744 [25524:SpotifyEventLoop] spotify.eventloop
  Notification received; processing events
DEBUG    2018-05-24 17:41:57,744 [25524:SpotifyEventLoop] spotify.session
  Notify main thread
DEBUG    2018-05-24 17:41:57,745 [25524:SpotifyEventLoop] spotify.eventloop
  Waiting 0.844s for new events
DEBUG    2018-05-24 17:41:57,745 [25524:SpotifyEventLoop] spotify.eventloop
  Notification received; processing events
DEBUG    2018-05-24 17:41:57,745 [25524:SpotifyEventLoop] spotify.session
  User info updated
DEBUG    2018-05-24 17:41:57,745 [25524:SpotifyEventLoop] spotify.session
  User info updated
DEBUG    2018-05-24 17:41:57,746 [25524:SpotifyEventLoop] spotify.eventloop
  Waiting 0.843s for new events

[chises]

i have excatly the same issue. i did not change anything.

client_id & client_secret are set in mopidy config. But no playlist are available.

mopidy log:

May 24 17:48:59 Ipse mopidy[24797]: INFO Logged in to Spotify in online mode
...

DEBUG 2018-05-24 17:44:25,959 [22798:MpdSession-17] mopidy.mpd.session
Request from [::ffff:127.0.0.1]:34480: command_list_begin
DEBUG 2018-05-24 17:44:25,960 [22798:MpdSession-17] mopidy.mpd.session
Request from [::ffff:127.0.0.1]:34480: load "spotify::playlist:6x5yNjb1UlnsNgPgLQJzFA"
DEBUG 2018-05-24 17:44:25,961 [22798:MpdSession-17] mopidy.mpd.session
Request from [::ffff:127.0.0.1]:34480: command_list_end
DEBUG 2018-05-24 17:44:25,968 [22798:MpdSession-17] mopidy.mpd.session
Response to [::ffff:127.0.0.1]:34480: ACK [50@0] {load} No such playlist

[AidanOD]

Same issue.
I can playback songs/artist/album from the uri but not playlists.

Was working nothing change on my end.

[kingosticks]

I currently see the same. The question is, is it a temporary issue or the next step in libspotify's slow and painful death? I have asked but I'm not hopeful they'll bother answering.

…

On Thu, 24 May 2018, 20:26 AidanOD, ***@***.***> wrote: Same issue. I can playback songs/artist/album from the uri but not playlists. Was working nothing change on my end. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#182 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA5DqDlIiZLn3mokQMNZHgW4L4w_fZyKks5t1wl9gaJpZM4UMgcP> .

[Koubi4cK]

Same here, came back from work and no more personnal playlists.

However i can acces the ones provided by spotify.

[blacklight]

Well, we really can't say that we hadn't been warned that libspotify was going to die...

Still, it'd be good if Spotify could at least provide developers with timelines or at least a heads up whenever they're going to kill some features.

Is there some viable alternative to support playlists using the web API or spotipy? I know that user playlists can be retrieved that way - would playback be possible as well without libspotify?

[kingosticks]

It's well and good warning users (which they also managed to screw up) but not much use if you are not going to provide a proper alternative.

You can lookup user playlists and everything else using the Web API so that's fine. But playback is not possible like we do now. They have not provided any replacement for this other than their entirely unsuitable web playback API which requires a web browser.

[blacklight]

It sounds like a possible (although dirty and slow) workaround, for the time being, might be to get the user playlists via web API, expand them and add/play item by item - at least libspotify doesn't break when adding/playing non-user-generated content so far.

[kingosticks]

It's definitely the plan and it's very usable, just nobody has done it yet. I don't expect Spotify to provide any other option so I wouldn't think of it as temporary workaround either.

I implemented a proof of concept some time ago but I didn't get around to implementing it properly. Things do slow down a little when loading large playlists but once cached it's fine (their API support etags too). There's no reason we can't store the cache and re-use it between runs too, just like libspotify does.

[blacklight]

Using the fix/incompatible_playlists (almost) fixes the problem in my case.
I can see the playlists again in my interface but they have no items when I load them.
It seems that SpotifyPlaylistsProvider.get_items() is never called (it used to be called before) and therefore the playlists content is never populated.

Any idea why this may happen?
Let me know if I can help in any way to speed up the rollout of this fix.

[kingosticks]

I can't think why that would be, it's up to the client to call get_items() and musicbox-webclient seemed to work OK with it. I'm going to try look at it again this weekend but anyone is very welcome to to have their own stab at implementing something better.

[blacklight]

I may have got the reason, in most of the cases you're assuming that the tracks field of the web playlist is a list (see kingosticks@9b892df#diff-2bb29d223c95fe3e36489c9fc18e5017R290) while instead it's a hash that looks like {"total":123, "href":"https://api.spotify.com/v1/users/me/playlists/<id>/tracks"}.

Maybe a previous implementation of the web API returned the tracks directly inside the playlist object, while the latest implementation requires you to get them by tracks URL.

[kamek-pf]

The broader question is, is there any other content provider maintaining a low level library allowing users to interact with their services ? Deezer or anything ?

[blacklight]

@kamek-pf the spotify-deezer repo has been shut down after a DMCA complaint started by Deezer itself. I have the repo still cloned locally but many API requests to Deezer are no longer working (and I don't feel like investigating or fixing it until Deezer changes its approach towards developers building stuff on top of their services).

mopidy-gmusic has issues as well as of today since the self-generated device_id is no longer working mopidy/mopidy-gmusic#195 - and it will probably break more in the next days/weeks as Google will move most of the GMusic services to the new YouTube Music service or however they want to call it.

Anyway, I have prepared a fix for the Spotify playlists issue on top of @kingosticks old fix/incompatible_playlists branch blacklight@e245c2d. It's probably not the ideal fix and it will slow down Mopidy startup a bit, especially on Raspberry (I'm sure that more caching can be done on the tracks retrieval through the web API), but at least it should make things work again:

git clone git@github.com:BlackLight/mopidy-spotify.git
cd mopidy-spotify
git checkout fix/incompatible_playlists
[sudo] python2 setup.py build install

[bpetrikovics]

Thank you, I've just tested and it works. :)

[quimbydogg]

@blacklight Thanks, I'm able to view my playlists again! Looking at mopidy -vvvvvvv I've noticed a few error messages associated with what looks to be the playlist lookups. Everything seems fine even though there there are a few ERROR messages sprinkled in with DEBUG/TRACE notes related to pulling in the playlist info. I'm not very familiar with mopidy or mopidy-spotify in general. Is this just my deployment or anything to be worried about?

ERROR 2018-05-25 18:26:22,395 [24859:SpotifyBackend-6] mopidy_spotify.playlists _get_playlist spotify:user:

[blacklight]

@quimbydogg those are just debug traces that were already in the branch, nothing to be worried about :)

[Maskmagog]

EDIT: Feel free to ignore this, I'm way out of my league, and I'll just wait and hope this fix finds its way to "normal" mopidy-spotify. Carry on :)

Hi, sorry for my total noob question :)
I read BlackLights (temporary?) solution above, but in what directory should I be in when executing those commands? (git clone etc). Raspberry Pi, Raspbian Jessie, using mopidy, mpc to play from spotify.

edit: I tried it in home/pi/ directory, but it didn't seem to solve it. My guess is that I should place it somewhere else, but I don't know where.

Background:
I've built an RPi music player for my kids, with different RFID stickers starting different Spotify playlist via mpc commands. This worked beautifully until recently, and I guess I found out why in this thread. I can still use mpc add spotify:track:3tGjDR45XGBw4xV7eBnXSo (for example), and then mpc play, but can't load playlists.

Big thanks to all of you for enabling us mere mortals to build fun stuff! :)

[blacklight]

@Maskmagog no worries, there's no such thing as silly questions, only silly answers ^^

git and shell basics troubleshooting is a bit OT here though, but feel free to drop by the #mopidy freenode channel for more support.

[L422Y]

@blacklight does this require using the spotify web client as well?

[blacklight]

@L422Y it's not required, mopidy-spotify can now leverage the web API as well (and IMHO mopidy-spotify and mopidy-spotify-web should sooner or later converge as they do overlap a lot now).

[L422Y]

Alright, well playlists still do not seem to work even with your adjusted code :(

[blacklight]

Can you post the output of mopidy -vvvv? Did you install from the fix/incompatible_playlist branch? Also did you make sure to remove any previous installation of mopidy-spotify (via apt or from /usr/[local]/lib/python2.7/site-packages)?

[Fubukimaru]

They are back with this fix :)

[L422Y]

@blacklight https://pastebin.com/TzMjMUcs
The normal error I receive is
loading: Office Playlist error: No such playlist
I cloned the repo and switched to the fix/incompatible_playlist branch, ceared the site-packages and ran:

python2 setup.py build install

Everything compiles and installs correctly, just can't play a playlist that worked last week and mpc lsplaylists returns nothing :(

EDIT: this ended up being a double whammy, I needed to upgrade from Ubuntu 14.04 LTS in order to get a newer version of Python w/ newer SSL support, otherwise the auth token system was failing. I'm not sure if that was the primary issue before I started messing with things, do-release-upgrade helped to get me up to a newer Ubuntu implementation. I've removed my followup comments as they were irrelevant.

[blacklight]

@L422Y I think your mopidy instance is still using the old mopidy-spotify code, not the newly installed branch. The new branch should print a bunch of (debug) error log lines to trace the status of the tracks loaded in each playlists and I can't see those lines in your log output.

You said that you had to create the site-packages folder - that shouldn't be the case, especially if mopidy-spotify was already installed. If you're using Ubuntu and you installed mopidy-spotify through apt then the installation folder is likely to be dist-packages instead. But in such a case, you'd better uninstall mopidy-spotify cleanly through apt and then install the new branch from the sources.

[thescottyq]

I am running Mopidy on a Raspberry Pi that uses MUSCIBOX from here - http://www.pimusicbox.com/

Does anyone know how I can get this fix onto the Mopidy installed on there please?

Any and all help would be much appreciated

[kingosticks]

@blacklight there are still changes going on in there and it hasnt had any review yet. So i wouldn't like to call it the ultimate solution.

[blacklight]

@kingosticks ok let me know if you need any help with tests or reviews.

[Elvilmalia]

I have tried this solution web-api-playlists-v2 but no go for me. Still can't see my personnal playlist.

[weskerfoot]

I've created a small commandline tool that just synchronizes your spotify playlists with MPD once. https://github.com/weskerfoot/Spotify-MPD-Sync

Can't promise that it is a perfect solution but I've been using it without trouble for a while. It clears your spotify playlists on the MPD side every time it syncs up if it finds that there are any differences (spotify is the source of truth).

I'm not sure if this could be integrated into mopidy-spotify because it is fairly slow if you have a lot of playlists/tracks, and it doesn't make sense to run it every time you start up mopidy (more like once a day or when you know you've changed things).

[MrLemur]

I can confirm the web-api-playlists-v2 branch works as expected, but start up take a very long time.

This is exacerbated by using Iris which needs to query every playlist to get the tracks.

[blacklight]

Any progress with merging this into master? It's been on hold for more than a year.

Should we wait until mopidy 3 and Python 3 integration before seeing this merged, or is it possible to provide a timeline for the merge? The situation with the forks is becoming quite chaotic, maybe the time to merge has come?

[damonhere]

I was also able to get the #web-api-playlists-v2 branch to work, but as @MrLemur noted it now takes a very long time to start up, especially on limited hardware. On my project raspberry pi zero W it takes just over 10 minutes to load, and I don't have very many playlists with very many songs.

[blacklight]

The slowness in the new playlists branch it's something reported by many users and something that I've noticed on all of my machines as well.

However I'm pretty sure by now that it's due to something changed in mopidy more than mopidy-spotify.

I've seen people reporting slowness in loading playlists both on my branch and on #web-api-playlists-v2. And that's something I also experience on both the branches on any version of mopidy >= 2.2.3 (tested on Arch, Debian and Raspbian), but things are still smooth on Spotify playlist load on a couple of Raspberry Pis where there's still Raspbian Jessie with the stock mopidy 2.1.0 installed.

@kingosticks do you also experience the same behaviour? Has anything changed in mopidy between 2.1 and 2.2 when it comes to playlist lookup? Maybe now mopidy also gets the metadata for all the tracks and that slows down the lookup?

[damonhere]

FYI I'm on mopidy 2.2.2, and although I don't have the exact timing for how long mopidy took before I installed web-api-playlist-v2, prior to installing this branch everything seemed ready to go immediately after boot. In fact, after installing the web-api-playlist-v2 I thought I had completely broken my mopidy installation as mpc and iris seemed not to connect to mopidy, but I left the pi on overnight and when I returned in the morning, "lo and behold" it was working!

Two relevant lines in my mopidy.log file are:
...
2019-07-19 19:19:29,799 INFO [1952:SpotifyBackend-4] mopidy_spotify.utils: Refresh Playlists took 138519ms
...
2019-07-19 19:27:02,600 INFO [1952:HttpServer] tornado.access: 101 GET /mopidy/ws/ (192.168.0.118) 230850.74ms

These two are definitely the longest, but not the only ones that take long.

[kingosticks]

Adding to the tracklist via MPD changed in v2.2.0 (mopidy/mopidy#1621). Yes, it does lookups now.

I was reluctant to complicate things any further by persisting the playlist cache data but that is what libspotify did, and 10 minutes doesn't sound great. I would suggest updating your version of pykka to the latest, there are some improvements to be had using that.

[blacklight]

@kingosticks I'm using pykka 2.0.0 here (same version as the one on Github). And even though the latest version makes the lookup a bit less heavy, the latency is not yet comparable with what it used to be on earlier versions of mopidy.

I understand that retrieving the metadata for all the tracks in a playlist makes things more consistent with the MPD protocol, but probably we shouldn't treat the lookup cost of a locally stored playlist in the same way we treat the lookup cost of a remote Spotify playlist with > 1000 items.

A couple of ideas:

• Make the new _lookup_playlist logic with metadata retrieval overridable by the backend, so that the full metadata of the tracks is only retrieved on demand (e.g. through lsinfo)?

• Caching the playlist metadata? I know that it will add complexity, but at this point it might be a better option than a 5-10 minutes latency to load a large playlist.

[kingosticks]

Caching the playlist metadata

It is all cached. The playlist endpoints of Spotify's Web API have max-age=0 which means you should technically always re-fetch them. However, they do support Etags so you can tell when nothing has changed and skip downloading all the data again. What I have implemented takes advantage of this and there should be no 5-10 min latency to load a large playlist (once the initial playlist load during startup has completed).

I have also overridden the max-age directive to keep the cache valid for a short length of time thus avoiding the etag request in some cases; I think I did 10 seconds. It's a bit of a hack and I'm not very happy with the idea but it does help, maybe not as much now we have Pykka v2.0.

[jarodwsams]

I can confirm the web-api-playlists-v2 branch works as expected, but start up take a very long time.

This is exacerbated by using Iris which needs to query every playlist to get the tracks.

New Mopidy and Mopidy-Spotify user here. Took me a while, but I finally found the references to the
fix/web-api-playlists-v2 branch above, and I'm happy to report that it works as expected for me as well.

[Glog78]

I get the following error with the branch (on arch)->
2019-07-22 23:13:04,691 ERROR [3858:SpotifyBackend-7] pykka: Unhandled exception in SpotifyBackend (**************************************):
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/pykka/_actor.py", line 186, in _actor_loop
self.on_start()
File "/usr/lib/python2.7/site-packages/mopidy_spotify/backend.py", line 69, in on_start
self.playlists.refresh()
File "/usr/lib/python2.7/site-packages/mopidy_spotify/playlists.py", line 65, in refresh
self._get_playlist(playlist_ref.uri)
File "/usr/lib/python2.7/site-packages/mopidy_spotify/playlists.py", line 53, in _get_playlist
self._backend._bitrate, as_items)
File "/usr/lib/python2.7/site-packages/mopidy_spotify/playlists.py", line 85, in playlist_lookup
web_playlist = web_client.get_playlist(uri)
File "/usr/lib/python2.7/site-packages/mopidy_spotify/web.py", line 428, in get_playlist
'market': 'from_token'})
File "/usr/lib/python2.7/site-packages/mopidy_spotify/web.py", line 385, in get_one
result.increase_expiry(self._extra_expiry)
AttributeError: 'dict' object has no attribute 'increase_expiry'
2019-07-22 23:13:04,692 ERROR [3858:MainThread] mopidy.commands: Actor died: SpotifyBackend (urn:uuid:6a56be9f-c5aa-4940-bea9-9930a2027049) stopped before handling the message

[jodal]

Fixed by the merge of PR #235.