Search query longer than 100 characters returns an error

[duracell013]

Describe the bug
The search method returns an error when the query string is longer than 100 characters. Is this a limitation of the Spotify web API?

Your code

sp = spotipy.Spotify(client_credentials_manager=SpotifyClientCredentials(), requests_timeout=10, retries=10)
string100 = 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbb'
sp.search(string100)
sp.search(string100 + 'z')

Expected behavior
The search method should be able to handle queries of more than 100 characters

Output for the first search with 100-character string [OK]

{'tracks': {'href': 'https://api.spotify.com/v1/search?query=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbb&type=track&offset=0&limit=10', 'items': [], 'limit': 10, 'next': None, 'offset': 0, 'previous': None, 'total': 0}}

Output for the second search with 101-character string [NOT OK]

HTTP Error for GET to https://api.spotify.com/v1/search returned 404 due to Not found.
Traceback (most recent call last):
  File "C:\Users\smagne\AppData\Local\Programs\Python\Python37-32\lib\site-packages\spotipy\client.py", line 175, in _internal_call
    response.raise_for_status()
  File "C:\Users\smagne\AppData\Local\Programs\Python\Python37-32\lib\site-packages\requests\models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api.spotify.com/v1/search?q=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbz&limit=10&offset=0&type=track

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<pyshell#62>", line 1, in <module>
    sp.search(string100+'z')
  File "C:\Users\smagne\AppData\Local\Programs\Python\Python37-32\lib\site-packages\spotipy\client.py", line 464, in search
    "search", q=q, limit=limit, offset=offset, type=type, market=market
  File "C:\Users\smagne\AppData\Local\Programs\Python\Python37-32\lib\site-packages\spotipy\client.py", line 210, in _get
    return self._internal_call("GET", url, payload, kwargs)
  File "C:\Users\smagne\AppData\Local\Programs\Python\Python37-32\lib\site-packages\spotipy\client.py", line 190, in _internal_call
    headers=response.headers,
spotipy.exceptions.SpotifyException: http status: 404, code:-1 - https://api.spotify.com/v1/search?q=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbz&limit=10&offset=0&type=track:
 Not found.

Environment:

• OS: Windows
• Python 3.7
• spotipy 2.13.0
• IDE: IDLE

[stephanebruckert]

I can confirm I'm seeing 404s for searches with queries longer than 100 characters since end of April 2022. It doesn't make any sense because many tracks+artist that long. It's also not documented in the doc.

Please upvote https://community.spotify.com/t5/Spotify-for-Developers/V1-API-Search-q-query-parameter-appears-to-have-a-limit-of-100/td-p/5398898

In spotipy I wonder how this should be handled, because 404 Not Found is wrong, it should be a 400 Bad Request. When len(query) > 100, should we:

• just add a warning while calling search()
• add a warning when receiving 404
• return an empty array when receiving 404
• modify the 404 and return a 400 instead?

[duracell013]

@stephanebruckert thanks for confirming the issue. I've upvoted the post on the Spotify API website.

[Razlif]

looks like spotify fixed the return code issue, they now return 400.
sent a query longer than 100 chars and got the following:

HTTP Error for GET to https://api.spotify.com/v1/search with Params: {'q': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbz', 'limit': 10, 'offset': 0, 'type': 'track', 'market': None} returned 400 due to Bad request.
Traceback (most recent call last):
  File "/home/ylifshit/projects/spotipy/spotipy/client.py", line 245, in _internal_call
    response.raise_for_status()
  File "/app/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.spotify.com/v1/search?q=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbz&limit=10&offset=0&type=track

[stephanebruckert]

This is good but yeah, the problem is still there.

Personally I'm ignoring queries that are > 100 but that's because my use case allows it. Trimming the query would be not only difficult but also opinionated, because all queries are personalised. In the case of a complex query, what to trim? a 50-char long artist, or a 80-char long track?

What we could do at first is display a warning when we get a 400 and the query is > 100. This way if one day the API changes the limit from 100 to another number, spotipy won't be preventing the request.
In the future we might want to create a util to cleanse queries. The idea to remove brackets rootshellz/Beatporter#15 (comment) is quite good.