[Patreon] JSONDecodeError

[zxgbs]

I think this was because the site changed
verbose log:
[gallery-dl][debug] Version 1.26.1
[gallery-dl][debug] Python 3.10.6 - Windows-10-10.0.19045-SP0
[gallery-dl][debug] requests 2.25.1 - urllib3 1.26.12
[gallery-dl][debug] Configuration Files ['%USERPROFILE%\gallery-dl.conf']
[gallery-dl][debug] Starting DownloadJob for 'https://www.patreon.com/posts/september-2023-91731970'
[patreon][debug] Using PatreonPostExtractor for 'https://www.patreon.com/posts/september-2023-91731970'
[patreon][debug] TLS 1.2 disabled.
[cookies][debug] Extracting cookies from C:\Users\123\AppData\Roaming\Mozilla\Firefox\Profiles\l91x86hv.default-release-1650697863926\cookies.sqlite
[cookies][info] Extracted 624 cookies from Firefox
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.patreon.com:443
[urllib3.connectionpool][debug] https://www.patreon.com:443 "GET /posts/september-2023-91731970 HTTP/1.1" 200 None
[patreon][error] An unexpected error occurred: JSONDecodeError - Expecting value: line 1 column 1 (char 0). Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[patreon][debug]
Traceback (most recent call last):
File "C:\Users\123\AppData\Local\Programs\Python\Python310\lib\site-packages\gallery_dl\job.py", line 124, in run
for msg in extractor:
File "C:\Users\123\AppData\Local\Programs\Python\Python310\lib\site-packages\gallery_dl\extractor\patreon.py", line 38, in items
for post in self.posts():
File "C:\Users\123\AppData\Local\Programs\Python\Python310\lib\site-packages\gallery_dl\extractor\patreon.py", line 327, in posts
post = self._extract_bootstrap(page)["post"]
File "C:\Users\123\AppData\Local\Programs\Python\Python310\lib\site-packages\gallery_dl\extractor\patreon.py", line 252, in _extract_bootstrap
return util.json_loads(text.extr(
File "C:\Users\123\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\123\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[Tobi823]

I can confirm the bug:

gallery-dl --verbose --cookies cookies-patreon-com.txt "https://www.patreon.com/posts/..."
[gallery-dl][debug] Version 1.26.1
[gallery-dl][debug] Python 3.11.5 - Linux-6.5.5-1-MANJARO-x86_64-with-glibc2.38
[gallery-dl][debug] requests 2.31.0 - urllib3 2.0.7
[gallery-dl][debug] Configuration Files ['${HOME}/.config/gallery-dl/config.json']
[gallery-dl][debug] Starting DownloadJob for 'https://www.patreon.com/posts/...'
[patreon][debug] Using PatreonPostExtractor for 'https://www.patreon.com/posts/...'
[patreon][debug] TLS 1.2 disabled.
[patreon][debug] Loading cookies from 'cookies-patreon-com.txt'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.patreon.com:443
[urllib3.connectionpool][debug] https://www.patreon.com:443 "GET /posts/... HTTP/1.1" 200 None
[patreon][error] An unexpected error occurred: JSONDecodeError - Expecting value: line 1 column 1 (char 0). Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[patreon][debug] 
Traceback (most recent call last):
  File "/home/hacker/.local/pipx/venvs/gallery-dl/lib/python3.11/site-packages/gallery_dl/job.py", line 124, in run
    for msg in extractor:
  File "/home/hacker/.local/pipx/venvs/gallery-dl/lib/python3.11/site-packages/gallery_dl/extractor/patreon.py", line 38, in items
    for post in self.posts():
                ^^^^^^^^^^^^
  File "/home/hacker/.local/pipx/venvs/gallery-dl/lib/python3.11/site-packages/gallery_dl/extractor/patreon.py", line 327, in posts
    post = self._extract_bootstrap(page)["post"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hacker/.local/pipx/venvs/gallery-dl/lib/python3.11/site-packages/gallery_dl/extractor/patreon.py", line 252, in _extract_bootstrap
    return util.json_loads(text.extr(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[Tobi823]

Maybe already fixed in master? #4715

[MarqFJA87]

Maybe already fixed in master? #4715

I installed the new master, now it says that it can't find the "campaign ID".

[Tobi823]

Looks like Patreon change the JavaScript structure. The variable page in https://github.com/mikf/gallery-dl/blob/master/gallery_dl/extractor/patreon.py#L253 contains the hole webpage as a string.

But in this var I can't find the substring "window.patreon.bootstrap," - I only similar data is window.patreon = {"bootstrap":

<!DOCTYPE html><html><head><meta charSet="utf-8"/><script type="text/javascript">
            window.patreon = {"bootstrap":{"campaign":{"data":{"attributes":{"creation_name":"...","currency":"USD","current_user_can_be_free_member":false,"current_user_is_free_member":false,"has_rss":false,"is_monthly":true,"offers_free_membership":true,"offers_paid_membership":true,"pay_per_name":"month","pledge_url":"/checkout/....","primary

[Tobi823]

Found the bug

[Tobi823]

https://github.com/mikf/gallery-dl/blob/master/gallery_dl/extractor/patreon.py#L253 must be changed from
page, "window.patreon.bootstrap,", "});") + "}") to
page, "window.patreon = {\"bootstrap\":", "},\"apiServer\"") + "}")

(or page, 'window.patreon = {"bootstrap":', '},"apiServer"') + "}"))

[Tobi823]

I created a PR

[variousinputs]

Tobi823: I can confirm the patch works fine. Fixed my issue! Thanks!

[liyiecho]

weibo also has this problem, when I download from weibo it show [weibo][error] Unable to download data: JSONDecodeError: Expecting value: line 2 column 3 (char 3)

[mikf]

@liyiecho open a separate issue for this and post a --verbose log.
All weibo tests still complete without any error.