[instagram] An unexpected error occurred: TypeError - the JSON object must be str, bytes or bytearray, not NoneType. #2644

viceroycowboy · 2022-05-31T18:52:20Z

[gallery-dl][debug] Version 1.22.0 - Git HEAD: 30e3d8883
[gallery-dl][debug] Python 3.9.13 - macOS-12.4-arm64-arm-64bit
[gallery-dl][debug] requests 2.27.1 - urllib3 1.26.9
[gallery-dl][debug] Starting DownloadJob for 'https://www.instagram.com/instagram/'
[instagram][debug] Using InstagramUserExtractor for 'https://www.instagram.com/instagram/'
[instagram][debug] Using InstagramPostsExtractor for 'https://www.instagram.com/instagram/posts/'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.instagram.com:443
[urllib3.connectionpool][debug] https://www.instagram.com:443 "GET /instagram/ HTTP/1.1" 200 None
[instagram][error] An unexpected error occurred: TypeError - the JSON object must be str, bytes or bytearray, not NoneType. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[instagram][debug] 
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.9/site-packages/gallery_dl/job.py", line 77, in run
    for msg in extractor:
  File "/opt/homebrew/lib/python3.9/site-packages/gallery_dl/extractor/instagram.py", line 49, in items
    for post in self.posts():
  File "/opt/homebrew/lib/python3.9/site-packages/gallery_dl/extractor/instagram.py", line 475, in posts
    user = self._extract_profile_page(url)
  File "/opt/homebrew/lib/python3.9/site-packages/gallery_dl/extractor/instagram.py", line 388, in _extract_profile_page
    data = self._extract_shared_data(page)["entry_data"]
  File "/opt/homebrew/lib/python3.9/site-packages/gallery_dl/extractor/instagram.py", line 380, in _extract_shared_data
    data = json.loads(shared_data)
  File "/opt/homebrew/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py", line 339, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not NoneType
[instagram][debug] Using InstagramStoriesExtractor for 'https://www.instagram.com/stories/instagram/'
[instagram][debug] Sleeping for 8.996 seconds

The text was updated successfully, but these errors were encountered:

rorgoroth · 2022-05-31T21:52:53Z

I'm getting the same, must be a site update because I've had to allow a new 3rd party cdn in uBlock for the site to work today.

romainlavisse · 2022-05-31T21:59:30Z

I got the same error when i try to download using cookies

mikf · 2022-06-01T09:05:56Z

Instagram is still working for me. Probably a geolocation-dependent change.
Could someone with this error do a run with --write-pages and post the generated 01_https_www.instagram.com_instagram_.dump file here, so that I can see what is different?

$ gallery-dl https://www.instagram.com/instagram/
/tmp/instagram/instagram/2850451812694367232.mp4
/tmp/instagram/instagram/2850399139174555561_2850398181513369749.mp4
...

rorgoroth · 2022-06-01T11:50:31Z

Interesting, it does work but like romainlavisse says it is when using cookies the error occurs.

Here is the dump anyway: 01_https_www.instagram.com_instagram_.dump

mikf · 2022-06-01T13:10:31Z

Thanks for the dump.

Being logged in or not doesn't matter for me, it still works all the same:

$ gallery-dl --cookies cookies-instagram-com.txt https://www.instagram.com/instagram/
/tmp/instagram/instagram/2850451812694367232.mp4
...
$ gallery-dl -u MYUSERNAME https://www.instagram.com/instagram/
/tmp/instagram/instagram/2850451812694367232.mp4
...

rorgoroth · 2022-06-01T13:40:31Z

Hm, I have no idea what's going on.

Logging in with -u -p works fine (although incredibly slow) but then fails with the same error as when using cookies.

romainlavisse · 2022-06-01T13:44:34Z

On my side i got the error when i download a full instagram, ive not try to dl only one image

rorgoroth · 2022-06-01T15:57:14Z

On my side i got the error when i download a full instagram, ive not try to dl only one image

Yeah I'm going for the full page too.
It works fine when specifying the post, for example:

>> gdl https://www.instagram.com/instagram/
[instagram][error] An unexpected error occurred: TypeError - the JSON object must be str, bytes or bytearray, not NoneType. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .

>> gdl https://www.instagram.com/p/CdqlFNiF58H/
./instagram/instagram/2840245600859430663.jpg

>>

- use different way to fetch user IDs - use new API endpoints for /tagged/ and single posts

mikf · 2022-06-01T20:14:41Z

This should be fixed with 2fb0193. I hope I haven't broken anything else ...

It no longer tries to fetch user IDs from a user's main profile page and uses some API endpoint instead.

rorgoroth · 2022-06-01T21:10:07Z

Thanks, that seems to fix the issue although the problem still persists on stories.

Slightly different results than before as both full profile story and specific story url are both broken.

romainlavisse · 2022-06-01T21:10:43Z

If i download once again the binary, it will have the update ?

ethanforvest · 2022-06-03T09:48:37Z

The error only gets when I try to download stories.

mikf · 2022-06-03T15:52:37Z

Stories are fixed as well (d0dc29f)

On an unrelated note: It is now possible to use user IDs instead of screen names (ee7cea8)

If i download once again the binary, it will have the update ?

The executables from https://github.com/mikf/gallery-dl/actions/workflows/executables.yml will.

rorgoroth · 2022-06-03T16:55:37Z

Unfortunately it's still kinda broken for me, now the error is as follows:

>> gallery-dl --verbose --cookies ~/.config/cookies.txt https://www.instagram.com/stories/username
[gallery-dl][debug] Version 1.22.1-dev - Git HEAD: 65ec766
[gallery-dl][debug] Python 3.10.4 - Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with
[gallery-dl][debug] requests 2.27.1 - urllib3 1.26.9
[gallery-dl][debug] Starting DownloadJob for 'https://www.instagram.com/stories/username'
[instagram][debug] Using InstagramStoriesExtractor for 'https://www.instagram.com/stories/username'
[instagram][error] An unexpected error occurred: AttributeError - 'NoneType' object has no attribute 'startswith'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[instagram][debug]
Traceback (most recent call last):
  File "/home/ray/.local/lib/python3.10/site-packages/gallery_dl/job.py", line 77, in run
    for msg in extractor:
  File "/home/ray/.local/lib/python3.10/site-packages/gallery_dl/extractor/instagram.py", line 49, in items
    for post in self.posts():
  File "/home/ray/.local/lib/python3.10/site-packages/gallery_dl/extractor/instagram.py", line 753, in posts
    reel_id = self._uid_by_screen_name(self.item)
  File "/home/ray/.local/lib/python3.10/site-packages/gallery_dl/extractor/instagram.py", line 155, in _uid_by_screen_name
    if screen_name.startswith("id:"):
AttributeError: 'NoneType' object has no attribute 'startswith'

fixing the fix ...

mikf · 2022-06-03T17:03:16Z

Sorry about that: 46d171c

rorgoroth · 2022-06-03T17:05:08Z

Ah, it's working great again.
Thank you so much!

ShadwDrgn · 2022-06-03T17:29:14Z

still crashing (for me) with the latest commit from master which seems to include that one:

    ~/tmp/gallery-dl/bin/instagram    master  ~/tmp/gallery-dl/bin/gallery-dl "https://www.instagram.com/instagram/" --cookies-from_browser brave -D . --verbose
[gallery-dl][debug] Version 1.22.0
[gallery-dl][debug] Python 3.10.4 - Linux-5.15.41-1-MANJARO-x86_64-with-glibc2.35
[gallery-dl][debug] requests 2.27.1 - urllib3 1.26.9
[gallery-dl][debug] Starting DownloadJob for 'https://www.instagram.com/instagram/'
[cookies][debug] Extracting cookies from /home/user/.config/BraveSoftware/Brave-Browser/Default/Cookies
[cookies][debug] Detected desktop environment: kde
[cookies][debug] Chosen keyring: kwallet
[cookies][debug] using kwallet-query to obtain password from kwallet
[cookies][debug] NetworkWallet = 'kdewallet'
[cookies][debug] Failed to read password from kwallet. Using empty string instead
[cookies][info] Extracted 3027 cookies from brave
[cookies][debug] cookie version breakdown: {'v10': 0, 'v11': 2970, 'other': 0, 'unencrypted': 57}
[instagram][debug] Using InstagramUserExtractor for 'https://www.instagram.com/instagram/'
[cookies][debug] Extracting cookies from /home/user/.config/BraveSoftware/Brave-Browser/Default/Cookies
[cookies][debug] Detected desktop environment: kde
[cookies][debug] Chosen keyring: kwallet
[cookies][debug] using kwallet-query to obtain password from kwallet
[cookies][debug] NetworkWallet = 'kdewallet'
[cookies][debug] Failed to read password from kwallet. Using empty string instead
[cookies][info] Extracted 3027 cookies from brave
[cookies][debug] cookie version breakdown: {'v10': 0, 'v11': 2970, 'other': 0, 'unencrypted': 57}
[instagram][debug] Using InstagramPostsExtractor for 'https://www.instagram.com/instagram/posts/'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.instagram.com:443
[urllib3.connectionpool][debug] https://www.instagram.com:443 "GET /instagram/ HTTP/1.1" 200 None
[instagram][error] An unexpected error occurred: TypeError - the JSON object must be str, bytes or bytearray, not NoneType. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[instagram][debug] 
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/gallery_dl/job.py", line 77, in run
    for msg in extractor:
  File "/home/user/.local/lib/python3.10/site-packages/gallery_dl/extractor/instagram.py", line 49, in items
    for post in self.posts():
  File "/home/user/.local/lib/python3.10/site-packages/gallery_dl/extractor/instagram.py", line 475, in posts
    user = self._extract_profile_page(url)
  File "/home/user/.local/lib/python3.10/site-packages/gallery_dl/extractor/instagram.py", line 388, in _extract_profile_page
    data = self._extract_shared_data(page)["entry_data"]
  File "/home/user/.local/lib/python3.10/site-packages/gallery_dl/extractor/instagram.py", line 380, in _extract_shared_data
    data = json.loads(shared_data)
  File "/usr/lib/python3.10/json/__init__.py", line 339, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not NoneType

mikf · 2022-06-03T17:47:39Z

[gallery-dl][debug] Version 1.22.0

The latest commit would have [gallery-dl][debug] Version 1.22.1-dev as version string.
1.22.0 is the latest version from the Releases page, but that doesn't include 2fb0193 etc yet.

edit: git master also doesn't recognize --cookies-from_browser anymore. The option name has been fixed to --cookies-from-browser there.

huskyhi · 2022-06-03T18:51:58Z

[gallery-dl][debug] Version 1.22.0

The latest commit would have [gallery-dl][debug] Version 1.22.1-dev as version string. 1.22.0 is the latest version from the Releases page, but that doesn't include 2fb0193 etc yet.

edit: git master also doesn't recognize --cookies-from_browser anymore. The option name has been fixed to --cookies-from-browser there.

Is there a way to install this version if your using a version of gallery-dl installed by pip?

mikf · 2022-06-03T18:59:23Z

From https://github.com/mikf/gallery-dl#pip:

python3 -m pip install -U -I --no-deps --no-cache-dir https://github.com/mikf/gallery-dl/archive/master.tar.gz

I'll also publish a new release this weekend.

FriendlyUser10 · 2022-06-04T04:12:02Z

From https://github.com/mikf/gallery-dl#pip:

python3 -m pip install -U -I --no-deps --no-cache-dir https://github.com/mikf/gallery-dl/archive/master.tar.gz

I'll also publish a new release this weekend.

Probably a dumb question, but there is no way to do this for those of us who use the executable, right? Would it be recommended to use the pip versions instead? Bit of a noob with all this stuff, so please excuse the ignorance.

EDIT: Sorry for the inconvenience, I figured it out; the filenames on fix page confused me as they're slightly different than the "normal" exe build download. Appreciate the help you guys do.

romainlavisse · 2022-06-06T10:05:59Z

works fine, thanks a lot man !

ethanforvest · 2022-06-07T11:47:34Z

Now it's working again after the new release (v1.22.1). Thank you!

mikf added a commit that referenced this issue Jun 1, 2022

[instagram] fix and update extractors (#2644)

2fb0193

- use different way to fetch user IDs - use new API endpoints for /tagged/ and single posts

mikf added a commit that referenced this issue Jun 3, 2022

[instagram] fix stories (#2644)

d0dc29f

mikf added a commit that referenced this issue Jun 3, 2022

[instagram] fix stories (#2644)

46d171c

fixing the fix ...

mikf added the site:change label Jun 3, 2022

mikf closed this as completed Jun 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[instagram] An unexpected error occurred: TypeError - the JSON object must be str, bytes or bytearray, not NoneType. #2644

[instagram] An unexpected error occurred: TypeError - the JSON object must be str, bytes or bytearray, not NoneType. #2644

viceroycowboy commented May 31, 2022 •

edited

Loading

rorgoroth commented May 31, 2022

romainlavisse commented May 31, 2022

mikf commented Jun 1, 2022 •

edited

Loading

rorgoroth commented Jun 1, 2022

mikf commented Jun 1, 2022

rorgoroth commented Jun 1, 2022

romainlavisse commented Jun 1, 2022

rorgoroth commented Jun 1, 2022

mikf commented Jun 1, 2022

rorgoroth commented Jun 1, 2022 •

edited

Loading

romainlavisse commented Jun 1, 2022

ethanforvest commented Jun 3, 2022

mikf commented Jun 3, 2022

rorgoroth commented Jun 3, 2022

mikf commented Jun 3, 2022

rorgoroth commented Jun 3, 2022

ShadwDrgn commented Jun 3, 2022 •

edited

Loading

mikf commented Jun 3, 2022 •

edited

Loading

huskyhi commented Jun 3, 2022

mikf commented Jun 3, 2022

FriendlyUser10 commented Jun 4, 2022 •

edited

Loading

romainlavisse commented Jun 6, 2022

ethanforvest commented Jun 7, 2022

[instagram] An unexpected error occurred: TypeError - the JSON object must be str, bytes or bytearray, not NoneType. #2644

[instagram] An unexpected error occurred: TypeError - the JSON object must be str, bytes or bytearray, not NoneType. #2644

Comments

viceroycowboy commented May 31, 2022 • edited Loading

rorgoroth commented May 31, 2022

romainlavisse commented May 31, 2022

mikf commented Jun 1, 2022 • edited Loading

rorgoroth commented Jun 1, 2022

mikf commented Jun 1, 2022

rorgoroth commented Jun 1, 2022

romainlavisse commented Jun 1, 2022

rorgoroth commented Jun 1, 2022

mikf commented Jun 1, 2022

rorgoroth commented Jun 1, 2022 • edited Loading

romainlavisse commented Jun 1, 2022

ethanforvest commented Jun 3, 2022

mikf commented Jun 3, 2022

rorgoroth commented Jun 3, 2022

mikf commented Jun 3, 2022

rorgoroth commented Jun 3, 2022

ShadwDrgn commented Jun 3, 2022 • edited Loading

mikf commented Jun 3, 2022 • edited Loading

huskyhi commented Jun 3, 2022

mikf commented Jun 3, 2022

FriendlyUser10 commented Jun 4, 2022 • edited Loading

romainlavisse commented Jun 6, 2022

ethanforvest commented Jun 7, 2022

viceroycowboy commented May 31, 2022 •

edited

Loading

mikf commented Jun 1, 2022 •

edited

Loading

rorgoroth commented Jun 1, 2022 •

edited

Loading

ShadwDrgn commented Jun 3, 2022 •

edited

Loading

mikf commented Jun 3, 2022 •

edited

Loading

FriendlyUser10 commented Jun 4, 2022 •

edited

Loading