MangaDex API 5.0

[kattjevfel]

As of yesterday MangaDex has opened their API again after the attacks. It's in an open beta and read-only, but most of the features that gallery-dl might be interested in is there.

Announcement: https://mangadex.org/index.html
API docs: https://api.mangadex.org/docs.html

[kattjevfel]

The web interface went live today, it's a bit slow and may fail to load at times due to the insane pressure but seems to work.

Old IDs like https://mangadex.org/title/34216 are automatically converted into the new format and redirects to like so: https://mangadex.org/title/008cda8e-2c41-4c89-8c97-62bd17af0fdf

[mikf]

A few changes to the old behavior:

• --chapter-range and --chapter-filter no longer work. --chapter-filter can be replaced with regular --filter, but there's no alternative for --chapter-range.
• artist, author, and group metadata entries now require additional HTTP requests and need to be enabled with the metadata option.
• there is now a lang option to filter by ISO 639-1 code server-side (e.g. -o lang=en)
• -A/--abort now works as expected (as does -T/--terminate)

and it does not auto-translate legacy IDs to new IDs (yet)

[kattjevfel]

It seems to work great, though I had to cancel it almost right away since my archive is of course using the legacy IDs. Is this something that will be addressed or should I just begin manually converting the IDs in my archive?

Also it seems like setting lang=en only works when in the config file, trying to set it in commandline just halts right away.

gallery-dl --ignore-config -o lang=en --verbose https://mangadex.org/title/008cda8e-2c41-4c89-8c97-62bd17af0fdf
[gallery-dl][debug] Version 1.18.0-dev
[gallery-dl][debug] Python 3.9.5 - Linux-5.12.9-zen1-1-zen-x86_64-with-glibc2.33
[gallery-dl][debug] requests 2.25.1 - urllib3 1.26.5
[gallery-dl][debug] Starting DownloadJob for 'https://mangadex.org/title/008cda8e-2c41-4c89-8c97-62bd17af0fdf'
[mangadex][debug] Using MangadexMangaExtractor for 'https://mangadex.org/title/008cda8e-2c41-4c89-8c97-62bd17af0fdf'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): api.mangadex.org:443
[urllib3.connectionpool][debug] https://api.mangadex.org:443 "GET /manga/008cda8e-2c41-4c89-8c97-62bd17af0fdf/feed?order%5Bvolume%5D=asc&order%5Bchapter%5D=asc&translatedLanguage%5B%5D=en&offset=0 HTTP/1.1" 200 None

EDIT: It seems now nothing works, or only select titles... Quite unstable :D

[mikf]

Is this something that will be addressed or should I just begin manually converting the IDs in my archive?

It doesn't appear to be is possible to convert new to old, only the other way round. There is /legacy/mapping, but it only accepts legacy integer IDs. So all gallery-dl archive entries are invalid and there's nothing that can be done except manually converting them.

Also it seems like setting lang=en only works when in the config file, trying to set it in commandline just halts right away.

Your example manga doesn't have any chapters in English, hence nothing is returned. -o lang=ru works.

[kattjevfel]

Is this something that will be addressed or should I just begin manually converting the IDs in my archive?

It doesn't appear to be is possible to convert new to old, only the other way round. There is /legacy/mapping, but it only accepts legacy integer IDs. So all gallery-dl archive entries are invalid and there's nothing that can be done except manually converting them.

I was more thinking of automating converting all my old entires to the new format, since I have ~29k mangadex entries in my archive, it'll take a while to go through by hand. But I suppose I'll look into making a script to automate it :P

EDIT: https://github.com/kattjevfel/scripts/blob/master/convertmangadexIDs.sh crude but very much functional.

[mikf]

Added the ability to download chapters from an account's follow feed (1197ee2) and the for that required ability to login (07c8adb).
I've also "reverted" the way chapters are handled to the old system where --chapter-filter etc work as expected, but one needs to use -T/--terminate or "skip": "terminate" to stop downloading. (53dab5c)

edit: Thanks, Firefox.

[Korbeil]

Hello around,

I'm panda__________ dev for MangaDex API. I'm coming around to know if you could use a clean User-Agent when requesting the MD API ? We had some occurrence of a guy using your library to update the manga he's reading and since we saw a old Firefox UA, we tough it was malicious and banned him. He replied to the ban explaining that he was using this library.

It doesn't appear to be is possible to convert new to old, only the other way round. There is /legacy/mapping, but it only accepts legacy integer IDs. So all gallery-dl archive entries are invalid and there's nothing that can be done except manually converting them.

There is a zip archive with all the legacy mapping ids available on the MangaDex Discord on channel #dev-talk-api

Also if you need any help ask in that channel, I'll be glad to help you out :)
Regards,
panda__________

[mikf]

I'm coming around to know if you could use a clean User-Agent when requesting the MD API ?

Sure, would something like gallery-dl/1.18.0 be OK?

old Firefox UA

Version 78.0 is the latest ESR release. The next one will be 91.0 on August 10th.

[Korbeil]

I'm coming around to know if you could use a clean User-Agent when requesting the MD API ?

Sure, would something like gallery-dl/1.18.0 be OK?

Sounds perfect 👍

[mikf]

Should this User-Agent header also be used for file downloads, or is it enough to use it only when querying the API?

[Korbeil]

Should this User-Agent header also be used for file downloads, or is it enough to use it only when querying the API?

Both would be awesome :)

[mikf]

Done (dfe1f49)

[Korbeil]

Done (dfe1f49)

Thanks ! 🎉

[kattjevfel]

Is there anything still on the old API? Otherwise I'll close this issue.

[mikf]

Everything has been moved to the new API, so yes, this should be closed.
I've just left this open in case anyone had some feedback concerning these changes.