Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pixiv] Fix tag write when tags setting is set to original #3675

Merged
merged 4 commits into from
Mar 2, 2023

Conversation

gmanley
Copy link
Contributor

@gmanley gmanley commented Feb 19, 2023

See this issue: #3674

The following command will fail on current master:
gallery-dl --ignore-config --write-tags --option "extractor.pixiv.tags=original" --verbose https://www.pixiv.net/en/artworks/105480642

The traceback:

Traceback (most recent call last):
  File "/Users/gray/source/gallery-dl/gallery_dl/job.py", line 102, in run
    self.dispatch(msg)
  File "/Users/gray/source/gallery-dl/gallery_dl/job.py", line 146, in dispatch
    self.handle_url(url, kwdict)
  File "/Users/gray/source/gallery-dl/gallery_dl/job.py", line 297, in handle_url
    callback(pathfmt)
  File "/Users/gray/source/gallery-dl/gallery_dl/postprocessor/metadata.py", line 101, in run
    self.write(fp, pathfmt.kwdict)
  File "/Users/gray/source/gallery-dl/gallery_dl/postprocessor/metadata.py", line 188, in _write_tags
    fp.write("\n".join(tags) + "\n")
TypeError: sequence item 0: expected str instance, dict found

The issue:

> /Users/gray/source/gallery-dl/gallery_dl/postprocessor/metadata.py(198)_write_tags()
    197
--> 198         fp.write("\n".join(tags) + "\n")
    199

[asyncio][debug] Using selector: KqueueSelector
ipdb> tags
[{'name': 'バーチャルYouTuber', 'translated_name': 'Virtual Youtuber'}, {'name': 'VTuber', 'translated_name': 'virtual YouTuber'}, {'name': 'ホロライブ', 'translated_name': 'Hololive'}, {'name': 'hololive', 'translated_name': None}, {'name': 'holoX', 'translated_name': None}, {'name': '沙花叉クロヱ', 'translated_name': 'Sakamata Chloe'}, {'name': 'バーチャルYouTuber100users入り', 'translated_name': 'Virtual Youtuber 100+ bookmarks'}]

When the tags setting for Pixiv is set to original the tags are returned as a list of dict objects. The code that is bombing out expects tags to be list of str.

There is already branching logic to handle the different formats that tags come back as from the extractors so I've extended that logic to handle when tags come back as list of dict objects. As you can see sometimes the values are None so there is handling to remove those from the returned list as well.

This will make the list of tags written just a flat list of strings, loosing the mapping between Japanese tag and the translated version, but that is just how the file is. Ideally I think it would be great to have this change be in the Pixiv extractor but if I flatten out the mapping there, then we'll loose the mapping in that variable that is useful in other parts, such as the json metadata write.

@gmanley gmanley changed the title [pixiv] Fix tag write when set tags setting is set to original [pixiv] Fix tag write when tags setting is set to original Feb 19, 2023
@mikf mikf merged commit 7610d9c into mikf:master Mar 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants