fiction.live: KeyError: 'contentRating'

[JimmXinu]

@HazelSh,

Reported here, there's another metadata issue with fiction.live support on some stories. Do you have time to look into it? Thanks.

  File "...\fanficfare\adapters\adapter_fictionlive.py", line 109, in extract_metadata
    self.story.setMetadata('rating', data['contentRating'])
KeyError: 'contentRating'

Examples:
https://fiction.live/stories/Golden-Empire-Chapter-1-Authority/c3ea820a-a8e6-4917-8067-f04d7eeca1ab
https://fiction.live/stories/Golden-Empire-Chapter-2-Purpose/a9b355a5-802d-4ac4-9afc-a89666375ac6
https://fiction.live/stories/Golden-Empire-Chapter-3-Ruin/5426e7f3-64d7-4eb5-8736-2299c7feddf8

[HazelSh]

Ok, digging into it. I know the site used to use a different system for ratings ('tAge'), but even very old stories seemed to have the new contentRating. Seems that this story doesn't have either? I really did not think it was optional.

Looking at the comments in the HTML, I think the site-generator defaults to Teen when there's no contentRating? So we probably should too. Patch soon.

At this point, I might just go ahead and add code to handle the inevitable story which doesn't have an author, or a publication date. It's actually too much to think that there might be a story without a storyID, right???

[HazelSh]

I think that should fix it. Can confirm that I have those stories working locally.

[dakswiggin]

Sorry, seems original reporter pointed out also one more issue:
"another(chapter 8 of the series, so easy to find by going to author, is mature so not sure I should post here) says 'unhashable type: 'dict'."
Afaik happens due to typo in

FanFicFare/fanficfare/adapters/adapter_fictionlive.py

Line 210 in ade0458

routes.append({{"id": r['_id'], "title": title}})

[muchtea]

Yep, that's a typo, that should be a simple rather than a double dict. Happened because I made a quick change and didn't test it before pushing. Thanks @dakswiggin for finding the issue.

[JimmXinu]

This is still happening?

FFF: DEBUG: 2020-09-17 19:34:42,016: calibre_plugins.fanficfare_plugin.fff_plugin(1093): url:https://fiction.live/stories/Golden-Empire-Chapter-3-Ruin/5426e7f3-64d7-4eb5-8736-2299c7feddf8
FFF: DEBUG: 2020-09-17 19:34:42,022: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1152): #####################################
pagecache(GET) MISS: https://fiction.live/api/node/5426e7f3-64d7-4eb5-8736-2299c7feddf8/
FFF: DEBUG: 2020-09-17 19:34:42,154: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1010): Encoding:utf8
FFF: ERROR: 2020-09-17 19:34:42,154: calibre_plugins.fanficfare_plugin.dialogs(714): Exception: {'title': 'Unknown', 'author_sort': ['Unknown'], 'author': ['Unknown'], 'comments': '', 'good': False, 'status': 'Error', 'showerror': True, 'calibre_id': None, 'begin': None, 'end': None, 'comment': "'contentRating'", 'url': 'https://fiction.live/stories/Golden-Empire-Chapter-3-Ruin/5426e7f3-64d7-4eb5-8736-2299c7feddf8', 'site': 'fiction.live', 'series': '', 'added': False, 'pubdate': None, 'publisher': None, 'uniqueurl': 'https://fiction.live/stories/Golden-Empire-Chapter-3-Ruin/5426e7f3-64d7-4eb5-8736-2299c7feddf8[None-None]', 'listorder': 2, 'collision': 'Update EPUB Always'}:'contentRating'
Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.dialogs", line 701, in do_loop
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1178, in prep_download_loop
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1060, in get_story_metadata_only
  File "calibre_plugins.fanficfare_plugin.fanficfare.adapters.base_adapter", line 297, in getStoryMetadataOnly
  File "calibre_plugins.fanficfare_plugin.fanficfare.adapters.adapter_fictionlive", line 99, in doExtractChapterUrlsAndMetadata
  File "calibre_plugins.fanficfare_plugin.fanficfare.adapters.adapter_fictionlive", line 109, in extract_metadata
KeyError: 'contentRating'

[HazelSh]

Can't reproduce, sorry. For me, that story downloads without error.

Are you sure that error is from the most recent test version? Line 109 is exactly where the issue would have been before the patch.

[dakswiggin]

@JimmXinu most recent test version published on mobileread.com doesn't include changes in adapter related to contentRating (compared git source with https://www.mobileread.com/forums/attachment.php?attachmentid=181987&d=1600349943)
Probably this cause issues reported by user.

[JimmXinu]

@dakswiggin is correct -- this was my mistake in uploading the test version. Sorry about that.