Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error while downloading story from Fiction.live (#2) #539

Closed
dakswiggin opened this issue Sep 5, 2020 · 7 comments
Closed

Error while downloading story from Fiction.live (#2) #539

dakswiggin opened this issue Sep 5, 2020 · 7 comments

Comments

@dakswiggin
Copy link

Let me add one more report for fiction.live here.
For story https://fiction.live/stories/Quest-The-Tribe/KTjKJ4yhPHT4LFrWk download end with error 'TypeError: cannot concatenate 'str' and 'NoneType' objects'
I've added some trace to allow get tags etc, info below:

Job: 0 Download 1 FanFiction Book(s) finished
Download 1 FanFiction Book(s)
FFF: INFO: 2020-09-05 22:53:28,010: calibre_plugins.fanficfare_plugin.jobs(53): FanFicFare v3.22.8
FFF: INFO: 2020-09-05 22:53:28,010: calibre_plugins.fanficfare_plugin.jobs(57): Adding jobs for URLs:
FFF: INFO: 2020-09-05 22:53:28,010: calibre_plugins.fanficfare_plugin.jobs(59): https://fiction.live/stories/Quest-The-Tribe/KTjKJ4yhPHT4LFrWk
FFF: INFO: 2020-09-05 22:53:43,348: calibre_plugins.fanficfare_plugin.jobs(95): Logfile for book ID None (Quest - The Tribe)
FFF: INFO: 2020-09-05 22:53:43,348: calibre_plugins.fanficfare_plugin.jobs(96): url:(https://fiction.live/stories/Quest-The-Tribe/KTjKJ4yhPHT4LFrWk) id:(None)
FFF: DEBUG: 2020-09-05 22:53:29,095: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://fiction.live/api/node/KTjKJ4yhPHT4LFrWk/
FFF: DEBUG: 2020-09-05 22:53:29,664: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1007): Encoding:utf8
FFF: DEBUG: 2020-09-05 22:53:29,696: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/yJkAnziR2OY8IDP5D5pf
FFF: DEBUG: 2020-09-05 22:53:30,266: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'Quest - The Tribe']
FFF: DEBUG: 2020-09-05 22:53:30,270: calibre_plugins.fanficfare_plugin.fanficfare.story(503): ['fiction.live']
FFF: DEBUG: 2020-09-05 22:53:30,272: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'fiction.live']
FFF: DEBUG: 2020-09-05 22:53:30,272: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'fiction.live']
FFF: DEBUG: 2020-09-05 22:53:30,273: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'https://fiction.live/stories/Quest-The-Tribe/KTjKJ4yhPHT4LFrWk']
FFF: DEBUG: 2020-09-05 22:53:30,276: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'https://fiction.live/user/lostmind/']
FFF: DEBUG: 2020-09-05 22:53:30,276: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'lostmind']
FFF: DEBUG: 2020-09-05 22:53:30,276: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u"lostmind"]
FFF: DEBUG: 2020-09-05 22:53:30,292: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'hiatus']
FFF: DEBUG: 2020-09-05 22:53:30,296: calibre_plugins.fanficfare_plugin.fanficfare.story(503): ['2015-09-26']
FFF: DEBUG: 2020-09-05 22:53:30,298: calibre_plugins.fanficfare_plugin.fanficfare.story(503): ['2020-08-23']
FFF: DEBUG: 2020-09-05 22:53:30,299: calibre_plugins.fanficfare_plugin.fanficfare.story(503): ['2020-09-05 22:53:29']
FFF: DEBUG: 2020-09-05 22:53:30,302: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'nsfw']
FFF: DEBUG: 2020-09-05 22:53:30,303: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'45']
FFF: DEBUG: 2020-09-05 22:53:30,305: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'278,316']
FFF: DEBUG: 2020-09-05 22:53:30,309: calibre_plugins.fanficfare_plugin.fanficfare.story(503): ['fiction.live']
FFF: DEBUG: 2020-09-05 22:53:30,313: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'KTjKJ4yhPHT4LFrWk']
FFF: DEBUG: 2020-09-05 22:53:30,315: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u"Quest - The Tribe"]
FFF: DEBUG: 2020-09-05 22:53:30,319: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'https://fiction.live/stories/Quest-The-Tribe/KTjKJ4yhPHT4LFrWk']
FFF: DEBUG: 2020-09-05 22:53:30,321: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'Rip. Tear. Lead. Ascend.']
FFF: DEBUG: 2020-09-05 22:53:30,325: calibre_plugins.fanficfare_plugin.fanficfare.story(503): ['flive']
FFF: DEBUG: 2020-09-05 22:53:30,326: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'FanFicFare v3.22.8']
FFF: DEBUG: 2020-09-05 22:53:30,328: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u"lostmind"]
FFF: DEBUG: 2020-09-05 22:53:30,335: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'']
FFF: DEBUG: 2020-09-05 22:53:30,345: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [446]
FFF: DEBUG: 2020-09-05 22:53:30,348: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'Light']
FFF: DEBUG: 2020-09-05 22:53:30,351: calibre_plugins.fanficfare_plugin.fanficfare.story(503): [u'Rip. Tear. Lead. Ascend.']
FFF: INFO: 2020-09-05 22:53:30,354: calibre_plugins.fanficfare_plugin.jobs(246): write to C:\Users\Daks\AppData\Local\Temp\calibre_3v2gej\fanficfare_hhqmvu\Quest - The Tribe-lostmind-b391cn.epub
FFF: INFO: 2020-09-05 22:53:30,355: calibre_plugins.fanficfare_plugin.fanficfare.writers.base_writer(182): Save directly to file: C:\Users\Daks\AppData\Local\Temp\calibre_3v2gej\fanficfare_hhqmvu\Quest - The Tribe-lostmind-b391cn.epub
FFF: DEBUG: 2020-09-05 22:53:30,357: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://fiction.live/api/anonkun/chapters/KTjKJ4yhPHT4LFrWk/1443299632528/1443394005808/
FFF: DEBUG: 2020-09-05 22:53:30,663: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1007): Encoding:utf8
FFF: DEBUG: 2020-09-05 22:53:31,161: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://fiction.live/api/anonkun/chapters/KTjKJ4yhPHT4LFrWk/1443394005809/1443475415384/
FFF: DEBUG: 2020-09-05 22:53:31,401: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1007): Encoding:utf8
FFF: DEBUG: 2020-09-05 22:53:31,426: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/3sOo3GRxaBXDnWUmH9YQ
FFF: DEBUG: 2020-09-05 22:53:32,446: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://fiction.live/api/anonkun/chapters/KTjKJ4yhPHT4LFrWk/1443475415385/1443556811645/
FFF: DEBUG: 2020-09-05 22:53:32,654: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1007): Encoding:utf8
FFF: DEBUG: 2020-09-05 22:53:32,875: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/4aoY2sxDSiDmex5qjRzM
FFF: DEBUG: 2020-09-05 22:53:33,604: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://fiction.live/api/anonkun/chapters/KTjKJ4yhPHT4LFrWk/1443556811646/1443646144887/
FFF: DEBUG: 2020-09-05 22:53:33,887: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1007): Encoding:utf8
FFF: DEBUG: 2020-09-05 22:53:33,947: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/66TQVAERG6dmDaLT0QSl
FFF: DEBUG: 2020-09-05 22:53:34,395: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/Fln5d4wbRuxh5zlaCzan
FFF: DEBUG: 2020-09-05 22:53:34,848: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/VJfWqnVTSiKxUCTv2qRL
FFF: DEBUG: 2020-09-05 22:53:35,381: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/uxQ2UpbWQiK1TPsGLv7D
FFF: DEBUG: 2020-09-05 22:53:35,963: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/9nLt2x11QRqkEwZCoOiS
FFF: DEBUG: 2020-09-05 22:53:36,561: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://fiction.live/api/anonkun/chapters/KTjKJ4yhPHT4LFrWk/1443646144888/1443736882503/
FFF: DEBUG: 2020-09-05 22:53:37,321: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1007): Encoding:utf8
FFF: DEBUG: 2020-09-05 22:53:37,397: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/YLEFhOMzR0OOWAlb8Aep
FFF: DEBUG: 2020-09-05 22:53:37,789: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/svFff9jcQ06BaFTk6IlA
FFF: DEBUG: 2020-09-05 22:53:38,344: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/Ik6m605hSmfQohEswnL2
FFF: DEBUG: 2020-09-05 22:53:40,085: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/KgaSU6baQW2yDLLzfGW8
FFF: DEBUG: 2020-09-05 22:53:40,688: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://fiction.live/api/anonkun/chapters/KTjKJ4yhPHT4LFrWk/1443736882504/1443891597678/
FFF: DEBUG: 2020-09-05 22:53:40,964: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1007): Encoding:utf8
FFF: DEBUG: 2020-09-05 22:53:41,150: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/jTWCIX7Toip1tTKXrStV
FFF: DEBUG: 2020-09-05 22:53:41,526: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/2cFuo5sIRQOWAeGDQdqg
FFF: DEBUG: 2020-09-05 22:53:41,848: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/pXrlBG1hTpRSxO23kdyW
FFF: DEBUG: 2020-09-05 22:53:42,276: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/fgbHFLETLCZcNDBcgwxA
FFF: DEBUG: 2020-09-05 22:53:42,714: calibre_plugins.fanficfare_plugin.fanficfare.configurable(1149): #####################################
pagecache(GET) MISS: https://www.filepicker.io/api/file/5kaJg2ZSzGsDCGPUMpnQ
FFF: INFO: 2020-09-05 22:53:43,180: calibre_plugins.fanficfare_plugin.jobs(350): Exception: {u'calibre_columns': {}, u'begin': None, u'site': 'fiction.live', u'added': False, u'good': False, u'collision': u'Update EPUB if New Chapters', u'end': None, u'status': u'Error', u'outfile': u'C:\Users\Daks\AppData\Local\Temp\calibre_3v2gej\fanficfare_hhqmvu\Quest - The Tribe-lostmind-b391cn.epub', u'calibre_id': None, u'tags': [u'hiatus'], u'url': u'https://fiction.live/stories/Quest-The-Tribe/KTjKJ4yhPHT4LFrWk', u'publisher': u'fiction.live', u'updatedate': datetime.datetime(2020, 8, 23, 8, 45, 17, 65000, tzinfo=SafeLocalTimeZone()), u'icon': u'dialog_error.png', u'password': '', u'timestamp': datetime.datetime(2020, 9, 5, 22, 53, 29, 88000, tzinfo=SafeLocalTimeZone()), u'title': u'Quest - The Tribe', u'all_metadata': {'numChapters': u'45', 'status': u'hiatus', 'seriesUrl': '', 'site': u'fiction.live', 'description': u'Rip. Tear. Lead. Ascend.', 'seriesHTML': '', 'output_css': '', 'cover_image': '', 'storyId': u'KTjKJ4yhPHT4LFrWk', 'formatext': '', 'language': '', 'langcode': '', 'dateUpdated': u'2020-08-23', 'words_added': '', u'tags': u'advancement, civilization, fantasy, plot, characterdevelopment, magic, civquest, realfiction, fictionalreality, factsandfiction, certifiedhistoryprofessors, advancedsociology, harem, stoneage, tg, history', 'lastupdate': u'Last Update Year/Month: 2020/08, Last Update: 2020/08/23', 'formatname': '', 'publisher': u'fiction.live', 'genre': u'', u'reader_input': u'Light', 'rating': u'nsfw', 'dateCreated': u'2020-09-05 22:53:26', 'authorId': u'7N2vJL5mc5gNaqi46', 'title': u'Quest - The Tribe', u'likes': u'446', u'live': '', 'ships': u'', u'key_tags': u'tribe, maleprotagonist, survival, smut, politics', 'series': '', 'storyUrl': u'https://fiction.live/stories/Quest-The-Tribe/KTjKJ4yhPHT4LFrWk', 'authorUrl': u'https://fiction.live/user/lostmind/', 'titleHTML': u"Quest - The Tribe", 'category': u'', 'sectionUrl': u'https://fiction.live/stories/Quest-The-Tribe/KTjKJ4yhPHT4LFrWk', 'version': u'FanFicFare v3.22.8', 'numWords': u'278,316', 'warnings': u'', 'datePublished': u'2015-09-26', 'author': u'lostmind', 'extratags': u'', 'siteabbrev': u'flive', 'authorHTML': u"lostmind", 'characters': u''}, u'is_adult': False, u'comment': u"cannot concatenate 'str' and 'NoneType' objects", u'uniqueurl': u'https://fiction.live/stories/Quest-The-Tribe/KTjKJ4yhPHT4LFrWk[None-None]', u'showerror': True, u'series': '', u'pubdate': datetime.datetime(2015, 9, 26, 23, 12, 4, 24000, tzinfo=SafeLocalTimeZone()), u'listorder': 0, u'author_sort': [u'lostmind'], u'comments': u'Rip. Tear. Lead. Ascend.', u'author': [u'lostmind'], u'username': 'NoneGiven'}:cannot concatenate 'str' and 'NoneType' objects
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.jobs", line 248, in do_download_for_worker
File "calibre_plugins.fanficfare_plugin.fanficfare.writers.base_writer", line 201, in writeStory
File "calibre_plugins.fanficfare_plugin.fanficfare.adapters.base_adapter", line 237, in getStory
File "calibre_plugins.fanficfare_plugin.fanficfare.adapters.base_adapter", line 389, in getChapterTextNum
File "calibre_plugins.fanficfare_plugin.fanficfare.adapters.adapter_fictionlive", line 243, in getChapterText
File "calibre_plugins.fanficfare_plugin.fanficfare.adapters.adapter_fictionlive", line 417, in format_readerposts
TypeError: cannot concatenate 'str' and 'NoneType' objects

FFF: INFO: 2020-09-05 22:53:43,348: calibre_plugins.fanficfare_plugin.jobs(100): 
Download Results:
Error https://fiction.live/stories/Quest-The-Tribe/KTjKJ4yhPHT4LFrWk cannot concatenate 'str' and 'NoneType' objects

FFF: INFO: 2020-09-05 22:53:43,348: calibre_plugins.fanficfare_plugin.jobs(124): 
Error
`https://fiction.live/stories/Quest-The-Tribe/KTjKJ4yhPHT4LFrWk`
@cryzed
Copy link
Contributor

cryzed commented Sep 5, 2020

I can reproduce this error, roll is obviously None here.

@JimmXinu
Copy link
Owner

JimmXinu commented Sep 5, 2020

Not sure why there's two issues(#538) open for this adapter, but tagging @HazelSh (who wrote all the code for supporting this site), to hopefully trigger a notification.

HazelSh pushed a commit to HazelSh/FanFicFare that referenced this issue Sep 6, 2020
null handling in format_readerposts -- closes JimmXinu#539
8844a8f
@HazelSh
Copy link

HazelSh commented Sep 6, 2020

Fixed locally and have that story working, will commit and push soon. Another case of not being cautious enough with the API -- turns out that in a list of user dice rolls, just because there's an entry with a uid doesn't mean that the value is present, or valid text.

@JimmXinu
Copy link
Owner

JimmXinu commented Sep 6, 2020

Should I try to cherry pick 8844a8f or will you be making a new PR when you're done?

@HazelSh
Copy link

HazelSh commented Sep 6, 2020

Please cherry pick it -- I looked into making a pull request, but github wanted to include the commits where the adapter was introduced as well, I think because that pull request wasn't github-merged even though you've included the patches and have the code. I need to look into getting it going on my end, I suspect I should delete that branch and rebase off your master version. I'm still pretty new to github, sorry.

@JimmXinu
Copy link
Owner

JimmXinu commented Sep 7, 2020

Yeah, I made some whitespace clean up changes after your last changes. It would be easier in future if you can submit changes from a new branch that is based off the original.

For example, I have a fork of calibre and use this script to force my fork back into sync before I start working on anything in it:

git fetch upstream
git checkout master
git reset --hard upstream/master
git push origin master --force
git gc

@HazelSh
Copy link

HazelSh commented Sep 7, 2020

yep, I'm getting my version sync'd. sorry about that.

narethdeer pushed a commit to narethdeer/FanFicFare that referenced this issue Sep 26, 2020
@narethdeer
null handling in format_readerposts -- closes JimmXinu#539
1ea4bff
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@cryzed @JimmXinu @dakswiggin @HazelSh