Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connection Closed rendering Facebook page #2

Open
rmassidda opened this issue Nov 13, 2018 · 2 comments
Open

Connection Closed rendering Facebook page #2

rmassidda opened this issue Nov 13, 2018 · 2 comments

Comments

@rmassidda
Copy link

An exception is raised inside generate_soup method.
This seems to happen only with Facebook pages, I isolated the get/render part of the code and it runs smoothly with other websites.
Could this be related to the parameters of Retry?

Raised exception:
File "~/Projects/livorno-wave-bot/test.py", line 20, in <module>
main()
File "~/Projects/livorno-wave-bot/test.py", line 15, in main
r.html.render(timeout=100) #here the Javascript is rendered in a mock browser
File "~/.local/lib/python3.7/site-packages/requests_html.py", line 583, in render
content, result, page = self.session.loop.run_until_complete(_async_render(url=self.url, script=script, sleep=sleep, wait=wait, content=self.html, reload=reload, scrolldown=scrolldown, timeout=timeout, keep_page=keep_page))
File "/usr/lib64/python3.7/asyncio/base_events.py", line 573, in run_until_complete
return future.result()
File "~/.local/lib/python3.7/site-packages/requests_html.py", line 538, in _async_render
page = await self.session.browser.newPage()
File "~/.local/lib/python3.7/site-packages/pyppeteer/browser.py", line 192, in newPage
return await self._defaultContext.newPage()
File "~/.local/lib/python3.7/site-packages/pyppeteer/browser.py", line 329, in newPage
return await self._browser._createPageInContext(self._id)
File "~/.local/lib/python3.7/site-packages/pyppeteer/browser.py", line 200, in _createPageInContext
'Target.createTarget', options)).get('targetId')
File "~/.local/lib/python3.7/site-packages/pyppeteer/connection.py", line 85, in send
raise ConnectionError('Connection is closed')
ConnectionError: Connection is closed

@Udinanon
Copy link
Owner

Udinanon commented Dec 6, 2018

Mhhh, I'm not sure, it looks like the connection isn't working, it might be a pyppeteer issue or it might be related to the Retry settings or something else in requests-html. I can't recreate the issue so I can't elaborate it forward, i suggest you ask directly at their githubs.
If you find anything or can give me more details on how this issue arises, I'll leave the issue open

@Udinanon
Copy link
Owner

Udinanon commented Jan 3, 2019

So i started getting the same error a few days ago and I've been working on either a fix or a workaround.
It appears on the Pyppeteer issue page [1] so it seems not to be my fault but I can't actually find a working fix.
I'm actually thinking about changing the rendering engine for JS, but right now I don't have much free time, maybe I'll be working on it in the spring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants