Session closed error when script is delayed more than 20 seconds

[zxwild]

When a script is delayed, the pyppeteer failed with the following error:
Session closed. Most likely the page has been closed.

Here is a simple example how to reproduce it:

1. Create browser
2. Select page (one is already created)
3. Request wikipedia main page
4. Try to get an element from the page, - all is ok.
5. Wait for 30 seconds
6. Try to get the same element from the page and you'll receive:
   Protocol Error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.

PS: On my machine, with a 19 seconds delay this script works ok, but with 20 and more - fails.

import asyncio
from pyppeteer import launch


async def main():
    browser = await launch({
        'headless': True,
        'args': [
            '--no-sandbox',
        ],
    })

    pages = await browser.pages()
    page = pages[0]
    await page.goto('https://wikipedia.org')

    valid_element = await page.waitForSelector('.central-featured')
    print(f'valid_element pass 1: {valid_element}')

    # Here we wait for 30 seconds, but actually 20 is enough on my machine
    # in other words with a 19 seconds delay this script should work, with 20 - shouldn't
    await asyncio.sleep(30)

    valid_element = await page.waitForSelector('.central-featured')
    print(f'valid_element pass 2: {valid_element}')

    await browser.close()

asyncio.get_event_loop().run_until_complete(main())

[zxwild]

Looks like it's related to websockets, browser._connection._ws, which is connected with default parameters:

ping_interval=20
ping_timeout=20

And somehow keepalive_ping_task doesn't work properly.

PS:
Just noticed #160 at least it helps to increase 20 seconds limit to 15 minutes at least in my case.
More comments are there.

Possible solutions:

• a fix from #159 [SEVERE] The communication with Chromium are disconnected after 20 seconds. #160 with pings removing
• dropdown websockets to "==6.0" (with some instability it seems)

[Lucifaer]

Good job!
I have found this problem too, and I rewrite my js code to python code like this:

async def scroll_page(page):
    cur_dist = 0
    height = await page.evaluate("() => document.body.scrollHeight")
    while True:
        if cur_dist < height:
            await page.evaluate("window.scrollBy(0, 500);")
            await asyncio.sleep(0.1)
            cur_dist += 500
        else:
            break

and the top solution seems works for me, but it doesn't solve the problem at all.
And your solution fundamentally solved the problem. Thanks! 👍

[stolati]

For those who want to hack before the patch arrives.

def patch_pyppeteer():
    import pyppeteer.connection
    original_method = pyppeteer.connection.websockets.client.connect

    def new_method(*args, **kwargs):
        kwargs['ping_interval'] = None
        kwargs['ping_timeout'] = None
        return original_method(*args, **kwargs)

    pyppeteer.connection.websockets.client.connect = new_method
patch_pyppeteer()

[uhbif19]

Do I understand correctly, that this issue is not solved, and monkeypatching is required now?

[zxwild]

@uhbif19 it looks like this project is abandoned by the core developer, so you can use any solution provided above, the easiest way is to dropdown websockets to the 6.0 version

[uhbif19]

@uhbif19 Get it, thanks.