Docker selenium issue?

[causefx]

Description

logs:

causefx@docker:/opt/docker$ docker logs southwest -f
Successfully logged in to *******s account

Successfully scheduled the following flights to check in for ******:
Flight from ***** to ****** at 2023-05-12 17:30:00 UTC
Flight from *****  to ******* at 2023-05-15 17:10:00 UTC

Successfully scheduled the following flights to check in for ******:
Flight from ****** to ******** at 2023-06-02 00:15:00 UTC
Flight from ******* to ******** at 2023-06-05 01:00:00 UTC

Process Process-1:1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/local/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/app/lib/checkin_handler.py", line 44, in _set_check_in
    self._wait_for_check_in(checkin_time)
  File "/app/lib/checkin_handler.py", line 60, in _wait_for_check_in
    self.checkin_scheduler.refresh_headers()
  File "/app/lib/checkin_scheduler.py", line 48, in refresh_headers
    webdriver.set_headers()
  File "/app/lib/webdriver.py", line 59, in set_headers
    driver = self._get_driver()
             ^^^^^^^^^^^^^^^^^^
  File "/app/lib/webdriver.py", line 142, in _get_driver
    driver = Chrome(
             ^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/seleniumwire/undetected_chromedriver/webdriver.py", line 61, in __init__
    super().__init__(*args, **kwargs)
  File "/usr/local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 429, in __init__
    super(Chrome, self).__init__(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 84, in __init__
    super().__init__(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 104, in __init__
    super().__init__(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 286, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 715, in start_session
    super(selenium.webdriver.chrome.webdriver.WebDriver, self).start_session(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 378, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 440, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: timeout: Timed out receiving message from renderer: 600.000
  (Session info: chrome=112.0.5615.165)

To Reproduce

Pull container and setup the config file

Expected Behavior

To check in for flight

Version

5.0

Additional context

Docker-Compose:

    southwest:
        container_name: southwest
        image: jdholtz/auto-southwest-check-in
        volumes:
          - ${DATADIR}/southwest/config/config.json:/app/config.json
        environment:
            - TZ=${TZ}
            - PUID=${PUID}
            - PGID=${PGID}
          

Config:

{
    "check_fares": true,
    "notification_urls": "mailtos://smtp.zoho.com/?from=smtp@*********&subject=Southwest&to=***********&user=************&pass=*********,
    "notification_level": 1,
    "retrieval_interval": 24,
    "accounts": [{"username": "***********", "password": "*********"}],
    "flights": []
}

[jdholtz]

Does this issue happen every time or just every so often?

[causefx]

not sure yet... i set it up yesterday and it tried checking in today... i manually checked in when it failed. when i added the verbose flag to the command key for compose and started it back up it picked up that i was checked in. I will leave it running with verbose and see if it checks in for my return flight. cheers!

[jdholtz]

It seems to be an issue that occurs every so often with Selenium. I might try switching to Playwright soon and see if that fixes a lot of the driver-related issues

[causefx]

makes sense... if you make the switch you will allow playwright driver url to be configured for docker?

[jdholtz]

Do you mean the “chromedriver_path” configuration option? If so, I will if it needs to be. I’m not too sure how Playwright works, so that might not even be needed

[causefx]

I run playwright in its own container at the moment and another project that can use it allows me to pass the websocket url to it via ENV so it can use that instance of Playwright.

- PLAYWRIGHT_DRIVER_URL=ws://docker.home.lab:5903/?stealth=1&--disable-web-security=true

[jdholtz]

Oh, gotcha. Yes, I will add compatibility for that if I end up going with Playwright.

[jdholtz]

I added more resistance to random errors like this while initializing the webdriver. Let me know if you run into any more issues related to this.

[causefx]

it seems fixed so far :) thank you!

[jdholtz]

Hey @causefx, I've been messing around with Playwright for a few hours and I am not able to bypass Southwest's bot protection. Unfortunately, I didn't find a Python library similar to undetected_chromedriver for Playwright either. Let me know if you have any suggestions or know of other projects that use Playwright to navigate around bot detection.

[causefx]

hey there @jdholtz, I don't know of a way to circumvent the bot detection with Playwright.... I guess we are just going to need to use Selenium. Thank you for trying to get it to work.