Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[🐛 Bug]: Selenium 4.11.x not working with Chronium. #12466

Closed
ikus060 opened this issue Aug 1, 2023 · 25 comments
Closed

[🐛 Bug]: Selenium 4.11.x not working with Chronium. #12466

ikus060 opened this issue Aug 1, 2023 · 25 comments

Comments

@ikus060
Copy link

ikus060 commented Aug 1, 2023

What happened?

Latest version 4.11.x is not working with Chronium. While the same code is working fine with 4.10.0.

How can we reproduce the issue?

from selenium import webdriver

# Start selenium driver
options = webdriver.ChromeOptions()
driver = webdriver.Chrome(options=options)
driver.get('http://example.com/')


### Relevant log output

```shell
./debugpy/launcher 49971 -- /home/ikus060/workspace/PDSL/rdiffweb.git/snippet.py 
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/ikus060/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/ikus060/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/ikus060/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/home/ikus060/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/ikus060/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/ikus060/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/home/ikus060/workspace/PDSL/rdiffweb.git/snippet.py", line 5, in <module>
    driver = webdriver.Chrome(options=options)
  File "/home/ikus060/workspace/PDSL/rdiffweb.git/.venv/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
    super().__init__(
  File "/home/ikus060/workspace/PDSL/rdiffweb.git/.venv/lib/python3.10/site-packages/selenium/webdriver/chromium/webdriver.py", line 56, in __init__
    super().__init__(
  File "/home/ikus060/workspace/PDSL/rdiffweb.git/.venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 206, in __init__
    self.start_session(capabilities)
  File "/home/ikus060/workspace/PDSL/rdiffweb.git/.venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 290, in start_session
    response = self.execute(Command.NEW_SESSION, caps)["value"]
  File "/home/ikus060/workspace/PDSL/rdiffweb.git/.venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 345, in execute
    self.error_handler.check_response(response)
  File "/home/ikus060/workspace/PDSL/rdiffweb.git/.venv/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /home/ikus060/.cache/selenium/chrome/linux64/115.0.5790.102/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
#0 0x55afbf2ddb13 <unknown>
#1 0x55afbf01d277 <unknown>
#2 0x55afbf047b88 <unknown>
#3 0x55afbf0444de <unknown>
#4 0x55afbf083f32 <unknown>
#5 0x55afbf08368f <unknown>
#6 0x55afbf07b1d3 <unknown>
#7 0x55afbf04f632 <unknown>
#8 0x55afbf0503de <unknown>
#9 0x55afbf2a297d <unknown>
#10 0x55afbf2a7213 <unknown>
#11 0x55afbf2b0628 <unknown>
#12 0x55afbf2a7c4a <unknown>
#13 0x55afbf2792ae <unknown>
#14 0x55afbf2c86d8 <unknown>
#15 0x55afbf2c887f <unknown>
#16 0x55afbf2d7098 <unknown>
#17 0x7f8c71cfbb43 <unknown>

Operating System

Ubuntu Linux 64bits

Selenium version

=4.11.0

What are the browser(s) and version(s) where you see this issue?

Chromium 115.0.5790.102 snap

What are the browser driver(s) and version(s) where you see this issue?

Not sure

Are you using Selenium Grid?

No

@github-actions
Copy link

github-actions bot commented Aug 1, 2023

@ikus060, thank you for creating this issue. We will troubleshoot it as soon as we can.


Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

@ikus060
Copy link
Author

ikus060 commented Aug 1, 2023

Seams v4.10 uses my local installation of chronium, while v4.11 is using the version downloaded into ~/.cache/selenium/chrome/linux64/115.0.5790.102/.

Not sure why it doesn't use the local version.

@ikus060
Copy link
Author

ikus060 commented Aug 1, 2023

The problem seams to reside in selenium-manager.

With v4.11 it doesn't find my local installation of chronium. So it download it.

ikus060@pop-os:~$ '/home/ikus060/workspace/PDSL/rdiffweb.git/.venv/lib/python3.10/site-packages/selenium/webdriver/common/linux/selenium-manager' --browser chrome --debug
DEBUG	Checking chromedriver in PATH
DEBUG	Running command: chromedriver --version
DEBUG	Output: "ChromeDriver 115.0.5790.102 (90efd4b0ad6aa15eeafcdabd5817ae939f7ba059-refs/branch-heads/5790_90@{#9})"
DEBUG	Running command: which chromedriver
DEBUG	Output: "/usr/bin/chromedriver"
DEBUG	Found chromedriver 115.0.5790.102 in PATH: /usr/bin/chromedriver
DEBUG	Checking chrome in PATH
DEBUG	Running command: which chrome
DEBUG	Output: ""
DEBUG	chrome not found in PATH
DEBUG	chrome has not been discovered in the system
DEBUG	Required browser: chrome 115.0.5790.102
DEBUG	Reading metadata from https://googlechromelabs.github.io/chrome-for-testing/known-good-versions-with-downloads.json
DEBUG	Downloading chrome 115.0.5790.102 from https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/115.0.5790.102/linux64/chrome-linux64.zip
DEBUG	chrome 115.0.5790.102 has been downloaded at /home/ikus060/.cache/selenium/chrome/linux64/115.0.5790.102/chrome
DEBUG	Required driver: chromedriver 115.0.5790.102
INFO	Driver path: /usr/bin/chromedriver
INFO	Browser path: /home/ikus060/.cache/selenium/chrome/linux64/115.0.5790.102/chrome
```

Versus, v4.10
```
DEBUG	Using shell command to find out chrome version
DEBUG	Running command: "google-chrome --version"
DEBUG	Output: ""
WARN	The version of chrome cannot be detected. Trying with latest driver version
DEBUG	Reading chromedriver version from https://chromedriver.storage.googleapis.com/LATEST_RELEASE
DEBUG	Required driver: chromedriver 114.0.5735.90
DEBUG	Running command: "chromedriver --version"
DEBUG	Output: "ChromeDriver 115.0.5790.102 (90efd4b0ad6aa15eeafcdabd5817ae939f7ba059-refs/branch-heads/5790_90@{#9})"
DEBUG	Running command: "which chromedriver"
DEBUG	Output: "/usr/bin/chromedriver"
WARN	Incompatible release of chromedriver (version 115.0.5790.102) detected in PATH: /usr/bin/chromedriver
DEBUG	Driver URL: https://chromedriver.storage.googleapis.com/114.0.5735.90/chromedriver_linux64.zip
DEBUG	File extracted to /home/ikus060/.cache/selenium/chromedriver/linux64/114.0.5735.90/chromedriver (15039112 bytes)
INFO	/home/ikus060/.cache/selenium/chromedriver/linux64/114.0.5735.90/chromedriver
```

@abdhamza
Copy link

abdhamza commented Aug 1, 2023

What happened?

Latest version 4.11.x is not working with Chronium. While the same code is working fine with 4.10.0.

How can we reproduce the issue?

from selenium import webdriver

# Start selenium driver
options = webdriver.ChromeOptions()
driver = webdriver.Chrome(options=options)
driver.get('http://example.com/')


### Relevant log output

```shell
./debugpy/launcher 49971 -- /home/ikus060/workspace/PDSL/rdiffweb.git/snippet.py 
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/ikus060/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/ikus060/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/ikus060/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/home/ikus060/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/ikus060/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/ikus060/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/home/ikus060/workspace/PDSL/rdiffweb.git/snippet.py", line 5, in <module>
    driver = webdriver.Chrome(options=options)
  File "/home/ikus060/workspace/PDSL/rdiffweb.git/.venv/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
    super().__init__(
  File "/home/ikus060/workspace/PDSL/rdiffweb.git/.venv/lib/python3.10/site-packages/selenium/webdriver/chromium/webdriver.py", line 56, in __init__
    super().__init__(
  File "/home/ikus060/workspace/PDSL/rdiffweb.git/.venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 206, in __init__
    self.start_session(capabilities)
  File "/home/ikus060/workspace/PDSL/rdiffweb.git/.venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 290, in start_session
    response = self.execute(Command.NEW_SESSION, caps)["value"]
  File "/home/ikus060/workspace/PDSL/rdiffweb.git/.venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 345, in execute
    self.error_handler.check_response(response)
  File "/home/ikus060/workspace/PDSL/rdiffweb.git/.venv/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /home/ikus060/.cache/selenium/chrome/linux64/115.0.5790.102/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
#0 0x55afbf2ddb13 <unknown>
#1 0x55afbf01d277 <unknown>
#2 0x55afbf047b88 <unknown>
#3 0x55afbf0444de <unknown>
#4 0x55afbf083f32 <unknown>
#5 0x55afbf08368f <unknown>
#6 0x55afbf07b1d3 <unknown>
#7 0x55afbf04f632 <unknown>
#8 0x55afbf0503de <unknown>
#9 0x55afbf2a297d <unknown>
#10 0x55afbf2a7213 <unknown>
#11 0x55afbf2b0628 <unknown>
#12 0x55afbf2a7c4a <unknown>
#13 0x55afbf2792ae <unknown>
#14 0x55afbf2c86d8 <unknown>
#15 0x55afbf2c887f <unknown>
#16 0x55afbf2d7098 <unknown>
#17 0x7f8c71cfbb43 <unknown>

Operating System

Ubuntu Linux 64bits

Selenium version

=4.11.0

What are the browser(s) and version(s) where you see this issue?

Chromium 115.0.5790.102 snap

What are the browser driver(s) and version(s) where you see this issue?

Not sure

Are you using Selenium Grid?

No

4.10.0 was stable. I shifted all my scripts back to 4.10.0 and now the scripts are working fine.

@diemol
Copy link
Member

diemol commented Aug 1, 2023

@bonigarcia @titusfortner do you know why Chromium is not being detected?

@titusfortner
Copy link
Member

I don't think SM is looking for Chromium? I also don't think it should...

I think if you want something other than a released/named version of Chrome, you'll need to pass in the location of chromium in options class.

@ikus060
Copy link
Author

ikus060 commented Aug 1, 2023

I think if you want something other than a released/named version of Chrome, you'll need to pass in the location of chromium in options class.

Far enough. Could I define the location of chronium and chronium driver ?

Both are available in PATH:

/usr/bin/chromium
/usr/bin/chromedriver

@titusfortner
Copy link
Member

Hmm, can you turn on logging so we can see what selenium manager is doing?

https://www.selenium.dev/documentation/webdriver/troubleshooting/logging/

I think if the driver is found on PATH, but SM didn't see a browser, it will use that driver and try to get the browser for that driver.

I'm trying to think what 4.10 would do... 4.10 would skip SM entirely. I can't remember if chromedriver looks for Chromium browsers if it can't find chrome, but that would make sense.

Ok, so, if chrome isn't found, should SM look to see if chromium is installed instead of treating it as nothing installed... Probably if that better matches 4.10 behavior.

@bonigarcia
Copy link
Member

The current implementation of Selenium Manager does not look for Chromium, just Chrome. But, of course, it can be implemented. If we want to support it, there are two possibilities:

  1. Include a new browser manager to explicitly specify Chromium (./selenium-manager --browser chromium).
  2. Modify the existing logic of the Chrome manager (./selenium-manager --browser chrome) to check Chromium (e.g., checking /usr/bin/chromium etc.) when Chrome is unavailable.

@bonigarcia
Copy link
Member

Could I define the location of chronium and chronium driver ?

Your chromedriver in the PATH is already being detected by Selenium Manager. Maybe the only thing you need to specify your browser binary (/usr/bin/chromium) using options (see doc).

@ikus060
Copy link
Author

ikus060 commented Aug 2, 2023

@bonigarcia Thanks alot for your support. It's well appreciated.

Are we seeing the same thing ?

image

@titusfortner
Copy link
Member

Yes, chromedriver did look for chromium and now also looks for chrome for testing - https://github.com/bayandin/chromedriver/blob/main/chrome/chrome_finder.cc#L34

I think Selenium Manager needs to do the same before downloading something new.

@ikus060
Copy link
Author

ikus060 commented Aug 2, 2023

As recommended, to support selenium <=4.10 and selenium>=4.11, I used the following code:

        # for selenium >=4.11, explicitly define location of chronium driver.
        if hasattr(webdriver, 'ChromeService'):
            chrome_service_cls = getattr(webdriver, 'ChromeService')
            service = chrome_service_cls(executable_path=shutil.which('chromedriver'))
        else:
            service = None
        driver = webdriver.Chrome(options=options, service=service)

This effectively skip selenium-manager, use the chromedriver from the PATH and automatically find chronium web browser.

@titusfortner
Copy link
Member

better option is to just always the location of the browser you want:

options.binary_location = "/path/to/chromium"

Selenium will get the right driver for whatever is there.

@titusfortner titusfortner added this to the 4.12 milestone Aug 2, 2023
@bonigarcia
Copy link
Member

I think Selenium Manager needs to do the same before downloading something new.

Selenium Manager already checks for the existence of Chrome before downloading CfT.

@titusfortner
Copy link
Member

Right, I'm saying it should also check for an installed CfT and an installed Chromium.
We should look for what chromedriver is currently looking for

@bonigarcia
Copy link
Member

I'm saying it should also check for an installed CfT

SM already checks CfT releases on the cache before downloading a new one.

and an installed Chromium

That is what I said above it is not implemented (see my first post in this thread).

@javierm
Copy link

javierm commented Aug 2, 2023

As a workaround, I've added a symbolic link called chrome to one of the folders in my PATH, pointing to the Chromium executable. Now I'm getting a warning and I'm not sure whether it's related to this issue; should I open a new one?

WARN Selenium [:selenium_manager] The chromedriver version (111.0.5563.146) detected in PATH 
at /usr/bin/chromedriver might not be compatible with the detected chrome version (111.0.5563.146);
currently, chromedriver 111.0.5563.64 is recommended for chrome 111.*, so it is advised to delete the
driver in PATH and retry 

So it looks like my version of chromedriver might not be compatible with the exact same version of chrome 🤔, and that an older version of chromedriver is recommended instead.

I'm using Ruby with selenium-webdriver 4.11. The code I'm working with can be found at consuldemocracy/consuldemocracy#5162.

@titusfortner
Copy link
Member

You can turn on debugging to see what it's seeing, but you have a chrome/chromium install with version 111 somewhere.

Selenium::WebDriver.logger.level = :debug

@javierm
Copy link

javierm commented Aug 3, 2023

I do. Sorry I didn't explain it correctly 🙏.

$ chromium --version
Chromium 111.0.5563.146 Arch Linux
$ chromedriver --version
ChromeDriver 111.0.5563.146 (2b5ec366bb5b68dacca9719d5d83cb7e61fb6431-refs/branch-heads/5563@{#1238})

And then I've created a chrome symlink at ~/bin/ pointing to /usr/bin/chromium.

Here's the output with debugging turned on:

DEBUG Selenium [:selenium_manager] Selenium Manager binary found at ~/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/selenium-webdriver-4.11.0/bin/linux/selenium-manager
DEBUG Selenium [:selenium_manager] Executing Process ["~/.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/selenium-webdriver-4.11.0/bin/linux/selenium-manager", "--browser", "chrome", "--output", "json", "--debug"]
DEBUG Selenium [:selenium_manager] Checking chromedriver in PATH
DEBUG Selenium [:selenium_manager] Running command: chromedriver --version
DEBUG Selenium [:selenium_manager] Output: "ChromeDriver 111.0.5563.146 (2b5ec366bb5b68dacca9719d5d83cb7e61fb6431-refs/branch-heads/5563@{#1238})"
DEBUG Selenium [:selenium_manager] Running command: which chromedriver
DEBUG Selenium [:selenium_manager] Output: "/usr/bin/chromedriver"
DEBUG Selenium [:selenium_manager] Found chromedriver 111.0.5563.146 in PATH: /usr/bin/chromedriver
DEBUG Selenium [:selenium_manager] Checking chrome in PATH
DEBUG Selenium [:selenium_manager] Running command: which chrome
DEBUG Selenium [:selenium_manager] Output: "~/bin/chrome"
DEBUG Selenium [:selenium_manager] Found chrome in PATH: ~/bin/chrome
DEBUG Selenium [:selenium_manager] Using shell command to find out chrome version
DEBUG Selenium [:selenium_manager] Running command: /usr/bin/chromium --version
DEBUG Selenium [:selenium_manager] Output: "Chromium 111.0.5563.146 Arch Linux"
DEBUG Selenium [:selenium_manager] Detected browser: chrome 111.0.5563.146
DEBUG Selenium [:selenium_manager] Required driver: chromedriver 111.0.5563.64
WARN Selenium [:selenium_manager] The chromedriver version (111.0.5563.146) detected in PATH at /usr/bin/chromedriver might not be compatible with the detected chrome version (111.0.5563.146); currently, chromedriver 111.0.5563.64 is recommended for chrome 111.*, so it is advised to delete the driver in PATH and retry
DEBUG Selenium [:selenium_manager] Driver path: /usr/bin/chromedriver
DEBUG Selenium [:driver_service] Executing Process ["/usr/bin/chromedriver", "--port=9515"]
DEBUG Selenium [:process] Starting process: ["/usr/bin/chromedriver", "--port=9515"] with {[:out, :err]=>#<IO:<STDOUT>>, :pgroup=>true}
DEBUG Selenium [:process]   -> pid: 4998
DEBUG Selenium [:driver_service] polling for socket on ["127.0.0.1", 9515]
Starting ChromeDriver 111.0.5563.146 (2b5ec366bb5b68dacca9719d5d83cb7e61fb6431-refs/branch-heads/5563@{#1238}) on port 9515
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.

@titusfortner
Copy link
Member

Ah, I didn't read your post closely enough.
The warning is because Selenium is looking for the latest version of a released chromedriver (*.64), but you aren't using a released chromedriver, rather a built driver from chromium, so Selenium is throwing a warning that it isn't what it expects. We could do more complicated logic here, but I think Boni already merged a commit so that it isn't as strict about the patch version.

@bonigarcia
Copy link
Member

Currently, the warning is shown when the driver in PATH is strictly different from the recommended driver. This is your case (you have chromedriver 111.0.5563.146, and the recommended version is 111.0.5563.64). But typically, you don't need to worry since the major version (111) matches. So we have changed the Selenium Manager logic to show only the warning message when the major version is different.

@titusfortner
Copy link
Member

@bonigarcia I think the second one here is easier, and more closely follows default (non-SM) behavior - #12466 (comment)

Can we track that somewhere else and close this one?

@bonigarcia
Copy link
Member

Can we track that somewhere else and close this one?

#12511

Copy link

github-actions bot commented Dec 8, 2023

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked and limited conversation to collaborators Dec 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

6 participants