[🐛 Bug]: Chrome 111 is not compatible with default HTTP Client

[jamesplawry]

What happened?

After the Chrome 111 update, you can no longer kick off a chromedriver instance unless you add an additional chrome option:

"--remote-allow-origins=*"

I submitted a bug with Google and they responded, and indicated this has to do with Selenium setting the origin header:

https://bugs.chromium.org/p/chromium/issues/detail?id=1422444

How can we reproduce the issue?

Just update to chrome 111, and try and run a UI selenium test.

I am using Java and Junit.  See the link to the Chrome ticket for more details on how I was personally starting the driver.

Relevant log output

Mar 07, 2023 8:18:50 PM org.openqa.selenium.remote.http.WebSocket$Listener onError
WARNING: Invalid Status code=403 text=Forbidden
java.io.IOException: Invalid Status code=403 text=Forbidden
	at org.asynchttpclient.netty.handler.WebSocketHandler.abort(WebSocketHandler.java:92)
	at org.asynchttpclient.netty.handler.WebSocketHandler.handleRead(WebSocketHandler.java:118)
	at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)


org.openqa.selenium.remote.http.ConnectionFailedException: Unable to establish websocket connection to http://localhost:64820/devtools/browser/27cc1ab2-7320-47bf-b503-1099b7052fcc
Build info: version: '4.4.0', revision: 'e5c75ed026a'
System info: host: 'sdg-mac334.local', ip: '2601:681:8004:57a0:0:0:0:5bfc%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '12.1', java.version: '17.0.4.1'
Driver info: driver.version: Driver

Operating System

Mac OS Monterey

Selenium version

4.4.0

What are the browser(s) and version(s) where you see this issue?

Chrome 111

What are the browser driver(s) and version(s) where you see this issue?

I'm using webdriver manager, so whichever the latest one its grabbing.

Are you using Selenium Grid?

No response

[github-actions]

@jamesplawry, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[OrKoN]

I think the issue might be related to netty/netty#9673 which seems to be fixed in netty 5 (I believe Selenium is using netty 4 still)

[titusfortner]

AsyncHttp Client has not been actively maintained (looks like this just changed), so we've been requesting people upgrade to the Java 11 HTTP client https://www.selenium.dev/blog/2022/using-java11-httpclient/

Looks like latest version of AsyncHttp still relies on Netty 4 and is compiled with Java 11 anyway? So I don't think continuing with that library is going to be the solution regardless.

[titusfortner]

Ok, we could also add this to the Service class the same way we are doing for Firefox. That probably is the right thing to do if we continue to support Java < 11

[Artur-]

Adding

   <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-http-jdk-client</artifactId>
        </dependency>

and

        System.setProperty("webdriver.http.factory", "jdk-http-client");

makes tests work again

[jfdez-s24]

Adding

   <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-http-jdk-client</artifactId>
        </dependency>

and

        System.setProperty("webdriver.http.factory", "jdk-http-client");

makes tests work again

@Artur- this will only work for versions starting from 4.5.0: https://www.selenium.dev/blog/2022/using-java11-httpclient/. I'm using 4.1.4, so still need to go with the workaround of -remote-allow-origins=*.

Is it possible to specify a placeholder for the port for the cases in which a random port is used? something like: -remote-allow-origins=http://localhost:{a-random-port-to-be-replaced-at-startup}

[LokeshGangaiah]

will there be a new ver of selenium to get this fix working ? current latest is 4.8.1 which was release on feb? so we will have new one sooner ?

[rodriguesxd7]

How can i add this new arg for the chrome options??

Is it something like this?

ChromeOptions options = new ChromeOptions();
options.addArguments("--remote-allow-origins=*");

??

Thanks guys

[jamesplawry]

How can i add this new arg for the chrome options??

Is it something like this?

ChromeOptions options = new ChromeOptions(); options.addArguments("--remote-allow-origins=*");

??

Thanks guys

Yep.

Then just start the driver with those options and you should be good to go.

[rodriguesxd7]

How can i add this new arg for the chrome options??
Is it something like this?
ChromeOptions options = new ChromeOptions(); options.addArguments("--remote-allow-origins=*");
??
Thanks guys

Yep.

Then just start the driver with those options and you should be good to go.

Doesn't work for me... could you help please?

Here are my code

ChromeOptions options = new ChromeOptions();
System.setProperty("webdriver.chrome.driver", "drivers/chromedriver.exe");
options.addArguments("--remote-allow-origins=http://localhost:8080/leiloes");
WebDriver driver = new ChromeDriver();
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.navigate().to("http://localhost:8080/leiloes");

Where am i wrong?

[vemulaharshini37]

WebDriver driver = new ChromeDriver(options); // update this mentioned line

[derRichter]

You have to wrote the Wildcard and not the website what you navigate
the option --remote-allow-origins=* is needed for the dev-port-connection an not for what you navigate after
replace "http://localhost:8080/leiloes" with *

[rodriguesxd7]

@vemulaharshini37 @derRichter
Thaaaanks guys, both work in my code... thank you so much!!!

[Faizan-Mamji]

Hello Everyone,

I have upgraded Java to 15 version.
Browser launching successfully for me if i am not using headless.

But when i am trying with headless it doesn't work for me.
Getting this below error.
Can anyone please help me to resolve this issue?

For reference i am adding the code as well.

 public WebDriver launchWebBrowser(String browserName, String appUrl) {
        System.setProperty("webdriver.http.factory", "jdk-http-client");
        DesiredCapabilities capss = new DesiredCapabilities();
        ChromeOptions options = new ChromeOptions();
        try {
            options.setAcceptInsecureCerts(true);
            options.setExperimentalOption("excludeSwitches", Arrays.asList("test-type"));
            options.addArguments("--headless=new");
            options.addArguments("--disable-web-security");
            options.addArguments("--no-sandbox");
            options.addArguments("start-maximized");
            options.addArguments("--disable-dev-shm-usage");
            options.addArguments("--disable-gpu");
            options.addArguments("--print-to-pdf");
            options.addArguments("--allow-running-insecure-content");
            capss.setCapability(ChromeOptions.CAPABILITY, options);
            capss.setCapability(CapabilityType.ACCEPT_INSECURE_CERTS, true);
            options.merge(capss);

            if (browserName.equalsIgnoreCase("chrome")) {
                WebDriverManager.chromedriver().setup();
                driverWeb = new ChromeDriver(options);

            } else if (browserName.equalsIgnoreCase("firefox")) {
                WebDriverManager.firefoxdriver().setup();
                driverWeb = new FirefoxDriver();
            }
            driverWeb.navigate().to(appUrl);
            driverWeb.manage().window().maximize();
            System.out.println("Browser size is " + driverWeb.manage().window().getSize());
            tWdriver.set(driverWeb);
            driverWeb.manage().timeouts().implicitlyWait(Duration.ofSeconds(Integer.parseInt(PropertyReader.getConfigValue(configFile, "implicitWait"))));

        } catch (Exception ex) {
            Assert.fail("Issue in launching browser - " + ex.getMessage());
        }
        return driverWeb;
    }

**Error I am Getting when running headless**

Starting ChromeDriver 111.0.5563.64 (c710e93d5b63b7095afe8c2c17df34408078439d-refs/branch-heads/5563@{#995}) on port 36269
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
Mar 13, 2023 9:47:38 AM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 111, so returning the closest version found: 110

junit.framework.AssertionFailedError: Issue in launching browser - no such window: target window already closed
from unknown error: web view not found
  (Session info: chrome=111.0.5563.64)
Build info: version: '4.8.1', revision: '8ebccac989'
System info: os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.16', java.version: '15.0.2'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Command: [8f586fb87f597bc9007bdb76dfbe218b, get {url=https://test-caseworker-portal.platform.rer.dev/}]
Capabilities {acceptInsecureCerts: true, browserName: chrome, browserVersion: 111.0.5563.64, chrome: {chromedriverVersion: 111.0.5563.64 (c710e93d5b63..., userDataDir: /var/folders/v8/mrd778lx16n...}, goog:chromeOptions: {debuggerAddress: localhost:54038}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: MAC, proxy: Proxy(), se:cdp: ws://localhost:54038/devtoo..., se:cdpVersion: 111.0.5563.64, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
Session ID: 8f586fb87f597bc9007bdb76dfbe218b

Please suggest the workaround for headless.

[poorSock]

You need to be at least on selenium version 4.5.0.
Works for me in headless with java 17 and selenium 4.5.0

Also: Why are there no "--" before your start-maximized option?

[Reshef1986]

WebDriver wd;
ChromeOptions options = new ChromeOptions();
options.addArguments("--remote-allow-origins=*");
wd =new ChromeDriver(options);

Wrote it like this and everything works

[titusfortner]

reminder: It's fixed in the latest version of Selenium

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.