Require Java 11 #12843

titusfortner · 2023-09-30T02:35:04Z

I'm sure there's a lot more that can be cleaned up that I'm missing, but I think this is the first step.

Description

Requires Java 11
Removes Netty Http client
Defaults to Native Java

Motivation and Context

Fixes #11526

codecov-commenter · 2023-09-30T03:04:04Z

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (b9bdff1) 56.51% compared to head (ac44568) 56.51%.
Report is 2 commits behind head on trunk.

❗ Current head ac44568 differs from pull request most recent head 44273c7. Consider uploading reports for the commit 44273c7 to get more accurate results

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##            trunk   #12843   +/-   ##
=======================================
  Coverage   56.51%   56.51%           
=======================================
  Files          86       86           
  Lines        5255     5255           
  Branches      187      187           
=======================================
  Hits         2970     2970           
  Misses       2098     2098           
  Partials      187      187

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

java/src/org/openqa/selenium/remote/http/HttpClient.java

joerg1985 · 2023-09-30T19:50:00Z

This test is using the NettyClient directly, this should be fixed:

selenium/java/test/org/openqa/selenium/remote/http/RetryRequestTest.java

Line 50 in 0b5cf2d

client = new NettyClient.Factory().createClient(config);

Does it make sense to have a bazel file in the jdk folder or could this just be part of the parent "../remote/http"?

titusfortner · 2023-09-30T21:45:43Z

Does it make sense to have a bazel file in the jdk folder

I don't know? I mostly copied what I saw Netty had done in the places that seemed to make sense. If you think we should change something, we probably should because I don't really know what I'm changing. 😆

joerg1985 · 2023-09-30T21:51:47Z

Does it make sense to have a bazel file in the jdk folder

I don't know? I mostly copied what I saw Netty had done in the places that seemed to make sense. If you think we should change something, we probably should because I don't really know what I'm changing. 😆

I am no bazel expert too, there might be a reason for it :D

joerg1985 · 2023-10-01T07:29:54Z

I did some changes to make the bazel build work, i hope this is okay for you

shs96c · 2023-10-01T09:04:40Z

Does it make sense to have a bazel file in the jdk folder or could this just be part of the parent "../remote/http"?

The "bazel way" is a build file per java package, so the jdk folder should definitely have its own BUILD.bazel file.

shs96c · 2023-10-01T09:06:56Z

The NettyDomainSocketClient needs to stay, as it's used for the Docker integration because the other HttpClient implementations can't use unix domain sockets.

joerg1985 · 2023-10-01T09:11:35Z

I had to remove the htmlunit-driver dependency to get rid of the asynchttpclient, this added a circular dependency what should be avoided. Beside some helper classes i could not find any use of the htmlunit-driver so i hope this is fine. Lets wait what the CI will tell us.

titusfortner · 2023-10-01T14:14:40Z

the other HttpClient implementations can't use unix domain sockets

Looks like we can transition those things when we require Java 17 next year: https://openjdk.org/jeps/380

titusfortner · 2023-10-01T14:25:43Z

@rbri is htmlunit-driver also moving to Java 11 requirement for the net release? Will that address the circular dependencies?

diemol

I don't understand why we need to make so many changes in a PR for this. We have always mentioned that moving completely to Java 11 is an iteration process.

Why don't we change the default http client, target Java 11 and that's it. The rest can be iterated little by little.

Long PRs are just too hard to review.

joerg1985 · 2023-10-01T18:07:01Z

@rbri is htmlunit-driver also moving to Java 11 requirement for the net release? Will that address the circular dependencies?

The circular dependency is now gone due to removal of the htmlunit-driver dependency.

joerg1985 · 2023-10-01T20:21:53Z

@diemol you are right about large PRs, but i think this PR is now ready. The CI is failing but this would also happen if only the default client has been changed. There is only one additional change here, the removal of the old netty client and it's dependencies (by removing the htmlunit-driver support).

diemol · 2023-10-01T20:36:02Z

However, almost all CI workflows in Java are failing. We cannot merge like that.

titusfortner · 2023-10-01T20:37:45Z

Yeah, I didn't want to keep distributing a library with a vulnerability, which is why I removed the netty client instead of just changing the default.

Thanks @joerg1985 for doing all the work to clean up after me. 😁

I'll take a look at the failures tomorrow

joerg1985 · 2023-10-01T20:38:47Z

Okay then there should be single PRs created for all of the changes.

java/src/org/openqa/selenium/chrome/ChromeDriverInfo.java

java/src/org/openqa/selenium/remote/http/netty/NettyDomainSocketClient.java

java/test/org/openqa/selenium/testing/TestUtilities.java

java/src/org/openqa/selenium/chrome/ChromeDriverInfo.java

java/test/org/openqa/selenium/remote/http/netty/NettyDomainSocketTest.java

pujagani · 2023-10-04T13:57:56Z

This also impacts the changes done as part of #11606. Hence the failing "DevToolsReuseTest" test. Earlier, this was not a problem since we used one instance of AsyncHttpClient and it was only closed when the JVM shuts down. With the JdkHttpClient, we use a new instance as required and close it too. Changes in #11345, which assigns the client to null is while closing causing the test failure and preventing re-use. It was done to prevent memory leaks. I am not sure about the right approach to address it. We might need to discuss/brainstorm over it.

diemol · 2023-10-04T15:35:12Z

This also impacts the changes done as part of #11606. Hence the failing "DevToolsReuseTest" test. Earlier, this was not a problem since we used one instance of AsyncHttpClient and it was only closed when the JVM shuts down. With the JdkHttpClient, we use a new instance as required and close it too. Changes in #11345, which assigns the client to null is while closing causing the test failure and preventing re-use. It was done to prevent memory leaks. I am not sure about the right approach to address it. We might need to discuss/brainstorm over it.

@pujagani is there a bug report for that? I can imagine that someone might bump into that situation when using the Java 11 client.

titusfortner · 2023-10-04T22:06:18Z

Ok, the NettyDomainSocketTest calls NettyClient.Factory() which pretty much needs all of Netty plus AsyncHttpClient.

So I think we need a different test if we need a test for it.

pujagani · 2023-10-05T05:03:23Z

@diemol I don't think there is one. Will double check else I will create it and link it here.

titusfortner · 2023-10-06T03:49:30Z

This is passing all the tests and looks right to me.

The only issue is that I removed NettyDomainSocketTest because it required AHC. If we need a test for that class, can someone please help add it?

pujagani · 2023-10-06T05:30:15Z

#12882 - Created an issue to track the regression

pujagani · 2023-10-06T06:03:31Z

This is passing all the tests and looks right to me.

The only issue is that I removed NettyDomainSocketTest because it required AHC. If we need a test for that class, can someone please help add it?

I looked into it, but I don't think the current test can be modified without having another client library. Since Java HTTP Client does not support UNIX sockets.

rbri · 2023-10-06T06:39:10Z

@rbri is htmlunit-driver also moving to Java 11 requirement for the net release? Will that address the circular dependencies?

The circular dependency is now gone due to removal of the htmlunit-driver dependency.

@joerg1985
first of all thanks for removing the outdated stuff. I think getting rid of the circular dependency is a step forward for selenium (see my comment on #12849).

Regarding the htmlunit-driver i think i have no chance - i have to move to Java 11. I like to stay with the support of the latest selenium version.
For HtmlUnit itself i like to stay at java8 if possible but i will see.
From my point of view we can go ahead with this making a compatible htmlunit-driver is a task on my desk like all the other htmlunit related stuff (of course any help is welcome....)

titusfortner · 2023-10-06T12:13:52Z

Since Java HTTP Client does not support UNIX sockets.

But we're testing with java 17 and I think Java 17 does support it? Can we create an implementation in test code only?

pujagani · 2023-10-09T09:14:04Z

Since Java HTTP Client does not support UNIX sockets.

But we're testing with java 17 and I think Java 17 does support it? Can we create an implementation in test code only?

Ahh yes! I think that can be done. I am not sure how can we create an implementation in test code only but will be happy to take a took.

* [java] require Java 11 * [java] remove netty http client and use native java client by default * [java] updated the list of exports * [java] use the default http client in tests * [java] removed --remote-allow-origins=* for the netty client * [java] removed the asynchttpclient and htmlunit-driver dependencies --------- Co-authored-by: Jörg Sautter <joerg.sautter@gmx.net>

…iumHQ#12843) Workaround was made as part of the fix (3f7f57c) the bug SeleniumHQ#11750 which was specific for netty client only. Since SeleniumHQ#12843 the netty client was removed.

… (#14134) * [java] Revert workaround for old netty http client (addendum to #12843) Workaround was made as part of the fix (3f7f57c) the bug #11750 which was specific for netty client only. Since #12843 the netty client was removed. * Running format script --------- Co-authored-by: Diego Molina <diemol@users.noreply.github.com> Co-authored-by: Diego Molina <diemol@gmail.com>

titusfortner added the C-java label Sep 30, 2023

titusfortner requested review from diemol, pujagani and joerg1985 September 30, 2023 02:35

joerg1985 reviewed Sep 30, 2023

View reviewed changes

java/src/org/openqa/selenium/remote/http/HttpClient.java Outdated Show resolved Hide resolved

joerg1985 force-pushed the java_11 branch from 7c14bff to 378c016 Compare October 1, 2023 07:11

diemol reviewed Oct 1, 2023

View reviewed changes

diemol requested changes Oct 1, 2023

View reviewed changes

java/src/org/openqa/selenium/chrome/ChromeDriverInfo.java Show resolved Hide resolved

java/src/org/openqa/selenium/remote/http/netty/NettyDomainSocketClient.java Outdated Show resolved Hide resolved

java/test/org/openqa/selenium/testing/TestUtilities.java Outdated Show resolved Hide resolved

pujagani requested changes Oct 3, 2023

View reviewed changes

java/src/org/openqa/selenium/chrome/ChromeDriverInfo.java Show resolved Hide resolved

java/test/org/openqa/selenium/remote/http/netty/NettyDomainSocketTest.java Show resolved Hide resolved

titusfortner force-pushed the java_11 branch 2 times, most recently from ff5f88e to b89ed8e Compare October 5, 2023 03:53

joerg1985 and others added 10 commits October 5, 2023 20:25

[java] removed the netty related tests

18bfd69

[java] use the default http client in tests

0009bf6

[java] removed --remote-allow-origins=* for the netty client

7ec1a26

[java] fixed the http client factory unit tests

9e42076

[java] removed the asynchttpclient and htmlunit-driver dependencies

aba70f6

[java] run the format script

613d068

[java] put NettyDomainSocket class and tests back

d1e2677

[java] remove unnecessary toggles on jdk-http-client

daa574b

update tests

530a1f5

put ruby back down to 11

44273c7

titusfortner force-pushed the java_11 branch from 74498c1 to 44273c7 Compare October 6, 2023 01:35

titusfortner requested review from pujagani, diemol and joerg1985 October 6, 2023 03:42

titusfortner merged commit 9b35af2 into trunk Oct 6, 2023
48 of 54 checks passed

titusfortner deleted the java_11 branch October 6, 2023 14:14

Zidious mentioned this pull request Oct 11, 2023

Selenium dropped support for Java 8 dequelabs/axe-core-maven-html#378

Open

kool79 mentioned this pull request Jun 13, 2024

[java] Revert workaround for old netty http client (addendum to #12843) #14134

Merged

x-way mentioned this pull request Aug 10, 2024

[Snyk] Upgrade selenium-webdriver from 4.18.1 to 4.23.0 x-way/cal-heatmap#544

Closed

This was referenced Sep 13, 2024

[Snyk] Upgrade selenium-webdriver from 4.17.0 to 4.23.0 organich/chrome-extensions-samples#1

Open

[Snyk] Upgrade selenium-webdriver from 4.17.0 to 4.24.0 organich/chrome-extensions-samples#3

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Require Java 11 #12843

Require Java 11 #12843

titusfortner commented Sep 30, 2023

codecov-commenter commented Sep 30, 2023 •

edited

Loading

joerg1985 commented Sep 30, 2023

titusfortner commented Sep 30, 2023

joerg1985 commented Sep 30, 2023

joerg1985 commented Oct 1, 2023

shs96c commented Oct 1, 2023

shs96c commented Oct 1, 2023

joerg1985 commented Oct 1, 2023

titusfortner commented Oct 1, 2023 •

edited

Loading

titusfortner commented Oct 1, 2023

diemol left a comment

joerg1985 commented Oct 1, 2023

joerg1985 commented Oct 1, 2023

diemol commented Oct 1, 2023

titusfortner commented Oct 1, 2023

joerg1985 commented Oct 1, 2023

pujagani commented Oct 4, 2023

diemol commented Oct 4, 2023

titusfortner commented Oct 4, 2023

pujagani commented Oct 5, 2023

titusfortner commented Oct 6, 2023 •

edited

Loading

pujagani commented Oct 6, 2023

pujagani commented Oct 6, 2023

rbri commented Oct 6, 2023

titusfortner commented Oct 6, 2023

pujagani commented Oct 9, 2023

Require Java 11 #12843

Require Java 11 #12843

Conversation

titusfortner commented Sep 30, 2023

Description

Motivation and Context

codecov-commenter commented Sep 30, 2023 • edited Loading

Codecov Report

joerg1985 commented Sep 30, 2023

titusfortner commented Sep 30, 2023

joerg1985 commented Sep 30, 2023

joerg1985 commented Oct 1, 2023

shs96c commented Oct 1, 2023

shs96c commented Oct 1, 2023

joerg1985 commented Oct 1, 2023

titusfortner commented Oct 1, 2023 • edited Loading

titusfortner commented Oct 1, 2023

diemol left a comment

Choose a reason for hiding this comment

joerg1985 commented Oct 1, 2023

joerg1985 commented Oct 1, 2023

diemol commented Oct 1, 2023

titusfortner commented Oct 1, 2023

joerg1985 commented Oct 1, 2023

pujagani commented Oct 4, 2023

diemol commented Oct 4, 2023

titusfortner commented Oct 4, 2023

pujagani commented Oct 5, 2023

titusfortner commented Oct 6, 2023 • edited Loading

pujagani commented Oct 6, 2023

pujagani commented Oct 6, 2023

rbri commented Oct 6, 2023

titusfortner commented Oct 6, 2023

pujagani commented Oct 9, 2023

codecov-commenter commented Sep 30, 2023 •

edited

Loading

titusfortner commented Oct 1, 2023 •

edited

Loading

titusfortner commented Oct 6, 2023 •

edited

Loading