Different performance between python and dotnet (firefox)

[xAaeEx]

Different performance between python and dotnet (firefox)

OS: Windows 10 Enterprise 2016 LTSB (Build: 1607)

Selenium Version:
3.14.1 (python) - 3.14.0 (dotnet), Visual Studio, Visual Studio Code

Browser:
Firefox

Browser Version:
63.0 64-Bit

Expected Behavior

Expected Behavior is that all Tests (with Firefox) behave the same in dotnet as in python environment.
Which they do on the execution level, but unfortunatly they behave different in performance which I don't comprehend.

Actual Behavior

Behavior in Firefox

The performance difference is big!

Browser	Time
Firefox (dotnet)	32.40 sec
Firefox (python)	11.825 sec

Example behavior in Chrome (same test)

It's basically the same (+/- some seconds).

Browser	Time
Chrome (dotnet)	10.87 sec
Chrome (python)	12.174 sec

Steps to reproduce this issue

On https://gist.github.com/xAaeEx/2da70dbdf6a55aeebd6ca7ba253341e0 you will find the Test(s) I created which opens Google search for SeleniumHQ and click on the first link.

Both tests are (almost) identical on python aswell as dotnet.

Can someone explain what is happening? Or if this is a firefox/marionette issue I should report?

If this is an issue, we should consider looking into it, as apperently many folks are using more and more dotnet to automate stuff (as am I).

[AutomatedTester]

Since Python is working faster than .NET and it's doing the same thing it suggests that it is not the remote end (geckodriver and Firefox) that is the issue.

Could you do a test where it is purely instantiating an object and see how long that takes. Under the hood this is starting the browser and getting everything in place. If it is the same at that point then perhaps there are issues with a specific command but if we can narrow that area first then we can move from there.

Ideally they should be within the same times.

[shs96c]

Which version of .Net are you using? There's a known performance problem with .Net Core 2, which you might be tickling in the firefox case.

I'd also be interested to know how long the tests took once the browser started --- is the problem with test execution once the browser is stood up, or is it in actually creating the instance in the first place?

[xAaeEx]

So I updated the tests (see gist above) to give me the time ellapsed when the driver is up and running until the test is complete:

	Time
python	1.6514158248901367s
dotnet	12.5151790s

Also here are all requirements I took to write the tests and the environments:

Requirements for Python:

Python 3.6.4

• unittest
• selenium (3.14.1)

Nuget packages for C#:

dotnet Core 2.1 (2.1.403)

• Microsoft.NET.Test.Sdk (15.8.0)
• MSTest.TestAdapter (1.3.2)
• MSTest.TestFramework (1.3.2)
• Selenium.WebDriver (3.14.0)

WebDrivers

geckodriver v0.23.0

chromedriver v2.43

[xAaeEx]

Also I noticed that the firefox browser is very slow when you do an findElement and give that element an input (click or sendKeys).

It's almost when you do test against InternetExplorer and it's slow motion.

[jimevans]

Yep. You’re definitely hitting the performance issue that is part of .NET Core. It doesn’t happen on Chrome, IE, or Edge, because the driver executables for each of those browsers (unlike geckodriver) listen on both the IPv4 and IPv6 loopback addresses. If you were to specify “::1” as the host for geckodriver with .NET, the problem would disappear.

I will attempt to resist the temptation to editorialize about the perceived slowness of the IE driver’s SendKeys when the root cause is well-known, and well-publicized.

[xAaeEx]

@jimevans Sorry man :) it was not intensional to post it as a rant.
It was just a way to describe the current issue. (I read about the 32/64-Bit problems)

[xAaeEx]

FYI:
I'm testing this issue with the newest dotnet core version (3.0.100-preview-010184) and it seems not to be fixed yet.

[xAaeEx]

FYI:

Firefox 66.0.5 (64-bit)
geckodriver 0.24.0 (win64)
dotnet core 3.0.100-preview5-011568 (64-bit)

Issue still remains

[jimevans]

The problem could be fixed in geckodriver, if the geckodriver team would listen on both the IPv4 and IPv6 loopback connections simultaneously (opening two sockets instead of one, as every other WebDriver remote-end implementation currently does).\

[sickvisionz]

I was having a similar issue, but in C#. The answer at https://stackoverflow.com/a/57720610/8056356 solved it for me.

[AutomatedTester]

Closing as the issue is with Geckodriver and Mozilla needs to fix it. From their side they can enable geckodriver to do that but the pass through to Firefox gets stuck.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.