FF w/marionette doesnt work in Capybara

[twalpole]

As of a future FF, Firefox driver in Selenium will no longer work, and support will have to be provided through marionette. Work has begun on support for marionette in the marionette branch of Capybara but there are still a lot of things broken and many tests that run fine separately but fail when run as part of the test suite. If anyone can figure out how to select/deselect a single option in a multiple select without selecting/deselecting the rest of the options info would be appreciated.

With selenium-webdriver 2.53.1 a driver can be configured to use marionette as

Capybara.register_driver :selenium do |app|
Capybara::Selenium::Driver.new(app, browser: :firefox, marionette: true)
end

also requires having wires in your path

[gmcnaughton]

Couple gotchas about the wires binary:

• Per the Mozilla docs:
  • "Even though the project has been renamed to GeckoDriver, many of the selenium clients look for the old name. You need to rename the binary file to 'wires' (the old name) and ensure it is executable."
• If running specs launches Wireshark instead of your browser, it's because selenium-webdriver is looking for any executable called 'wires*' and is finding wireshark in your PATH before wires.
  • Make sure geckodriver-0.8.0-OSX has been renamed wires, made executable (chmod u+x wires), and is somewhere in your PATH before wireshark.

[twalpole]

Another gotcha is that multiple sessions don't work because selenium-webdriver doesn't set a unique marionette-port for each instance of firefox launched - this means the second sessions instance of wires/geckodriver connects/attempts to connect to the first sessions instance of firefox and the first session then goes nuts -- actions occurring twice, random errors, etc.

[juanibiapina]

Another gotcha is that this call errors:

Capybara.register_driver :firefox_custom do |app|
  profile = Selenium::WebDriver::Firefox::Profile.new
  profile["browser.download.useDownloadDir"] = true
  Capybara::Selenium::Driver.new(app, :browser => :firefox, :profile => profile, :marionette => true)
end

unknown option: profile

Without the profile option it "works" as before.

[tucknats]

So is there a way to work with a custom profile?

[gmcnaughton]

@juanibiapina good point, I got that error too! I haven't found a workaround for specifying a profile yet. (In our case, we're using a profile to set a custom user-agent).

[twalpole]

@juanibiapina Looking through the selenium and geckodriver code it looks like profile is now supposed to be passed through firefox_profile in the capabilities --- https://github.com/SeleniumHQ/selenium/blob/master/rb/lib/selenium/webdriver/remote/capabilities.rb#L245 and https://github.com/mozilla/geckodriver/blob/4cbbc79ad487a7b1d8defd66be2923af6febf33e/src/marionette.rs#L331 -- unfortunately the special handling (encoding) for firefox_profile doesn't appear to have made it into the W3CCapabilities class, which I believe needs to be used for capabilities when using marionette - https://github.com/SeleniumHQ/selenium/blob/master/rb/lib/selenium/webdriver/remote/w3c_capabilities.rb#L187 so you may need to encode them yourself

[twalpole]

FYI - all current changes on the marionette branch have been merged into the master branch -- This does not mean marionette works fully with Capybara, we are dependent on a number of fixes in selenium and geckodriver before it will be fully usable.

[gmcnaughton]

@twalpole Thanks! 👍

[twalpole]

Update: geckodriver 0.9.0 fixes the multiple session issues but appears to break a lot of other things since it appears to now return attribute values with no way to access attribute properties (with selenium-webdriver 2.53.4) -- Basically I think marionette is going to be unusable with Capybara until selenium-webdriver v3 launches.

[mockdeep]

Yet another gotcha, geckodriver explicitly does not support Firefox 47. Is there an alternative that gets selenium working on Firefox 47, or do we just wait until 48 comes out?

[twalpole]

@mockdeep That really just means they won't fix any issues that only occur with 47 - It "works" with 47, if we define "works" as starts up firefox and some things happen, but it's definitely not fully functional with the latest selenium-webdriver on any current version of firefox (47, 48 beta, 49 alpha) at the moment.

[mockdeep]

@twalpole unfortunately it doesn't do much for me. It boots up the browser and doesn't get past the login page. Do we have any sense at what point there will be a stable alternative?

[twalpole]

@mockdeep That really depends on the selenium-webdriver release schedule. From looking at the code I assume it will come in selenium-webdriver v3 which I would expect to be released relatively soon after Firefox 48 if that really no longer works with FirefoxDriver. For now you should just stay with FF 47.0.1 and selenium-webdriver 2.54.3 and not use marionette/geckodriver

[mockdeep]

@twalpole selenium-webdriver 2.54.3 doesn't work for me with FF 47. I get an unable to connect error:

Selenium::WebDriver::Error::WebDriverError:
  unable to obtain stable firefox connection in 60 seconds (127.0.0.1:7055)

[twalpole]

@mockdeep Whoops typo -- it's selenium-webdriver 2.53.4 - and FF 47.0.1 . The .0.1 is very important 47.0.0 doesn't work -- It's what the capybara tests are currently run with and works fine - works locally too.

[mockdeep]

Ah, okay. I'm still on 47.0, so will report back when then next release comes through.

[twalpole]

Selenium-webdriver released 3.0.0.beta1 today - running it with Capybara, the latest geckodriver (0.9.0), and FF 48beta7 things are still massively broken - https://travis-ci.org/jnicklas/capybara/jobs/148403787 . A large part of that is due to the fact there is no way to access the current value of an element (it only returns the attribute not property), but there's also things like clicking on an option in a select doesn't change the select, mouse actions don't work, etc. Hopefully there are a few more betas coming.

[twalpole]

FF48 released today - Until selenium-webdriver and geckodriver provide a few more features it's basically unusable with Capybara so don't update.

[titusfortner]

Basically I think marionette is going to be unusable with Capybara until selenium-webdriver v3 launches.

The great thing about geckodriver now is that its usability is completely independent of the Selenium release cycle. Part of the reason for the missing features is because it is implementing everything entirely based on the new w3c spec, which is still in draft. The Mozilla team has made a ton of progress recently, but Selenium 3 will likely be released before they've fixed the major issues you've identified.

[twalpole]

@titusfortner It's independent of the Selenium release cycle if we write a driver directly against geckodriver. If we continue using Selenium then a lot of it is still dependent on the Selenium release cycle. My personal view is that releasing Selenium 3 before it has comparable feature support with the Selenium 2.x and FirefoxDriver is just a recipe for tons of bug reports, but thats not my decision to make :)

[titusfortner]

geckodriver will be just like chromedriver, completely independent of Selenium releases. The browser manufacturer will keep their latest drivers compatible with the latest version of Firefox. The only conflation with a Selenium release is that 3+ defaults to geckodriver instead of the legacy driver, which as we've discussed in the PR is configurable. :)

My point is merely that the release of selenium v3 will not make marionette any more or less usable as your comment seemed to indicate.

At the point that the latest Firefox version doesn't work with legacy driver, it doesn't make sense for that to continue to be the default, and the Selenium team gets to punt the bug reports to Mozilla who are hard at work dealing with them. Also, this change in default was at Mozilla's request. :)

[twalpole]

@titusfortner The issue is that while geckodriver can change how its features work, unless Selenium 3 is providing access to them then we are tied to the selenium releases - as per my comment on PR# 1744 - Providing access to the element value and the element properties would actually make marionette eminently more usable (although the select issues would still be a problem until the next FF release). Also, I'm not in any way arguing that the move to marionette isn't a good thing - it definitely is - i'm just stating that it's not yet really usable for most people with Capybara.

[twalpole]

Update : Geckodriver 0.11.1 was released - current failures in the capybara test suite when using geckodriver 0.11.1, master branch of selenium-webdriver and firefox 50.0b5 - https://travis-ci.org/jnicklas/capybara/jobs/166497395 .
Summary:

1. I would hope the passing text to a system prompt modal will be fixed before Selenium 3 releases
2. mouseMoveTo errors are dependent on FF implementing the webdriver actions API and will prevent drag_drop/hover/right_click/double_click from working - timeframe unknown
3. send_keys/fill_in - The webdriver spec changed and basically rules out complex combinations of characters being sent through as an elements value (modifiers, etc) - This may just be a limitation that has to be accepted. It may be possible to workaround the limitation for send_keys once FF implements the webdriver actions API for keys - timeframe unknown

So if you have no need for system prompt modals, mouse interaction more than a standard left click, or modifier keys being sent to elements then feel free to give the latest versions a try

[titusfortner]

Because marionette is a w3c implementation and Geckodriver is still
considered alpha, the Selenium team does not plan to hold up the 3.0
release for Firefox issues. These are Selenium 4.0 concerns.

On Oct 10, 2016 12:23, "Thomas Walpole" notifications@github.com wrote:

Update : Geckodriver 0.11.1 was released - current failures in the
capybara test suite when using geckodriver 0.11.1, master branch of
selenium-webdriver and firefox 50.0b5 - https://travis-ci.org/
jnicklas/capybara/jobs/166497395 .

Summary:

1. I would hope the passing text to a system prompt modal will be fixed
   before Selenium 3 releases
2. mouseMoveTo errors are dependent on FF implementing the webdriver
   actions API and will prevent drag_drop/hover/right_click/double_click
   from working - timeframe unknown
3. send_keys/fill_in - The webdriver spec changed and basically rules out
   complex combinations of characters being sent through as an elements value
   (modifiers, etc) - This may just be a limitation that has to be accepted.
   It may be possible to workaround the limitation for send_keys once FF
   implements the webdriver actions API for keys - timeframe unknown

So if you have no need for system prompt modals, mouse interaction more
than a standard left click, or modifier keys being sent to elements then
feel free to give the latest versions a try

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1710 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAvb4f17rhVGzelqgYul5Xrvp4_VSn9mks5qypClgaJpZM4IznnW
.

[twalpole]

@titusfortner The only issue I stated I hoped would be fixed before Selenium 3 release is the system prompt modal text - which is defined in the w3c spec to pass the text as a "value" parameter - https://w3c.github.io/webdriver/webdriver-spec.html#dfn-send-alert-text . Appears selenium-webdriver doesn't do that. The other issues are all dependent on FF feature implementation - hence the "timeframe unknown"

[titusfortner]

Yes, that was a recent change in the spec to require everything to pass results in "value", but what is causing the text to alert failure is this: https://bugzilla.mozilla.org/show_bug.cgi?id=1255906
The new 'value' expectation also broke the window size / position when going through the standalone server for Firefox / geckodriver (that one still works with Ruby directly).

[twalpole]

@titusfortner While the mozilla bug is part of the issue, I believe the current issue being seen is that selenium-webdriver passes the text as a "text" parameter - https://github.com/SeleniumHQ/selenium/blob/master/rb/lib/selenium/webdriver/remote/w3c_bridge.rb#L174 - rather than a "value" parameter as the spec requires and it appears geckodriver expects. That then produces a "Missing 'value' parameter" error. Not sure whether changing that key from "text" to "value" would break the other drivers though.

[lucascaton]

selenium-webdriver 3 has been released!
https://rubygems.org/gems/selenium-webdriver/versions/3.0.0

[twalpole]

@lucascaton Yep - and if you want to use it with Chrome it should work fine. If you want to use it with new versions of Firefox (>= 48) then the limitations of no system prompt modals, mouse interaction more than a standard left click, or complex sets of characters including modifier keys being sent to elements still apply and there's nothing Capybara can do about it. Apparently you should be able to use it with Firefox 45ESR and 47.0.1 by specifying the marionette: false option in the capabilities

[dunban1]

I figure since the question of porting the old FireFox profile config to Marionette was brought up in this thread it couldn't hurt to follow up on what happened.

@gmcnaughton did you ever get the profile options working? I am having trouble determining if I am passing through the profile options correctly. The old way of configuring the browser using Capybara was the following:

useragent = "Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) " \
                     "AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 " \
                     "Mobile/13B143 Safari/601.1"
Capybara.register_driver :firefox do |app|
  profile = Selenium::WebDriver::Firefox::Profile.new
  profile['general.useragent.override'] = useragent
  Capybara::Selenium::Driver.new(app, :browser => :firefox, :profile => profile, :marionette => false)
end

now with marrionette set as true (as you know) the above code no longer works. So I am trying the following code:

Capybara.default_driver = :selenium
Capybara.register_driver :selenium do |app|
  useragent = "Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) " \
                "AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 " \
               "Mobile/13B143 Safari/601.1"
 profile = { 'general.useragent.override' => useragent }
 ffProfile = { "prefs" => profile }
 desired_caps = Selenium::WebDriver::Remote::Capabilities.firefox(
  {
     marionette: true,
      firefox_profile: ffProfile
    }
  )
  Capybara::Selenium::Driver.new(
    app,
    browser: :firefox,
    desired_capabilities: desired_caps
  )
end

I use a javascript in the page to see if the user agent has changed:

alert(navigator.userAgent.toLowerCase())

How did you encode the prefs to work like before with profile?

[twalpole]

@dunban1 Try something like

Capybara.register_driver :firefox do |app|
  profile = Selenium::WebDriver::Firefox::Profile.new
  profile['general.useragent.override'] = useragent
  desired_caps = Selenium::WebDriver::Remote::Capabilities.firefox(
    {
      marionette: true,
      firefox_options: { profile: profile.as_json.values.first }
    }
  )
  Capybara::Selenium::Driver.new(app, :browser => :firefox, desired_capabilities: desired_caps)
end

[gmcnaughton]

@dunban1 Good question! I haven't had a chance to try again. Personally we're still blocked by mozilla/geckodriver#93 (support for self-signed certificates), so we're still running our test suite under FF 45.

@twalpole Thanks for the sample code (and all your work on this)!

[dunban1]

Thank you for your responses @twalpole and @gmcnaughton

I get the following error with the code snippet:

rack-test requires a rack application, but none was given (ArgumentError)

when I add this line Capybara.default_driver = :selenium at the top. I get the following error instead:

undefined method `visit' for #<Proc:0x007ff5346f5de8> (NoMethodError)

The registration of the driver seems to be failing and it tries to use the fall back default which fails as well without proper configuration?

Perhaps there is another way that I should be looking at instead? For example, I have explored getting this to work is by referencing a pre-made FireFox profile. Is there a way to do this in Capybara with Marrionette?

I have been able to configure FireFox directly using selenium by linking the correct FireFox profile in the following code:

require 'selenium-webdriver'
profile = Selenium::WebDriver::Firefox::Profile.new '/Users/dunban1/Library/Application Support/Firefox/Profiles/s1xdknox.testprofile'
driver = Selenium::WebDriver.for :firefox, :profile => profile

But I have no idea how to register the driver once I configured it directly. I imagine Capybara is designed to instantiate the driver only inside the register_driver. If not is there a way to get this working should passing the firefox_options object ultimately not work?

[twalpole]

@dunban1 This is getting well off topic for the issue at hand. The README has a number of examples for calling register_driver, if those aren't enough please ask your howto questions on the mailing list as requested in the README.

[dunban1]

For anyone that finds this thread and was looking for an answer to how to use firefox_options to configure the FireFox profile. The conversation shall be continued here: https://groups.google.com/forum/#!topic/ruby-capybara/AyAcBX9-lIE

@gmcnaughton FYI if you're eventually going to stop using FF47 once Marrionette stabilizes I would imagine you will need this answer as well.

[twalpole]

I'm closing this issue since it's documented in the README and Capybara will be swapping to default to Chrome when Capybara 3 releases

[twalpole]

System prompt modals are fixed in the selenium-webdriver master branch so selenium-webdriver > 3.0.5 when released should have that functionality. That just leaves mouse interaction other than a standard left click, or modifier keys being sent to elements as the features that would limit use of current FF with selenium-webdriver

[rafiqdeen]

For Capybara try this steps After downloading geckodriver and Added it into Path:

Make sure your selenium webdriver is '3.0.0'

Capybara.default_driver = :selenium
Capybara.register_driver :selenium do |app|
browser = (ENV['BROWSER'] || 'firefox').to_sym
Capybara::Selenium::Driver.new(app, browser: browser, marionette: true)
end

[twalpole]

@rafiqdeen ??? Do you have a problem using that? Yes it will "work" but the features mentioned previously will not work (i.e.. no hover, no drag_to, no right_click, no double_click, etc, etc)

[jeremylynch]

Is there any update on this issue? Has anything got capybara + selenium + geckodriver working?

[titusfortner]

@mrjlynch - it works just fine unless you need more complicated keyboard/mouse interactions. That part of the spec was recently agreed on and Mozilla is working hard to finish that implementation.

Oh, and I thought I'd already released all of the important updates. I'll get selenium-webdriver 3.0.6 out soon with the system prompt modal fix; sorry for that delay.

[jeremylynch]

@titusfortner how do you install geckodriver?
Here is what I tried in vain:

1. Download Geckodriver from here: https://github.com/mozilla/geckodriver/releases
2. Unzip and run the executable (geckodriver)
3. Run export PATH=$PATH:downloads/geckodriver

I also tried renaming geckodriver to wired (and updating path accordingly)

[titusfortner]

Options:

1. Find a path that is already on PATH and stick geckodriver there
2. Do what you did but use a full path instead of a relative path
3. Do: gem install webdrivers and it will manage all of the drivers for you. It automatically makes sure that you are on the latest version.

[jeremylynch]

Thanks @titusfortner, got it working (thanks to option 3)
It seems very buggy.

unknown command: :mouse_move_to

Do you know when chrome will be supported?

[titusfortner]

Glad you got it working. What is buggy? Chrome is already supported it just isn't the default. The latest geckodriver finally supports mouse movements and Lucas has a pr to selenium with an implementation based on specs, but it isn't quite ready yet.

[twalpole]

@mrjlynch You need to specify :chrome for the browser, and install chromedriver. The only features that currently fail with Capybara and the newest version of selenium-webdriver with the newest version of FF are

1. system prompt modals - fixed in the master branch of selenium, will be in the next release
2. mouse interaction more than a standard left click - as @titusfortner mentioned it's coming alone
3. complex sets of characters including modifier keys being sent to elements - the spec has been finalized and as soon as geckodriver matches that spec it should be ok.
4. A recently discovered one that only affects Capybara master - equality with elements returned from evaluate_script - being worked on currently, would expect it to be fixed in the next selenium-webdriver release

[twalpole]

#drag_to and #hover support are fixed as of geckodriver 0.15 selenium-webdriver 3.3 and firefox 53b1 . - # right_click and #double_click will also probably work if the element is in the visible window (none of these appear to scroll elements out of view into view)