Chromedriver issue on Heroku CI

[hfjallemark]

Issue

I am getting a weird error when running Wallaby with ChromeDriver on Heroku CI:

test can sign in valid credentials (TestAppWeb.SignInTest)
test/test_app_web/features/sign_in_test.exs:6
** (Wallaby.JSError) There was an uncaught javascript error:
file:/// 0:8 Uncaught
code: |> sign_in_with("user1@account1.com", "pass1")
stacktrace:
  (wallaby) lib/wallaby/experimental/chrome/logger.ex:8: Wallaby.Experimental.Chrome.Logger.parse_log/1
  (elixir) lib/enum.ex:783: Enum."-each/2-lists^foreach/1-0-"/2
  (elixir) lib/enum.ex:783: Enum.each/2
  (wallaby) lib/wallaby/driver/log_checker.ex:12: Wallaby.Driver.LogChecker.check_logs!/2
  (wallaby) lib/wallaby/browser.ex:926: Wallaby.Browser.visit/2
  test/test_app_web/features/sign_in_test.exs:20: TestAppWeb.SignInTest.sign_in_with/3
  test/test_app_web/features/sign_in_test.exs:8: (test)

Locally everything works. I tried using both stable and beta channels for Chrome and ChromeDriver on Heroku. Also tried disabling all JavaScripts but still getting the same issue. Any ideas what I could be doing wrong?

Test Code & HTML

Test:

defmodule TestAppWeb.SignInTest do
  use TestAppWeb.FeatureCase, async: true

  import Wallaby.Query

  test "can sign in valid credentials", %{session: session} do
    session
    |> sign_in_with("user1@account1.com", "pass1")
    |> assert_has(css(".heading1", text: "Dashboard"))
  end

  defp sign_in_with(session, email, password) do
    session
    |> visit("/sign-in")
    |> fill_in(text_field("E-mail"), with: email)
    |> fill_in(text_field("Password"), with: password)
    |> click(button("Sign in"))
  end
end

Configuration:

config :wallaby, driver: Wallaby.Experimental.Chrome

[aaronrenner]

Hi @hfjallemark. 👋

This is a tough issue because Chrome isn't sending back enough information in its error log message to be useful. It sounds like Chrome is running into a javascript error in CI, but we may need to do more investigation to see if Wallaby can raise a more complete error message.

In the meantime, you could temporarily update your config to

config :wallaby, 
  driver: Wallaby.Experimental.Chrome,
  js_errors: false

to get CI to pass until you figure out what JS error might be occurring. I know this is less than ideal, but hopefully it will get you unstuck.

[hfjallemark]

Yep that’s what I did and then it worked. It’s weird that I only get this error in Heroku CI — is there any way to debug further what the issue may be?

[hfjallemark]

@aaronrenner any idea how to dig deeper into this? It's strange because it works perfectly fine locally.

[aaronrenner]

@hfjallemark I'll do a bit of research to see we can expose more lines of that log message.

[hfjallemark]

Hey @aaronrenner, were you able to expose some more log messages?

[aaronrenner]

@hfjallemark, I put some IO.inspect/1 statements in the wallaby code locally, but it turns out the "uncaught ... " message is all that chromedriver is returning. 😞

%{
  "level" => "SEVERE",
  "message" => "http://localhost:38277/errors.html 15:8 Uncaught",
  "source" => "javascript",
  "timestamp" => 1580875317342
}

Unfortunately, the WebDriver logging endpoint is not even part of the newer W3C webdriver spec, and fails when chromedriver is running in W3C mode. I wish I had better news here... Would it be possible to switch back to phantomjs temporarily to see if it would give you more info on the error?

[hfjallemark]

I’ll give that a try and see. Weird thing is that the same code works locally (macOS) with Chromedriver but not on Heroku (Ubuntu I believe).