[SEVERE] test_api v0.4.6 causes "Timed out waiting for Chrome to connect."

[felangel]

Running dart run build_runner test --fail-on-severe with test_api 0.4.5 consistently runs the tests within 2-3 seconds whereas test_api 0.4.6 consistently exceeds the 30s timeout with:

Failed to load "test/my_test.dart": Timed out waiting for Chrome to connect.
  package:test/src/runner/browser/browser_manager.dart 132:7  BrowserManager.start.<fn>
  dart:async/zone.dart 1420:47                                _rootRun
  dart:async/zone.dart 1328:19                                _CustomZone.run
  dart:async/future_impl.dart 865:34                          Future.timeout.<fn>
  dart:async/zone.dart 1420:47                                _rootRun
  dart:async/zone.dart 1328:19                                _CustomZone.run
  dart:async/zone.dart 1236:7                                 _CustomZone.runGuarded
  dart:async/zone.dart 1276:23                                _CustomZone.bindCallbackGuarded.<fn>
  dart:async/zone.dart 1428:13                                _rootRun
  dart:async/zone.dart 1328:19                                _CustomZone.run
  dart:async/zone.dart 1260:23                                _CustomZone.bindCallback.<fn>
  dart:async-patch/timer_patch.dart 18:15                     Timer._createTimer.<fn>
  dart:isolate-patch/timer_impl.dart 395:19                   _Timer._runTimers
  dart:isolate-patch/timer_impl.dart 426:5                    _Timer._handleMessage
  dart:isolate-patch/isolate_patch.dart 184:12                _RawReceivePortImpl._handleMessage

Adding:

test_api: ">=0.4.0 <0.4.6"

Resolves the timeouts.

Tests Failing with timeout: https://github.com/felangel/bloc/runs/3982800253
Commit with fix: felangel/bloc@fc591f6
Tests Passing: https://github.com/felangel/bloc/runs/3982912418

Related Issues:

• Internet Explorer tests time out on windows #1614
• Browser launch timeout needs to be increased or customizable #1610
• Provide more info on "Timed out waiting for <browser> to connect." #1616

I'm guessing the bug was introduced in #1604 but I'm not totally sure.

Originally posted by @felangel in #1610 (comment)

[gmpassos]

Just to highlight for investigation:

I speculate that this is where the bug was introduced:

https://github.com/dart-lang/test/blob/master/pkgs/test_core/lib/src/runner.dart#L261

I can't confirm it yet, but it's one of the significant changes from 0.4.5 to 0.4.6.

[ivoleitao]

I have the same exact error. All my browser tests cases started to fail

[d-markey]

Same here.

• Successfull tests https://github.com/d-markey/squadron/actions/runs/1376187261 after forcing test package version to 1.18.2
• Failed tests https://github.com/d-markey/squadron/actions/runs/1376256548 after retrying with test package version 1.19.1

[gmpassos]

Browser tests are definitely broken!

The title should have a "SEVERE" prefix.

[renggli]

Same here, all my browser tests are broken:

• https://github.com/petitparser/dart-petitparser/actions/runs/1376686101
• https://github.com/renggli/dart-data/actions/runs/1376630317
• https://github.com/renggli/dart-more/actions/runs/1376739069
• https://github.com/renggli/dart-rx/actions/runs/1376730260
• https://github.com/renggli/dart-statemachine/actions/runs/1376640900
• https://github.com/renggli/dart-xml/actions/runs/1376706479

[marandaneto]

same for https://github.com/getsentry/sentry-dart/runs/3984790347?check_suite_focus=true

dev_dependencies:
  test: ">=1.18.2 <1.19.0"

is a workaround

[athomas]

This also affects dart-lang packages, e.g.:
https://github.com/dart-lang/characters/actions/runs/1376610144

[jakemac53]

Ah yes, looks like I got hit by dart-lang/pub#3126 again :(. We need to commit a local .pubignore or something to prevent this in the future until a fix is out for the pub validator :(.

[jakemac53]

This should be fixed in test version 1.19.2.