Protractor synchronization fails if multiple versions of selenium-webdriver are in node_modules #3505
Comments
|
It did work once with
While I agree with that statement, this is not the core problem. Like I wrote in the bug report, protractor will just pass all tests. By the way, this will also happen if I don't even have the application running. I can literally see that a browser window opens with the chrome error page - still the tests pass. Please let me know what I can do for you to help your investigation |
|
As promised, here is the actual error message I get But I think that this is a rather small issue compared to blindly passing tests |
|
Would it be possible for you to post a an example repository with the issue? I started with mgechev/angular2-seed and set it up to use ts-node in the protractor config the way you have it, but I'm still not able to reproduce the problem. |
|
I will try. It's late now here, I will do so as soon as i wake up. By the way, I am not even using angular-seed. I am running on angular-cli |
|
I found a quick way for you to reproduce this bug. # terminal 1
# clone angular2-seed repo
$ git clone https://github.com/mgechev/angular2-seed.git
$ cd angular2-seed
# checkout the issue mentioned in mgechev/angular2-seed#1185 to be faulty
$ git checkout 9a8bf12eef0f28d310ecc0e568f6c5235422749e
# sidenote: At this point I opened package.json and changed
# "dependencies.zone.js": "^0.6.12"
# to
# "dependencies.zone.js": "0.6.12"
# because 0.6.17 is buggy. Don't know if this changes anything
$ npm install
# once after install
$ npm run webdriver-update
# then anytime to reproduce
$ npm run webdriver-start
# terminal 2
$ npm run serve.e2e
# terminal 3
$ npm run e2e
# terminal 2: Quit server using CTRL+C
$ ^C
# terminal 3:
$ npm run e2eNo webserver running, but tests will still pass. Now go to // Line 7
it('should have a title', () => {
// Line 8 [---]
expect(browser.getTitle()).toEqual('Welcome to angular2-seed!');
// Line 8 [+++]
expect(browser.getTitle()).toEqual('surely not the correct title!');
// Line 9
});
// ...You will see that the tests will still pass, no matter whether you started the webserver or not. If you then do the following: // Line 7
it('should have a title', () => {
// Line 8 [---]
expect(browser.getTitle()).toEqual('surely not the correct title!');
// Line 8 [+++]
expect(true).toBe(false);
// Line 9
});
// ...The test will fail (assuming that you've started the webserver, otherwise it didn't recompile). I hope this helps somehow. Please let me know if I can help any further or if you need anything. |
|
Update: Same bevaiour on Debian |
|
Thanks for the repro steps. I am able to reproduce this. I fixed the error handling, and this is the actual error: Also, it's worth noting that even though the tests are marked as passing (which is a bug), protractor exits with a non-zero exit code, indicating something went wrong. I'll try to figure out what's going on. |
|
That's strange, because if it would return a non-zero exit code, my CI tests wouldn't pass. Update: I have tested what you said, and for me, it does return 0 as exit code. I have a script in my $ npm run e2e config/protractor.conf.js && echo "previous exit code was 0"I get the following output: Spec started
app
✓ should load
# ......
✓ this test should definetly fail
Executed 6 of 6 specs SUCCESS in 0.125 sec.
[19:53:34] I/launcher - 0 instance(s) of WebDriver still running
[19:53:34] I/launcher - chrome #01 passed
previous exit code was 0 I would also get the default npm error stuff if the script that was ran using a |
|
It looks like this is related to the upgrade to selenium-webdriver 2.53.2, but it's a little hard to track down. If you have a minute, could you try installing protractor with 'npm install -g protractor' and launching your tests with that one? |
heathkit
changed the title
|
I can try tomorrow. It's already late. I did not have this issues with |
|
Ok, I've tracked it down. The issue has to do with npm install non-determinism. That seed project depends on gulp-protractor ^2.4.0, which depends on protractor 3.3.0, which in turn depends on selenium-webdriver 2.52.0. It also has protractor ^4.0.0, which depends on selenium-webdriver 2.53.0. The simplest repo for this would be to start with any angular project, and do You can use 'npm ls' to confirm that you have two versions of selenium-webdriver. I'll keep working on this to make sure we don't fall over in this case (or at least throw a helpful error message). In the meantime, if you upgrade to gulp-protractor ^3.0.0 you should be fine. |
|
Well, I am not even using gulp. I am using angular-cli. I just found that the guys at angular-seed faced the same issue and that was an easy way for me to show you how to reproduce this bug |
|
The problem isn't gulp exactly, it's just that your package.json contains both Basically, whatever the project, if you do |
|
I just checked my npm ls output. |
|
You may well be the first angular cli user to install Protractor 4.0.4 and file a bug on this. So thanks for that! I sent angular-cli a PR to update, and we'll have a fix for this issue in the next Protractor release. In the meantime, you can install Protractor 4 globally and run your tests with the protractor command instead of the npm script. I might have a better workaround after a bit more investigation. |
|
All right then! I will try it tomorrow :) Thank you so much for your quick and competent help. You are doing a great job! |
|
Here's the smoking gun: So jasminewd2 and protractor are using different instances of selenium webdriver. It turns out this was sort of known (see #2790 and angular/jasminewd #43). The fix is to make jasminewd take a webdriver instance from Protractor instead of doing require(), so we can guarantee they work with the same control flow. I'll be putting out a PR soon. Also, it's interesting to note that this is a jasmine-specific issue - Mocha users aren't affected. |
Fixex angular/protractor#3505, which was caused by Protractor and Jasminewd finding different webdriver instances through require(), and thus using different ControlFlows.
Fixes angular#3505 and 2790.
ghost
mentioned this issue
|
I can confirm that this issue is related to |
Fixex angular/protractor#3505, which was caused by Protractor and Jasminewd finding different webdriver instances through require(), and thus using different ControlFlows.
Fixes angular#3505 and angular#2790, which is caused by JasmineWd and Protractor using different controlflow instances
|
4.0.5 is released and includes the fix for this. |
|
Awesome, thank you so much! Will try tomorrow :) [Edit] Just tried it, and it works. Thank you very much for that quick fix |
Bug report
Setup
6.3.13.10.34.0.4 & 4.0.3Chrome (52.0.2743.116 (Official Build) (64-bit))Ubuntu 16.04Bug description
Whenever I run my e2e tests, my browser opens and immediately closes. Protractor will then claim, that all tests have passed, which just can't be. I implemented a test like this to test this:
Although this test should typically timeout, it will pass just fine in a split second.
If I add a spec like
expect(true).toBe(false), the test will fail by the way.I also had a
beforeEachset up, withbrowser.ignoreSynchronization = true. After removing this line, all tests will pass like before, but protractor will fail anyway with this error:(The error message above this sentence is copied from below referenced issue, since I currently can't access my own logs. I will insert my actual error message above this one as soon as I have time)
This also seems to be highly relevant, since it describes the very same behaviour: mgechev/angular-seed#1185
Sadly, this issue was closed without further investigation.
Since this issue seems to be quite rare, I assume that it is caused by some faulty dependency or so. Therefore I thought that maybe also my
package.jsonfile could help:Reproduction
See #3505 (comment)
If you need more information, please let me know. I will try to get you everything you need to reproduce and hopefully eventually fix this
The text was updated successfully, but these errors were encountered: