Issue with ngAnimate when using Selenium-TestNG for testing our application #13619

zann12 · 2015-12-23T10:49:24Z

Hello all,

I am currently facing to a random failure behavior on my Selenium which only occurs when my application has the ngAnimate module as dependency: angular.module('myApp', [ 'ngAnimate' ]);.

The angular-animate library seems to be too slow when Selenium is testing our application. We don't reproduce this behavior when we use our application manually.
When the input text failed with the ngMaxlength constraint, we can observe the ng-valid-maxlength class is removed and the ng-invalid-maxlength class is added but the ng-invalid class is not pushed in the dom and the ng-valid class is kept in the DOM instead of to be removed.

You will find below some parts of sources code, but you also can find the all projects (simple angular app and selenium/TestNG test) here:

Test with Selenium & TestNG: https://github.com/zann12/angular-animate-with-selenium.git
The application: https://github.com/zann12/angular-animate-with-simple-app.git

Libs:

angular: 1.3.13
angular-animate: 1.3.13

index.html

<!DOCTYPE html>
<html lang="en" class="no-js"> 
<head> 
    <link rel="stylesheet" href="app.css">

    <!-- bower:js -->
    <script src="bower_components/jquery/dist/jquery.js"></script>
    <script src="bower_components/angular/angular.js"></script>
    <script src="bower_components/angular-animate/angular-animate.js"></script>
    <script src="app.js"></script>
    <!-- endinject -->

</head>
<body ng-app="myApp">

<div ng-init="model.textvalue='Modify me!';model.maxlength=15">
    <label>A simple Input with two constraints directives (ng-minlength and ng-maxlength)</label>
    <br>
    <input
            id="input#id1"
            ng-model="model.textvalue"
            class="text-value"
            ng-minlength="4"
            ng-maxlength="model.maxlength">
    <br><br>
    <input
            id="input#id2"
            ng-model="model.maxlength">
</div>

</body>
</html>

Selenium Test

@Test(priority = 1, invocationCount = 100)
public void validatorsOnQuickTest() {
this.singleDriver.get("http://localhost:10000/Validator_bug/app/index.html#/view1");
    WebElement t = this.singleDriver.findElement(By.id("input#id1"));
    WebElement tMax = this.singleDriver.findElement(By.id("input#id2"));
    QuickTest.clearInputValue(tMax);
    tMax.sendKeys("10");
    QuickTest.clearInputValue(t);
    t.sendKeys("aaaaaaaaaaa");
    Assert.assertTrue(this.hasError(t));  // "aaaaaaaaaaa".length > maxLength (10) => An error is expected
}

private boolean hasError(WebElement element) {
    /*
     * An error must be visible on an invalid component only when:
     * #1. The 'ng-invalid' CSS class is set by angular
     * AND
     * #2. the model is not pristine (because in our case we don't need to show
     *      the error if there is no action from the user). 
     */
    String classes = element.getAttribute("class");
    if (classes == null || classes.length() == 0) {
      return false;
    }
    // split classes to avoid conflict with names!
    String[] cssClasses = classes.split(" ");

    if (ArrayUtils.contains(cssClasses, "ng-invalid") // #1.
        && !ArrayUtils.contains(cssClasses, "ng-pristine") // #3.
    ) {
      return true;
    }
    return false;
}

Selenium / WITH ngAnimate / with an input which contains the following two constraints: ngMaxlength & ngMinlength

App.js

angular.module('myApp', [ 'ngAnimate' ]);

Part of Index.html

<input
        id="input#id1"
        ng-model="model.textvalue"
        class="text-value"
        ng-minlength="4"
        ng-maxlength="model.maxlength">

Selenium Test Result

Default test
Tests run: 100, Failures: 47, Skips: 0

Selenium / WITH ngAnimate / with an input which contains only one constraint: ngMaxlength

App.js

angular.module('myApp', [ 'ngAnimate' ]);

Part of Index.html

<input
        id="input#id1"
        ng-model="model.textvalue"
        class="text-value"
        ng-maxlength="model.maxlength">

Selenium Test Result

Default test
Tests run: 100, Failures: 6, Skips: 0

Selenium / WITHOUT ngAnimate / with an input which contains the following two constraints: ngMaxlength & ngMinlength

App.js

angular.module('myApp', []);

Part of Index.html

<input
        id="input#id1"
        ng-model="model.textvalue"
        class="text-value"
        ng-minlength="4"
        ng-maxlength="model.maxlength">

Selenium Test Result

Default test
Tests run: 100, Failures: 0, Skips: 0

Selenium / WITHOUT ngAnimate / with an input which contains only one constraint: ngMaxlength

App.js

angular.module('myApp', []);

Part of Index.html

<input
        id="input#id1"
        ng-model="model.textvalue"
        class="text-value"
        ng-maxlength="model.maxlength">

Selenium Test Result

Default test
Tests run: 100, Failures: 0, Skips: 0

Resume

I use Selenium to test an input which contains always the following directive attribute: ngMaxlength.

ngAnimate	ngMinlength	Test Result
Disabled	Not Used	SUCCESS
Disabled	Used	SUCCESS
Enabled	Not Used	FAILED (6 failures)
Enabled	Used	FAILED (47 failures)

When I use both the ngMaxlength directive attribute and the ngMinlength directive attribute, the number of failures is upper than if I do not used the ngMinlength directive attribute.

Why the angular-animate fails with our application, with our tests?
Can someone give me some light on why we have this behavior please?

Thank you very much.

The text was updated successfully, but these errors were encountered:

Narretz · 2015-12-23T11:59:41Z

Thanks for the report. Can you please test with the latest 1.3.20? Or even better 1.4.9? 1.3.x only gets critical fixes.

zann12 · 2015-12-23T14:45:22Z

Thank you for your prompt message. I test with angular 1.3.20 and angular-animate 1.3.20, and also with angular 1.4.8 and angular-animate 1.4.8. Unfortunately the same bad behavior occurs.

zann12 · 2016-01-04T15:16:47Z

Hello Narretz. Have you reproduced this behavior? What is the status for this issue? Thank you.

Narretz · 2016-01-04T15:26:19Z

I haven't tested this yet. So the app on github shows this behavior? I can clone and run it and the error will appear?

zann12 · 2016-01-04T15:27:20Z

Yes I hope so.

zann12 · 2016-01-14T08:57:56Z

Hi Narretz, What about this issue? Our team is currently blocked with this issue and we don't know how to override it.
Regards.

frfancha · 2016-01-14T09:59:30Z

We have some (unrelated to this specfic problem) "speed" problem with selenium.
All our inputs to the browser from Selenium are followed by a sleep of 50ms and that solves everything for us.

Narretz · 2016-01-14T11:36:20Z

@zann12 I'll take a look right now. But don't forget, this is not a support forum, this tracker is only for bugs and feature requests. Sorry if I gave off a different impression.

Ok, regarding your project - I see two github repos, but I have actually no idea how to use them together. So unless you can give me these instructions, I can't test this.

What you can try is to disable animations in your selenium tests by adding a test-specific config. For example, with protractor it would look like this: http://stackoverflow.com/questions/26584451/how-to-disable-animations-in-protractor-for-angular-js-appliction

In any case, end to end tests for angular apps are much easier with protractor - you should consider using that over plain Selenium.

Narretz · 2016-01-21T21:49:19Z

Sorry, I can't help without clear instructions on the project. Closing. Feel free to reopen if you have more info for me.

Narretz added component: ngAnimate needs: more info labels Dec 23, 2015

Narretz added this to the Purgatory milestone Dec 23, 2015

Narretz added the branch: 1.3.x label Dec 23, 2015

Narretz added the branch: 1.4.x label Dec 31, 2015

Narretz modified the milestones: 1.4.x, Purgatory Dec 31, 2015

Narretz self-assigned this Dec 31, 2015

Narretz added needs: investigation and removed needs: more info labels Jan 4, 2016

Narretz closed this as completed Jan 21, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue with ngAnimate when using Selenium-TestNG for testing our application #13619

Issue with ngAnimate when using Selenium-TestNG for testing our application #13619

zann12 commented Dec 23, 2015

Narretz commented Dec 23, 2015

zann12 commented Dec 23, 2015

zann12 commented Jan 4, 2016

Narretz commented Jan 4, 2016

zann12 commented Jan 4, 2016

zann12 commented Jan 14, 2016

frfancha commented Jan 14, 2016

Narretz commented Jan 14, 2016

Narretz commented Jan 21, 2016

Issue with ngAnimate when using Selenium-TestNG for testing our application #13619

Issue with ngAnimate when using Selenium-TestNG for testing our application #13619

Comments

zann12 commented Dec 23, 2015

Libs:

index.html

Selenium Test

Selenium / WITH ngAnimate / with an input which contains the following two constraints: ngMaxlength & ngMinlength

App.js

Part of Index.html

Selenium Test Result

Selenium / WITH ngAnimate / with an input which contains only one constraint: ngMaxlength

App.js

Part of Index.html

Selenium Test Result

Selenium / WITHOUT ngAnimate / with an input which contains the following two constraints: ngMaxlength & ngMinlength

App.js

Part of Index.html

Selenium Test Result

Selenium / WITHOUT ngAnimate / with an input which contains only one constraint: ngMaxlength

App.js

Part of Index.html

Selenium Test Result

Resume

Narretz commented Dec 23, 2015

zann12 commented Dec 23, 2015

zann12 commented Jan 4, 2016

Narretz commented Jan 4, 2016

zann12 commented Jan 4, 2016

zann12 commented Jan 14, 2016

frfancha commented Jan 14, 2016

Narretz commented Jan 14, 2016

Narretz commented Jan 21, 2016