Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Appium tests on Sauce all failing #145

Open
janpio opened this issue May 10, 2019 · 7 comments
Open

CI: Appium tests on Sauce all failing #145

janpio opened this issue May 10, 2019 · 7 comments

Comments

@janpio
Copy link
Member Author

janpio commented May 10, 2019

I checked older, successful CI runs of cordova-plugin-camera, and it seems the Appium tests haven't been run for quite some time:

...
cordova-paramedic: tests have been completed
cordova-paramedic: Tests finished
Start running Appium tests...
Running Appium tests on Sauce Labs
Couldn't find Appium tests, skipping...
Completed tests at 5:21:17 PM
Getting saucelabs jobs details...
...

https://travis-ci.org/apache/cordova-plugin-camera/builds/528500727?utm_source=github_status&utm_medium=notification

Somehow "Couldn't find Appium tests, skipping..." got fixed in the last 5 days.

@janpio
Copy link
Member Author

janpio commented May 10, 2019

Could find a successful CI run with Appium tests actually being executed for the contacts plugin:
https://travis-ci.org/apache/cordova-plugin-contacts/jobs/491749438
3 months ago.


On further looking:

A Android run for the camera plugin that succeeded:
https://travis-ci.org/apache/cordova-plugin-camera/jobs/514613162
~1 month ago

Before that, ~2 months ago, we have successful Appium runs for iOS and Android:
https://travis-ci.org/apache/cordova-plugin-camera/jobs/507356980
https://travis-ci.org/apache/cordova-plugin-camera/jobs/507356981
https://travis-ci.org/apache/cordova-plugin-camera/jobs/507356982
https://travis-ci.org/apache/cordova-plugin-camera/jobs/507356983
https://travis-ci.org/apache/cordova-plugin-camera/jobs/507356984


So the question now is: What changed between these successful runs and now?

@janpio
Copy link
Member Author

janpio commented May 10, 2019

  • We changed the OS versions the tests are run on (all new iOS versions as old simulators are gone)
  • We added an environment for the tests to run on (local instead of just saucelabs)
  • We changed the whole travis.yml structure (but this shouldn't influence things)
  • We changed the OS version configuration details (saucelabs parameters like appiumVersion etc)
  • We rewrote good parts of paramedic, that is used to actually run the tests

@janpio
Copy link
Member Author

janpio commented May 10, 2019

Similar OS/version combinations being run then and now are: Andorid 4.4, 5.1, 6.0, 7.0.
For these combinations debugging makes most sense, as theoretically it should still work.

@janpio
Copy link
Member Author

janpio commented May 10, 2019

Looking into the failures we are getting a bit, they look different at first, but turn out to be the same thing:

iOS:

- Failed: Failed to start a session

Android:

- ReferenceError: lastFailureReason is not defined

Turns out the Android error comes from this line:
https://github.com/apache/cordova-plugin-camera/blob/295e928784e2a9785982fbf15b05215a317774df/appium-tests/android/android.spec.js#L353

Here lastFailureReason is indeed undefined, it doesn't appear anywhere else. Otherwise this would also output:

Failed to start a session

Why though... no idea yet.

@janpio
Copy link
Member Author

janpio commented Jun 19, 2019

@janpio
Copy link
Member Author

janpio commented Jun 20, 2019

As a temporary workaround, the Appium tests were removed from camera. A new PR will add them back: apache/cordova-plugin-camera#469

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant