Functional Tests - Core Framework

General purpose automation framework for native mobile apps based on Appium.

Demo Project

NativeScript/functional-tests-demo

Features

• Manage emulators, simulators and physical devices.
  • Create emulators and simulators
  • Install/Uninstall applications
  • Start/Stop/Restart applications
  • Get console logs
  • Get screenshots
  • Simulate environment (like geo location for example)
  • [Android] Get performance stats (memory usage, startup time, application size)
  • All of the above managed via configs files
• Interaction with native apps on iOS Simulators, Android Emulators as well as physical Android and iOS devices.
  • Locate elements via accessibility of the mobile platforms
  • Cross-platform locators that work for both Android and iOS (and also with different backend automation)
  • Locate elements via images and OCR
  • Perform gestures
• Flexible image verification.
  • Verify your page looks as expected
  • Verify particular element on screen looks as expected
  • Allow setting tolerance as % and number of pixels
  • Logic to exclude status bar from comparison
  • Logic that helps when you have shadows or pixels that looks almost the same, but RGB is different
• Locating elements by image and OCR (based on Sikuli).
• Logs, Reports, Artifacts
  • Console log from devices, emulators and simulators
  • Appium server logs
  • Test execution logs
  • Screenshots
  • Junit style xml reports with meta info that helps generating html reports with images.

Supported Platforms

Host

Tests based on this framework can be executed from Windows, Linux and macOS machines.

However, there are some limitations:

• Testing on iOS Simulators and devices can be done only from macOS hosts.
• Android testing is possible from any hosts, however some features do not work on Windows:
  • Executing tests on multiple Android emulators/simulators simultaneously.
  • Create emulators feature.

Mobile

Tests based on this framework can be executed against Android Emulators, iOS Simulators, Android physical devices and iOS physical devices.

Supported versions:

• iOS: 8+
• Android: 4.2+

...and yes, we work on iOS 10 and Android 7.1 :)

Most likely we also work on Android < 4.2, but we can not confirm since we test only Andoird 4.2+

Prerequisites and Setup

Prerequisites

Prerequisites to run tests based on this framework.

• The latest Node LTS release
• JDK 1.8+
• Appium
• (Android only) Android SDK
• (iOS only) Latest Xcode

Environment Setup

Please read Environment Setup document.

NativeScript Functional Tests

This framework is used for testing all sample apps @NativeScript.

Tests are available in NativeScript/functional-tests repository.

Issues

All work-items (issues, features, questions) are handled in GitHub.

If you find valid issue, please log it (as good is the report as good is the chance to fix it).

Clarification:

We do not guarantee all issues will be fixed, but we will be happy to see pull requests.

Build and Contribute

Build

Gradle is used as build system.

Project use Gradle Wrapper so global installation of Gradle is not required.

Run tasks

• Unix-like: ./gradlew <task>
• Windows: gradlew.bat <task>

List of tasks:

• tasks - Displays the tasks runnable from root project.
• build - Assembles and tests this project.
• check - Runs all style checks in config/checkstyle/checkstyle.xml; outputs reports in ./build/reports/.
• fatJar - Assembles a jar archive with dependencies; outputs a .jar file in ./build/libs/.

Contribute

If you see an area for improvement, want to fix some git issues or just have an idea for a new feature, we will appreciate your pull requests

Just make sure check task passes successfully.