-
Notifications
You must be signed in to change notification settings - Fork 24.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
add RNTester-E2E: tests for iOS and Android via Appium, WDIO and Jest (…
…#36267) Summary: The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release. This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests! This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E). ### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684) ### Extra notes We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading. ### Things that will be handled as follow up work * upgrade to WDIO 8 * [an automated yarn run-e2e-test script](#36267 (comment)) * [a small refactoring into a js script](#36267 (comment)) ## Changelog: [INTERNAL] [ADDED] - Added first working configuration for e2e testing Pull Request resolved: #36267 Test Plan: You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md). CircleCI jobs will be added. Reviewed By: cipolleschi Differential Revision: D46225747 Pulled By: cortinico fbshipit-source-id: e5428c439afd84ed3dd219d6ada538263120d7c6
- Loading branch information
1 parent
74a1822
commit 3c196fb
Showing
16 changed files
with
3,923 additions
and
897 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,124 @@ | ||
| # RNTester E2E folder | ||
|
|
||
| In this folder we have a the setup for running E2E testing in RNTester via the usage of [Appium](https://appium.io/) and [WebDriverIO](https://webdriver.io/) and [Jest](https://jestjs.io/). | ||
|
|
||
| - [Setting up locally](#setting-up-locally) | ||
| - [(one-off) Setting up Appium](#one-off-setting-up-appium) | ||
| - [Building RNTester app](#building-rntester-app) | ||
| - [Building for iOS](#building-for-ios) | ||
| - [Building for Android](#building-for-android) | ||
| - [Setting up the RNTester E2E folder](#setting-up-the-rntester-e2e-folder) | ||
| - [Testing the RNTester app E2E](#testing-the-rntester-app-e2e) | ||
| - [Adding new tests (and project structure)](#adding-new-tests-and-project-structure) | ||
|
|
||
| ## Setting up locally | ||
|
|
||
| ### (one-off) Setting up Appium | ||
|
|
||
| The first step you need to do is to ensure to install the tooling: | ||
|
|
||
| ```bash | ||
| npm install appium@2.0.0 -g | ||
| appium driver install uiautomator2 | ||
| appium driver install xcuitest | ||
| ``` | ||
|
|
||
| > More details about drivers in Appium [here](https://appium.github.io/appium/docs/en/2.0/guides/managing-exts/) and [here](https://appium.github.io/appium/docs/en/2.0/quickstart/uiauto2-driver/) | ||
| You should not need to run install commands for drivers separately more than once, even if you bump the dep in package.json. | ||
|
|
||
| ### Building RNTester app | ||
|
|
||
| Building manually *.app* and *.apk* is required to run automation tests on local environment. | ||
|
|
||
| 0. *(optional)* If you previously built RNTester, you may need to clean up build files and Pods: | ||
|
|
||
| ```bash | ||
| yarn test-e2e-local-clean && yarn install | ||
| ``` | ||
|
|
||
| 1. Step 1: install packages for the repository, then navigate in the rn-tester folder | ||
|
|
||
| ```bash | ||
| cd react-native | ||
| yarn install | ||
| cd packages/rn-tester | ||
| ``` | ||
|
|
||
| Now, depending on the platform, there are some specific steps | ||
|
|
||
| #### Building for iOS | ||
|
|
||
| 0. Make sure you have Bundler `gem install bundler` - we use it ensure installing the right version of CocoaPods locally. | ||
| 1. Install Bundler and CocoaPods dependencies: `bundle install` then `bundle exec pod install` or `yarn setup-ios-hermes` for RNTester with Hermes. In order to use JSC instead of Hermes engine, run: `USE_HERMES=0 bundle exec pod install` or `setup-ios-jsc` instead. | ||
| 2. You can build app with React Native CLI or manually with Xcode: | ||
| 1. To build with React Native CLI: | ||
| 1. Run `npx react-native build-ios --mode Debug --scheme RNTester --buildFolder /path/to/build-folder`, replace `/path/to/build-folder` with the real path. | ||
| 2. Copy the built app using `mv` - `mv /path/to/build-folder/Build/Products/Debug-iphonesimulator/RNTester.app ~/react-native/packages/rn-tester-e2e/apps` or manually. | ||
| 2. To build with Xcode, open the generated `RNTester.xcworkspace` and build. | ||
| 1. Find the **RNTester.app** in `~/Library/Developer/Xcode/DerivedData/RNTesterPods-{id}/Build/Products/Debug-iphonesimulator` | ||
| 2. Copy the app to the following directory `~/react-native/packages/rn-tester-e2e/apps`. | ||
| 3. Change its name to: `rn-tester.app` | ||
|
|
||
| #### Building for Android | ||
|
|
||
| 0. You'll need to have all the [prerequisites](https://reactnative.dev/contributing/how-to-build-from-source#prerequisites) (SDK, NDK) for Building React Native installed. | ||
| 1. Start an Android emulator. | ||
| 2. Build the app via | ||
| ```bash | ||
| # In order to not use Hermes engine, run `yarn install-android-jsc` instead. | ||
| yarn install-android-hermes | ||
| yarn start | ||
| ``` | ||
| *Note: Building for the first time can take a while.* | ||
| 3. Find the **app-*-debug.apk** in `~/react-native/packages/rn-tester/android/app/build/outputs/apk/hermes/debug` | ||
| 4. Copy the app `app-*-debug.apk` to the following directory `~/react-native/packages/rn-tester-e2e/apps` | ||
| 5. Change its name to: `rn-tester.apk` | ||
| ### Setting up the RNTester E2E folder | ||
| In `react-native/packages/rn-tester-e2e` open the following file | ||
| ```bash | ||
| /react-native/packages/rn-tester-e2e/e2e-config.js | ||
| ``` | ||
| And modify lines L24->L39 to reflect your local setup configuration (ex. `platformVersion`, `deviceName`). Make sure to **not** commit this change if you send a PR to add tests. | ||
| ## Testing the RNTester app E2E | ||
| After you have done all the above correctly, and you have the Android/iOS apps in the `rn-tester-e2e/apps` folder, in a dedicated terminal window, run: | ||
| ```bash | ||
| appium --base-path /wd/hub | ||
| ``` | ||
| This will start the Appium server - you will need this to keep running. | ||
| Then open a second terminal window and start the Metro terminal from the `packages/rn-tester` folder, via `yarn start --reset-cache`. This terminal window also needs to keep running. | ||
| Now, make sure that the iOS simulator/the Android emulator is up and running. | ||
| Finally, you can open a third terminal window and run: | ||
| ```bash | ||
| yarn test-android-e2e # for android | ||
| yarn test-ios-e2e # for ios | ||
| ``` | ||
| Now you should see the RNTester app being open, and the defined test being run. | ||
| ## Adding new tests (and project structure) | ||
| This project has 2 main folders: | ||
| - `apps`, where, as you have seen above, the iOS/Android RNTester apps need to be put so that appium will pick them and install in the emulator/simulator consistently. | ||
| - `tests`, where the tests and referencing files all live. The substructure is as follows: | ||
| - `screens` -> in this folder, you will find `*.screen.js` files, where each file represents a navigation screen for RNTester. So there are 3 root ones (`apis`, `bookmarks`, `components`) and then for subscreens, there's a folder with the same name - currently, that's only `components` that contains `buttonComponent.screen.js`. The content of these files is what was earlier mentioned as "references": they provide an easy way to define all elements present in said screen, so that they can be used for tests. | ||
| - `specs` -> this folder follows a similar 1:1 mapping to the RNTester screens, but for the tests: for each screen (or subscreen) there's a dedicated `*.test.js` file (such as `buttonComponentScreen.test.js`). Ideally, in this file the Jest tests are standard, leveraging the `*.screen.js` counterpart for the details of defining how Appium/WDIO can reach those elements on screen. | ||
|
|
||
| When adding a new test, please ensure that you follow this pattern and add the relevant test in the right screen file / screen test file. Use the files mentioned above as examples. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Put the *.app and *.apk files here. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| /** | ||
| * Copyright (c) Meta Platforms, Inc. and affiliates. | ||
| * | ||
| * This source code is licensed under the MIT license found in the | ||
| * LICENSE file in the root directory of this source tree. | ||
| * | ||
| * @flow | ||
| * @format | ||
| */ | ||
|
|
||
| module.exports = { | ||
| presets: [['@babel/preset-env', {targets: {node: 'current'}}]], | ||
| plugins: ['@babel/plugin-transform-flow-strip-types'], | ||
| }; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| /** | ||
| * Copyright (c) Meta Platforms, Inc. and affiliates. | ||
| * | ||
| * This source code is licensed under the MIT license found in the | ||
| * LICENSE file in the root directory of this source tree. | ||
| * | ||
| * @flow | ||
| * @format | ||
| */ | ||
|
|
||
| const path = require('path'); | ||
|
|
||
| type Capabilities = { | ||
| platformName: 'Android' | 'iOS', | ||
| 'appium:platformVersion': string, | ||
| 'appium:deviceName': string, | ||
| 'appium:app': string, | ||
| 'appium:automationName': 'UiAutomator2' | 'XCUITest', | ||
| 'appium:newCommandTimeout'?: number, | ||
| }; | ||
|
|
||
| let capabilities: Capabilities; | ||
|
|
||
| const android = { | ||
| platformName: 'Android', | ||
| 'appium:platformVersion': '13.0', | ||
| 'appium:deviceName': 'Android Emulator', | ||
| 'appium:app': path.join(process.cwd(), '/apps/rn-tester.apk'), | ||
| 'appium:automationName': 'UiAutomator2', | ||
| 'appium:newCommandTimeout': 240, | ||
| }; | ||
|
|
||
| const ios = { | ||
| platformName: 'iOS', | ||
| 'appium:platformVersion': '16.4', | ||
| 'appium:deviceName': 'iPhone 14', | ||
| 'appium:automationName': 'XCUITest', | ||
| 'appium:app': path.join(process.cwd(), '/apps/rn-tester.app'), | ||
| }; | ||
|
|
||
| // check that E2E_DEVICE exists, is a string and its either "ios" or "android" | ||
| if (!process.env.E2E_DEVICE) { | ||
| throw new Error('E2E_DEVICE environment variable is not defined'); | ||
| } else if (typeof process.env.E2E_DEVICE !== 'string') { | ||
| throw new Error('E2E_DEVICE environment variable is not a string'); | ||
| } else if ( | ||
| process.env.E2E_DEVICE !== 'ios' && | ||
| process.env.E2E_DEVICE !== 'android' | ||
| ) { | ||
| throw new Error('E2E_DEVICE environment variable is not "ios" or "android"'); | ||
| } | ||
|
|
||
| if (process.env.E2E_DEVICE === 'android') { | ||
| capabilities = android; | ||
| } | ||
|
|
||
| if (process.env.E2E_DEVICE === 'ios') { | ||
| capabilities = ios; | ||
| } | ||
|
|
||
| export default capabilities; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| /** | ||
| * Copyright (c) Meta Platforms, Inc. and affiliates. | ||
| * | ||
| * This source code is licensed under the MIT license found in the | ||
| * LICENSE file in the root directory of this source tree. | ||
| * | ||
| * @flow | ||
| * @format | ||
| */ | ||
|
|
||
| module.exports = { | ||
| testTimeout: 120000, | ||
| bail: 0, | ||
| setupFilesAfterEnv: ['./jest.setup.js'], | ||
| testMatch: ['**/specs/**/*.js'], | ||
| maxWorkers: 1, | ||
| }; |
Oops, something went wrong.