Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve E2E testing for Android by retrying single tests #48260

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
+19 −12
Draft

Improve E2E testing for Android by retrying single tests #48260

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/actions/maestro-android/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ runs:
steps:
- name: Installing Maestro
shell: bash
run: export MAESTRO_VERSION=1.36.0; curl -Ls "https://get.maestro.mobile.dev" | bash
run: export MAESTRO_VERSION=1.39.5; curl -Ls "https://get.maestro.mobile.dev" | bash
- name: Set up JDK 17
if: ${{ inputs.install-java == 'true' }}
uses: actions/setup-java@v4
Expand All @@ -61,8 +61,10 @@ runs:
with:
api-level: 24
arch: x86
ram-size: '4096M'
ram-size: '8192M'
heap-size: '2048M'
disk-size: '10G'
cores: '4'
disable-animations: false
avd-name: e2e_emulator
script: node .github/workflow-scripts/maestro-android.js ${{ inputs.app-path }} ${{ inputs.app-id }} ${{ inputs.maestro-flow }} ${{ inputs.flavor }} ${{ inputs.working-directory }}
Expand Down
2 changes: 1 addition & 1 deletion .github/actions/maestro-ios/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ runs:
steps:
- name: Installing Maestro
shell: bash
run: export MAESTRO_VERSION=1.36.0; curl -Ls "https://get.maestro.mobile.dev" | bash
run: export MAESTRO_VERSION=1.39.5; curl -Ls "https://get.maestro.mobile.dev" | bash
- name: Installing Maestro dependencies
shell: bash
run: |
Expand Down
6 changes: 3 additions & 3 deletions .github/workflow-scripts/maestro-android.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,10 +57,10 @@ async function main() {
});
metroProcess.unref();
console.info(`- Metro PID: ${metroProcess.pid}`);
}

console.info('Wait For Metro to Start');
await sleep(5000);
console.info('Wait For Metro to Start');
await sleep(5000);
}

console.info('Start the app');
childProcess.execSync(`adb shell monkey -p ${APP_ID} 1`, {stdio: 'ignore'});
Expand Down
7 changes: 4 additions & 3 deletions .github/workflows/test-all.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -432,13 +432,13 @@ jobs:
uses: ./.github/actions/build-android
with:
release-type: ${{ needs.set_release_type.outputs.RELEASE_TYPE }}
run-e2e-tests: ${{ github.ref == 'refs/heads/main' || contains(github.ref, 'stable') || inputs.run-e2e-tests }}
run-e2e-tests: true #${{ github.ref == 'refs/heads/main' || contains(github.ref, 'stable') || inputs.run-e2e-tests }}

test_e2e_android_rntester:
# Temporarily disable RNTester tests on Android as they are quite flaky and they make CI always red
# if: ${{ github.ref == 'refs/heads/main' || contains(github.ref, 'stable') || inputs.run-e2e-tests }}
if: ${{ contains(github.ref, 'stable') || inputs.run-e2e-tests }}
runs-on: ubuntu-latest
# if: ${{ contains(github.ref, 'stable') || inputs.run-e2e-tests }}
runs-on: 4-core-ubuntu
needs: [build_android]
strategy:
fail-fast: false
Expand All @@ -461,6 +461,7 @@ jobs:
run: ls -lR ./packages/rn-tester/android/app/build/outputs/apk/${{ matrix.jsengine }}/${{ matrix.flavor }}/
- name: Run E2E Tests
uses: ./.github/actions/maestro-android
timeout-minutes: 120
with:
app-path: ./packages/rn-tester/android/app/build/outputs/apk/${{ matrix.jsengine }}/${{ matrix.flavor }}/app-${{ matrix.jsengine }}-x86-${{ matrix.flavor }}.apk
app-id: com.facebook.react.uiapp
Expand Down
4 changes: 2 additions & 2 deletions packages/rn-tester/.maestro/flatlist.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ appId: ${APP_ID} # iOS: com.meta.RNTester.localDevelopment | Android: com.facebo
id: "Flatlist"
- tapOn:
id: "Basic"
# - assertVisible:
# id: "item_550"
- assertVisible:
id: "item_550"
# - repeat:
# while:
# notVisible:
Expand Down
6 changes: 5 additions & 1 deletion packages/rn-tester/.maestro/text.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,18 @@ appId: ${APP_ID} # iOS: com.meta.RNTester.localDevelopment | Android: com.facebo
id: "Text"
direction: DOWN
speed: 60
timeout: 60000
visibilityPercentage: 100
- tapOn:
id: "Text"
- assertVisible: "Text"
- scrollUntilVisible:
element:
id: "background-border-width"
direction: DOWN
speed: 10
speed: 20
visibilityPercentage: 100
timeout: 60000
- assertVisible: "Text with background color only"
- assertVisible: "Text with background color and uniform borderRadii"
- assertVisible: "Text with background color and non-uniform borders"
Expand Down
Loading