Skip to content

Conversation

@christopherferreira9
Copy link
Contributor

@christopherferreira9 christopherferreira9 commented Sep 16, 2025

Description

  • Replace bash e2e runner with Node ESM.
  • Add per-split duplication for new/modified specs to detect flakiness (base + 2 retries).
  • Remove Octokit dependency; use native fetch + GitHub GraphQL.

What changed

  • run-e2e-tags-gha.mjs: runs tests based on the Test Suite Tag passed on + ensures new and modified tests run 3 times to avoid the introduction of flaky tests

Behavior

  • If skip-e2e-quality-gate label present: no test duplication.
  • If no PR or no changed specs: no duplication.
  • duplication only for ADDED/MODIFIED specs selected in the current split.

Testing
A new dummy test was added in order to showcase the spec file duplication.
Example logs from this run (no label applied and a new added test):

🚀 Running wallet-platform-android-smoke-2 tests on android
  Using TEST_SUITE_TAG: SmokeWalletPlatform
  🚀 Starting E2E tests...
  Running split 2 of 2
  Searching for tests with pattern: SmokeWalletPlatform
  Running split 2 of 2
  📋 Found 9 matching test files in total
  🧪 Duplicated for flakiness check: e2e/specs/multisrp/dummy-flakyness-retry-1.spec.ts
  🧪 Duplicated for flakiness check: e2e/specs/multisrp/dummy-flakyness-retry-2.spec.ts
  🧪 After duplication (per split), total selected files: 6
  🔍 Running 6 tests for split 2 of 2:
    - e2e/specs/multisrp/dummy-flakyness.spec.ts                 --> Original test file
    - e2e/specs/multisrp/dummy-flakyness-retry-1.spec.ts    --> Retry 1
    - e2e/specs/multisrp/dummy-flakyness-retry-2.spec.ts   --> Retry 2
    - e2e/specs/multisrp/export-srp-from-account-actions.spec.ts
    - e2e/specs/multisrp/export-srp-from-settings.spec.ts
    - e2e/specs/wallet/incoming-transactions.spec.ts
  🚀 Running matching tests for split 2...
  🤖 Running Android tests on GitHub Actions
  yarn run v1.22.22
  $ IS_TEST='true' NODE_OPTIONS='--experimental-vm-modules' detox test -c android.github_ci.release --headless --record-logs all e2e/specs/multisrp/dummy-flakyness.spec.ts e2e/specs/multisrp/dummy-flakyness-retry-1.spec.ts e2e/specs/multisrp/dummy-flakyness-retry-2.spec.ts e2e/specs/multisrp/export-srp-from-account-actions.spec.ts e2e/specs/multisrp/export-srp-from-settings.spec.ts e2e/specs/wallet/incoming-transactions.spec.ts

Example logs from this run (with label applied):

  🚀 Running wallet-platform-android-smoke-2 tests on android
  Using TEST_SUITE_TAG: SmokeWalletPlatform
  🚀 Starting E2E tests...
  Running split 2 of 2
  Searching for tests with pattern: SmokeWalletPlatform
  Running split 2 of 2
  📋 Found 9 matching test files in total
  ⏭️  skip-e2e-quality-gate detected; skipping flaky duplication
  🔍 Running 4 tests for split 2 of 2:
    - e2e/specs/multisrp/dummy-flakyness.spec.ts
    - e2e/specs/multisrp/export-srp-from-account-actions.spec.ts
    - e2e/specs/multisrp/export-srp-from-settings.spec.ts
    - e2e/specs/wallet/incoming-transactions.spec.ts
  🚀 Running matching tests for split 2...
  🤖 Running Android tests on GitHub Actions
  yarn run v1.22.22
  $ IS_TEST='true' NODE_OPTIONS='--experimental-vm-modules' detox test -c android.github_ci.release --headless --record-logs all e2e/specs/multisrp/dummy-flakyness.spec.ts e2e/specs/multisrp/export-srp-from-account-actions.spec.ts e2e/specs/multisrp/export-srp-from-settings.spec.ts e2e

Changelog

CHANGELOG entry:

Related issues

Fixes:

Manual testing steps

Feature: my feature name

  Scenario: user [verb for user action]
    Given [describe expected initial app state]

    When user [verb for user action]
    Then [describe expected outcome]

Screenshots/Recordings

Before

After

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

@github-actions
Copy link
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@metamaskbot metamaskbot added the team-qa QA team label Sep 16, 2025
@github-actions github-actions bot added size-L and removed size-M labels Sep 16, 2025
@christopherferreira9 christopherferreira9 changed the title tests: refactors the e2e sh script to mjs tests: refactors the e2e sh script to mjs for flakiness detection Sep 16, 2025
@christopherferreira9 christopherferreira9 changed the title tests: refactors the e2e sh script to mjs for flakiness detection test: refactors the e2e sh script to mjs for flakiness detection Sep 16, 2025
@christopherferreira9 christopherferreira9 added the no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed label Sep 16, 2025
@christopherferreira9 christopherferreira9 marked this pull request as ready for review September 16, 2025 17:22
cursor[bot]

This comment was marked as outdated.

@christopherferreira9 christopherferreira9 added the skip-e2e-quality-gate Skips the E2E Quality gate (flakiness detection) label Sep 16, 2025
cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

@christopherferreira9 christopherferreira9 removed the skip-e2e-quality-gate Skips the E2E Quality gate (flakiness detection) label Sep 17, 2025
@christopherferreira9 christopherferreira9 added the skip-e2e-quality-gate Skips the E2E Quality gate (flakiness detection) label Sep 17, 2025
cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

@sonarqubecloud
Copy link

@christopherferreira9 christopherferreira9 added this pull request to the merge queue Sep 24, 2025
Merged via the queue into main with commit 174aa4c Sep 24, 2025
53 checks passed
@christopherferreira9 christopherferreira9 deleted the christopher/migrate-e2e-sh-scripts branch September 24, 2025 09:35
@github-actions github-actions bot locked and limited conversation to collaborators Sep 24, 2025
@metamaskbot metamaskbot added the release-999.9.0 Issue or pull request that will be included in release 999.9.0 label Sep 24, 2025
@XxdpavelxX XxdpavelxX added release-7.57.0 Issue or pull request that will be included in release 7.57.0 and removed release-999.9.0 Issue or pull request that will be included in release 999.9.0 labels Sep 24, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed release-7.57.0 Issue or pull request that will be included in release 7.57.0 size-L skip-e2e-quality-gate Skips the E2E Quality gate (flakiness detection) team-qa QA team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants