Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding smoke and regression tags to tests #5546

Merged
merged 30 commits into from
Feb 6, 2023
Merged

Conversation

cortisiko
Copy link
Member

@cortisiko cortisiko commented Jan 18, 2023

Development & PR Process

  1. Follow MetaMask Mobile Coding Standards
  2. Add release-xx label to identify the PR slated for a upcoming release (will be used in release discussion)
  3. Add needs-dev-review label when work is completed
  4. Add needs-qa label when dev review is completed
  5. Add QA Passed label when QA has signed off

Description

The time it takes to build and launch tests on bitrise is >90 mins. To prevent tests from timing out on bitrise, we need to add specific tags @smoke @regression on the different E2E tests. We need a better way to specify tags. We should create a constants file for all of our tags. then have a command line flag to run specific tags. For example, if we want to run all tests with the smoke tag we wun do something like this:

yarn test:android:qa --tags=smoke

Screenshots/Recordings

If applicable, add screenshots and/or recordings to visualize the before and after of your change

Issue

Progresses #???

Checklist

  • There is a related GitHub issue
  • Tests are included if applicable
  • Any added code is fully documented

@cortisiko cortisiko requested a review from a team as a code owner January 18, 2023 21:25
@cortisiko cortisiko linked an issue Jan 18, 2023 that may be closed by this pull request
@Fatxx Fatxx assigned Fatxx and unassigned Fatxx Jan 19, 2023
@smoke
Copy link

smoke commented Jan 27, 2023

@cortisiko Hey, sorry to bother on this PR, but can I kindly ask you to mark tags as code using
` e.g. @smoke @regression instead of just @smoke @regression.
It is not a big deal, but otherwise GitHub uses those as handles and you risk "summoning" someone unintentionally.

I love Metamask it is great product, keep up the good work!

@cortisiko
Copy link
Member Author

oh my apologies @smoke, I was unaware that there was a gh profile associated with " @ smoke". I will format the text in the ticket description now.

@Fatxx Fatxx changed the title adding smoke and regression tags to tests Adding smoke and regression tags to tests Jan 30, 2023
@github-actions
Copy link
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@Fatxx
Copy link
Contributor

Fatxx commented Feb 3, 2023

As an increment to this PR I added the following:

  • Splited build_android_qa and wdio_android_e2e_test in to separate workflows, at the end of build_android_qa wdio_android_e2e_test is triggered in a separate build this will help us avoid build timeouts in Bitrise;
  • Added support for the following environment variables: BROWSERSTACK_DEVICE, BROWSERSTACK_OS_VERSION, BROWSERSTACK_TAG_EXPRESSION and BROWSERSTACK_APP_URL in wdio/config/android.config.browserstack.js they can now be declare when triggering a build in Bitrise which enable more customisation
  • Updated docs: https://www.notion.so/E2E-Testing-361fee8ff65048f0b18d6d3dddc5ad9c

@Fatxx Fatxx added the needs-dev-review PR needs reviews from other engineers (in order to receive required approvals) label Feb 3, 2023
@Fatxx Fatxx merged commit bce6a95 into main Feb 6, 2023
@Fatxx Fatxx deleted the Appium/add-tags-to-feature-files branch February 6, 2023 15:53
@github-actions github-actions bot locked and limited conversation to collaborators Feb 6, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Appium needs-dev-review PR needs reviews from other engineers (in order to receive required approvals) team-mobile-platform
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add tags to e2e tests and enable running tests by specific label
6 participants