Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use package:patrol for Android e2e tests. #1137

Merged
merged 33 commits into from
Nov 6, 2023
Merged

Conversation

Jonas-Sander
Copy link
Collaborator

@Jonas-Sander Jonas-Sander commented Oct 29, 2023

Use package:patrol for integration/e2e tests for Android.

The biggest benefit is that each test can run from a completely clean state, which is not possible with the integration_test package. Because of this we shouldn't have the issues with e.g. the auth state being kept for each test (or logging out and back in which breaks stuff).

For other platforms I kept our old tests (renamed to integration_test/integration_test_old.dart):

  • Web should theoretically work as long as no native interaction is used, but I couldn't set it up practically. There is an open issue for web in their repo.
  • iOS should work, but I couldn't get it running.
  • macOS is never mentioned, so I don't know if it's possible to run tests for mac.

From https://patrol.leancode.co/:

Patrol lets you access native features of the platform that the Flutter app is running on. Finally, you can interact with permission dialogs, notifications, WebViews, change device settings, toggle Wi-Fi, and much more – and you can code this very easily in plain Dart.

Patrol also provides a new custom finder system that extends Flutter's default finders, making them shorter and easier to understand. Patrol's custom finders, coupled with Hot Restart, make writing integration tests dramatically faster, easier and more fun!

Command to running the tests:

patrol build android \
 --flavor prod \
 --dart-define USER_1_EMAIL=example@sharezone.net \
 --dart-define USER_1_PASSWORD=foobar \
 -t integration_test/app_test.dart

@github-actions github-actions bot added platform: android feature: navigation Navigation inside the app (e.g. switching to a different screen). testing feature: onboarding The steps (setting username, courses, etc.) after creating a new account. feature: authentification Logging in/out (anonymous, sign-in with X, etc.) and registration. dependencies Changing, updating, adding or removing one or more dependencies. w: dashboard-page Page that shows a summary of all important things (homeworks, events, etc.). labels Oct 29, 2023
@github-actions
Copy link

github-actions bot commented Oct 29, 2023

Visit the preview URL for this PR (updated for commit c31de56):

https://sharezone-test--pr1137-patrol-e2e-tests-rzzd2cae.web.app

(expires Thu, 09 Nov 2023 11:28:44 GMT)

🔥 via Firebase Hosting GitHub Action 🌎

Sign: 4cb3ae61e1e018abfd9841fd3239f5b49ccc034b

@github-actions github-actions bot added the ci/cd label Oct 29, 2023
github-merge-queue bot pushed a commit that referenced this pull request Oct 30, 2023
…test`. (#1138)

As long as I'm still working on #1137 this might help speed up the
current android integration tests.
@Jonas-Sander Jonas-Sander changed the title Use package:patrol for e2e tests. Use package:patrol for Android e2e tests. Nov 2, 2023
@Jonas-Sander
Copy link
Collaborator Author

@nilsreichardt Ready for review.

I couldn't get iOS set up, so for now I left it as is.
Maybe you could try your luck sometime in the future with setting it up.

Also the macOS job seems to be making problems, do you know how it can be fixed?

@Jonas-Sander Jonas-Sander marked this pull request as ready for review November 2, 2023 21:42
# The secrets already contain the base64 encoded values. We don't
# encode them in the workflow file, because we could easily leak the
# base64 encoded version in logs.
USER_1_EMAIL_BASE64: ${{ secrets.INTEGRATION_TEST_USER_1_EMAIL_DART_DEFINE_BASE64 }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After merging this PR, we can delete INTEGRATION_TEST_USER_1_EMAIL_DART_DEFINE_BASE64 from our secrets, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so, yes

Copy link
Member

@nilsreichardt nilsreichardt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nilsreichardt
Copy link
Member

iOS should work, but I couldn't get it running.

Even, if we can get Patrol running for iOS, we can't re-use the tests for Android because on iOS you can't clear the app state after each test. Because of our sign-out bug, we still need on iOS one large e2e test.

@Jonas-Sander Jonas-Sander added this pull request to the merge queue Nov 6, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 6, 2023
@Jonas-Sander Jonas-Sander added this pull request to the merge queue Nov 6, 2023
Merged via the queue into main with commit 210a8b6 Nov 6, 2023
27 checks passed
@Jonas-Sander Jonas-Sander deleted the patrol-e2e-tests branch November 6, 2023 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/cd dependencies Changing, updating, adding or removing one or more dependencies. documentation feature: authentification Logging in/out (anonymous, sign-in with X, etc.) and registration. feature: navigation Navigation inside the app (e.g. switching to a different screen). feature: onboarding The steps (setting username, courses, etc.) after creating a new account. platform: android testing w: dashboard-page Page that shows a summary of all important things (homeworks, events, etc.).
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants