Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[auick_action_ios] Retries multiple times to not fail ci when there is a flake #3823

Merged
merged 4 commits into from
Apr 27, 2023
Merged

[auick_action_ios] Retries multiple times to not fail ci when there is a flake #3823

merged 4 commits into from
Apr 27, 2023

Conversation

cyanglaz
Copy link
Contributor

The only way that we found to test quick action menu is to long press for a x second. Previously, we pressed it for 2 seconds but it is sometimes too long and the quick action menu is disappeared.

In this PR, we:

  1. Reduce the press time to 1.5 seconds.
  2. Retry 3 times.

fixes flutter/flutter#125509

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the relevant style guides and ran the auto-formatter. (Unlike the flutter/flutter repo, the flutter/packages repo does use dart format.)
  • I signed the CLA.
  • The title of the PR starts with the name of the package surrounded by square brackets, e.g. [shared_preferences]
  • I listed at least one issue that this PR fixes in the description above.
  • I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy, or this PR is exempt from version changes.
  • I updated CHANGELOG.md to add a description of the change, following repository CHANGELOG style.
  • I updated/added relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or this PR is test-exempt.
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

Chris Yang added 2 commits April 25, 2023 14:54
@cyanglaz cyanglaz requested a review from hellohuanlin as a code owner April 26, 2023 01:09
@cyanglaz cyanglaz added override: no versioning needed Override the check requiring version bumps for most changes override: no changelog needed Override the check requiring CHANGELOG updates for most changes labels Apr 26, 2023
@cyanglaz cyanglaz changed the title [Quick_action_ios] Retries multiple times to not fail ci when there is a flake [auick_action_ios] Retries multiple times to not fail ci when there is a flake Apr 26, 2023
hellohuanlin
hellohuanlin reviewed Apr 26, 2023
View reviewed changes
packages/quick_actions/quick_actions_ios/example/ios/RunnerUITests/RunnerUITests.swift
@@ -5,6 +5,9 @@
import XCTest

private let elementWaitingTime: TimeInterval = 30
private let quickActionPressDuration: TimeInterval = 1.5
// Max number of tries to open the quick action menu if failed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe add a short explanation why we need to retry?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

packages/quick_actions/quick_actions_ios/example/ios/RunnerUITests/RunnerUITests.swift
let actionOne = springboard.buttons["Action one"]
if !actionOne.waitForExistence(timeout: elementWaitingTime) {
var actionOne: XCUIElement?
for _ in 1...quickActionMaxRetries {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may wanna write a helper function?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@cyanglaz
Copy link
Contributor Author

cyanglaz commented Apr 26, 2023 •
edited
Loading

Offline discussion: Maybe start with 2 seconds and reduce the press duration over the following retires.
2 seconds the first try, 1.8 seconds the next try, 1.6 seconds the next try etc.

We'd only reduce duration when we see the DeleteButton.
Maybe also increase duration if we see neither the action menu and DeleteButton.

@hellohuanlin
Copy link
Contributor

yeah that's probably better

@cyanglaz cyanglaz requested a review from hellohuanlin April 26, 2023 22:02
hellohuanlin
hellohuanlin approved these changes Apr 26, 2023
View reviewed changes
packages/quick_actions/quick_actions_ios/example/ios/RunnerUITests/RunnerUITests.swift
pressDuration -= pressDurationRetryAdjustment
} else {
// Neither action button nor delete button was found, we need a longer press.
pressDuration += pressDurationRetryAdjustment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me. though the drawback is that it makes the test a bit complicated, as test code itself is not tested. i would simply try like 1, 1.33, 1.66 and 2. but up to you.

@cyanglaz cyanglaz added the autosubmit Merge PR when tree becomes green via auto submit App label Apr 27, 2023
@auto-submit auto-submit bot merged commit cf8dd12 into flutter:main Apr 27, 2023
@jmagman jmagman mentioned this pull request Apr 27, 2023
Merged
@cyanglaz cyanglaz deleted the quick_action_flake branch April 28, 2023 16:49
sybrands-place pushed a commit to sybrands-place/packages that referenced this pull request May 1, 2023
Merge branch 'main' into webview_patchedv5
9965c00 
* main: (46 commits)
  [go_router] Cleans up route match API and introduces dart fix (flutter#3819)
  [camerax] Add `LifecycleOwner` Proxy (flutter#3837)
  [file_selector] Add getDirectoryPaths implementation for Windows (flutter#3704)
  [various] Update Android example min SDKs (flutter#3847)
  Bump github/codeql-action from 2.2.12 to 2.3.2 (flutter#3838)
  [camerax] Implement Image Streaming (flutter#3454)
  [various] update agp and gradle for all examples in packages (flutter#3822)
  Update xcode to 14c18 (flutter#3774)
  [camera_android] Add NV21 as an image stream format flutter#3277 (flutter#3639)
  [go_router] Remove unused navigator keys (flutter#3708)
  [image_picker] Move I/O operations to a separate thread (flutter#3506)
  [pigeon]: Bump org.jetbrains.kotlin:kotlin-gradle-plugin from 1.8.20 to 1.8.21 in /packages/pigeon/platform_tests/test_plugin/android (flutter#3824)
  [pigeon] Reland: Add an initial example app (flutter#3832)
  Roll Flutter from c9004ff to 66fa4c5 (68 revisions) (flutter#3830)
  [various] Conditionalize the namespace in all Android plugins (flutter#3836)
  [in_app_pur]: Bump com.android.billingclient:billing from 5.1.0 to 5.2.0 in /packages/in_app_purchase/in_app_purchase_android/android (flutter#3672)
  [auick_action_ios] Retries multiple times to not fail ci when there is a flake (flutter#3823)
  Swap some iOS package CODEOWNERS (flutter#3793)
  [various] Add `targetCompatibility` to build.gradle (flutter#3825)
  [various] Set cmake_policy versions (flutter#3828)
  ...
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request May 1, 2023
Closed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request May 1, 2023
@engine-flutter-autoroll
cf8dd12 [auick_action_ios] Retries multiple times to not fail ci when…
cc12260 
… there is a flake (flutter/packages#3823)
@engine-flutter-autoroll engine-flutter-autoroll mentioned this pull request May 1, 2023
Merged
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request May 1, 2023
@engine-flutter-autoroll
cf8dd12 [auick_action_ios] Retries multiple times to not fail ci when…
5714c4c 
… there is a flake (flutter/packages#3823)
auto-submit bot pushed a commit to flutter/flutter that referenced this pull request May 1, 2023
@engine-flutter-autoroll
Roll Packages from 7e3f5da to de6131d (41 revisions) (#125811)
ff248d2 
Roll Packages from 7e3f5da to de6131d (41 revisions)

flutter/packages@7e3f5da...de6131d

2023-05-01 reidbaker@google.com I122213 update non examples (flutter/packages#3846)
2023-04-29 47866232+chunhtai@users.noreply.github.com [go_router] Cleans up route match API and introduces dart fix (flutter/packages#3819)
2023-04-29 43054281+camsim99@users.noreply.github.com [camerax] Add `LifecycleOwner` Proxy (flutter/packages#3837)
2023-04-29 stuartmorgan@google.com [file_selector] Add getDirectoryPaths implementation for Windows (flutter/packages#3704)
2023-04-29 stuartmorgan@google.com [various] Update Android example min SDKs (flutter/packages#3847)
2023-04-28 49699333+dependabot[bot]@users.noreply.github.com Bump github/codeql-action from 2.2.12 to 2.3.2 (flutter/packages#3838)
2023-04-28 43054281+camsim99@users.noreply.github.com [camerax] Implement Image Streaming (flutter/packages#3454)
2023-04-28 reidbaker@google.com [various] update agp and gradle for all examples in packages (flutter/packages#3822)
2023-04-28 54558023+keyonghan@users.noreply.github.com Update xcode to 14c18 (flutter/packages#3774)
2023-04-28 andrewjohncoutts@gmail.com [camera_android] Add NV21 as an image stream format #3277 (flutter/packages#3639)
2023-04-28 gautier.bayzelon@gmail.com [go_router] Remove unused navigator keys (flutter/packages#3708)
2023-04-28 JeroenWeener@users.noreply.github.com [image_picker] Move I/O operations to a separate thread (flutter/packages#3506)
2023-04-28 49699333+dependabot[bot]@users.noreply.github.com [pigeon]: Bump org.jetbrains.kotlin:kotlin-gradle-plugin from 1.8.20 to 1.8.21 in /packages/pigeon/platform_tests/test_plugin/android (flutter/packages#3824)
2023-04-28 stuartmorgan@google.com [pigeon] Reland: Add an initial example app (flutter/packages#3832)
2023-04-28 engine-flutter-autoroll@skia.org Roll Flutter from c9004ff to 66fa4c5 (68 revisions) (flutter/packages#3830)
2023-04-28 stuartmorgan@google.com [various] Conditionalize the namespace in all Android plugins (flutter/packages#3836)
2023-04-27 49699333+dependabot[bot]@users.noreply.github.com [in_app_pur]: Bump com.android.billingclient:billing from 5.1.0 to 5.2.0 in /packages/in_app_purchase/in_app_purchase_android/android (flutter/packages#3672)
2023-04-27 ychris@google.com [auick_action_ios] Retries multiple times to not fail ci when there is a flake (flutter/packages#3823)
2023-04-26 magder@google.com Swap some iOS package CODEOWNERS (flutter/packages#3793)
2023-04-26 stuartmorgan@google.com [various] Add `targetCompatibility` to build.gradle (flutter/packages#3825)
2023-04-26 stuartmorgan@google.com [various] Set cmake_policy versions (flutter/packages#3828)
2023-04-26 stuartmorgan@google.com [path_provider] Allow `win32` up to version 4.x (flutter/packages#3820)
2023-04-26 stuartmorgan@google.com [tool] Move Android lint checks (flutter/packages#3816)
2023-04-25 evace93@gmail.com [google_maps_flutter_android] Fix Android lint warnings (flutter/packages#3751)
2023-04-25 49699333+dependabot[bot]@users.noreply.github.com [in_app_pur]: Bump androidx.annotation:annotation from 1.5.0 to 1.6.0 in /packages/in_app_purchase/in_app_purchase_android/android (flutter/packages#3381)
2023-04-25 jason-simmons@users.noreply.github.com Update test golden images for the latest Skia roll (flutter/packages#3787)
2023-04-25 stuartmorgan@google.com [various] Adds Android namespace (flutter/packages#3791)
2023-04-25 reidbaker@google.com [shared_preferences] Update gradle/agp in example apps (flutter/packages#3809)
2023-04-24 43640732+dancamdev@users.noreply.github.com [go_router] Adds name to TypedGoRoute (flutter/packages#3702)
2023-04-22 10687576+bparrishMines@users.noreply.github.com [webview_flutter] Adds support to receive permission requests (flutter/packages#3543)
2023-04-21 stuartmorgan@google.com [google_sign_in] Fix Android Java warnings (flutter/packages#3762)
2023-04-21 stuartmorgan@google.com [local_auth] Fix enum return on Android (flutter/packages#3780)
2023-04-21 stuartmorgan@google.com [pigeon] Warn when trying to use enums in collections (flutter/packages#3782)
2023-04-21 10687576+bparrishMines@users.noreply.github.com [webview_flutter_android] [webview_flutter_wkwebview] Platform implementations for supporting permission requests (flutter/packages#3792)
2023-04-21 scheglov@google.com [pigeon] Update for compatibility with a future change to the analyzer. (flutter/packages#3789)
2023-04-21 10687576+bparrishMines@users.noreply.github.com [camera_android] Fix Android lint warnings  (flutter/packages#3716)
2023-04-21 10687576+bparrishMines@users.noreply.github.com [webview_flutter_platform_interface] Adds method to receive permission requests (flutter/packages#3767)
2023-04-21 magder@google.com [image_picker_ios] In unit test write and read kCGImagePropertyExifUserComment property (flutter/packages#3783)
2023-04-21 widrans@gmail.com [go_router_builder] Fixed the return value of the generated push method (flutter/packages#3650)
2023-04-21 JeroenWeener@users.noreply.github.com [image_picker] Mention `launchMode: singleInstance` in README (flutter/packages#3759)
2023-04-21 stuartmorgan@google.com Revert "[pigeon] Add an initial example app" (flutter/packages#3785)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages-flutter-autoroll
Please CC flutter-ecosystem@google.com,rmistry@google.com on the revert to ensure that a human
...
nploi pushed a commit to nploi/packages that referenced this pull request Jul 16, 2023
@nploi
[auick_action_ios] Retries multiple times to not fail ci when there i…
b51e4c9 
…s a flake (flutter#3823)

The only way that we found to test quick action menu is to long press for a x second. Previously, we pressed it for 2 seconds but it is sometimes too long and the quick action menu is disappeared.

In this PR, we:
1. Reduce the press time to 1.5 seconds.
2. Retry 3 times.

fixes flutter/flutter#125509
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hellohuanlin hellohuanlin hellohuanlin approved these changes

Assignees
No one assigned
Labels
autosubmit Merge PR when tree becomes green via auto submit App override: no changelog needed Override the check requiring CHANGELOG updates for most changes override: no versioning needed Override the check requiring version bumps for most changes p: quick_actions platform-ios
Projects
None yet
Milestone
No milestone
2 participants
@cyanglaz @hellohuanlin