Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[auick_action_ios] Retries multiple times to not fail ci when there is a flake #3823

Merged
merged 4 commits into from
Apr 27, 2023
Merged

[auick_action_ios] Retries multiple times to not fail ci when there is a flake #3823

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
572b158
draft
Apr 25, 2023
3080c6f
retries
Apr 25, 2023
a15dfdb
adjust wait time
Apr 26, 2023
33e6db4
comments
Apr 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 50 additions & 24 deletions packages/quick_actions/quick_actions_ios/example/ios/RunnerUITests/RunnerUITests.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,20 @@
import XCTest

private let elementWaitingTime: TimeInterval = 30
// The duration in when pressing the app icon to open the
// quick action menu. This duration is undocumented by Apple.
// The duration will be adjusted with `pressDurationRetryAdjustment` if
// this duration does not result in the quick action menu opened.
private let quickActionPressDuration: TimeInterval = 1.5
// If the previous try to open quick action menu did not work,
// a new try with adjust the press time by this value.
// The adjusment could be + or - depends on the result of the previous try.
private let pressDurationRetryAdjustment: TimeInterval = 0.2
// Max number of tries to open the quick action menu if failed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe add a short explanation why we need to retry?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

// This is to deflake a situation where the quick action menu is not present after
// the long press.
// See: https://github.com/flutter/flutter/issues/125509
private let quickActionMaxRetries: Int = 4;

class RunnerUITests: XCTestCase {

Expand All @@ -25,22 +39,8 @@ class RunnerUITests: XCTestCase {
func testQuickActionWithFreshStart() {
let springboard = XCUIApplication(bundleIdentifier: "com.apple.springboard")
let quickActionsAppIcon = springboard.icons["quick_actions_example"]
if !quickActionsAppIcon.waitForExistence(timeout: elementWaitingTime) {
XCTFail(
"Failed due to not able to find the example app from springboard with \(elementWaitingTime) seconds. Springboard debug description: \(springboard.debugDescription)"
)
}

quickActionsAppIcon.press(forDuration: 2)

let actionTwo = springboard.buttons["Action two"]
if !actionTwo.waitForExistence(timeout: elementWaitingTime) {
XCTFail(
"Failed due to not able to find the actionTwo button from springboard with \(elementWaitingTime) seconds. Springboard debug description: \(springboard.debugDescription)"
)
}

actionTwo.tap()
findAndTapQuickActionButton(buttonName: "Action two", quickActionsAppIcon: quickActionsAppIcon, springboard: springboard);

let actionTwoConfirmation = exampleApp.otherElements["action_two"]
if !actionTwoConfirmation.waitForExistence(timeout: elementWaitingTime) {
Expand Down Expand Up @@ -73,24 +73,50 @@ class RunnerUITests: XCTestCase {
)
}

quickActionsAppIcon.press(forDuration: 2)
findAndTapQuickActionButton(buttonName: "Action one", quickActionsAppIcon: quickActionsAppIcon, springboard: springboard);

let actionOne = springboard.buttons["Action one"]
if !actionOne.waitForExistence(timeout: elementWaitingTime) {
let actionOneConfirmation = exampleApp.otherElements["action_one"]
if !actionOneConfirmation.waitForExistence(timeout: elementWaitingTime) {
XCTFail(
"Failed due to not able to find the actionOne button from springboard with \(elementWaitingTime) seconds. Springboard debug description: \(springboard.debugDescription)"
"Failed due to not able to find the actionOneConfirmation in the app with \(elementWaitingTime) seconds. Springboard debug description: \(springboard.debugDescription)"
)
}

actionOne.tap()
XCTAssert(actionOneConfirmation.exists)
}

let actionOneConfirmation = exampleApp.otherElements["action_one"]
if !actionOneConfirmation.waitForExistence(timeout: elementWaitingTime) {
private func findAndTapQuickActionButton(buttonName: String, quickActionsAppIcon: XCUIElement, springboard: XCUIElement) {
var actionButton: XCUIElement?
var pressDuration = quickActionPressDuration
for _ in 1...quickActionMaxRetries {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may wanna write a helper function?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

if !quickActionsAppIcon.waitForExistence(timeout: elementWaitingTime) {
XCTFail(
"Failed due to not able to find the example app from springboard with \(elementWaitingTime) seconds. Springboard debug description: \(springboard.debugDescription)"
)
}
quickActionsAppIcon.press(forDuration: pressDuration)
actionButton = springboard.buttons[buttonName]
if actionButton!.waitForExistence(timeout: elementWaitingTime) {
// find the button, exit the retry loop.
break
}
let deleteButton = springboard.buttons["DeleteButton"]
if deleteButton.waitForExistence(timeout: elementWaitingTime) {
// Found delete button instead, we pressed too long, reduce the press time.
pressDuration -= pressDurationRetryAdjustment
} else {
// Neither action button nor delete button was found, we need a longer press.
pressDuration += pressDurationRetryAdjustment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me. though the drawback is that it makes the test a bit complicated, as test code itself is not tested. i would simply try like 1, 1.33, 1.66 and 2. but up to you.

}
// Reset to previous state.
XCUIDevice.shared.press(XCUIDevice.Button.home)
}
if (!actionButton!.exists) {
XCTFail(
"Failed due to not able to find the actionOneConfirmation in the app with \(elementWaitingTime) seconds. Springboard debug description: \(springboard.debugDescription)"
"Failed due to not able to find the \(buttonName) button from springboard with \(elementWaitingTime) seconds. Springboard debug description: \(springboard.debugDescription)"
)
}

XCTAssert(actionOneConfirmation.exists)
actionButton!.tap();
}
}