Skip to content

[Testing] Fix for flaky UITests in CI - 2 #31763

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

[Testing] Fix for flaky UITests in CI - 2 #31763

wants to merge 8 commits into from

Conversation

@TamilarasanSF4853
Copy link
Contributor

@TamilarasanSF4853 TamilarasanSF4853 commented Sep 24, 2025

This PR addresses UI test image failures in the main branch and includes updates to improve rendering and test stability across platforms.

  • WebViewShouldNotMirrored: Updated to use a new static web link and added VerifyInternetConnectivity to ensure internet connection before running the test.
  • HeaderFooterHorizontalViewWorks: Changed the scroll strategy to Gesture for better reliability.

Test cases:

  • WebViewShouldNotMirrored
  • HeaderFooterHorizontalViewWorks

@dotnet-policy-service dotnet-policy-service bot added community ✨ Community Contribution partner/syncfusion Issues / PR's with Syncfusion collaboration labels Sep 24, 2025
@jsuarezruiz
Copy link
Contributor

jsuarezruiz commented Sep 24, 2025

/azp run MAUI-UITests-public

@azure-pipelines
Copy link

azure-pipelines bot commented Sep 24, 2025

Azure Pipelines successfully started running 1 pipeline(s).

@TamilarasanSF4853 TamilarasanSF4853 marked this pull request as ready for review September 24, 2025 17:05
Copilot AI review requested due to automatic review settings September 24, 2025 17:05
Copilot AI reviewed Sep 24, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR addresses flaky UI test failures in the CI pipeline by improving test reliability and rendering stability for WebView and CollectionView tests.

  • Added internet connectivity verification for WebView tests to ensure network availability before execution
  • Changed scroll strategy from Auto to Gesture for better reliability in CollectionView horizontal scrolling tests
  • Updated WebView test to use a more stable static web link (example.com) instead of a GitHub issue URL

Reviewed Changes

Copilot reviewed 3 out of 5 changed files in this pull request and generated no comments.

File Description
Issue30575.cs (Tests) Added VerifyInternetConnectivity() call before WebView test execution
CollectionViewUITests.HeaderAndFooter.cs Changed scroll strategy from Auto to Gesture for improved reliability
Issue30575.cs (HostApp) Updated WebView URL to example.com and added isInternetRequired flag

jsuarezruiz
jsuarezruiz suggested changes Sep 25, 2025
View reviewed changes
src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue30575.cs

[Test]
[Category(UITestCategories.WebView)]
public void WebViewShouldNotMirrored()
Copy link
Contributor

@jsuarezruiz jsuarezruiz Sep 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

VisualTestUtils.VisualTestFailedException :
Snapshot different than baseline: WebViewShouldNotMirrored.png (1.90% difference)
image

Failing on Android, Mac and Windows.

Copy link
Contributor Author

@TamilarasanSF4853 TamilarasanSF4853 Oct 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsuarezruiz I have added the pending snapshots in the latest commit.

@rmarinho
Copy link
Member

rmarinho commented Oct 2, 2025

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 2, 2025

Azure Pipelines successfully started running 3 pipeline(s).

@sheiksyedm sheiksyedm added the area-testing Unit tests, device tests label Oct 3, 2025
@rmarinho
Copy link
Member

rmarinho commented Oct 3, 2025

/rebase

@github-actions github-actions bot force-pushed the fix-main-flaky-uitest branch from 04cb5ce to 1184149 Compare October 3, 2025 06:31
@rmarinho
Copy link
Member

rmarinho commented Oct 3, 2025

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 3, 2025

Azure Pipelines successfully started running 3 pipeline(s).

@rmarinho
Copy link
Member

rmarinho commented Oct 5, 2025

Test is still failing ...

https://dev.azure.com/xamarin/public/_build/results?buildId=151521&view=ms.vss-test-web.build-test-results-tab&runId=5035643&resultId=100046&paneView=debug

@rmarinho rmarinho mentioned this pull request Oct 5, 2025
Open
@rmarinho
Copy link
Member

rmarinho commented Oct 6, 2025

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 6, 2025

Azure Pipelines successfully started running 3 pipeline(s).

@rmarinho
Copy link
Member

rmarinho commented Oct 6, 2025

/rebase

@github-actions github-actions bot force-pushed the fix-main-flaky-uitest branch from d073c7f to dfb00ff Compare October 6, 2025 14:19
@jsuarezruiz
Copy link
Contributor

jsuarezruiz commented Oct 9, 2025

/azp run MAUI-UITests-public

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 9, 2025

Azure Pipelines successfully started running 1 pipeline(s).

jsuarezruiz
jsuarezruiz suggested changes Oct 15, 2025
View reviewed changes
Copy link
Contributor

@jsuarezruiz jsuarezruiz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WebViewShouldNotMirrored test failing on Windows and Mac.

Example, Mac:

VisualTestUtils.VisualTestFailedException :
Snapshot different than baseline: WebViewShouldNotMirrored.png (3.34% difference)
image

MartyIX
MartyIX reviewed Oct 16, 2025
View reviewed changes
src/Controls/tests/TestCases.HostApp/Issues/Issue30575.cs Outdated
namespace Maui.Controls.Sample.Issues;

[Issue(IssueTracker.Github, 30575, "FlowDirection RightToLeft causes mirrored content in WebView", PlatformAffected.UWP)]
[Issue(IssueTracker.Github, 30575, "FlowDirection RightToLeft causes mirrored content in WebView", PlatformAffected.UWP, isInternetRequired: true)]
Copy link
Contributor

@MartyIX MartyIX Oct 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change no longer necessary with the HtmlWebViewSource below?

Copy link
Contributor Author

@TamilarasanSF4853 TamilarasanSF4853 Oct 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MartyIX I have updated the test case in the latest commit.

@TamilarasanSF4853
Copy link
Contributor Author

TamilarasanSF4853 commented Oct 16, 2025

WebViewShouldNotMirrored test failing on Windows and Mac.

Example, Mac:

VisualTestUtils.VisualTestFailedException :
Snapshot different than baseline: WebViewShouldNotMirrored.png (3.34% difference)
image

@jsuarezruiz I have updated the test case in the latest commit. Could you please rerun the CI?

@jsuarezruiz
Copy link
Contributor

jsuarezruiz commented Oct 28, 2025

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 28, 2025

Azure Pipelines successfully started running 3 pipeline(s).

jsuarezruiz
jsuarezruiz reviewed Nov 4, 2025
View reviewed changes
Copy link
Contributor

@jsuarezruiz jsuarezruiz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TamilarasanSF4853 Could you rebase to fix the conflicts?

@TamilarasanSF4853
Copy link
Contributor Author

TamilarasanSF4853 commented Nov 5, 2025

@TamilarasanSF4853 Could you rebase to fix the conflicts?

@jsuarezruiz I have rebased the branch and resolved the conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

+2 more reviewers

@MartyIX MartyIX MartyIX left review comments

@jsuarezruiz jsuarezruiz jsuarezruiz requested changes

Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

area-testing Unit tests, device tests community ✨ Community Contribution partner/syncfusion Issues / PR's with Syncfusion collaboration

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@TamilarasanSF4853 @jsuarezruiz @rmarinho @MartyIX @sheiksyedm