Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Windows] Review currently failing Windows tests #15530

Open
17 of 32 tasks
Tracked by #15113
PureWeen opened this issue Jun 8, 2023 · 7 comments
Open
17 of 32 tasks
Tracked by #15113

[Windows] Review currently failing Windows tests #15530

PureWeen opened this issue Jun 8, 2023 · 7 comments
Assignees
@jfversluis @jsuarezruiz
Labels
area-testing Unit tests, device tests platform/windows 🪟 s/triaged Issue has been reviewed t/bug Something isn't working testing-flakiness
Milestone
Backlog

Comments

@PureWeen
Copy link
Member

PureWeen commented Jun 8, 2023 •
edited by jsuarezruiz
Loading

Description

As part of #15527 I've skipped the tests listed below. All of these tests were failing on the original PR. We don't necessarily need to fix these issues for .NET8 but we should establish if we should fix these for .NET8 or just fix the tests themselves.

In addition after merging #15629 we're now running the DeviceTests for Windows, these have been disabled because they were failing and we should look into them.

Core
At the time of writing the whole Core project test runs have been disabled because it would crash while running, so that is something we need to fix, other than that, these tests are failing:

  • WebViewPlaysHtml5Video
  • Dashed Stroke Initializes Correctly (all variations)
  • Background Initializes Correctly (all variations)
  • Stroke Initializes Correctly (all variations)
  • CancelButtonColorInitializeCorrectly
  • StrokeShape Initializes Correctly (all variations)
  • ThumbColorUpdatesCorrectly
  • CursorPositionDoesntResetWhenNativeTextValueChanges
  • BackgroundUpdatesCorrectly
  • SettingSemanticHintMakesElementAccessible
  • DisconnectHandlerDoesntCrash
  • SettingSemanticDescriptionMakesElementAccessible
  • Enable Core DeviceTests Project (Enable WinUI Core Tests on CI #15885)
  • CursorPositionDoesntResetWhenNativeTextValueChanges
  • SettingSemanticDescriptionMakesElementAccessible (Activity Indicator)
  • SettingSemanticHintMakesElementAccessible (Activity Indicatr)
  • SwipeViewHandlerTests

BlazorWebView
#16187

Essentials

  • CanOpenUri (sms, tel and mailto)
  • CanOpen (sms, tel and mailto)
  • IsSupported (App Actions)
  • Vibrate
  • Vibrate_Cancel
  • Charge_Level

Controls
Just like Core, the Controls projects also crashes somewhere along the test run and never completes. We need to investigate why and enable it for running on CI again

  • Enable Controls DeviceTests Project

Unpackaged
Then we need to run this full suite also for unpackaged apps. I think we can make this a variation by leveraging the device parameter that is in place for iOS and Android and add that to the matrix for Windows

  • Enable unpackaged DeviceTests Project

Random

  • Read Windows app package ID from the csproj instead of providing it as a parameter
  • Check if we still need the RunHeadless property
@PureWeen PureWeen added this to the .NET 8 milestone Jun 8, 2023
@PureWeen PureWeen mentioned this issue Jun 8, 2023
Merged
@Eilon Eilon added the area-testing Unit tests, device tests label Jun 8, 2023
@Vroomer

This comment was marked as off-topic.

Sign in to view
@jonathanpeppers
Copy link
Member

@PureWeen are these running on CI now? It probably not very impactful to fix these tests unless we are running them?

@PureWeen
Copy link
Member Author

@jonathanpeppers we're getting CI spun up here soon so I'm just trying to gauge what to do about these failures for tracking purposes

  • Ignore the test because it's not relevant on windows
  • Fix the bug
  • fix the test

Or some combination of those

@jonathanpeppers
Copy link
Member

Let me know when Windows device tests are running on CI, as I probably won't fix the ones above until then.

They are probably not real bugs, but something with the tests themselves.

@PureWeen
Copy link
Member Author

@jonathanpeppers sounds good!
We have them marked as skip for now.

@hartez hartez mentioned this issue Jun 12, 2023
Merged
jonathanpeppers added a commit to jonathanpeppers/maui that referenced this issue Jul 5, 2023
@jonathanpeppers
[tests] make Windows memory leak tests consistent
d2f903e 
Fixes part of: dotnet#15530

These would fail randomly for me when I ran them. But after reviewing
memory snapshots, I didn't find an actual issue the tests were just
flaky.

I found I could rework the tests to do:

    await AssertionExtensions.Wait(() =>
    {
        GC.Collect();
        GC.WaitForPendingFinalizers();
        return !pageReference.IsAlive;
    }, timeout: 5000);

And now they pass every time for me locally.

So it waits up to 5 seconds, and quits early if the page is gone.

If this still passes on all platforms -- we should just use this
instead.
@jonathanpeppers jonathanpeppers mentioned this issue Jul 5, 2023
Merged
PureWeen pushed a commit that referenced this issue Jul 5, 2023
@jonathanpeppers
[tests] make Windows memory leak tests consistent (#15998)
bf1a9c0 
Fixes part of: #15530

These would fail randomly for me when I ran them. But after reviewing
memory snapshots, I didn't find an actual issue the tests were just
flaky.

I found I could rework the tests to do:

    await AssertionExtensions.Wait(() =>
    {
        GC.Collect();
        GC.WaitForPendingFinalizers();
        return !pageReference.IsAlive;
    }, timeout: 5000);

And now they pass every time for me locally.

So it waits up to 5 seconds, and quits early if the page is gone.

If this still passes on all platforms -- we should just use this
instead.
@jfversluis jfversluis mentioned this issue Jul 13, 2023
Merged
4 tasks
@Redth Redth modified the milestones: .NET 8, .NET 8 GA Jul 18, 2023
@samhouts samhouts added the t/bug Something isn't working label Jul 31, 2023
@PureWeen PureWeen mentioned this issue Aug 12, 2023
Open
40 tasks
@PureWeen PureWeen assigned jfversluis and unassigned jsuarezruiz Oct 26, 2023
@PureWeen PureWeen modified the milestones: .NET 8 SR1, .NET 8 SR2 Nov 3, 2023
@PureWeen PureWeen moved this from Todo to In Progress in MAUI SDK Ongoing Nov 3, 2023
@jfversluis
Copy link
Member

@jonathanpeppers not sure you're still interested and just caught this now... But I think all Windows tests are now running on CI. So if you have still something to do here, do it now! ✨

@PureWeen PureWeen modified the milestones: .NET 8 SR1, .NET 8 SR2 Nov 22, 2023
@Redth Redth removed this from MAUI SDK Ongoing Dec 5, 2023
@Redth Redth modified the milestones: .NET 8 SR2, Backlog Dec 5, 2023
@ghost
Copy link

ghost commented Dec 5, 2023

We've added this issue to our backlog, and we will work to address it as time and resources allow. If you have any additional information or questions about this issue, please leave a comment. For additional info about issue management, please read our Triage Process.

@jsuarezruiz jsuarezruiz mentioned this issue Jan 24, 2024
Merged
@jaosnz-rep jaosnz-rep added the s/triaged Issue has been reviewed label Jan 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jfversluis jfversluis

@jsuarezruiz jsuarezruiz

Labels
area-testing Unit tests, device tests platform/windows 🪟 s/triaged Issue has been reviewed t/bug Something isn't working testing-flakiness
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests