WinUI3 - ManipulationStarting Event only occurs on first touch -> e.Container Property is null in following manipulation events

[chrkon]

Describe the bug

I want to read the Container Property from the ManipulationEventArgs. But I get a null reference Exception for this property if I touch the Screen a second time.
As a second effect the Touch Position contains an offset. It seems that the Position is relative to the parent control.

Steps to reproduce the bug

1. create a sample WinUI 3 App with a rectangle as content.
2. Set manipulationMode = "All" in the code
3. Add eventhandler for all "Manipulation" events
4. Write a DebugOutput into each eventhadler.
5. run the app
6. do pan or rotate gestures on screen (first time)
7. do the same a second time
8. check the debug output

Result:
I use a SwapChainPanel and I have set the manipulation mode property to "All".
The first time I do a pan gesture on the UiElement I receive the following events in this order:

1. OnManipulationStarting
2. OnManipulationStarted
3. OnManipulationDelta ............... (n times)
4. OnManipulationInertiaStarting
5. OnManipulationDelta .... (m times)
6. OnManipulationCompleted

This seems to be OK, but (second tap result!):
If I do this a second time or more often, the "OnManipulationStarting" event handler is not called !
And then the Container Property is null!

Expected behavior

On the second call and on any further calls the OnManipulationStarting event is missing and the Container Property is null!

I would expect the same behaviour as in the first call.

Screenshots

No response

NuGet package version

WinUI 3 - Windows App SDK 1.0

Windows app type

• UWP
• Win32

Device form factor

Desktop

Windows version

Windows 11 (21H2): Build 22000

Additional context

If another UIElement gets the focus and the user set the focus back to the swapChainPanel again, the next tap will call the Manipulationstarting event.

It feels like this behaviour mentioned in the WinUI documentation:
https://docs.microsoft.com/en-us/windows/winui/api/microsoft.ui.xaml.uielement.manipulationstarting?view=winui-3.0#windows8-behavior
But I use it on Windows 11!

[chrkon]

A new information on this issue: If you make a single tap (or just a mouse click), then the next pan gesture will send all Manipulation Events (including the ManipulationStarting event).

[ojhad]

@JJBrychell FYI

[chrkon]

This problem is still open and I need to know if you also see this problem and if you already have it on the todo list?
Otherwise I have to add a patch for this in our software. But I don't want do this, because it seems to be a bug in WinUi 3.

[michael-hawker]

I can also repro this with 1.0.3 on Win 10 21H1 19044.1826. The same exact code on UWP runs fine, but in the Windows App SDK, only the first ManipulationStarting event is fired.

This effects two new components we've been building within the Windows Community Toolkit. One is a base class for 3 components including our refactored GridSplitter which has been a widely used control (being a WPF polyfill as well).

@ryandemopoulos is it possible to raise the priority of this and get this looked into? Hopefully it's a smaller issue, but would great to see in a hotfix, as seems troublesome to work-around.

[chrkon]

Has anything happened in this matter since I created it on 17 March 2022 ?
I noticed that @michael-hawker confirmed this behaviour, but I don't see any reaction to it.
Unfortunately, I am not in a position to look for the error myself. Therefore, I am trying to see if I can somehow circumvent this behaviour.
But I still hope, that somebody can find the root cause for this Issue.

[chrkon]

I have written a workaround for this problem. Maybe it will help someone else, too.

This is the way I use the patch:

  public void HandleManipulationDeltaEvent(object sender, ManipulationDeltaRoutedEventArgs e)
    {
        var pos = e.Position; //  do not remove this line. This line is correct if the patch is no longer needed.
        pos = WinUI3Patch6847.GetPositionEvenIfContainerIsMissing(e); // Remove this line, after Issue 6847 is solved.
        Debug.WriteLine($"TouchPos={pos.X:f1} ; {pos.Y:f1} | object={e.Container}");
    }

And this is the source code of the patch:

    /// <summary>
    /// This Class is a patch for the Win UI 3 Issue 6847 
    /// <see cref="https://github.com/microsoft/microsoft-ui-xaml/issues/6847"/>
    /// </summary>
    internal static class WinUI3Patch6847
    {
        private static FrameworkElement? _container;

        private static Vector3 GetOffset(FrameworkElement? element)
        {
            if (element == null) return Vector3.Zero;
            var offset = element.ActualOffset;
            if (element.Parent != null) offset += GetOffset(element.Parent as FrameworkElement);
            return offset;
        }

        /// <summary>
        /// Retrieves the correct position from the event args, even if the container 
        /// field in subsequent event args is null. The container should not be null 
        /// in the first occurrence.
        /// </summary>
        /// <param name="e">
        /// Manipulation delta routed event information.
        /// </param>
        /// <remarks>
        /// The container property is null, if the focus of the touched UI element was
        /// not changed after the first occurrence. This is an issue in the WinUI3 
        /// Framework and it is well known as Issue 6847.
        /// This function will patch this behaviour and returns the correct position.
        /// It can be removed when the Issue in the WinUI 3 framework is solved.
        /// </remarks>
        /// <returns>
        /// The correct position, even if container is missing.
        /// </returns>
        public static Point GetPositionEvenIfContainerIsMissing(ManipulationDeltaRoutedEventArgs e)
        {
            var position = e.Position;
            if (e.Container != null)
            {
                _container = e.Container as FrameworkElement;
            }
            else if (_container != null)
            {
                var offset = GetOffset(_container);
                position.X -= offset.X;
                position.Y -= offset.Y;
            }
            return position;
        }
    }

This patch works for me, but I will not close this issue, because I think it schould be solved in the Framework it self.

[michael-hawker]

Have a standalone repro project here: ManipulationStartedBug6847.zip

In UWP, if I drag a bit and then click on the red square I get this and see the clear set of events (first a drag and then a single click without movement):

But with the WinUI 3, we get this instead:

First drag works fine, but then on the click the ManipulationStarting event is missing, you can see there's only PointerPressed.

Sometimes when doing a drag the ManipulationStarting will appear, but not always:

And you can see in that case that the e.Container is indeed missing in that scenario.

These events are always firing correctly in UWP.

---

From what I can tell, after the first successful drag in the WinAppSDK scenario, every subsequent drag will fail to fire the event and have the container set. However, if you click without dragging it can reset the behavior and get a good sequence back (which will then fail again each subsequent time).

It seems consistent and repeatable. Would indicate probably something in the initialization/finalization/reset code of a manipulation is causing an issue here with the logic?

[michael-hawker]

Above repro was tested on 1.2.230217.4, can confirm still an issue with 1.3.230228005-preview1 as well.

[JJBrychell]

The issue has been found and a fix is in progress. A typo occurred when Xaml switched the way that gesture recognition occurs a while back. UWP applications still use the older method so they were not affected. However, WinUI 3 applications will not see the ManipulationStarting event on first pointer down following a completed Manipulation because the previous manipulation is in an invalid state (mostly completed, but not completely reset) and Xaml thinks the manipulation starting event has already been sent.

[michael-hawker]

Updated to 1.3 release and tested with our SizerBase controls, looks like this is resolved now, though didn't try my original repro project. Thanks!

[codendone]

Fixed and available in 1.3: https://www.nuget.org/packages/Microsoft.WindowsAppSDK/1.3.230331000