[Wasm] When Css Style "clip-path" applied to Canvas, Canvas receive wrong Pointer Event.

[easy28]

Current behavior

Consider below example, GreenCanvas inside the RedCanvas, and I set css style "clip-path" on GreenCanvas : GreenCanvas.SetCssStyle("clip-path", "path('M300,0 500,50 300,100z')");

When pointer moving on red area, GreenCanvas wrongly received the pointer move events.

This error start from Uno.UI/ Uno.UI.WebAssembly version 5.4.0-dev.1485 . before this version it's correct.

        <Canvas Width="500" Height="100" Background="Red" x:Name="RedCanvas" PointerMoved="Canvas_PointerMoved" >
            <Canvas  Width="500" Height="100" Background="Green" x:Name="GreenCanvas" PointerMoved="Canvas_PointerMoved"> </Canvas>
        </Canvas>

        public TestPanel()
        {
            this.InitializeComponent();
            GreenCanvas.SetCssStyle("clip-path", "path('M300,0 500,50 300,100z')");
        }

        private void Canvas_PointerMoved(object sender, PointerRoutedEventArgs e)
        {
            Console.WriteLine((sender as Canvas).Name + " moved");
            e.Handled = true;
        }

Thank you!

Expected behavior

No response

How to reproduce it (as minimally and precisely as possible)

No response

Workaround

No response

Works on UWP/WinUI

None

Environment

Uno.UI / Uno.UI.WebAssembly / Uno.UI.Skia

NuGet package version(s)

No response

Affected platforms

WebAssembly

IDE

Visual Studio 2022

IDE version

No response

Relevant plugins

No response

Anything else we need to know?

No response

[MartinZikmund]

@ramezgerges could it be related to hit testing changes, which are now purely managed?

[ramezgerges]

I'm pretty sure that moving WASM pointers to managed means that any native clipping won't affect hit-testing at all (since we do our own hit tests), but I didn't work on that, so I'm not sure. Maybe @dr1rrb can chime in?

[easy28]

So can we have a setting to switch between managed and native pointers in WASM?

Thank you

[Youssef1313]

@easy28 IMO, it's in general not a good idea to alter "clip-path" yourself. For example, if we implemented #13782, then our own layout clipping will be competing on the same property that you are trying to set (though it won't happen for Canvas specifically as it's never layout clipped by us).

I think a better way to achieve the same rendering would be something like this:

<Canvas Width="500" Height="100" Background="Red" x:Name="RedCanvas">
    <Path Width="500" Height="100" Fill="Green" x:Name="GreenPath" Data="M 300,0 500,50 300,100z" />
</Canvas>

Can you try that and let us know if pointers don't work properly with that code?

Note: the above will also work on all platforms, while what you have will only work on Wasm

[easy28]

@Youssef1313 In fact, we want clip an image by "clip-path", not just show a shape.

<Canvas Width="500" Height="500" Background="Red" x:Name="RedCanvas" PointerMoved="Canvas_PointerMoved" >
    <Image Source="/Assets/Sample.jpeg" PointerMoved="Canvas_PointerMoved" x:Name="InnerImage" />
</Canvas>

InnerImage.SetCssStyle("clip-path", "path('M300,0 500,50 300,100z')");

[ramezgerges]

@easy28 This isn't as powerful as clip-path, but you can do something similar with a Path and an ImageBrush:

	<Canvas Width="500" Height="500" Background="Red" x:Name="RedCanvas">
		<Path>
			<Path.Fill>
				<ImageBrush ImageSource="/Assets/Sample.jpeg" />
			</Path.Fill>
			<Path.Data>
				M300,0 500,50 300,100z
			</Path.Data>
		</Path>
	</Canvas>

[easy28]

@ramezgerges @Youssef1313 This method still can't clip UIElement e.g. Canvas. We have a big project rely on 'clip-path' and it works perfect before Uno.UI version 5.4.0-dev.1441. After this version, it can't work correctly. So could you please add a setting to switch between managed and native pointers.

Thank you very much!

[MartinZikmund]

@easy28 unfortunately I doubt switching between managed and native pointers is possible, as it is a completely different infrastructure involved (am I correct @dr1rrb?)

[easy28]

is it posible to add a setting (switch between managed and native pointers) in the Wasm Project File?

[github-actions]

This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don't have enough information to take action. We don't monitor discussions on closed issues thus please open a new GitHub issue if you need the team to revisit this matter.

[easy28]

@dr1rrb @MartinZikmund is it posible to add a setting (switch between managed and native pointers) in Uno.UI.FeatureConfiguration ? so we can use back the native pointers in wasm.

Thanks a lot

[Xiaoy312]

So we did some major update regarding how the pointer are handled in wasm back in 5.4 (#17633). Instead of handling pointer events from every single view, we are now handling them centrally at document.body level with uno taking care of dispatching, rather than the browser. Now of course there are some discrepancies, as you have noticed. However, this is not something that can be easily reverted as it addresses others problem, and we can neither make it a toggle.

That said, we can compromise with what we already have or take the best of both worlds. If the end goal was to clip-path an <Image>, while having its hitbox matches the clipped shape, we can combine your approach and the workaround proposed above. The idea is to clip-path the image (or its container) and make them not hit-visible, and then superpose a path with the same clip as the hitbox for the pointer events. (Here, unlike clip-path which uno are blind to, the Path.Data is something known by us, so we can dispatch the event properly.)

ClippableImage.cs

using Microsoft.UI.Xaml.Input;
using Path = Microsoft.UI.Xaml.Shapes.Path;

namespace u18458;

#if __WASM__

[TemplatePart(Name = TemplateParts.InnerImage, Type = typeof(Image))]
[TemplatePart(Name = TemplateParts.HitBoxPath, Type = typeof(Path))]
public partial class ClippableImage : Control
{
	private static class TemplateParts
	{
		public const string InnerImage = nameof(InnerImage);
		public const string HitBoxPath = nameof(HitBoxPath);
	}

	private Image? _image;
	private Path? _hitbox;

	public event PointerEventHandler? ClippedPointerEntered;
	public event PointerEventHandler? ClippedPointerExited;
	public event PointerEventHandler? ClippedPointerMoved;
	public event PointerEventHandler? ClippedPointerPressed;

	#region DependencyProperty: Source

	public static DependencyProperty SourceProperty { get; } = DependencyProperty.Register(
		nameof(Source),
		typeof(ImageSource),
		typeof(ClippableImage),
		new PropertyMetadata(default(ImageSource)));

	public ImageSource? Source
	{
		get => (ImageSource?)GetValue(SourceProperty);
		set => SetValue(SourceProperty, value);
	}

	#endregion
	#region DependencyProperty: ClipPath

	public static DependencyProperty ClipPathProperty { get; } = DependencyProperty.Register(
		nameof(ClipPath),
		typeof(string),
		typeof(ClippableImage),
		new PropertyMetadata(default(string), OnClipPathChanged));

	public string? ClipPath
	{
		get => (string?)GetValue(ClipPathProperty);
		set => SetValue(ClipPathProperty, value);
	}

	#endregion

	public ClippableImage()
	{
		this.Loaded += (s, e) => UpdateClip();
	}
	protected override void OnApplyTemplate()
	{
		base.OnApplyTemplate();

		_image = GetTemplateChild(TemplateParts.InnerImage) as Image;
		_hitbox = GetTemplateChild(TemplateParts.HitBoxPath) as Path;
		if (_hitbox is { })
		{
			_hitbox.PointerMoved += (s, e) => ClippedPointerEntered?.Invoke(this, e);
			_hitbox.PointerExited += (s, e) => ClippedPointerExited?.Invoke(this, e);
			_hitbox.PointerMoved += (s, e) => ClippedPointerMoved?.Invoke(this, e);
			_hitbox.PointerPressed += (s, e) => ClippedPointerPressed?.Invoke(this, e);
		}

		UpdateClip();
	}

	private static void OnClipPathChanged(DependencyObject sender, DependencyPropertyChangedEventArgs e) => (sender as ClippableImage)?.UpdateClip();

	private void UpdateClip()
	{
		if (!IsLoaded) return;

		if (!string.IsNullOrEmpty(ClipPath))
		{
			this.SetCssStyle("clip-path", $"path('{ClipPath}')");
		}
		else
		{
			this.ClearCssStyle("clip-path");
		}

		if (_hitbox is { })
		{
			_hitbox.Data = ClipPath;
		}
	}
}
#endif

ClippableImage.xaml

make sure to reference this in your app.xaml directly or indirectly

<ResourceDictionary xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
					xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
					xmlns:local="using:u18458">

	<Style TargetType="local:ClippableImage">
		<Setter Property="Template">
			<Setter.Value>
				<ControlTemplate TargetType="local:ClippableImage">
					<Grid Background="{x:Null}">
						<Image x:Name="InnerImage" Source="{TemplateBinding Source}" IsHitTestVisible="False" />
						<Path x:Name="HitBoxPath" Fill="Transparent" />
					</Grid>
				</ControlTemplate>
			</Setter.Value>
		</Setter>
	</Style>

</ResourceDictionary>

usage:

<Canvas x:Name="RedCanvas2"
		Background="Red"
		Width="500"
		Height="100"
		PointerMoved="Canvas_PointerMoved">
	<local:ClippableImage Width="500"
						  Height="100"
						  Source="https://placehold.co/500x100/orange/white"
						  ClipPath="M300,0 500,50 300,100z"
						  ClippedPointerMoved="Canvas_PointerMoved" />
</Canvas>

full solution: u18458.zip

[easy28]

hi Xiaoy312,

Thanks for the solution, now the main problem is performance, our project have more than 1000+ clipped images, with extra Grid & Path layer may double the load time, and other runtime performance down.

Is it posible to add ClipPath attribute to UIElement in wasm, and consider the ClipPath when uno handle dispatching? (Since you can properly dispatch event to Path with Path.Data)

Thank you very much!

[Xiaoy312]

That would be unlikely to happen, as we try mostly to respect the winui api. If I had to suggest, you can implement custom js listener on click/pointerdown. To cross between cs and js, you can use [JSImport] and [JsExport], see: https://platform.uno/docs/articles/external/uno.wasm.bootstrap/doc/features-interop.html.

Or, in the Canvas_PointerMoved, you can check if the pointer is inside of the path to decide if the event should be handled. (Bu, Im not sure how you can hit-test that.)

[easy28]

Thanks for your advice, I will try these methods.

[Xiaoy312]

@easy28 I have updated the sample with the native event injection approach:
u18458.zip

// just make sure the helper is initialized:
this.Loaded += (s, e) => NativeJsEventHelper.InitializeAsync(); // with this being Shell or MainPage

// then you can subscribe like so:
NativeJsEventHelper.AddEventListener(MyImage, "click", OnNativeClicked);

[easy28]

@Xiaoy312 That's great , so we can implement NativePointerPressed / NativePointerMoved / NativePointerReleased event
as well?

Thanks a lot

[Xiaoy312]

You can implement the same for "NativePointerPressed" via pointerdown and "NativePointerReleased" via pointerup following the click example.
For NativePointerMoved that would be pointerover, but you would likely have to pass back the pointer x,y coords & possibly need to remap it into winui coords space.

If you need further assistance in these areas, please share with us what you intent with do with these events (some examples or existing pointer handlers), so we can better update the sample.