Question: [Using Composition with Canvas]

[ifgotmoney]

Hello,

I want to build a desktop app with WinUI3, to allow draw some simple diagrams like flowsheet, etc, show texts along with lines or blocks.

in WPF I can either inherit from DrawingVisual object or Shape object. but I don't find enough examples to do this in C++/WinRT + WinUI3.

Question 1: what's the proper way to do it in WinUi3? I'm trying to use Microsoft.UI.Composition to do this, but I don't find a similar class like DrawingVisual, in WPF by inherit DrawingVisual, I can draw shapes or paths and render text in derived classes.

Question 2: should I use Microsoft.UI.Xaml.Shapes.Shape class as base instead? 2000 lines might be expected as the worst case in the future, to be zoomed in or out, or moved , selected something like that. is there any performance concern if I use Microsoft.UI.Xaml.Shapes.Shape ?

Question 3: should I use Microsoft.UI.Composition.CompositionSpriteShape as base class? or other class(please advise), if so, should it be a runtimeclass ,or just a normal C++ class? is *.idl file required?

Question 4: I want to render text along with these shapes, what's the right way to do so. just simple description is fine to me.

too less resource in C++/WinRT, what a pity.....
I have found some like this, and now just trying to do it in C++/WinRT.

Thank you!

[kmgallahan]

Another option: https://github.com/microsoft/Win2D

[StephenLPeters]

@codendone and @MikeHillberg Thoughts?

[chrkon]

Another option: https://github.com/microsoft/Win2D

Win2D is a good Option. Just drawing on a SwapChainPanel.
See Introduction or Win2D-WinUI3 WithoutControls

[ifgotmoney]

#microsoft/Win2D#869 (comment)

Thank you for your replies above. I have tried with win2D, with either CanvasControl and CanvasSwapChainPanel , successfully draw about 2000 circles on the canvas, this is fine to me. but I have a problem here, and I'm not sure it's a Win2D problem or a WinUi3 problem.

the problem is : I have hooked up with the PointerPressed ``PointerMoved and PointerReleased events of both Canvas, and trying to move selected CanvasGeometry, it also works. But there's a significant lag between the cursor and the object being dragged/moved. but for example in MS Paint in win10, when drawing line or drag some geometry by using mouse, there's almost no lag between the draged object and the mouse cursor.

I had experienced same thing in WPF previously. Do I miss something? why the object being dragged cannot catch up with the mouse cursor?

I hope below sketch can clearly show my problem.

[ifgotmoney]

A snap video to show the lag between mouse cursor and dragged object

1.mp4

[kmgallahan]

It is likely because although Win2D uses immediate mode to draw things, the WinUI event system still only pumps out updates at whatever rate it can handle - which will always be slower than more directly hooking up to mouse events via Win32 in native code. Plus after getting the events you have additional code to process and then ask Win2D to update the position.

That event loop probably still operates at or around the rendering rate of WinUI.

Also note that running in Debug vs. Release mode can have a considerable impact on performance.

I'm unsure if you can bypass WinUI to listen more directly for input events.

Figuring out one of these might help:

https://github.com/microsoft/detours

https://docs.microsoft.com/en-us/previous-versions/windows/desktop/bb153251(v=vs.85)

Edit: You could maybe test using a rapid 1-5ms loop to constantly retrieve the pointer location via Win32 once it enters a Win2D canvas instead of relying on the event pump.

@codendone does this sound about right?

[chrkon]

Hi @ifgotmoney, it seems to me, that the moving procedure just starts when the pointer leaves the circle. Do you get the pointerMoved event directly after you start moving?

I my project I use the ManipulationDelta event to recognize the movement. I use it to scroll the panel content manually. From the ManipulationDeltaRoutedEventArgs I use the Delta.Translation Property to get the coordinates.

Just as Info: If you use this way and you get an offset in the Delta.Position property, check if the Container property is null. This can happen if the ManipulationStarting event is missing. (see: #6847)

[codendone]

Much of this likely comes down to how expensive is it to handle each input event and render the updated content. Measuring those two steps on a Release build might help identify room for optimization.

For example, XAML will send all pointer input it receives, and orders the pointer input vs. rendering as a queue. If, for example, 5 pointer input events get queued and the handling of that first event requests rendering, then rendering will be added to the queue after the other 4 input events which still need to be processed before finally rendering. If the handling of these input events is expensive, this can result in longer-than-desired delays before rendering, producing a low framerate during the drag and possible latency. Windows will automatically coalesce pointer input if too many queue up, which can help the drag catch up by skipping some input events. Because of all this, you should try to keep the cost of input handling low during a scenario like the topic of this thread.

@kmgallahan's suggestion to listen more directly for input events might potentially be helpful for optimizing input processing cost if you have a complex element tree which requires a lot of work to process for each event, such as to hit-test an element in a large and deep element tree. Using SwapChainPanel.CreateCoreIndependentInputSource could help if this is an issue, though I suspect it's not a significant contributor here, particularly if this circle dragging repro was part of a sample.

When are the commands sent to Win2D to update its rendering, and how many commands are required for each update? If it is fully redrawing on each input, this could lead to low framerate which could contribute to noticeable latency. Redrawing less frequently may help, such as during XAML's CompositionTarget.Rendering event (which should be registered only when you want callbacks every render frame and unregistered when you no longer need it, to avoid burning CPU unnecessarily). And depending on how the redraw is requested, if there are different options to request a partial update that may also help (I'm not familiar enough Win2D to know if this is relevant or possible).

Another thing worth considering is when the Win2D rendering is happening. I expect Win2D CanvasControl renders on the UI thread, which could be affected by input handling delays or anything else keeping the UI thread busy. I don't know if CanvasSwapChainPanel has a background thread rendering option; if it does, that could reduce latency.

[ifgotmoney]

Thank you all above buddies, I have solved most part of above issues except memory leak when dynamically create CanvasCachedGeometry and draw it like this

m_stroke = win2D::Geometry::CanvasCachedGeometry::CreateStroke(m_uncachedgeometry,2);
...

ds.DrawCachedGeometry(m_fill, Windows::UI::Colors::Yellow());
ds.DrawCachedGeometry(m_stroke, Windows::UI::Colors::Black());

with out above part, and re create geometries when needed there's no leaks, but if try above code, the memory consumption goes up every time when call above code, and never release, until set the m_uncachedgeometry to nullptr; not sure what happens there.

but this seems a win2D issue so I'd like to close this issue, if you want comment please fell free to re open it.