NEW: MouseEvents for InputSystem - Sending data from package & Samples and Tests #2207

ritamerkl · 2025-07-24T20:10:48Z

Description

This PR introduces sending pointer data (e.g. mouse, pen, touch...) to native for the OnMouse events driven by the InputSystem. It also adds tests and samples for the OnMouse events working with the InputSystem. The behaviour on native side is introduced with this PR and both PR's should land together, or one (shortly) after the other.

Testing status & QA

Only manual testing for the sample.

Overall Product Risks

Complexity: 1
Halo Effect: 1

Comments to reviewers

Please review together with https://github.cds.internal.unity3d.com/unity/unity/pull/73198.

Checklist

Before review:

Changelog entry added.
- Explains the change in Changed, Fixed, Added sections.
- For API change contains an example snippet and/or migration example.
- JIRA ticket linked, example (case %%). If it is a private issue, just add the case ID without a link.
- Jira port for the next release set as "Resolved".
Tests added/changed, if applicable.
- Functional tests Area_CanDoX, Area_CanDoX_EvenIfYIsTheCase, Area_WhenIDoX_AndYHappens_ThisIsTheResult.
- Performance tests.
- Integration tests.
Docs for new/changed API's.
- Xmldoc cross references are set correctly.
- Added explanation how the API works.
- Usage code examples added.
- The manual is updated, if needed.

During merge:

Commit message for squash-merge is prefixed with one of the list:
- NEW: ___.
- FIX: ___.
- DOCS: ___.
- CHANGE: ___.
- RELEASE: 1.1.0-preview.3.

After merge:

Create forward/backward port if needed. If you are blocked from creating a forward port now please add a task to ISX-1444.

codecov-github-com · 2025-07-24T22:43:04Z

Codecov Report

All modified and coverable lines are covered by tests ✅

@@             Coverage Diff             @@
##           develop    #2207      +/-   ##
===========================================
- Coverage    68.15%   68.15%   -0.01%     
===========================================
  Files          367      367              
  Lines        53662    53663       +1     
===========================================
- Hits         36576    36574       -2     
- Misses       17086    17089       +3

Flag	Coverage Δ
linux_2021.3_pkg	`5.45% <ø> (ø)`
linux_2021.3_project	`70.39% <ø> (ø)`
linux_2022.3_pkg	`5.23% <ø> (ø)`
linux_2022.3_project	`65.24% <ø> (ø)`
linux_6000.0_pkg	`5.23% <ø> (-0.01%)`	⬇️
linux_6000.0_project	`67.99% <ø> (ø)`
linux_6000.1_pkg	`5.23% <ø> (ø)`
linux_6000.1_project	`68.05% <ø> (+0.02%)`	⬆️
linux_6000.2_pkg	`5.23% <ø> (ø)`
linux_6000.2_project	`68.04% <ø> (+0.01%)`	⬆️
linux_trunk_pkg	`?`
mac_2021.3_pkg	`5.45% <ø> (ø)`
mac_2021.3_project	`70.40% <ø> (ø)`
mac_2022.3_pkg	`5.22% <ø> (ø)`
mac_2022.3_project	`65.26% <ø> (ø)`
mac_6000.0_pkg	`5.23% <ø> (ø)`
mac_6000.0_project	`68.06% <ø> (ø)`
mac_6000.1_pkg	`5.23% <ø> (ø)`
mac_6000.1_project	`68.06% <ø> (ø)`
mac_6000.2_pkg	`5.23% <ø> (ø)`
mac_6000.2_project	`68.05% <ø> (ø)`
win_2021.3_pkg	`5.45% <ø> (ø)`
win_2021.3_project	`70.48% <ø> (ø)`
win_2022.3_pkg	`5.23% <ø> (ø)`
win_2022.3_project	`65.34% <ø> (ø)`
win_6000.0_pkg	`5.23% <ø> (ø)`
win_6000.0_project	`68.13% <ø> (ø)`
win_6000.1_pkg	`5.23% <ø> (ø)`
win_6000.1_project	`68.13% <ø> (ø)`
win_6000.2_pkg	`5.23% <ø> (ø)`
win_6000.2_project	`68.13% <ø> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
.../com.unity.inputsystem/InputSystem/InputManager.cs	`88.77% <ø> (ø)`

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

jfreire-unity

Nice work @ritamerkl ! I like the approach and the sample is of reasonable size to test the main functionality. Also really nice to have plenty of tests.

However:

This functionality needs to be guarded with defines in asmdef, as this will only be available from 6.3 forward AFAIK. So if I try this PR without the trunk PR, I'll get compilation errors.
I would like to see at least 1 test with Touch as well, particularly with more than 1 touch. Mouse and Pen are pretty similar as they are a single pointer.

Assets/Tests/InputSystem/CoreTests_MouseEvents.cs

Packages/com.unity.inputsystem/CHANGELOG.md

Assets/Samples/MouseEvents/OnMouseEventsTest.cs

jfreire-unity · 2025-07-25T08:01:52Z

Nice work @ritamerkl ! I like the approach and the sample is of reasonable size to test the main functionality. Also really nice to have plenty of tests.

However:

This functionality needs to be guarded with defines in asmdef, as this will only be available from 6.3 forward AFAIK. So if I try this PR without the trunk PR, I'll get compilation errors.

I would like to see at least 1 test with Touch as well, particularly with more than 1 touch. Mouse and Pen are pretty similar as they are a single pointer.

Also there's some CI failures that will need to be addressed. Not sure if it's only because of the lack of defines or if there's any impact on pointer devices functionality.

ritamerkl · 2025-07-25T08:54:33Z

WIP: working on adding touch samples (multi-touch) & parallel pointers (eg. touch&pen).

LeoUnity

This is looking great!
I'm happy to see us improving our feature parity! Also thanks for investing the time on the tests!
I've left a few comments, but I'll leave it up to you to decide which ones to address

Assets/Samples/MouseEvents/OnMouseEventsTest.cs

LeoUnity · 2025-07-28T14:35:57Z

Assets/Tests/InputSystem/CoreTests_MouseEvents.cs

+    [Category("MouseEvents")]
+    public IEnumerator PenEvents_CanReceiveOnMouseDown()
+    {
+        var pen = InputSystem.AddDevice<Pen>();


It seems like we are duplicating the exact same tests for both mouse and pen, I'm wondering if we can parameterize that and just have a single test...

Agree to previous comment, this isn't necessarily the case any longer either right? "That we have to queue into NativeInputRuntime"?

Done: 937d7b6

untitled folder/Il2CppOutputProject/Source/il2cppOutput/il2cpp-compile.traceevents

ekcoh

I see 3 files under "untitled folder", the folder including those files should be removed from this PR right?

ekcoh · 2025-08-22T13:26:53Z

Packages/com.unity.inputsystem/InputSystem/InputManager.cs

@@ -3646,6 +3647,9 @@ private unsafe void OnUpdate(InputUpdateType updateType, ref InputEventBuffer ev
            ////       mess in the event buffer
            ////       same goes for events that someone may queue from a change monitor callback
            InvokeAfterUpdateCallback(updateType);
+            //send pointer data to backend for OnMouseEvents
+            if (Pointer.current != null && gameIsPlaying)


@ritamerkl Have you checked if Input Manager supports OnMouseXXX events from a script that is marked with [ExecuteInEditorMode]? That could be relevant here...

I don't think so, since the InputManager MouseEvents were only sent from the InputModuleRegistration player loop callback when ::IsWorldPlayingThisFrame() was true, which basically is only true if in playmode, not editmode. Unless I miss the way [ExecuteInEditorMode] is working now.

OK, then I believe we might be safe, would to good to make sure we haven't overlooked something like that though.

ekcoh · 2025-08-22T13:28:07Z

Packages/com.unity.inputsystem/InputSystem/InputManager.cs

@@ -3646,6 +3647,9 @@ private unsafe void OnUpdate(InputUpdateType updateType, ref InputEventBuffer ev
            ////       mess in the event buffer
            ////       same goes for events that someone may queue from a change monitor callback
            InvokeAfterUpdateCallback(updateType);
+            //send pointer data to backend for OnMouseEvents
+            if (Pointer.current != null && gameIsPlaying)
+                NativeInputSystem.SetMouseEventsData(Pointer.current.press.isPressed, Pointer.current.press.wasPressedThisFrame, Pointer.current.position.x.value, Pointer.current.position.y.value);


Shouldn't this be routed via INativeInputRuntime interface and its NativeInputRuntime and TestInputRuntime implementations?

NativeInputRuntime is the name of the Bindings class in the module, I fail to understand what exactly you are referring to.
I placed it here to have all updated data after the inputsystem Update, this happens for test as well.

I referred to the fact that this is calling NativeInputSystem directly. If you look at NativeInputRuntime.cs you will see it implements INativeInputRuntime as an abstraction of the module to allow stubbing it away in e.g. tests. Hence InputTestRuntime also exists stubbing these calls. It's basically a mocking strategy that has historically been used. If this is possible to auto-test without mocking, it should right? Then I see no reason why this would be just fine, but just wanted to let you know in case you had not noticed since it seems SetMouseEventsData is part of Input module based on where its placed/mapped?

And NativeInputRuntime is not the bindings class, it's just an abstraction within the package. The bindings class (P/Invoke) is called NativeInputSystem which is the one you are calling here

I was adding this same comment. We should call this as m_Runtime.SetMouseEventsData(...);. This avoids making other classes (like this one) dependent on the bindings as well. This way we only have NativeInputRuntime class as a single dependent of the Module bindings (NativeInputSystem) and make it consistent across the codebase as is.

And by making it part of IInputRuntime interface we leverage the mocking strategy @ekcoh mentioned as well. So I think this should be improved.

…echnologies/InputSystem into mouseevents-samles-tests

…inter device

ekcoh · 2025-08-27T18:13:34Z

Assets/Samples/MouseEvents/OnMouseEventSample.cs

+using UnityEngine.InputSystem.EnhancedTouch;
+using Touch = UnityEngine.InputSystem.EnhancedTouch.Touch;
+
+public class OnMouseEventSample : MonoBehaviour


If the intent here is to serve as a sample (and maybe also our own way to manually test or debug this), I would suggest that we have all the supported implicit script mouse event callback functions present in this file.

E.g. missing OnMouseOver, OnMouseEnter, OnMouseExit, OnMouseUp, OnMouseUpAsButton

ekcoh · 2025-08-27T18:22:06Z

Assets/Tests/InputSystem/CoreTests_MouseEvents.cs

@@ -0,0 +1,199 @@
+#if UNITY_6000_4_OR_NEWER


Great you added automated tests for all the events! This way we protect ourselves from regressions in this area. Similarly I think there is a lot to gain from having them covered in the sample as well.

ekcoh · 2025-08-27T18:23:10Z

Assets/Tests/InputSystem/CoreTests_MouseEvents.cs

+using UnityEngine.InputSystem;
+using UnityEngine.TestTools;
+using Is = UnityEngine.TestTools.Constraints.Is;
+


Nitpick: Redundant empty row

ekcoh · 2025-08-27T18:24:55Z

Assets/Tests/InputSystem/CoreTests_MouseEvents.cs

+
+    private static HashSet<Type> _testDevices = new HashSet<Type> { typeof(Mouse), typeof(Pen), typeof(Touchscreen) };
+    // OnMouseOver/Exit and Hover events are not supported for touch
+    private static HashSet<Type> _testDevicesNoTouch = new HashSet<Type> { typeof(Mouse), typeof(Pen) };


Isn't a HashSet overkill here (dynamic allocation etc) when function with a switch statement with 2 cases would be enough?

I can't see (or am I missing something) that we use lookup in the tests? If not, then I would recommend a plain static readonly array for these. (Less allocation, leaner, simpler) and also likely much faster. Also, when using HashSet, the iteration order is "random"

ekcoh · 2025-08-27T18:29:48Z

Assets/Tests/InputSystem/CoreTests_MouseEvents.cs

+    [Category("MouseEvents")]
+    public IEnumerator MouseEvents_CanReceiveOnMouseDown([ValueSource(nameof(_testDevices))] Type pointerType)
+    {
+        var pointer = (Pointer)InputSystem.s_Manager.AddDevice(pointerType);


Nitpick: It looks like this

var pointer = (Pointer)InputSystem.s_Manager.AddDevice(pointerType); InputSystem.AddDevice(pointer);

Is repeated for all the tests. Can't we just embed it into SetUpScene() and have it as an out parameter from that function to reduce redundant code?

ekcoh · 2025-08-27T18:32:57Z

Assets/Tests/InputSystem/CoreTests_MouseEvents.cs

+        gameObject.AddComponent<OnMouseEventsTest>();
+        var vec = Camera.main.WorldToScreenPoint(gameObject.transform.position);
+
+        Set(pointer, new Vector2(vec.x, vec.y), true, false);


Throughout all of these tests I see we have the same pattern, e.g. Set(...); yield return null; (repeat)

I would be reassuring we also verified this works correctly without the yields, e.g. when events are not in separate frames. Maybe it's possible to parameterise the test with another variable e.g. bool yieldBetweenSet so that we could test both scenario by having the yields not happening when that is false. Maybe in that case we only need to yield at the end to trigger script callbacks?

Based on how this have been done in InputManager.cs I wonder if my comment above doesn't make sense. It seems we only call the script once per frame which I wonder if its valid. E.g. that way there wouldn't be any events for e.g. two events in the same frame. Is this really how it works with InputManager? This is the main thing to sort out before I approve. Maybe its useful to test InputManager vs Input System with forced 1 FPS frame rate to sort this out?

ekcoh · 2025-08-27T18:49:39Z

Assets/Tests/InputSystem/CoreTests_MouseEvents.cs

+
+    void Set(Pointer pointer, Vector2 point, bool pressed, bool released)
+    {
+        if (pointer is Touchscreen touchscreen)


Is Pen/Stylus falling under the same behaviour as mouse? Or is it excluded from this functionality somehow?

ekcoh · 2025-08-27T18:55:16Z

Packages/com.unity.inputsystem/CHANGELOG.md

@@ -16,6 +16,7 @@ however, it has to be formatted properly to pass verification tests.

 ### Added
 - Exposed MediaPlayPause, MediaRewind, MediaForward keys on Keyboard.
+- Added OnMouse events for the InputSystem from editor version 6.3, including samples and tests.


I would suggest saying "Added support for (MonoBehavior OnMouse-events) [https://docs.unity3d.com/ScriptReference/MonoBehaviour.html] when running the Input System on Unity 6.3 or newer."

Then I would make a separate entry for the sample linking to it. It makes more sense to say Unity 6.3 IMO since editor might indicate it would only be supported in editor but this is for players as well right?

ekcoh · 2025-08-27T19:05:11Z

Packages/com.unity.inputsystem/InputSystem/InputManager.cs

+            //send pointer data to backend for OnMouseEvents
+#if UNITY_6000_4_OR_NEWER
+            if (Pointer.current != null && gameIsPlaying)
+                NativeInputSystem.SetMouseEventsData(Pointer.current.press.isPressed, Pointer.current.press.wasPressedThisFrame, Pointer.current.position.x.value, Pointer.current.position.y.value);


Thanks for putting this together @ritamerkl, I added a few suggestions and my only main concern is still whether this will behave similar to Input Manager when we only call into the SetMouseEventData(...) function once every frame - this looks incorrect to me. I would have expected that this would not be based on a polling-based evaluation like this here but rather sit inside the event processing loop above or from within the device state updates after event compression. Does this work for e.g. 1 FPS?

Maybe I am overlooking something.... but was Input Manager only calling this once every frame?

ekcoh · 2025-08-27T19:07:43Z

I see CI is failing with "Packages\com.unity.inputsystem\InputSystem\InputManager.cs(3710,35): error CS0117: 'NativeInputSystem' does not contain a definition for 'SetMouseEventsData'" but this would be expected until the feature is available in Unity.

ritamerkl · 2025-08-28T09:13:30Z

I see CI is failing with "Packages\com.unity.inputsystem\InputSystem\InputManager.cs(3710,35): error CS0117: 'NativeInputSystem' does not contain a definition for 'SetMouseEventsData'" but this would be expected until the feature is available in Unity.

It is weird since this is wrapped in defines for unity 600.4 or newer, but still fails on 600.3 🤔

jfreire-unity · 2025-08-28T13:50:43Z

Packages/com.unity.inputsystem/InputSystem/InputManager.cs

@@ -3703,6 +3704,11 @@ private unsafe void OnUpdate(InputUpdateType updateType, ref InputEventBuffer ev
            ////       mess in the event buffer
            ////       same goes for events that someone may queue from a change monitor callback
            InvokeAfterUpdateCallback(updateType);
+            //send pointer data to backend for OnMouseEvents
+#if UNITY_6000_4_OR_NEWER
+            if (Pointer.current != null && gameIsPlaying)


This means we will be calling this every frame, regardless of having received an event from a Pointer device or not, right?

We need to look at how its implemented under the hood, there might even be timeout logic in there which would require it to be once a frame.

there might even be timeout logic in there which would require it to be once a frame.

I'm not sure I understand what you mean with this. Could you specify it a bit?

jfreire-unity · 2025-08-29T07:13:57Z

Assets/Tests/InputSystem/CoreTests_MouseEvents.cs

+        var pointer = (Pointer)InputSystem.s_Manager.AddDevice(pointerType);
+        InputSystem.AddDevice(pointer);


I think it's great we have tests with the Pointer abstraction. But could we have also some real device tests, for Touch, Pen and Mouse devices to make sure the events are generated for these devices?

You can take advantage of TestCase attribute pattern already used in other tests in our codebase. Something like:

[TestCase("Pointer")] [TestCase("Pen")] [TestCase("Touchscreen")] [TestCase("Mouse")] MouseEvents_CanReceiveOnMouseUp(string deviceLayout) { var device = InputSystem.AddDevice(layoutName); .... }

This removes the need for private static devices such as HashSet _testDevices

ekcoh · 2025-08-29T08:04:24Z

Packages/com.unity.inputsystem/InputSystem/InputManager.cs

@@ -3703,6 +3704,11 @@ private unsafe void OnUpdate(InputUpdateType updateType, ref InputEventBuffer ev
            ////       mess in the event buffer
            ////       same goes for events that someone may queue from a change monitor callback
            InvokeAfterUpdateCallback(updateType);
+            //send pointer data to backend for OnMouseEvents
+#if UNITY_6000_4_OR_NEWER


I would advise against this. Instead, create an asmdef symbol that defines the feature/capability, e.g. UNITY_INPUTSYSTEM_HAS_MOUSE_SCRIPT_EVENTS_SUPPORTED, then let that symbol be conditionally compiled based on the specific version that contains the native changes, e.g. 6000.0.4.a7, that is more specific and also removes versions from the source code which only cares about a certain API being a certain way instead. And it centralises the version number mess. At least this is what we agreed upon in #2176, and the same technique was used in #2200 to handle the API difference in module API and behaviour .

ritamerkl added 4 commits July 17, 2025 17:30

added a very basic mouse events sample

857e304

added MouseEventScript

3cc804f

added mouse events test for pen and mouse

dcf359a

add changelog

b1cf800

ritamerkl requested review from Pauliusd01, LeoUnity and jfreire-unity July 24, 2025 20:11

jfreire-unity requested changes Jul 25, 2025

View reviewed changes

jfreire-unity reviewed Jul 25, 2025

View reviewed changes

Assets/Tests/InputSystem/CoreTests_MouseEvents.cs Outdated Show resolved Hide resolved

Packages/com.unity.inputsystem/CHANGELOG.md Outdated Show resolved Hide resolved

Assets/Samples/MouseEvents/OnMouseEventsTest.cs Outdated Show resolved Hide resolved

ritamerkl added 2 commits July 25, 2025 10:47

added defines and outsourced code duplication

b0201e1

mention editor version in changelog

bab6e35

ritamerkl and others added 4 commits July 25, 2025 11:14

remove enhanced touch enable

c3e5dc3

Merge branch 'develop' into mouseevents-samles-tests

8f2c237

added touch test

167eb0f

added more touch tests

ff22909

LeoUnity approved these changes Jul 29, 2025

View reviewed changes

added sending pointer data to native for OnMouse events

18b1cc8

ritamerkl changed the title ~~NEW: MouseEvents for InputSystem Samples & Tests~~ NEW: MouseEvents for InputSystem - Sending data from package & Samples and Tests Aug 22, 2025

ekcoh requested changes Aug 22, 2025

View reviewed changes

ekcoh reviewed Aug 22, 2025

View reviewed changes

ritamerkl and others added 6 commits August 22, 2025 16:21

Merge branch 'develop' into mouseevents-samles-tests

042bc00

remove build

e4e88bb

Merge branch 'mouseevents-samles-tests' of https://github.com/Unity-T…

77c89af

…echnologies/InputSystem into mouseevents-samles-tests

refactored OnMouseEventSample

6429e90

added defines

9e161b3

refactored tests to send events to package queue and reuse for all po…

937d7b6

…inter device

ritamerkl requested review from ekcoh and jfreire-unity August 27, 2025 15:09

ekcoh reviewed Aug 27, 2025

View reviewed changes

jfreire-unity reviewed Aug 28, 2025

View reviewed changes

jfreire-unity reviewed Aug 29, 2025

View reviewed changes

ekcoh reviewed Aug 29, 2025

View reviewed changes

set data and trigger sending events both from package

b9fcced

		var pointer = (Pointer)InputSystem.s_Manager.AddDevice(pointerType);
		InputSystem.AddDevice(pointer);

NEW: MouseEvents for InputSystem - Sending data from package & Samples and Tests #2207

Are you sure you want to change the base?

NEW: MouseEvents for InputSystem - Sending data from package & Samples and Tests #2207

Uh oh!

Conversation

ritamerkl commented Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing status & QA

Overall Product Risks

Comments to reviewers

Checklist

Uh oh!

codecov-github-com bot commented Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jfreire-unity left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jfreire-unity commented Jul 25, 2025

Uh oh!

ritamerkl commented Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LeoUnity left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ekcoh left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ekcoh Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jfreire-unity Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ritamerkl commented Jul 24, 2025 •

edited

Loading

codecov-github-com bot commented Jul 24, 2025 •

edited

Loading

ritamerkl commented Jul 25, 2025 •

edited

Loading

ekcoh Aug 27, 2025 •

edited

Loading

jfreire-unity Aug 28, 2025 •

edited

Loading

ritamerkl commented Aug 28, 2025 •

edited

Loading

ekcoh Aug 29, 2025 •

edited

Loading