Pointer EventManager state

[alvarosabu]

Description

As a developer using TresJS, I would like to have an Event Management solution with the following features:

• Support for:
  • onClick
  • onContextMenu (rightClick)
  • onDoubleClick
  • onWheel
  • onPointerDown
  • onPointerUp
  • onPointerLeave
  • onPointerMove
  • onPointerCancel
  • onLostPointerCapture
• Event prioritization
• primitive pointer events
• Event bubbling and propagation Event bubbling #501 TresGroups Pointer events missing #426

Propagation through intersected objects

Raycasting-Based Interaction: Tres should use Three.js's raycasting to determine which objects are interacted with. A ray is cast from the camera through the mouse position into the 3D space, and intersections with objects are calculated.

Simulated Bubbling: When an event occurs, Tres might propagate it through objects based on their spatial arrangement (like from child to parent), but this is based on the raycast hits and not a strict parent-child hierarchy as in the DOM.

Meaning that stop propagation is based on occlusion

If the object is a Group or a model consistent with several meshes, the same concept applies, the closest mesh to the camera stops the propagation

Suggested solution

Current solution uses:

• useRaycaster https://github.com/Tresjs/tres/blob/main/src/composables/useRaycaster/index.ts
• usePointerEventHandler https://github.com/Tresjs/tres/tree/main/src/composables/usePointerEventHandler

Register of events is being done here
https://github.com/Tresjs/tres/blob/main/src/composables/usePointerEventHandler/index.ts#L57-L62

const registerObject = (object: Object3D & EventProps) => {
  const { onClick, onPointerMove, onPointerEnter, onPointerLeave } = object

  if (onClick) objectsWithEventListeners.click.set(object, onClick)
  if (onPointerMove) objectsWithEventListeners.pointerMove.set(object, onPointerMove)
  if (onPointerEnter) objectsWithEventListeners.pointerEnter.set(object, onPointerEnter)
  if (onPointerLeave) objectsWithEventListeners.pointerLeave.set(object, onPointerLeave)
}

  // to make the registerObject available in the custom renderer (nodeOps), it is attached to the scene
  scene.userData.tres__registerAtPointerEventHandler = registerObject
  scene.userData.tres__deregisterAtPointerEventHandler = deregisterObject

  scene.userData.tres__registerBlockingObjectAtPointerEventHandler = registerBlockingObject
  scene.userData.tres__deregisterBlockingObjectAtPointerEventHandler = deregisterBlockingObject

These are then used on the renderer by saving them on the userData of the scene object

insert(child, parent) {
  if (parent && parent.isScene) scene = parent as unknown as TresScene

  const parentObject = parent || scene

  if (child?.isObject3D) {
    if (
      child && supportedPointerEvents.some(eventName => child[eventName])
    ) {
      if (!scene?.userData.tres__registerAtPointerEventHandler)
        throw 'could not find tres__registerAtPointerEventHandler on scene\'s userData'

      scene?.userData.tres__registerAtPointerEventHandler?.(child as Object3D)
    }
  }

https://github.com/Tresjs/tres/blob/main/src/core/nodeOps.ts#L102

Desired solution

A state/store to manage the events

Alternative

No response

Additional context

No response

Validations

• I agree to follow this project's Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.

[andretchen0]

Imagine each mesh below is visible on screen and not occluded by any other meshes.

<TresCanvas :on-click="() => console.log('Scene')">
  <TresGroup :on-click="() => console.log('Group')">
    <Sphere :on-click="() => console.log('Sphere')">
        <Torus :on-click="() => console.log('Torus')" />
        <TorusRing />
    </Sphere>
    <Box />
  </TresGroup>
</TresCanvas>

What should be in the console when ...

• Torus is clicked?
• TorusRing is clicked?
• Sphere is clicked?
• Box is clicked?
• the background is clicked?

[alvarosabu]

Hi @andretchen0 regarding your example

• TresCanvas is not meant to have a click event
• Sphere, Torus, and TorusRing are meant to be meshs and siblings ? You can pass geometries and materials to a mesh, but not children to a sphere right?

In the case that you have a group and the children are not occluding, (example you click on the sphere inside of the group) then it will trigger the console for Sphere and then the on for Group

In r3f (we don't have stopPropagation yet) If you do <Sphere @click="(e) => e.stopPorpagation()"> then only the Sphere should prompt, group will remain silent.

[andretchen0]

Hey @alvarosabu ,

• You can pass geometries and materials to a mesh, but not children to a sphere right?

Maybe we're talking about different things. I should have included working code. Here's a StackBlitz with the kind of nesting I'm talking about.

So, with that kind of setup, assume Box2 doesn't have an event handler, but its parent (Box1) does.

<Box> <!-- Box0 -->
    <Box :on-click="() => alert('Box1')"> <!-- Box1 -->
        <Box /> <!-- Box2 -->
    </Box>
</Box>

Is Box2 clickable? If so and it's clicked, will Box1's :on-click be triggered?

Assuming we're doing events like the DOM does, it seems to me that the answer to both is "yes". I just want to make sure my understanding is correct.

[garrlker]

<TresCanvas :on-click="() => console.log('Scene')">
  <TresGroup :on-click="() => console.log('Group')">
    <Sphere :on-click="() => console.log('Sphere')">
        <Torus :on-click="() => console.log('Torus')" />
        <TorusRing />
    </Sphere>
    <Box />
  </TresGroup>
</TresCanvas>

Note if we follow convention, then we would also log Scene in these examples because all events would bubble up to the canvas. I'm assuming we aren't doing that in my answers

Do we want to let canvas receive all events thought 🤔
Can you all let me know what you think about that

Q/A

What should be in the console when ...

Torus is clicked?

Torus
Sphere
Group

TorusRing is clicked?

Sphere
Group

Sphere is clicked?

Sphere
Group

Box is clicked?

Group

the background is clicked?

<Box> <!-- Box0 -->
    <Box :on-click="() => alert('Box1')"> <!-- Box1 -->
        <Box /> <!-- Box2 -->
    </Box>
</Box>

Is Box2 clickable? If so and it's clicked, will Box1's :on-click be triggered?

Assuming we're doing events like the DOM does, it seems to me that the answer to both is "yes". I just want to make sure my understanding is correct.

Yes if you click Box2, Box1 would receive the event.

To add on to this, if you lined up the view so that Box2 was occluding Box1/Box0 then clicked on Box2(with no stopProgation), Box1's event handler would get called twice

• Once because Box2's event would bubble up to Box1 since it is Box2's parent
• Secondly, because once Box2's onClick event finishes propagating up the scene the next object hit by the ray will also have an onClick event fired

[andretchen0]

@garrlker Thanks for the answers and clarifications. That's making sense now.

About bubbling events to Scene, I think it's really handy. One (useful to me) pattern for DOM events is to attach a handful of listeners to the document body and to handle events bubbled up from children there. In our case, that could be handled with a TresGroup wrapping all other objects, but I guess I don't currently see a reason why we wouldn't also bubble all the way to Scene. (That doesn't mean there isn't a good reason not to do that though!)

Maybe relevant: R3F has an event called onPointerMissed that fires when no mesh is hit by a click/touch.

---

Maybe relevant to the broader discussion of how to implement events:

I haven't looked at the R3F source, but at least in this R3F StackBlitz onPointerEnter and onPointerLeave don't fire unless the pointer is moved.

[garrlker]

@garrlker Thanks for the answers and clarifications. That's making sense now.

Anytime!

About bubbling events to Scene, I think it's really handy. One (useful to me) pattern for DOM events is to attach a handful of listeners to the document body and to handle events bubbled up from children there. In our case, that could be handled with a TresGroup wrapping all other objects, but I guess I don't currently see a reason why we wouldn't also bubble all the way to Scene. (That doesn't mean there isn't a good reason not to do that though!)

I also think it could be handy, thought I'm not quite sure how I'd use it yet. I think it's better we include it than not

Maybe relevant: R3F has an event called onPointerMissed that fires when no mesh is hit by a click/touch.

Good catch! We definitely want to include that event

I haven't looked at the R3F source, but at least in this R3F StackBlitz onPointerEnter and onPointerLeave don't fire unless the pointer is moved.

Those events don't fire unless the pointer is moved, by default

You can force the ray cast to fire with the mouse's last know coordinates in a useFrame callback, and if a moving object moves into those coordinates those events will be triggered

[andretchen0]

Those events don't fire unless the pointer is moved, by default

Gotcha! Thanks!

[andretchen0]

Editing to make the thread less noisy: #426.

Thanks @Tinoooo .

[Tinoooo]

@andretchen0 What you are describing is what was discussed in #426 . I agree with you. The fix should be provided in the scope of this issue.

[andretchen0]

Related: #527

[garrlker]

It's still WIP, but I've pushed up my changes so that you all can start reviewing the code and test out the changes

There is a playground example wired up at /raycaster/propogation. Wire up as many pointer events as you can and have fun :)

So far I've implemented

Events

• onClick ✅
• onContextMenu ✅
• onPointerDown ✅
• onPointerUp ✅
• onPointerLeave/onPointerOut ✅
• onPointerEnter/onPointerOver ✅
• onDoubleClick ✅
• onWheel ✅
• onPointerCancel ❌
• onLostPointerCapture ❌
• onPointerMissed ❌

Features

Primitives are supported (and possibly anything else in the scene judging by #527 ) ✅

• Event Prioritization ❌
• TresCanvas Events ✅
• Forced Raycasts ❔- might work, I've written the forceUpdate function, but still figuring out how to expose it to a component for testing

Event Propogation

• Raycast ✅
• Scene Hierarchy ✅

Event HMR is broken right now so that needs to be fixed before merge as well

Besides that, I wasn't able to create this in a store-ish approach. I unhooked the current system so I could work on it free from bias but the useEventStore ended up being a similar approach to the current usePointerEventHandler

@alvarosabu / @Tinoooo I may need a bit of guidance on how you two were wanting the use of said store to look like to try and work this event system into what ya'll were expecting.

Changes

1. Now we call intersectObjects() over the whole scene. We needed to support any object whether or not it had an event listener and that was the easiest way to do it. This is why Primitives/Sprites/etc are now supported. I've got some ideas for how to make this more performant that involve being extra smart in the renderer and nodeOps, but those are outside of the scope of this PR for now

2. useEventStore completely sidesteps the nodeOps Object3d insert/remove operations by only executing when eventHooks fire in useRaycaster

3. This supports the .stop event modifier for blocking events, and this supports multiple event listeners on a single object3D