Implement controls with pointerEvents

[arodic]

Recently, OrbitControls and TrackballControls were updated to use pointerEvents but multiple regressions happened (#20191, #20193, #20205, broken pen support and more...) reverted in #20194.

I completely and overwhelmingly support transition to pointerEvents. It is an amazing API that will solve most of the unnecessary conditionals when dealing with different input devices and it will make the controls much simpler. It enables developers to write code that works with any pointer device, including the future ones. I would like to suggest new implementations that take full advantage of pointerEvents API.

In my experience, switch ( event.pointerType ) is unnecessary for most use cases - it defeats the purpose of using pointerEvents. You can treat pointer events as input-device-agnostic API.

In other words, switching to pointerEvents is great, but it should be done without dealing with pointerType unless really necessary - which is almost never. Here are some basic gestures that are possible with pointerEvents without caring about pointerType:

clicktouch-and-drag

You treat single pointermove with primary button pressed the same in all cases. From UX perspective touch-drag is same as primary click-and-drag. Don't care about pointerType, only that there is only one pointer and the primary button is pressed.

hover

Mouse/pen hover is basically pointermove without any buttons pressed. This is impossible on traditional touch device. So essentially same as above, but no buttons pressed. Don't care about pointerType.

Hover might work on certain touchscreen devices that support proximity hover, but I never tried.

multi-touch-and-drag

This is when your pointermove has more than one pointer. This is impossible with mouse/pen since only touchscreen supports multiple pointers. This is where you count the pointers and add special case for multi-touch gestures with 2, 3 or more pointers. Don't care about pointerType or buttons.

Technically, multiple mouse cursors are possible, but completely insane.

non-primary button click-and-drag

This can happen when mouse/pen non-primary buttons are pressed. This should also happen only in a single pointer scenario since touchscreen doesn't have non-primary buttons. Here you simply check which buttons are pressed. Don't care about pointerType.

contextmenu

You dont need to use pointerEvents for this since this old API already works reliably on all pointer devices. You might want to preventDefault() on this one if you need secondary button action for something special.

Use pointerCapture.

For continuous gestures you should use setPointerCapture() on pointerdown and releasePointerCapture() on pointerup. This ensures that all pointermove events in between are going straight to your listener and nowhere else. This is much cleaner than what we used to do with mouse events in the past (no more adding events to window, stoppingPropagation etc).

[mrdoob]

Akiii! Can't agree more.

In my experience, switch ( event.pointerType ) is unnecessary for most use cases - it defeats the purpose of using pointerEvents. You can treat pointer events as input-device-agnostic API.

In other words, switching to pointerEvents is great, but it should be done without dealing with pointerType unless really necessary - which is almost never.

Yes, the current implementation is supposed to be transitional. I didn't want to overhaul the whole thing and avoid breaking too many things (and they still did break 😅).

[Mugen87]

@mrdoob I'm absolutely fine with making this change however I'm not 100% happy how this transition happens right now. The current approach just confirms all devs how say three.js does not care enough about not breaking things when introducing new stuff.

[arodic]

Off topic...

three.js does not care enough about not breaking things

I have been mostly lurking recently but I didn't get the sense that people don't care enough. I think threejs just grew too fast and testing didn't catch up. The high frequency release cycle doesn't help either.

Also, innovation and stability are sometimes counteractive. If we want to innovate fast, we need the flexibility to break things, as long as there are ways to detect and time to fix.

Perhaps we can solve this by spreading releases apart (including alpha and beta) while simultaneously improving testing?

[mrdoob]

Back to the topic.

@arodic I've implemented pointer events in TransformControls: 10c9e41

Please, take a look if you have a minute. The pointer.button logic was a bit hard to understand.

[Mugen87]

Added a note in the migration guide for r120 to notify users about this change:

https://github.com/mrdoob/three.js/wiki/Migration-Guide#r119--r120

[arodic]

@mrdoob looks good!

Two suggestions for future improvements:

1. Consider using setPointerCapture() instead of attaching listeners to scope.domElement.ownerDocument.

2. Consider setting scope.domElement.style.touchAction = 'none' instead of setting up touchstart listener and preventDefault().

[Mugen87]

Consider setting scope.domElement.style.touchAction = 'none' instead of setting up touchstart listener and preventDefault().

As mentioned here #20223 (comment), this does not work in Safari/iOS.

[mrdoob]

Well, the last time I tested it (March this year) it didn't work.

[Mugen87]

Unfortunately, I have no iOS device for testing. However, "Can I use" says the support is still limited. Meaning iOS Safari only supports auto and manipulation.

https://caniuse.com/#feat=css-touch-action

[arodic]

Oh well, preventDefalut(), it is. I'll update #20219

[mrdoob]

I just tried domElement.style.touchAction = 'none' and seems to work.
https://raw.githack.com/mrdoob/three.js/dev/examples/webgl_multiple_scenes_comparison.html

[mshamaseen]

Just for clarification: I was using mousedown event before migrating to r120 and it was working well, now after migration, it won't respond anymore.
You need to use pointerdown instead.

[cereschen]

In the pointerdown event, let's simulate the mouse event
it's work

  e.target.dispatchEvent(new MouseEvent('mousedown'),e)
  e.target.dispatchEvent(new MouseEvent('mouseup'),e)

[Mugen87]

Small update: DragControls, OrbitControls and TrackballControls are now fully migrated to pointer events. So similar to TransformControls they do not use mouse* and touch* events anymore.

Consider setting scope.domElement.style.touchAction = 'none' instead of setting up touchstart listener and preventDefault().

This suggestion is now implemented.

Consider using setPointerCapture() instead of attaching listeners to scope.domElement.ownerDocument.

This is something that still needs to be investigated.