Mouse/touch event handling bug

[dhowe]

I'm not positive that this is a bug, but it sure is confusing that the code below results in 3 very different behaviors in the browser, depending on whether devtools is open and on the device selected.

To recreate:

• paste code below into the online editor in chrome browser
• drag square around with mouse: sketch works as expected
• open dev-tools and try the same: mouse is now out of sync
• now open device-emulator with touch device: rect can no longer be dragged

Three behaviors (with mouse-events):

1. With no js console closed, sketch works as expected
2. with console opened but no device emulator, rect and mouse are out of sync
3. with a touch-device in device emulator, mousePressed is never called and the rect is undraggable

With touch-events (comment mouse-events and uncomment touch-events below), the only difference is that in case C, we get behavior B (rect & mouse are out of sync)

  let x = 100, y = 100, active = false;

  function setup() {
    createCanvas(400, 400);
  }

  function draw() {
    background(0);
    rect(x, y, 50, 50);
  }

  //function touchEnded() {
  function mouseReleased() {
    //console.log('mouseReleased');
    active = false;
  }

  //function touchStarted() {
  function mousePressed() {
    //console.log('mousePressed', mouseX-pmouseX);
    active = (mouseX > x && mouseX < x + 50 && mouseY > y && mouseY < y + 50);
  }

  //function touchMoved() {
  function mouseDragged() {
    //console.log('mouseDragged:',mouseX-pmouseX);
    if (active) {
      x += mouseX - pmouseX;
      y += mouseY - pmouseY;
    }
  }

[limzykenneth]

I first thought this is a window resize problem but it still behaves weird when I pop the inspector window out on its own.

It is also exclusive to Chrome when inspector is opened and Safari without anything else opened, Firefox doesn't seem to exhibit any weird behaviour.

I'm not sure what caused this problem but it may be the framerate and event loop getting out of sync or something like that.

[dhowe]

Thanks for taking a look @limzykenneth ... any thoughts on next steps? Meanwhile, as you've confirmed, I'll label as a bug...

[limzykenneth]

I'll need to do some more tests, I'll report back if I find any insights.

[limzykenneth]

It is pretty much an issue of framerate and event loop not matching. Specifically, _updateNextMouseCoords is called by the event handler while _updateMouseCoords is called every frame and such there is the possibility of mouseX/Y having been updated with new values but pmouseX/Y haven't yet and so the difference accumulates.

To test, if _updateMouseCoords is called in _updateNextMouseCoords just before setting mouseX/Y the sketch above will work as expected but

line(mouseX, mouseY, pmouseX, pmouseY);

will no longer work as expected (see #1471 for some info).

I have no idea why the behaviour is so weird on Chrome, probably a Chrome bug. I also have no idea how to reconcile the two. We probably will prefer to update these values per frame but _updateNextMouseCoords rely on having an event object in order to update values so it needs to get that event object somehow, while at the same time using the event handler for events is more idiomatic. Without fundamentally changing the way these values are updated, I'm drawing a blank on how to solve this. Maybe some one else can have a crack at this.

[dhowe]

@limzykenneth thanks for the research -- this is clearly part of the problem, but in my testing mousePressed() doesn't even fire when on a touch-device and dragging (case 3 above)...

[zenzoa]

I am seeing the same thing with a game I'm working on in p5js. I put console.log's in mousePressed() and mouseReleased(), and on Safari on my iPad Pro, it only prints the mouseReleased one. I was able to get touchStarted() to work though, so for now I'm duplicating the mousePressed() code in there.

[ykabusalah]

Has anyone been assigned to this? Could I tackle this?

[limzykenneth]

@ykabusalah No assignment has been made yet as we are still figuring out where the problem is and where any potential solution is. You can have a look at the problem and let us know here what you think. Thanks.

[trych]

I have a sketch, where pmouseX does not work at all, but has a certain constant – but seemingly random – value on each frame (like 107 or 366, no idea where these numbers are coming from). This value then stays with pmouseX, no matter how much I move the mouse. Is this the same issue, or should I open a new issue for that?

Oh and btw. this issue happens on all three browsers for me: Chrome, Firefox and Safari.