Normalizing MouseWheel in different browsers/platforms

[Omustardo]

This issue is to discuss the difference, and possible reconciliation, between mouse scrolling on different platforms and browsers.
Relevant code: https://github.com/goxjs/glfw/blob/master/browser.go#L234

When run on desktop, the scroll wheel has a delta of 1 per tick in my experience. When run in the browser I experience a delta of 10 per tick. It seems that the tick value also varies by browser according to:
http://stackoverflow.com/questions/5527601/normalizing-mousewheel-speed-across-browsers

Looking into it further, it appears my browser actually uses 120 per tick. http://phrogz.net/js/wheeldelta.html
Given the ideal of a single piece of code being able to run in the same fashion on both desktop and canvas, I suggest scroll wheel ticks be normalized to a delta of 1 per tick.

Possible solutions:

1. In the stack overflow link above, the top suggestion of simplifying the delta to -1, 0, or 1 seems reasonable at a glance, but based on comments it seems there are issues with different hardware like track pads that also uses the wheel event. It would also limit extremely fast scrolling to one tick per callback. I tested how often this might occur by spinning my mouse wheel as fast as I could manage while recording events in the chrome dev console. The most concentrated snippet of multiple events was:
   14 syscall.go:43 http:0: got scroll event: 0 -10
   syscall.go:43 http:0: got scroll event: 0 -20
   35 syscall.go:43 http:0: got scroll event: 0 -10
   syscall.go:43 http:0: got scroll event: 0 -20
   39 syscall.go:43 http:0: got scroll event: 0 -10
   syscall.go:43 http:0: got scroll event: 0 -20
   6 syscall.go:43 http:0: got scroll event: 0 -10
   syscall.go:43 http:0: got scroll event: 0 -20
   32 syscall.go:43 http:0: got scroll event: 0 -10
   The number of events with multiple mouse wheel ticks in a single callback is relatively small. It isn't negligible though.

2. Per browser support. Infeasible unless there's list that can be automatically kept up to date and imported statically.

3. http://stackoverflow.com/a/30134826/3184079
   Facebook published a solution licensed under BSD-3. Many of the stackoverflow comments recommend it, and it could be translated to Go without difficulty.
   https://github.com/facebook/fixed-data-table/blob/master/src/vendor_upstream/dom/normalizeWheel.js

4. Keep track of the smallest delta seen, and divide all others by its absolute value. Simple, but I don't know if it works in all cases.

I'm going to sleep on this and give it more thought.

[dmitshur]

Thanks for reporting this!

Given the ideal of a single piece of code being able to run in the same fashion on both desktop and canvas, I suggest scroll wheel ticks be normalized

I completely agree that goxjs/glfw should try to normalize and return consistent values on all platforms it supports. Ideally it'd be the browser's responsibility to return consistent values, but if they don't, we have to take it on.

a delta of 1 per tick.

I'll look into that and think about it. Off the top of my head, I can't confirm if that value makes the most sense.

Edit: I've realized we have glfw itself on desktop to follow, so we'll normalize on whatever value it provides on desktop.

Possible solutions:

Thanks for listing these.

I like 3 the most. That file seems to have great information, assuming as it's decently up to date. Even if not, we can still use it as a great starting point and update any issues as people report them.

I think it would make a lot of sense to port that JS code into a small Go package, and then import it in goxjs/glfw. That way, the responsibility to handle browser differences falls completely on that library, and it can be used by goxjs/glfw as well as any other code that needs to handle scroll events in the browser.

For information, I primarily used macOS and Chrome during development, so goxjs/glfw supports that best. I've probably tried it on Firefox and a few other browsers, but all on macOS.

When run on desktop, the scroll wheel has a delta of 1 per tick in my experience. When run in the browser I experience a delta of 10 per tick.

Can you file another issue that's specific to that problem, and include full details (OS, browser, etc.). Let's continue to use this issue for the high level discussion of the problem.

[dmitshur]

By the way, I have the following small test program for testing scroll events in the browser:

https://dmitri.shuralyov.com/projects/touch/scroll.html

Then there's glfw events test application:

https://godoc.org/github.com/goxjs/glfw/test/events

That can be used to compare scroll events on desktop and in browser. The browser version (might be out of date though) is up at:

https://dmitri.shuralyov.com/projects/glfw-events/ (see console for output)

[Omustardo]

Thanks for the helper utilities. I translated the js function and put it in https://github.com/Omustardo/wheel
I can move it somewhere else if desired.

In doing the translation, I made a one logic change in the code, and one restructure to fit with Golang requirements.

• The logic change dealt with the DOMMouseScroll/MouseScrollEvent which is handled in the facebook code. This is separate from the wheel event that goxjs/glfw handles. The MouseScrollEvent is from very old versions of firefox so I suggest we continue to not handle it.

• The restructure was changing to a switch statement instead of:

  pX = sX * PIXEL_STEP;
  pY = sY * PIXEL_STEP;

  if ('deltaY' in event) { pY = event.deltaY; }
  if ('deltaX' in event) { pX = event.deltaX; }

  if ((pX || pY) && event.deltaMode) {
    if (event.deltaMode == 1) {          // delta in LINE units
      pX *= LINE_HEIGHT;
      pY *= LINE_HEIGHT;
    } else {                             // delta in PAGE units
      pX *= PAGE_HEIGHT;
      pY *= PAGE_HEIGHT;
    }
  }

I did this because we can't support if ('deltaY' in event) { pY = event.deltaY; } due to Go's default values, unless we bypass the DeltaX/DeltaY field and look in the object using .Get("deltaY").

I also needed to modify goxjs/glfw/browser.go's wheel handler locally in order to apply the change:

document.AddEventListener("wheel", false, func(event dom.Event) {
	we := event.(*dom.WheelEvent)

	if w.scrollCallback != nil {
		dx, dy, _, _ := wheel.Normalize(*we)
		go w.scrollCallback(w, -dx, -dy)
	}

	we.PreventDefault()
})

I can submit a PR with that if everything looks good.

I also attempted unit tests, but haven't yet figured out how to avoid panics when using a js.Object's Get function.
When testing manually it appears to work as expected, but I only tested on my system (windows 10 + chrome).

[Omustardo]

I did a bit more manual testing, and I think either I did something very wrong, or the facebook solution doesn't work in all common cases. Testing on windows 10 + chrome on my laptop resulted in a delta value of 150, which normalized to 1.25.

I'm more and more convinced that simply checking the sign and returning -1, 0, or 1 is the most effective.

The downside would be:

• systems that batch mouse wheel events - saving up small events over a few ticks and putting them into one larger one.
• if people scroll extremely quickly it will cancel out the events that are caught in the same tick.

On the other hand, it's likely the most future proof solution since it doesn't look at any javascript object fields and doesn't try to be "smart" about it.

[dmitshur]

unless we bypass the DeltaX/DeltaY field and look in the object using .Get("deltaY").

Doing that should be absolutely fine in this case, since it's needed.

I'm more and more convinced that simply checking the sign and returning -1, 0, or 1 is the most effective.

The downside would be:

• systems that batch mouse wheel events - saving up small events over a few ticks and putting them into one larger one.
• if people scroll extremely quickly it will cancel out the events that are caught in the same tick.

I don't think that would be acceptable. I want to preserve smooth, high-precision scrolling with momentum, which currently works fine on macOS.

When run on desktop, the scroll wheel has a delta of 1 per tick in my experience. When run in the browser I experience a delta of 10 per tick.

Can you file another issue that's specific to that problem, and include full details (OS, browser, etc.).

Would you mind doing that?

[Omustardo]

unless we bypass the DeltaX/DeltaY field and look in the object using .Get("deltaY").

Done. The code still doesn't handle all situations but at least it follows the existing code more closely. I haven't yet found any better way to deal with all of these different environments.

I want to preserve smooth, high-precision scrolling with momentum, which currently works fine on macOS.

Yea. That's another situation that it definitely wouldn't work for.

Can you file another issue that's specific to that problem, and include full details (OS, browser, etc.).

Done.

[Omustardo]

Related discussions and solutions:

http://stackoverflow.com/a/11738705/3184079
From the original stackoverflow link in the first post of this thread. This solution keeps a sample of 500 deltas and scales anything new by the 33rd percentile.
This has a few issues.
This is not intuitive to users. 500 delta values is a lot unless you're spinning the scroll wheel quickly or using a trackpad. During the time where it's sampling, the amount that you actually scroll will continue to change.
It doesn't ever resample. If you're using a mouse while sampling is done, and then switch to a trackpad, the values won't adjust to the trackpad. It could be changed to continuously sample, but it still means there will be a period between switching input devices that will be strange.

jquery/jquery-mousewheel#36
https://github.com/jquery/jquery-mousewheel/blob/master/jquery.mousewheel.js
Long discussion with a few interesting examples. The actual jquery solution is very similar to the facebook solution, but with some neat features like keeping track of the lowest delta and using it to normalize other deltas. I need to find someone with a mac to test it, but I don't think it works for inertial scrolling just by reading the code.

https://github.com/monospaced/hamster.js
monospaced/hamster.js#1
A discussion that mostly fizzles out, but references:
https://developer.mozilla.org/en-US/docs/Web/Events/wheel#Listening_to_this_event_across_browser
I'm not convinced that this will work. It seems like it would only work with firefox and chrome with the standard 3 vs 120 values.

https://groups.google.com/forum/#!topic/gwt-forplay/1_JqN9EXCvg
http://forplay-code-reviews.appspot.com/35002/diff/1/core/src/forplay/html/HtmlMouse.java
Good discussion, but final solution is purely hardcoded checks for OS and Browser. It won't work for inertial scrolling.

darsain/sly#67
https://github.com/darsain/sly/blob/master/src/sly.js#L1566
darsain/sly@0c4d251#diff-d5b1c2cb742651d6e51f162dac378e32
This explains the exact issue we're encountering, and describes how the mac trackpad breaks everything.
According to the discussion, this code works and handles inertial scrolling too!
The third link is the change that added inertial scroll handling. From what I can tell, it handles basic mouse events as most other solutions - by dividing by either 3 or 120 depending on the browser. For inertial scrolling, it appears to group deltas in 200ms intervals and returns +1, 0, or -1.
This solution is reasonable for web browsing, but isn't a good solution for anything realtime, and doesn't mimic the desktop behavior that we're looking for.

https://github.com/cubiq/iscroll/blob/master/src/wheel/wheel.js#L16
Looks like another example of grouping events within a small time period.

[dmitshur]

Thanks for providing those references.

From looking over that, I'm starting to see that the situation is a hot mess, and it's likely some different browsers on different OSes report different values.

I've done some testing on macOS, using github.com/goxjs/glfw/test/events in desktop and in browser. I can see that the scroll events coming from my trackpad seem to match. The lowest tick value is 0.1 on both. However, using a Logitech G502 mouse (hah, same one you have), the lowest tick (before scroll acceleration kicks in) is 0.1 on desktop but 0.4 in browser. I'm not sure if it's a problem for larger values when scroll acceleration takes place. It's hard to measure without a point of reference.

I've asked for more specific information about the issue you ran into in #11. Let's avoid trying to do too much too quickly, and focus on resolving a specific reproducible issue first.

[Omustardo]

The 0.1 minimum value for trackpad input on the desktop is likely due to the change a few years ago which specifically affects MacOS inputs:
glfw/glfw#95
https://github.com/glfw/glfw/blob/5655e26315bede6d714ba9a9e087c04ab00e2b49/src/cocoa_window.m#L607
https://github.com/glfw/glfw/blob/master/src/cocoa_window.m#L621
In the browser, the 0.1 minimum value is due to dividing deltas by 10 when the deltaMode==pixel.
Ignoring those adjustments, it looks like the values match which is what we want.

Here's another little info dump:

• There are multiple types of mouse wheel events.
  (wheel, mousewheel, MozMousePixelScroll, DOMMouseScroll, etc)
  WheelEvent is the current standard interface: https://developer.mozilla.org/en-US/docs/Web/API/WheelEvent
  The wheel event is a type that implements this interface. wheel is also the only type of wheel event goxjs/glfw currently handles.
  https://github.com/goxjs/glfw/blob/master/browser.go#L234
  The dom library that goxjs/glfw depends on also only looks at the fields based on the official specification of a WheelEvent:
  https://github.com/dominikh/go-js-dom/blob/master/events.go#L317
  deltaMode, deltaX, deltaY, deltaZ
  I suggest we continue to ignore any non-wheel event as it complicates the issue significantly, and all major browsers support the wheel event.

• Nonstandard fields exist in implementations of WheelEvent.
  It appears these are for backwards compatibility, so in theory we should be able to continue ignore them.
  https://developer.mozilla.org/en-US/docs/Web/Events/mousewheel#wheelDelta_wheelDeltaX_and_wheelDeltaY_value

• GLFW and Browsers get events from the OS which then triggers an event in goxjs/glfw.
  In glfw this is done here: https://github.com/glfw/glfw/blob/master/src/cocoa_window.m#L621
  Of note is the handling of hasPreciseScrolling deltas, originally added in: Add support for precise scrolling deltas glfw/glfw#95
  Other than that, GLFW appears to pass along the raw deltas from the OS.

Here are the bug trackers for implementing wheel on major browsers:
https://bugs.chromium.org/p/chromium/issues/detail?id=227454
https://bugs.webkit.org/show_bug.cgi?id=94081
https://bugzilla.mozilla.org/show_bug.cgi?id=719320#c20
I didn't find them particularly helpful to solve this issue, but thought they'd be worth listing as a reference.

[Omustardo]

At this point I don't think there's any "good" way to solve this issue. There are just too many settings and inputs that the browser "simplifies" for us - leaving us with too little information to get back to the original OS event. One example is the number of lines scrolled per mouse step.
If you set your OS to scroll two lines per mouse step, and then test it, you'll get something like this:
Desktop GLFW: 1.0
Chrome: deltaMode=0 deltaY=X where X is dependent on something I'm unsure of. I've seen 33.333 and 41.666 on my different devices.
Firefox: mode=1 deltaY=-2

Now if you change your OS to scroll one line per mouse step:
Desktop GLFW: 1.0
Chrome: deltaMode=0 deltaY=X/2
Firefox: mode=1 deltaY=-1

The problem is that we have no way of detecting what the OS level scroll settings are, much less dealing with how each browser interprets them. That leaves bad solutions - the effective of which I can think of at this point is to keep track of the minimum value seen so far, assume that it is the base delta value, and use it to normalize other deltas.

• Pros
  • Browser Independent
  • Future proof(?), or at least won't break if some constants get changed
  • Works for both trackpad, single events, and grouped wheel events
• Cons
  • Mixed input won't be perfect - i.e. using a mouse and trackpad at the same time. We could adjust it to only look at events within the past ~5 seconds, or only since the last consistent period of scrolling began. In my experience, even scrolling as fast as possible fires off a few minimum value scroll deltas before it gets going quickly.
  • This would ignore any OS & browser defined wheel modifiers (like Firefox's about:config mousewheel.default.delta_multiplier_y). That's definitely fine for browser modifiers since we don't want them. I'm not sure if there are any OS modifiers that affect desktop GLFW.
  • Requires keeping track of state and a bit of data processing.

In general I'm surprised by the lack of existing solutions for this issue. It seems like a common problem - in particular for existing cross-browser projects. Maybe I'm just searching for the wrong things, but all of the solutions seem to be very incorrect - dividing wheelDelta's by 120 in Chrome, etc.

Edit: I realized that this solution doesn't maintain the behavior seen with hasPreciseScrolling seen in desktop GLFW due to the 0.1 multiplier. Since there isnt a way to get that flag in the browser (as far as I know), this solution will result in higher values than expected. Is multiplying by 0.1 when hasPreciseScrolling in desktop GLFW necessary? If that weren't done, I think this would work.

[dmitshur]

The 0.1 minimum value for trackpad input on the desktop is likely due to the change a few years ago which specifically affects MacOS inputs: glfw/glfw#95

Hehe, I wonder if you noticed, I'm the author of that PR. :)

another little info dump
bug trackers for implementing wheel on major browsers

Thanks for those, good to have for reference.

At this point I don't think there's any "good" way to solve this issue.

I've done some investigation for #11 and I'm arriving at a similar conclusion.

I personally feel the best thing to do is wait and let the browsers sort out their inconsistencies. Given most (all?) browsers today are evergreen, meaning they forcibly-auto-update on regular intervals, it seems plausible that the situation will improve over time. Also, given that many browsers are open source, it may be a wiser investment of time to try to fix problems at the root, in the browsers, than to try to apply hacks at a higher application layer.

Let me ask what's your stance on this, since you're quite active on this issue.

Is resolving this and #11 a curiosity for you, and are you looking for a good, simple, general end solution? Or are you motivated to resolve #11 in a shorter time frame because of a critical business need, i.e., the inconsistency is blocking you from making progress on a project?

That'd be helpful for me to know.

[Omustardo]

Hehe, I wonder if you noticed, I'm the author of that PR. :)

Yea, although to be fair, you've contributed to nearly every github repo I visit so it wasn't much of a surprise :P

I personally feel the best thing to do is wait and let the browsers sort out their inconsistencies. Given most (all?) browsers today are evergreen, meaning they forcibly-auto-update on regular intervals, it seems plausible that the situation will improve over time. Also, given that many browsers are open source, it may be a wiser investment of time to try to fix problems at the root, in the browsers, than to try to apply hacks at a higher application layer.

I agree we should wait for browsers to converge on standards. The only solutions we can do by looking at wheel events just aren't very good. I'd be fine putting them in a standalone application but they don't belong in a library like this that people expect standard behavior from. On that note, it would probably be worth adding a TODO or comment linking to this issue at
https://github.com/goxjs/glfw/blob/master/browser.go#L234
and maybe even https://github.com/goxjs/glfw/blob/master/desktop.go#L149
so users of this library know to expect different behavior on browsers.

Let me ask what's your stance on this, since you're quite active on this issue.
Is resolving this and #11 a curiosity for you, and are you looking for a good, simple, general end solution? Or are you motivated to resolve #11 in a shorter time frame because of a critical business need, i.e., the inconsistency is blocking you from making progress on a project?

Answered in #11

Thanks for being so responsive on this issue. I'm glad to have gone through the effort to do it right, even if a fix isn't currently feasible.

[krisj]

are you aware of this neat little piece of work: https://github.com/d4nyll/lethargy ?
seems to me that it tackles this whole mess quiet nicely.

[dmitshur]

Thanks for sharing @krisj. I did not know about it.

I tried it just now, and the demo did not work at all reliably for me. In fact, it worked very poorly and did not detect many valid scroll events I made.

From https://github.com/d4nyll/lethargy#how-does-it-work:

Lethargy keeps a record of the last few wheelDelta values that is passed through it, it will then work out whether these values are decreasing (decaying), and if so, concludes that the scroll event originated from inertial scrolling, and not directly from the user.

It's just an approximation that tries to work out whether a scroll event is inertial or not. That's not even a problem we care about solving for our issue.

So, this doesn't change our status, which is that we're not doing much aside waiting for browsers to sort out this mess, and/or contributing there ourselves.

[trusktr]

Related web spec issue:

• Needs alternative WheelEvent.deltaMode, WheelEvent.delta(X|Y|Z) w3c/uievents#181