VSYNC API: Support for VSYNC OFF / 120Hz+ / variable refresh displays / etc

[blurbusters]

EDIT: May 2017 -- COMPLETELY REWRITTEN. Now that I am an Invited Expert to W3C Web Platform Working Group, I've edited this to remove duplication with #785 -- And to add variable refresh rate support. As owner of Blur Busters, I also have an article that refers to this github item

EDIT: June 2017 -- Proposed new API is written here: PHASE 1 and PHASE 2. Comments are encouraged. Priority has been raised because of Apple's new 120Hz iPads

Introduction of VSYNC limitations in current browsers

With the need for a stronger VSYNC to synchronize animations to the display's refresh rate.

HTML API for Uncapped Framerate Support This is also called "VSYNC OFF". More than 80% of users in the now-$1-billion-dollar eSports gaming industry use "VSYNC OFF" because it reduces input lag. Scientific applications also sometimes require this for low-latency tests. Photodiode oscilloscope measurements show that 1000fps @ 60Hz has at ~8ms less input latency than 60fps @ 60Hz. (Almost half a frame-time less lag, because waiting-on-vsync can force a frame to be delayed). In some cases, lag can spike to a full frametime less lag, e.g. 16ms less input lag. Chrome already supports this via a command line option "--disable-gpu-vsync" but is not enabled via JavaScript API

HTML API for Variable Refresh Rate (VRR) Support This includes VESA Adaptive-Sync, HDMI 2.1 VRR, FreeSync, G-SYNC. They all function essentially the same at the application level. During full-screen mode, there needs to be a path of future support of optional JavaScript-controlled decoupling of updatesrates from fixed frequencies. This is to allow useragent apps to support emerging variable refresh rate display technologies for Canvas2D/WebGL/

HTML API for Bypass WDM compositing In full screen mode (not windowed mode, or "fullscreen borderless windowed mode") it is possible for applications of multiple platforms to bypass WDM compositing and decrease input lag by another full frame.

HTML API for Discovery Of Above Even today, there's no reliable way to detect the refresh rate of a monitor in a web browser (except with clever rAF() heuristics in Chrome, FireFox, Opera and Safari). There needs to be JavaScript ability to easily query for refresh rate, VSYNC ON/OFF support, variable refresh rate support, and low-lag full screen mode (bypass WDM compositing).

Historically, end of section 5 of W3C timing standard -- https://www.w3.org/TR/animation-timing/ -- recommends synchronization to refresh rate (VSYNC) -- and work on #785 has fixed some shortcomings of the wording in HTML 5.2. This is great for many applications, but not all.

Competition gamers/eSports like using "VSYNC OFF" in full screen video games. The ability to do a W3C compliant HTML 5.2 Javascript API equivalent of turning on/off "--disable-gpu-vsync" would be highly favoured by full-screen WebGL game developers, to bring browser latencies (buttons-to-pixels) down. Ideally, this should be made possible in both canvas2D and with WebGL.

Many displays support other refresh rates, such as 75Hz. Also, there are many displays other than 60Hz, including gaming displays (120Hz, 144Hz). And VR displays such as Oculus and HTC Vive operate at 90Hz which Chrome supports (360 degree YouTube videos). Also, future TV standards (e.g. 8K 120Hz) may introduce 120Hz as a more standardized refresh rate in coming years.

It is extremely important to standardize how animations behave at refresh rates other than 60Hz, as synchronizing animations exactly to refresh rate is important, regardless of refresh rates. (Even 115fps @ 120Hz results in unsmooth motion), but also still provide an API to optionally decouple update rates from display refresh rate.

Browser comparison on refresh rate support:
www.testufo.com/browser.html

Browser-based motion tests for analysis (test these at refresh rates other than 60Hz):
www.testufo.com
www.vsynctester.com
www.testufo.com/photo
www.testufo.com/eyetracking
www.testufo.com/animation-time-graph

• NOTE1: Don't forget to consider there are multi-monitor desktop computers. Different refresh rates for different monitors is possible. In other words (for multi monitor systems), the refresh rate of the specific display that the browser window is running on. Windows already changes the desktop manager update rate everytime the window moves to a different-refresh-rate display. Some browsers adapt correctly (automatically changes callback rate of requestAnimationFrame() when this happens) when windows are moved, but others mess up (suddenly goes janky / stuttery).*

• NOTE2: Don't forget emerging variable refresh rate displays including G-SYNC, FreeSync, VESA Adaptive-Sync, and HDMI 2.1 VRR. (The next XBox supports variable refresh rate -- Microsoft confirmed) Instead of software waiting for the display refresh, the display waits for the software. The display immediately refreshes the frame delivered by software -- so refresh rate runs at frame rate. E.g. If you get 53fps, the display is 53Hz. Etc. The refresh rate of the display changes dynamically dozens of times per second to exactly match the refresh rate to the framerate, for stutter-free changes to framerate (simulated animation -- test this on Windows/Mac with all other browser tabs closed for the most accurate visual simulation). Eventually, there may need to be a long-term path for browsers becoming "VRR-aware" at least in full-screen mode, very useful for WebGL 3D games.

• NOTE3: WebGL games can use wglSwapIntervalEXT(1) and glSwapIntervalEXT(0) to enable/disable synchronizing to VSYNC. But, they are still hampered by the latency of the window compositing manager. Most operating systems permit a full-screen mode that bypasses windows compositing delays, so this could be a feature (flag). In this particular case, if you both (A) bypass compositing and (B) use glSwapIntervalEXT(0) ... for WebGL applications, you'd get the 'tearing' effects (while gaining useful "eSports-friendly" latency-reducing effects) of running framerates far higher than refresh rate.

For wording fixes to requestAnimationFrame() rate, see #785.
(This is related wording clarifications to existing VSYNC-driven requestAnimationFrame() -- but this is separate from the additional features listed here.)

VSYNC API Requirements For HTML 5.2 or HTML 5.3

Modifications to a VSYNC API needs to solve the following, at least several or all of:

• Ability to detect whether requestAnimationFrame() or canvas (WebGL, etc) is dropping frames
  In other words, frames missing the current refresh cycle of the monitor. Aka dropped/skipped/throttled frames during refresh rate synchronization. Many games and certain kinds of applications need self-awareness to know if frames have missed their refresh rate. Websites created by multiple people including www.testufo.com (my creation) and www.vsynctester.com (@duckware creation) use useragent detection & heuristics. The heuristics is needed to make a guess whether or not a frame was missed. And to make a wild guess whether requestAnimationFrame() is running at a rate different from the display refresh rate. Understandably, this is kind of a witchcraft of clever coding that should be avoided. The existence of this API would also double as confirming that the browser supports VSYNC (currently www.testufo.com/browser.html uses useragent detection to achieve the same thing).
  NOTE: An accurate VSYNC time in requestAnimationFrame() callback parameter is another way to allow useragents to detect dropped frames.

• Ability to enable framerates far higher than refresh rate
  Basically, a JavaScript method of being able to enable/disable the equivalent of Chrome's "--disable-gpu-vsync" flag. At the absolute minimum during full screen mode, maybe as an additional parameter for the existing HTML fullscreen API. This is very important for two emerging major things:

• • REASON 1 -- Letting framerate vary as performance allows, makes things more compatible with future VRR displays. Letting browsers decide to render as fast as possible, and then letting the display automatically decide to raise/lower refresh rate to match the current render-rate / update-rate. Browser vendors can then decide whether or not they want to support the emerging variable refresh rate displays such as G-SYNC / FreeSync / VESA Adaptive-Sync / HDMI 2.1 VRR.

• • REASON 2 -- JavaScript developer access to access an ultra low latency mode for gaming. In eSports gaming (where crazy gamers play CS:GO at 1000fps on NVIDIA GTX Titans) -- the use of 1000fps@60Hz has 8 milliseconds less input lag than 60fps@60Hz. This is consistent with experience by gaming authors. Also, latency were recently done in "--disable-gpu-vsync" and some GPUs such as NVIDIA GTX1080 was able to run the web browser at framerates in excess of 1,000 frames per second on several test pages. This means browser performance is reaching eSports territory. High speed video / input latency benchmarks showed less lag with buttons-to-pixels when framerates far exceed refresh rate. This is because at 1000fps, the frame seen by eyes was rendered only 1ms ago -- the overkill framerate meant that whatever hit the screen was the freshest possible pixel, thus reducing mouse lag & keyboard lag for specialized low latency applications. In fact, in the billion-dollar eSports gaming industry -- Bloomberg News, Forbes, New York Times. I have attended some of these events myself and noticed that more than 90% of paid competition game players use VSYNC OFF (disable GPU vsync) during gaming such as Counterstrike:GO, etc. If we were able to enable Javascript APIs to do this, browsers would more easily (over long term) enter the niche market of olympics-style "milliseconds-matters" races of tight reaction times and simultaneous-draw situations.

• Ability to query whether or not Windows' compositing is bypassed
  Full screen mode can bypass window compositing manager (adds 1 frame of lag -- 16ms). This is important for reducing latency during high-precision applications, eSports gaming applications, Mac can do compositing in sub-frame latency (Safari has 1 frame of lag), while Windows enforces an additional wait (Chrome has 2 frame of lag). Being able to let applications query for this lag, would enhance latency awareness for web applications, and allow "eSports-ready" benchmarking in pure Javascript. If compositing is false then "tearing artifacts" will appear during VSYNC OFF (--disable-gpu-vsync) which is normal. Tearing artifacts look like this (simulated). Generally, the higher the framerate (e.g. 1000fps@60Hz), the less visible tearing artifacts becomes during true compositing-bypassed direct-to-display VSYNC OFF operation (eSports league lowest latency mode).

• Ability to to return the current maximum refresh rate of the display that the browser window is currently on
  While this can be roughly achieved by counting requestAnimationFrame() (except on IE/Edge -- see Strengthen requestAnimationFrame() spec for VSYNC / refresh rate matching in HTML 5.2 #785), there should be an easier mechanism to query. Should ideally be must be multimonitor aware -- different displays can run at diverging refresh rates. (As one example, windows's desktop manager changes update rate to the refresh rate of the monitor that the largest surface area of the window is on). Several browsers have very flawed behaviour on multimonitor systems with divergent refresh rates (including systems with two different 'kinds' of displays: A 60Hz monitor and a 90Hz Oculus VR goggles). Divergent refresh rates are still niche but getting common enough to merit departure away from the "one-refresh-rate" assumption prewritten into earlier standards. Also, if querying the refresh rate of a VRR display (Variable Refresh Rate such as G-SYNC, FreeSync, VESA Adaptive-Sync, and HDMI 2.1 VRR) -- that automatically matches refreshrate to current framerate -- then it should return the maximum refresh rate that the current VRR display is capable of.
  (NOTE: This one might be a reasonable substitute/alternative to missed-frame detection. Knowing the refresh rate means we can count frames per second within a browser page, and compare to refresh rate -- although it's possible to have a smooth 60fps@60Hz and a stuttery 60fps@60Hz when frames don't exactly line-up with display refresh cycles)

There are other APIs that may be needed, but this is the type of "update-rate and latency knowledge" that high-performance JavaScript applications need. This will be required for both WebGL and non-WebGL modes (e.g. HTML5 canvas).

UPDATE: I am now an Invited Expert in W3C Web Platform Working Group. I welcome gaming industry programmers / those in this industry to collaborate on this topic. You can follow up here, but I can also be directly reached at mark[at]blurbusters.com for ideas, suggestions, and technical discussions if you're not a github member. This VSYNC API text is also an article on BlurBusters.

[blurbusters]

Coincidentially, the Chrome team just targeted 90fps for viewing via Oculus VR goggles;

http://arstechnica.com/gadgets/2016/05/chrome-dev-asserts-browser-is-viable-vr-platform-targets-90-fps-rendering/

Tests on my systems using new modern built-in GPUs (Recent 2015-era and later IGPs ... ordinary builtin GPUs, not even Radeon/NVIDIA highend stuff!) is now able to run Google Earth (angled 3D view) at ~100 frames per second in WebGL in most web browsers. So refute any claims of "there's not enough performance"... On my now-midrange GPU (3-year old GeForce GPU), I'm able to achieve 144fps @ 144Hz with Google Earth at 1920x1080p, in most typical not-too-steep-angled Google Earth 3D views.

Considering some browsers, e.g. EDGE, hardcode to 60fps, even at higher refresh rates, it is critical that browsers SHOULD NOT use hardcoded frame rates, and MUST provide a way to synchronize to VSYNC.

Also, browsing SHOULD try to provide an optional method of reducing latency:
-- Ability to turn off VSYNC if allowed by the display (for low-latency gaming, at exchange of gaining tearing artifacts). This is common in many PC video games.
-- Bypassing display compositing, to reduce scripting-to-display latency. Many full-screen video games use a native full screen mode that has one-frame-less latency (e.g. 1/60th second faster at 60Hz) than running in windowed mode. Low latency mode should be up to browser implementers to implement, but there SHOULD be provisions to cover this in browser standardization. e.g. Going into full screen mode (F11) should optionally try to automatically use the operating system's lowest latency mode if possible where not compromising user experience
-- Alternatively if behaviour differences warrants it: Provide an HTML API to provide an optional flag/argument to JavaScript .requestFullscreen() such as .requestFullscreen("gaming") .... since some OS low latency full screen modes often changes behaviour such as disables overlapped windows (popup windows) when running in this type of full screen mode.

[joeblew99]

Vsync support would be very useful.
Its one of the reasons I can't recommend building a graphics intensive project using pure web technologies at the moment.

[blurbusters]

---copied from requestAnimationFrame() thread #159 since it is extremely useful here too .... They seem distinct, because it seems to be a decision between revert #159 (keep requestAnimationFrame) or go with #375 (create a new VSYNC API).

For those testing:

Note to those people "assuming 60fps is enough" (in a "640K ought to be enough" fashion):
There is a lot of incorrect assumption that the human eye can only see 60 images per second when it is not really that simple -- lots of /indirect/ artifacts can occur that are still visible at far over >200fps@200Hz --
(1) For example, motion side effects like eye-tracking-based motion blur caused by a non-infinite refresh rate (e.g. demo at www.testufo.com/eyetracking ...). (Note: I'm the creator of all the TestUFO.com motion tests, including this finite-refresh-rate-induced optical illusion) -- it is a great example of an indirect side effect caused by the finiteness of the refresh rate -- and behavior changes at 60Hz, 120Hz, 144Hz 165Hz...
(2) Vision researchers has long found this out, and a great article is written by VR authors -- Oculus' Michael Abrash, "Down the VR Rabbit Hole: Fixing Judder" at http://blogs.valvesoftware.com/abrash/down-the-vr-rabbit-hole-fixing-judder/ -- demonstrating discrete-refreshrate-related artifact still shows up in some situations even at 1000fps@1000Hz ... Trying to perfectly replicate a Holodeck needs an elimination of finite-refreshrate-induced optical illusions, for 100% perfect immersion, and it's a quite difficult thing to achieve...

I frequently reported to FireFox and Chrome about refreshrate/framerate bugs:
https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=(site:mozilla.org+OR+site:chromium.org)+(rejhon+OR+%22blurbusters%22)+60+120++

A popular animation test pattern used by browser programmers (at least the FireFox and Chrome teams) for requestAnimationFrame() debugging is this one http://www.testufo.com/animation-time-graph --

It uses Javascript High Resolution Time http://www.w3.org/TR/hr-time/ to benchmark the time between VSYNC intervals (by using requestAnimationFrame() as the signal-tick of VSYNC) ...

Two to three years ago, the Chrome programmers obtained a gaming monitor (capable of multiple refresh rates up to 144Hz), and fixed many refresh-rate and fluidity problems, and to this date, Chrome has been the favourite web browser for smooth motion.

Fluidity tests:
Also, here are web browser fluidity comparison benchmarks (how long browsers stay smooth with increasing screen rendertimes):
http://www.blurbusters.com/blur-busters-120hz-web-browser-tests/

These were done by using these tests to enable a hidden "Delay" setting.
http://www.testufo.com/#test=animation-time-graph&easteregg=1
Increasing the delay causes the javascript to do a busywait-loop for the specified number of milliseconds before beginning to paint. It simulates longer-than-expected rendering (e.g. intensive 3D graphics). At the time, Chrome maintained perfect fluidity longer than most of the others.

Hope this helps in motion fluidity testing!

P.S. OpenGL already provide options to synchronize to VSYNC (OpenGL "swap interval", which WebGL also seems to use) but not everyone wants to do animations using WebGL. So what do we do, when requestAnimationFrame() #159 disappears?

[blurbusters]

Great news -- here's an additional testing tool:

Here's a great VSYNC tester for requestAnimationFrame: www.vsynctester.com
It uses the formula here: http://www.vsynctester.com/howtocomputevsync.html

This is a more detailed compliance tester for VSYNC than www.testufo.com
(Select "Animation Timing Graph")

Attempting to detect VSYNC in a web browser is still very opaque, and it depends on requestAnimationFrame() which is assumed to return approximately near VSYNC time -- which means such sites needs to use timing-calculations heuristics to figure out if VSYNC is being kept or missed.

However, requestAnimationFrame timing is currently not properly standardized well in HTML 5.1 or HTML 5.2 -- see #785 for ongoing discussion.

[blurbusters]

As early headway to #375 I have made a candidate git commit (no functional changes; just standards solidification) to my fork of HTML 5.2 (waiting for W3C approval). See #785 for more info -- it is semi-related to this as requestAnimationFrame() synchronizes to VSYNC in most browsers.

I would also nominate @duckware to provide further feedback on this item, since he is the creator of www.vsynctester.com and has a huge deal of amazing commentary based on browser testing, as follows:

Commentary of www.vsynctester.com test results:

• Details of Chrome is Broken
• Details of FireFox is Broken
• Details of EDGE is Broken

I have updated the original #375 text with refreshed requirements.

[blurbusters]

Update: Here's a screenshot of 2000fps in Chrome using less than 20% CPU.

1. 5-year old computer with a 5-year-old discrete NVIDIA GPU card. Considered a midrange gaming computer today. This is the performance of a $200 NVIDIA GeForce GPU add-on card.
2. This is a screenshot of www.testufo.com/photo in Chrome using command line option "--disable-gpu-vsync"

What this demonstrates, is that quadruple-digit framerates is often effortless on desktop browsers running on gaming machines, as I have encountered many programmers (even developers working on some user agents) who still think browsers are not capable of performing at such framerates.

I include this screenshot, because a few years ago, I received an email from Microsoft's Internet Explorer team (before Edge) claiming browsers could not perform sufficiently to do 120fps. Five years later, the arbitrary framerate cap built-in to IE continues today -- and Edge has inherited a hard-coded framerate cap on requestAnimationFrame(). Despite several bug reports to Microsoft over the years, even today 2017's Windows Creators Update -- this specific user agent is artificially capped to be unable to run at 120 frames per second even if it is just 1% of CPU on an AC-power-connected 5-year-old gaming desktop computer with no resource constraints. Edge is doing much better in many ways, but the other browser vendors fixed this four years ago.

[blurbusters]

Apple's ProMotion on the 120Hz iPad -- with variable refresh rate support -- has dramatically raised the priority of both #375 and #785. I have moved this from #785 -- Here's my first crack at organizing the beginnings standardization attempt that accommodates the following:

Table of Contents

We need the quickest and easiest way to support a wide variety of use cases, preferably with no modifications to browser APIs where possibly avoided, except for (8):

(1) Easiest support for frame rates intentionally decoupled from refresh rate (higher & lower)
(2) Easiest support for fixed-custom-Hz (e.g. video, animations)
(3) Easiest support for dynamically-varying-Hz (e.g. games)
(4) Easiest support for regular HTML
(5) Easiest support for <video>
(6) Easiest support for <canvas> via WebGL
(7) Easiest support for <canvas> via requestAnimationFrame()
(8) Adds improved discovery (e.g. query for the display refresh rate).

Assumptions

• Graphics driver is responsible for variable refresh rate responsibilities.
• App simply specifies sync source (e.g. video to sync refresh rate to) or simply deliver frame buffers (e.g. to let graphics driver begin refresh cycles immediately)
• Refresh rate changes are 100% seamless disruption free (no visible blackout / no visible mode change)
• APIs don't necessarily need to be modified to support variable refresh rate. Existing APIs may handle this, if designed properly, simplifying HTML 5.2 responsibilities.
• You may be able to simply deliver frame buffers to trigger a refresh cycle (e.g. Direct3D Present() or OpenGL glFlush() which has been repurposed by GSYNC/FreeSync to instantaneously trigger an immediate refresh cycle. No specific custom API for variable refresh rate)
• You may only be able to specify a sync source (e.g. custom fixed frame rate, e.g. 24fps = 24Hz)
• You may not be able to support all the above simultaneously (e.g. ProMotion API limitations)
• OS upgrades can theoretically change support (e.g. iOS 12 might later enable true variable refresh rate for OpenGL games)
• Input-lag catch: Trying to deliver frames (e.g. Present() or glFlush()) faster than the maximum refresh rate, simply executes a "wait for VSYNC" style behavior. Be warned that this often increases input lag (adds frame buffering delays), so it's ideal that application software throttles the delivery of frames to the driver to minimize the time between an input read (touchscreen/keyboard) and the triggering of an immediate refresh -- to reduce latencies. The lag-increase-effect, of hitting the max frame rate, is well documented at places like www.blurbusters.com/gsync/preview2 using a high-speed camera.
• Some platforms only enable variable refresh rate via 3D framebuffers (OpenGL or Direct3D). This is another reason why it's easiest to enable VRR only in full screen mode. User agent compositor may need to copy the browser buffer into an OpenGL/Direct3D buffer and then glFlush()/Present() to force an asynchronous refresh cycle to occur immediately (and as a bonus: lower click/tap latency!)

User Experience Complications

• Multiple elements can run at different frame rates (multiple videos, canvas, etc)
• Multiple windows or tabs can exist, with different ideal update rates
• Mouse pointer can become erratic (increases in judder at lower refresh rate)
• Scrolling can become erratic (increase in judder if synchronizing to refresh rate of a small embedded advertisement video)
• VRR might only work in Direct3D or OpenGL rather than desktop window manager.
• OS may override the refresh rate at all times (e.g. globally force a lower refresh rate during power saver)
• Input lag is often lowest when running at higher frame rates. Good related Microsoft Research video: https://www.youtube.com/watch?v=vOvQCPLkPt4

Necessary Simplifications for Easily Adding VRR Support to Browsers

• Support synchronized/variable refresh rate for full screen or maximized mode.
• Synchronize to dominant update rates (e.g. full screen video OR full screen canvas OR full screen HTML). This is because we don't want small embedded video ads degrading mouse pointer / scrolling. See User Experience Considerations.

Performance & Power Saver Considerations

• An operating system (independently of browser) may globally force a reduced maximum screen refresh rate during a "Power Saver" mode*
• An operating system (independently of browser) may change the variable refresh rate range to save power. Browser may still have access to lower variable refresh rates (e.g. 24Hz for 24fps).
• In a "Settings" or "Power Management" or "Control Panel", some platforms may have a "Power Saver" type mode that lowers the maximum refresh rate, AND/OR trigger more frequent switches to lower refresh rate, AND/OR reduce the variable refresh rate range, AND/OR run at a fixed lower Hz (disabled variable refresh rate support).
• If continuously dynamic variable refresh rate exists, then browser engine may use dynamic variable refresh rate to eliminate stutters during processing spikes. Letting framerate gracefully degrade 60fps-59fps-58fps-57fps-58fps-59fps-60fps in a completely stutter-free and judder-free way is a major attraction of existing gaming variable refresh rate displays. This is good for temporary spikes in rendering complexity during scrolling and animations. Supporting this on many platforms, is often just as simple as simply delivering a composited frame to the graphics driver (letting the graphics driver immediately refresh the display). Slow rendering simply delays delivery of the frame, with longer intervals between frame deliveries.
• By default, there is no arbitrary frame rate cap, nor the user should be asked for permission to run at the highest refresh rate. The max refresh rate of the user's display is declared the user's opt-in of running at that frame rate. When a mobile device (iPad) includes 120Hz support, we always trust the operating system's power management of the 120Hz screen. However, it is acceptable to ask the user for permission before running at framerates higher than maximum refresh rate (e.g. when needing to do 500fps for a low-latency eSports application)
• Existing browser engines often target CPU% or power consumption targets (without using a hard-coded frame rate cap). That means low-overhead animations can be allowed to run at full frame rates (e.g. 120fps at 5% GPU) while high-overhead animations might be capped at, say, 25% GPU or 300 milliwatts (letting the dynamic variable refresh rate handle frame rate slowdowns from performance limitations) or other power-limiting API mechanism. Many existing browsers already do this at 60fps and deciding to slow down (e.g. 30fps) if too much resources are consumed. Likewise, the same would occur for 120fps gracefully degrading to lower frame rates.

[blurbusters]

Potential HTML 5.2 Variable Refresh Rate Standardization Path

PHASE 1

Simplified Goals

• Zero new APIs added initially
• Theoretical browser-vendor path without additional discovery APIs, command arguments, etc
• Leave most Variable Refresh Rate (VRR) responsibilities to OS / drivers (including automatic VRR-related power management).

Simplified Rules

• Automatically engage javscript-controlled VRR support only in full screen mode
• Automatically sync refresh to the first or biggest visible viewport (e.g. <video> or <canvas> tag)
• At top level, compositing allowed to use VRR to eliminate performance stutter & to idle on non-animated pages (if not done automatically by OS).
• Limiting VRR support to full-screen mode simplifies platform considerations for certain platforms (e.g. needing to use Direct3D/OpenGL to signal the display to immediately begin an asynchronous refresh cycle, even for plain text browsing or playing videos)

Satisfies The Following

PARTIAL - (1) Easiest support for frame rates intentionally decoupled from refresh rate (higher & lower)

In full screen or maximized mode, the browser vendor makes decision to synchronize to the dominant viewport. The refresh cycle workflow is either:

1. individually triggered refreshes: triggered on delivery of framebuffer (video frame, webGL frame, requestAnimationFrame); if supported. OR
2. custom fixed Hz: triggered by OS automatically (e.g. automatically by ProMotion fixed H.264 decoder rate during full screen videos)

No HTML coding changes to existing HTML apps are necessary.
"Partial" support denotes lack of support for frame rates higher than max Hz of display

RESOLVED - (2) Easiest support for fixed-custom-Hz (e.g. video, animations)

Resolved via (1) above.

RESOLVED - (3) Easiest support for dynamically-varying-Hz (e.g. games)

Resolved via (1) above.

Unresolved - (4) Easiest support for regular HTML

The easy path has no support for JavaScript specifying a custom HTML5 compositor rate (frame rate = refresh rate) since that requires adding a new API.
Exception: However, the top-level compositor may gracefully slow down the rate (for power management or for stutter-reduction), letting the graphics driver or OS immediately refresh the screen. A device "power saver" mode may automatically use a lower refresh rate, or performance-demands spikes may slow down the compositor frame rate (and thus, the refresh rate of a variable refresh rate)

RESOLVED - (5) Easiest support for <video>

Resolved via (1) above.

RESOLVED - (6) Easiest support for <canvas> via WebGL

Resolved via (1) above.

RESOLVED - (7) Easiest support for <canvas> via requestAnimationFrame()

Resolved via (1) above.

Special Note: about App-requested refresh rate: requestAnimationFrame already can be made to run via timer (e.g. if Javascript wants a 45Hz refresh rate, it simply runs requestAnimationFrame() at 45 times a second as already possible today; the OS/driver simply executes the refresh cycle upon delivery of a frame buffer). You'd simply call requestAnimationFrame inside a timer event, and the browser would immediately trigger the call to the callback. That way, no HTML API change is needed to allow a browser app developer do a custom refresh rate with <canvas>. Five years ago, you can already do this with existing HTML, run animation at a timer-based 45fps, but it stutters a lot (45fps at 60Hz is bad) -- however, if you do 45fps on a variable refresh rate display, it plays naturally stutterfree, with no app or browser intervention, except simply by delivering the framebuffer to the operating system on a timer. That's it. Very easy.

Special Note: about Gracefully slowed-down frame rates due to performance: Say, a game tries to run at 60fps but doesn't have enough performance, and it only runs at 53fps instead. This is what happens if requestAnimationFrame() is called inside the requestAnimationFrame() callback, then the rate will max out at the display's maximum refresh rate except when throttled for any reason. On a variable refresh rate, whenever framerate slows down, it is not necessary to run requestAnimationFrame at fixed intervals. Variable refresh rate can de-stutter / de-jitter erratic frame-delivery timings, as long as game time rendering is in sync with refresh cycle times. Prevailing practice is most games & animations already do this, in order to keep motionspeed consistent independently of frame rates. As a side-effect, such logic is automatically compatible with stutterfree variable-framerate motion on a variable frame rate display. For those unfamiliar --
it's actually almost miraculous to see existing 15-year-old source code, not invented for variable refresh rate -- whenever it is struggling to run at its fastest framerate -- still remarkably successfully stutterlessly run at exactly 53 frames per second at exactly 53 Hertz on a variable refresh rate display. This is in thanks to the operating system simply triggering refresh cycles on frame buffer delivery. Even if the software is 15 years old, the operating system can successfully give it variable refresh rate support! As long as the animations are already designed to run off gametime (and automatically adapt to fluctuating frame rates as they already do 15 years ago), it's usually beautifully compatible with variable refresh rate: The stutters miraculously disappears, much like this animation demo: www.testufo.com/stutter#demo=gsync

Unresolved - (8) Adds improved discovery (e.g. query for the display refresh rate).

There is often a need to query for the refresh rate of a display (fractionally too -- use a float!).
This is in Phase 2 (which can be done concurrently with Phase 1)

[blurbusters]

Potential HTML 5.2 Variable Refresh Rate Standardization Path

PHASE 2

Some elements of Phase 2 might be done before Phase 1 (e.g. refresh rate discoverability, throttle discoverability)

Please see Phase 1 first.

Phase 2 Addition of APIs to browsers

APIs can initially have prefixes during incubation ("moz", "o", "webkit", etc)

• Add API to discover current refresh rate
• Add API to discover current animation throttle
• Add API to discover whether display supports VRR
• Add API to discover whether VSYNC is ON/OFF
• Add API to disable VSYNC to permit frame rates above refresh rate.
• Add API to discover whether DWM compositor is being bypassed for lower lag

API to discover refresh rate: screen.hz

Existing Similar Practice: Currently we already have screen.width and screen.height which returns the dimensions of the monitor. (For a multi-monitor setup, existing browsers currently returns the resolution of the monitor that the browser window is on).
Proposed API: The API is proposed to be "screen.hz" (readonly) because "hz" is (mostly) more language neutral than "refreshrate" and easier to type. For multimonitor, it applies to the monitor that the browser window is currently on.

• It SHOULD return a floating-point value where possible, since refresh rate can be fractional.
• On a variable refresh rate display, this always returns the current maximum bounds of the variable refresh rate range. (e.g. If the current range is set to 24.000Hz-120.000Hz, then it should return "120.000000"). This is the prevailing practice on other pre-existing VRR APIs on other platforms, to simply return maximum refresh rate, and let software decide on a lower refresh rate simply by timing frame buffer deliveries (even erratically/asynchronously too!).
• If this is a fixed-frequency refresh rate change (e.g. 15Hz power saver mode, 24Hz movie mode, etc) where the software still needs to synchronize to the refresh cycles, then screen.hz should update to return this value.
• The value "screen.hz" shall be read-only. On a true continuously variable refresh rate display, there is not necessarily a need to change this value to a lower refresh rate. Simply delivering framebuffers slower to the graphics driver, is all that is needed, since the refresh cycles are triggered upon frame buffer deliveries (e.g. triggered upon completions of requestAnimationFrame()) from the application to the OS/driver.

API to discover throttle: window.getCurrentAnimationRateLimit()

• This is necessary because peer reviewed scientific papers are now beginning to use precision web browser animations (cite: [1],[2]). Also, webkit change r215070, which is undiscoverable to useragents, broke embedded 30fps-versus-60fps demos (see this)
• This is a sidebuddy API call to window.requestAnimationFrame(). This is to enable discoverability of an animation rate throttle, e.g. running in background mode (e.g. 1Hz) or running in embedded mode (e.g. 30Hz, as per Apple's modification r215070 to WebKit engine -- see the beginning part of https://bugs.webkit.org/show_bug.cgi?id=165694 ...) ... To solve the problems of animations being unaware of being throttled less than refresh rate (given web animations are now being used in peer reviewed science papers), it is now critical to add discoverability of the throttle. For example, Microsoft Edge 15 as of 2017 currently also has an animation throttle limit of 105 even when the monitor is running at a higher refresh rate, which is not reported to apps running within the browser. Such undiscoverable throttles creates major problems for scientific-precision animations.
• The value is allowed to change dynamically on the fly. The number may change during background or offscreen operation (e.g. 1 Hz) and suddenly go back to full rate matching "screen.hz" (e.g. "60" or "120") during foreground operation.
• This is simply a maximum throttle. This does not take into account ordinary frame drops (e.g. "30" throttle may still result in 25fps or 26fps if a few frames are dropped for example). Frame drops can be self-detected.

API to discover Variable Refresh support: screen.vrr

Existing Similar Practice: Currently we already have screen.width and screen.height
Proposed API: The API should be screen.vrr (readonly) returning true|false whether the display is capable currently in variable refresh rate mode. This does not necessarily mean that Javascript currently has control over the VRR, since it may only be available to videos. VRR stands for Variable Refresh Rate, which is the current generic terminology. Be noted, that on a multi-monitor system, not all monitors may support VRR. Both screen.hz and screen.vrr will apply to current monitor that the browser window is on (moving the window already updates screen.height and screen.width in prevailing practice)

API to discover or configure VSYNC ON/OFF: screen.vsync

See this original post for more info why VSYNC OFF can be beneficial for certain important use cases.

Existing Similar Practice: The Chrome browser has a VSYNC OFF command line option, "--disable-gpu-vsync" to permit frame rates far above refresh rates. Basically, renders will run at an unthrottled rate (as much as the current CPU allotment & power management plan permits).
Proposed API: The API should be screen.vsync (readonly OR read/write) returning true|false whether VSYNC is turned off.

• Attempting to write screen.vsync = false would be the JavaScript equivalent of launching Chrome with "--disable-gpu-vsync".
• During screen.vsync = true browser frame rates are limited to max refresh rate.
• During screen.vsync = false browser frame rates would run at the fastest rate permissible, much like a video game's VSYNC OFF setting.
• During VSYNC OFF, the compositor would run at the fastest rate, WebGL, and requestAnimationFrame() would callback at the fastest rate. At the render completion, the next render cycle begin immediately. The net result is framerates far above refresh rate, with correspondingly reduced keyboard/mouse/touchscreen lag useful for critical fast-response / eSports / etc.
• If VSYNC is successfully enabled/disabled, the "screen.vsync" setting will update to the value written to it. If not successful, "screen.vsync" will behave as a read-only value (much like "screen.width" and "screen.height") -- the next read of the value would result in an unchanged value from the attempted write.
• USER PERMISSION RECOMMENDED: Running at uncapped frame rates far above refresh rates, potentially >1000fps, can be very power-consuming, and hog system resources. One person suggests a theoretical very rare kind of certain timing attacks are theoretically possible (e.g. timing the length of draw operations) but this is something already possible with shader benchmarking as well as forcing animation frames to take longer than 1/60sec (to time draw operations without needing VSYNC OFF). However, WebGL is far more insecure than supporting VSYNC OFF, but this is being mentioned anyway for completeness' sake -- alongside full screen mode security considerations -- and another rationale for permission-based operation for requesting VSYNC OFF. Thus, this should be opt-in. This should be a user-permission request similiar to full screen mode requests. Upon first call by a specific web page to screen.vsync = true should automatically prompt the user whether to permit VSYNC OFF, for ultra-high framerates needed for low-latency / eSports applications.

API to discover whether DWM compositor is being bypassed: screen.dwm

Existing Similar Practice: Full-screen-exclusive mode for Direct3D and OpenGL already bypasses desktop window manager compositor (DWM) in full-screen applications (Even most browsers currently don't use the full-screen-exclusive mode, even in full screen mode -- it's more a borderless window stretched to full screen of the window manager instead)
Proposed API: The API should be screen.dwm (readonly) returning true|false whether desktop window manager compositor is currently being bypassed.

• This could be made to happen only during full screen mode
• It happens that variable refresh rate is supported under Windows only via graphics drivers via Direct3D and OpenGL frame buffer presentation APIs (such as Present() or glFlush() triggering the immediate beginnings of a refresh cycle).
• Using exclusive full screen mode presents a brand new opportunity to reduce keyboard/mouse input lag by using Direct3D/OpenGL to display the browser framebuffer (even for text, or for video frames). You'd gain variable refresh rate support, and lower input lag.
• screen.dwm might become true only during full screen mode.
• Discovery of this permits browers to automatically detect if the system is currently configured to the lowest latency. The simultaneous existence of "screen.vsync = false" and "screen.dwm = false" confirms the absolute theoretical lowest input-lag situation possible, and confirms that the web browser is eSports-ready for ultra-high-performance competitive purposes (where Olympics-style millisecond differences in reaction times matter a lot, such as simultaneous-draw situations in FPS shooter games)
• To be discussed: Should this be a read/write? By default, full screen mode still lets other windows overlap (if you Alt+Tab another window onto the top of a browser window). In exclusive-full-screen-mode, used by videogames for reduced lag, you lose this ability, and also the ability to bring up the Start Menu as quickly (During bringing up the Start Menu in the middle of a PC videogame -- the game either minimizes, goes windowed, or goes "borderless full screen windowed" with compositor temporarily reenabled). Perhaps the JavaScript developer should do a "screen.dwm = false" right after requestFullScreen() and make it permission-based too, as part of the requestFullScreen() call, since it can be somewhat disruptive to use the exclusive full screen mode.
• To be discussed: The behavior of "screen.vsync = false" simultaneously with "screen.vrr = true" will need to be defined later. Ideally "screen.vsync = false" may simply turn off VRR, or simply produce non-VRR behavior once framerates exceeds maximum refresh rate (like GSYNC + VSYNC OFF currently does -- tearing only occurs when framerates are above refresh rate).

Also, bypassing the compositor enables tearing during VSYNC OFF (which is normal). The ability to know we're in the lowest possible-latency mode can signal that the browser is "eSports compatible" (competitive gaming).

---

End of Phase 2. For a simpler beginning, please see Phase 1 first.

[AshleyScirra]

Have you considered the case of rendering at 30 FPS on a fixed-vsync 60 Hz display? Some game developers using our framework are keen for this, since if the system isn't fast enough, dropping from 60 to 30 FPS looks better than the irregular update intervals you can get at something like 45 FPS. It could also help save battery if the developer wants that.

You did mention a way of using timers to get a specific framerate, but the key thing about this is it isn't a specific framerate, it's a way of dropping to half-vsync, 1/3-vsync rate etc. For example on a fixed 80 Hz display we would want to drop to 40 FPS, not an uneven 30 FPS, if the system can't achieve full v-sync rate. As long as an integer divisor of the v-sync rate is used it should look smooth.

In the case of a fixed 120 Hz display (I know the iPad is variable-rate, but imagine a fixed rate for the time being), if the system can't keep up ideally we would progressively drop the framerate to 60 FPS, then 40 FPS, then 30 FPS, then 24 FPS, etc.

[blurbusters]

Have you considered the case of rendering at 30 FPS on a fixed-vsync 60 Hz display? Some game developers using our framework are keen for this, since if the system isn't fast enough, dropping from 60 to 30 FPS looks better than the irregular update intervals you can get at something like 45 FPS. It could also help save battery if the developer wants that.

Good points too.

You did mention a way of using timers to get a specific framerate, but the key thing about this is it isn't a specific framerate, it's a way of dropping to half-vsync, 1/3-vsync rate etc. For example on a fixed 80 Hz display we would want to drop to 40 FPS, not an uneven 30 FPS, if the system can't achieve full v-sync rate. As long as an integer divisor of the v-sync rate is used it should look smooth.

There's many methodologies. Using a timer is just one ecommended methodology for a VRR displays. For a fixed-Hz display, in certain browsers, it is also already possible to decimate framerate using requestAnimationFrame() one can check the timestamps (see www.vsynctester.com and www.testufo.com for examples) and decide to skip rendering a frame. If 1/60sec elapsed from last rAF() call, skip rendering (just duplicate current framebuffer), if 1/30sec elapsed from last rAF() call.

Instead of using a timer to aim for 30fps, it's better to use requestAnimationFrame() and skip VSYNC strategicaly instead (by checking the time argument -- to determine if 1/30sec or 1/60sec passed since the last requestAnimationFrame() ... This is because timers are never perfect. A timer might be 30.000Hz while the actual refresh rate is 30.1578Hz .... Refresh rates on displays are never 'perfect'. And we've got the 29.97Hz-versus-30Hz situation as well as 59.94Hz-versus-60Hz situation. Timers work very well with true variable refresh rate displays (100% asynchronous) since the display simply synchronizes to the software timer! But timers are pretty bad for fixed-frequency Hz, due to the drift. (Which creates intermittent frameskips and sawtooth input lag graphs (keyboard/mouse) due to the slewing differential....)

So the proper way of skipping frames is setting the next requestAnimationFrame() callback inside requestAnimationFrame() in order to keep it synchronized to VSYNC -- as is usual situation for VSYNC-synchronized animations -- and then skipping rendering strategically. However, there can be browser-specific problems with this approach -- some of them decimate suddenly 60-to-30 and others simply execute the requestAnimationFrames as fast as possible if framerate slows down.

Also, thinking further, standards can be tweaked to also improve:

• Specify browser behavior during situations of no drawing inside requestAnimationFrame()
  Browser behavior is sometimes weird when immediately returning from requestAnimationFrame() without drawing anything. The W3C standard should be clarified that if nothing happens inside requestAnimationFrame(), the existing framebuffer is recycled. That way, people like you, can be more assured of programmatically skipping frames. requestAnimationFrame() could be considered like a VSYNC timer that has occasionally skipped VSYNCs, and you'd look at the time argument to check whether or not a VSYNC was skipped, to aim at a 30fps rate.

• The 'time argument of requestAnimationFrame needs to be accurate VSYNC time instead of now() time
  This currently already done in Chrome, but not in FireFox yet (see this comment). By having this accurate VSYNC time, you can at least more accurately detect whether or not VSYNCs were missed or not (to decide whether to render or skip -- to aim at a target frame rate). TestUFO does it 'heuristically' (and alas, useragent detection too! -- sigh) but ideally just enough discovery APIs should be built in W3C HTML standard to let the app do its job.

That said, I'd like your comments. How difficult has it been for you to aim at a specific framerate in the light of randomly fluctuating requestAnimationFrames() (in situations where it occasionally skips VSYNC due to performance, etc)

In the case of a fixed 120 Hz display (I know the iPad is variable-rate, but imagine a fixed rate for the time being), if the system can't keep up ideally we would progressively drop the framerate to 60 FPS, then 40 FPS, then 30 FPS, then 24 FPS, etc.

Yes. This is a very common use case. The proposed stuff here doesn't prevent the ability to do this (see above). This will, however, be helped by minor clarifications in the reset of the W3C HTML standard.

BTW, I also covered framerate-slowdown algorithms should rightfully be developer choice (you), in a separate comment in a different issue. A consistent 30fps is definitely better than erratic 30-60-30-60-30, but it's also a developer preference to decide to do 60-60-59-58-57-58-59-60-60 too. You're very right, that as a Framework Developer, that this should be a developer choice. Browser engine developers SHALL NOT dictate a specific framerate-slowdown algorithm that cannot be overriden by app developers.

Wording should not dictate a preference of framerate-slowdown algorithm. This should be developer choice
Yes, sometimes this is best. But even for mainstream situations, blanket-recommending 60fps suddenly go to 30fps is very problematic. Sometimes this is actually better, but in many cases, this suddenly adds input lag that throws off aiming in certain kinds of gaming. In many use cases, like brief framerate slowdowns in video games, it's preferable to go in a sequence 60-60-60-60-58-55-50-54-57-59-60-60-60fps -- during brief performance issues, like a momentary complex moment in a video game. A gradual slowdown to 55fps has less input-lag than suddenly going to 30fps, for HTML gaming. However, at other times, it's definitely preferable to go to 30fps than 60fps. In a perfect ideal world, this is a developer-specified behavior (e.g. HTML5 API to specify preferred update-rate degradation behavior). But in the abscence of this, it's not W3C's place to suggest one-size-fits-all specific update-rate degradation behavior.
Also, I've developed simple HTML games before -- Here's a simplified mathematical example: Trying archery at a moving-target of 960 pixels/second, 60fps is 16 pixel steps per frame (combined with 1/60sec lag) and 30fps is 32 pixel steps per frame (combined with 2/60sec lag). If you aim your arrow at the moment after an unexpected 30fps slowdown due to performance -- your archery-shoot (aimed during 60fps) may have an unintentional browser-enforced "+2/60sec" lag modifier instead of a "+1/60sec" lag modifier. Your arrow misses by 16 pixels because of a 1/60sec time delta of aimed button press versus aimed moving target. In this situation, sudden 60-to-30fps slowdown created an unexpected input lag that threw off your aiming. Remember, human reaction times are typically 100-200ms, and if a browser slowdown occurs during this time window, your aiming is off because of an unwanted mandated browser half-framerate slowdown (i.e. a mere millisecond missing a 60fps deadline due to performance, enforces another forced 1/60sec wait before displaying the frame (if browser doing a mandated 60-to-30 slowdown) -- unwanted sudden lag change). As a result, for this specific HTML5 game, you want to override that, and enable the option of "play at maximum framerate as browser performance allows, without half-framerate slowdowns". In this case, user experience improves during this specific particular use-case of "usually 60fps with a few slowdowns". While many use cases are more complex than this, hopefully this simplified example helps developers understand. One has to realize the "60-to-30-slowdown" technique can be great but it is not a one-size-fits-all-apps solution.

---

At the end of the day, the standard needs to make it possible for the app developer to choose a framerate decimation mechanism. As seen in the 120Hz browser tests of 2013, there is inconsistent implementation between browser vendors (sudden or gradual slowdown algorithms). I believe (and I hope) that solving the bolded bullets above, would help this developer-choice problem in the simplest, minimal change to HTML standards (hopefully).

Also, refresh rate may vary seamlessly (no visible mode changes) in background, refresh rate may change, more frequently, and at will, by the operating system (e.g. power management) since there is no visible disruption to user for slow/static display.

It has already started to happen. You know, in the Olden Days, refresh rate changes were disruptive events that flashed the screen. Today, refresh rate can seamlessly change dozens of times a second with no visible disruptions (as GSYNC / FreeSync already does) -- by allowing software to asynchronously trigger individual refresh cycles. Now -- whether between multiple different fixed-Hz (power management 15Hz->movie 24Hz->movie 48Hz->regular 60Hz->better 120Hz) or as a true (asynhcronous/dynamic) variable refresh rate display -- these are the two major "variable refresh rate" scenarios of consideration (seamless jumps between multiple fixed Hz, or seamlessly continuously dynamic refresh rate).

Operating systems and drivers are beginning to blithely change refresh rates in the background (due to power management, or due to synchronizing variable refresh rate to a specific foreground application window -- e.g. GSYNC on windowed applications). Which means bystander applications are subject to deal with continual refresh rate changes beyond their control. Sometimes 60 Hz is still virtualized so applications don't know, but increasingly the actual refresh rate is being forced visible to app (like how www.testufo.com varies in framerate in Chrome whenever running alongside a GSYNC'd window application -- because Google Chrome is forced into this variable refresh rate situation that it was never originally designed for; and it stutters somewhat badly in this situation when the window desktop manager suddenly decides to runs at a variable update rate whenever a windowed GSYNC application is running).

Because this is happening now, already, and because of Apple iPad's VRR support -- I think it's now important that HTML 5.2 resolves the variable refresh rate standardization without waiting for HTML 5.3

When aiming at a fixed framerate on fixed-Hz variability (timers only work perfectly well with true VRR). The hopping between fixed-Hz values would interfere with VSYNC divisor factors. Instead of 120Hz, it might be suddenly 15Hz (e.g. power management). But an app developer could use PHASE2's recommended "screen.hz" discovery to determine current refresh rate, to decide on a VSYNC divisor on the fly (even every requestAnimationFrame() if desired).

Things to consider

• Some browsers do 60-to-30fps suddenly on a feather of extra load above 1/60sec
• Some browsers gradually reduce, 60fps-59fps-58fps on a feather of extra load above 1/60sec
• App developers want the choice of either, for very good very different reasons.
• VSYNC divisors are desired, but watch the gotcha of background refresh rate changes (power management)

It's possible to do some of this already (using very-browser-specific timing tricks). What we need is consistency. We need to think carefully, what changes to HTML 5.2 (if any) is needed to ensure consistency of control across browsers. Comments welcome.

[blurbusters]

Further talks have indicated that a throttle is completely unavoidable -- that's totally understandable, because of:

• Hidden but not yet-paused tab (rAF can run at 1 Hertz)
• Power saver mode (rAF can run at reduced frequencies)
• Idle machine (rAF might slow down after a minute of unattended operation)

As a result, discoverability of a throttle is being added to PHASE 2, and that discoverability means parts of PHASE 2 might come before PHASE 1

Discoverability of the throttle could be a call (e.g. ".getCurrentAnimationRateLimit()" in addition to ".requestAnimationFrame()") or a higher level property (e.g. "screen.animationrate"). Since this is a value that can dynamically change frequently even on a single-monitor system, and it is not really a "screen" property, I've decided to suggest it be a sidebuddy method call. (Personally: I'm fine with either approach, depending on combined browser vendor and W3C consensus).

On this related topic, I was very concerned to see that Apple has landed a revision (r215070) that provides a non-discoverable throttle in requestAnimationFrame to 30 callbacks per second when used embedded -- which breaks one of my websites (embedded frame rate comparisions). See https://bugs.webkit.org/show_bug.cgi?id=165694 for an example of the side effect.

As a result, this is why discoverability is needed (see Phase 2 -- now edited to include throttle discoverability) -- so animations know whenever they're being throttled, and be able to adjust accordingly. People are already using web browsers for peer-reviewed scientific motion tests [1],[2], making discoverability of animation precision even more essential.

Due to elements of Phase 2 being started before Phase 1, I'm now considering splitting #375 into two separate items (the variable refresh rate support category, and the discoverability category).

[chaals]

@mdrejhon would you be able to provide a PR for this?

[blurbusters]

PR?

The Blur Busters equivalent of a Press Release is at: https://www.blurbusters.com/blur-busters-working-on-changes-to-html-5-2/

How can I help?

I'm happy to collaborate on tech docs / proposals / etc. I've created spec documents such as XMPP Extension XEP-0301: In-Band Real-Time Text so I have experience in specs-document writing. You can see my name is listed in there, and I'm all familiar with the nuances of spec writing (including things like RFC2119 terminology, etc)

Maybe a new W3C proposal worked together to officially add a proper sync API framework. I'm happy to volunteer my hours on this, please use me!

Chromium recently added command line options for unofficial experimental full-screen-mode GSYNC/FreeSync support but it requires command line options. With no possibility to access in JavaScript, and they gave the excuse that there's no standard.

So.... Let's work together and create one by mutual consensus!

We need some way to access one or more of the following via Javascript:

• Discovery (finding out what the system supports)
• Modifications to frame rate capping behaviours (e.g. rAF() -- IE/Edge still has a 105fps limit)
• VSYNC OFF support (yes, even with tearlines. Yes, yes, again, there's use cases.)
• VRR support (GSYNC / FreeSync / etc)
• Etc. (Anything in the sphere of sync support)

One possible route forward is this could be done as part of the full screen API, with proper permissions confirmation, if need be. All options are on the table, as long as there's a way to let javascript developers access these features for many use cases.

Email mark[at]blurbusters.com (or discuss here)

[chaals]

Sorry, Pull Request. I.e. edit a piece of the spec to make a text change...

[fjorgemota]

just out of curiosity...any recent news about this proposal?

[blurbusters]

New Chrome VRR Trick

I just discovered I can unexpectedly force Chrome into a VRR mode with these conditions:

• GSYNC with an NVIDIA card
• Enable windowed GSYNC
• Enable GPU acceleration in browser chrome://gpu
• Configure GSYNC default for applicataions to override VSYNC OFF mode
• Run chrome --disable-gpu-vsync --disable-frame-rate-limit
• Go full screen (F11 or fullscreen API) to automatically enable GSYNC override of VSYNC OFF
• Create an animation using an ordinary timer (instead of requestAnimationFrame)
  Make sure only run one timer event (and no other animations, no animated GIFs, no video) since they interfere with each other. This will ensure that chrome only repaints during the timer event.
• Chrome successfully plays animation smoothly if the timer is within the variable refresh rate range -- a JavaScript timer of 53.2 Hz causes the display to be 53.2 Hz

Please note, that the command line seems to be problematic on AMD Radeon cards. Or maybe it was a newer Chrome version that introduced a bug. I don't think many know about the secret "--disable-frame-rate-limit" option which usefully causes requestAnimationFrame() to run at about 1500-2000fps on a modern GPU on TestUFO. It makes everything feel delightfully ultralow lag and I use it for some scientific testing. The frame rate is successfully throttled by using a simple Javascript timer, and the timer automatically sets the refresh rate of a GSYNC display -- very clever.

(That's the way VRR is supposed to work: A variable refresh rate monitor is slaving to the software, and only refreshes immediately when the software delivers the frame. The fact that Javascript now successfully dynamically/asynchronously controls the refresh cycles on a monitor -- is quite neat, indeed, in this undocumented trick that works in certain versions of Chrome).

Trick may not be reliable in all versions of Chrome but this is useful for incubating / testing. Also this MIGHT work with "GSYNC Compatible" mode (FreeSync monitors on an NVIDIA card are now supported via a DisplayPort connection).

Also.... in this Chrome VRR trick in full screen mode:

• GOOD NEWS! Full screen WebGL apps automatically runs GSYNC correctly by default. As long as the WebGL is the only refreshing window, the monitor syncs to the WebGL frame rate.
• GOOD NEWS! Full screen video playback windows automatically runs GSYNC correctly by default. Odd frame rates like 48fps Hollywood 59.94fps and 60.00fps etc run correctly without stutter.

Condition: It has to be the only/dominant repainting element (nothing else repainting), and it will then correctly sync.

So, apparently, at least one version of Chrome is already programmed in a way that is "reasonably" GSYNC friendly. We just need to expose this functionality to Javascript programmers. And need to figure out how to handle a custom frame rate for requestAnimationFrame since the Javascript programmer is fully in control of the refresh rate of the monitor, so theoretically requestAnimationFrame could have a frame rate parameter for VRR displays (or undefined to match max-Hz of display, letting it gracefully degrade depending on how much processing occurs within requestAnimationFrame).

[blurbusters]

Sorry, Pull Request. I.e. edit a piece of the spec to make a text change...

I am now more prepared (have learned more) to begin doing this stuff. Would it be possible for me to re-apply as a Invited Expert, to allow me to begin doing this?

Thanks to this recent discovery of an accidental undocumented "Chrome GSYNC" trick that has allowed me to incubate some ideas, I'd love to submit a potential Chrome VRR proposal that is exposed in perhaps a permission-approve API (like fullscreen API).

This provides an opportunity to simplfiy the VSYNC API somewhat since it may be easier than expected to expose GSYNC/FreeSync to Chrome, as long as some special considerations are followed with a custom developer-configurable frame rate throttle that allows javascript to set the refresh rate for requestAnimationFrame. (And also, access to VSYNC OFF API for non-GSYNC users)

The pre-existing internal hardcoded 60.00 framerate throttle already exists in most web browsers when there's no ability to access the operating system's refresh rate. (For cross-platform compiling). Like Chrome browser running without GPU acceleration, in software rendering mode on a 144Hz monitor -- the browser runs at 60fps because it's configured itself to that frame rate throttle on an assumption of a lower-common-denominator display. This actually could work in our favour; we simply let a Javascript developer configure this throttle for requestAnimationFrame -- and voila -- becomes the VRR refresh rate since the variable-refresh monitor automatically syncs to the update rate of the browser. While also allowing VSYNC OFF developers to uncap this frame rate or set ultrahigh rates (e.g. 300fps) for low-lag browser esports application purposes and scientific browser application purposes. So hits multiple birds with one simpler stone.

From this angle, might be the easiest simplification of a VSYNC API. With refresh-rate control API rolled into the permission-mechanism of the fullscreen API.

Thoughts welcome?

[siusin]

Thanks @mdrejhon .

We're closing this issue on the W3C HTML specification because the W3C and WHATWG are now working together on HTML, and all issues are being discussed on the WHATWG repository.

If you filed this issue and you still think it is relevant, please open a new issue on the WHATWG repository and reference this issue (if there is useful information here). Before you open a new issue, please check for existing issues on the WHATWG repository to avoid duplication.

If you have questions about this, please open an issue on the W3C HTML WG repository or send an email to public-html@w3.org.