fast plotting lib

[leeoniya]

hey Adam,

i saw your comment on css-tricks.com:

I’m currently using multiple canvases to model dune buggy data in realtime and it has to deal with many thousands of points. The CPU is constantly at 100% when using it and I’ve been trying to find ways of reducing the load.

i have a project you might be interested in: https://github.com/leeoniya/uPlot

particularly, this demo: https://leeoniya.github.io/uPlot/demos/stream-data.html

cheers!
Leon

P.S. i found you via:

1. https://old.reddit.com/r/javascript/comments/e5z56e/making_an_audio_waveform_visualizer_with_vanilla/
2. https://css-tricks.com/making-an-audio-waveform-visualizer-with-vanilla-javascript/
3. http://adam.teaches.engineering
4. https://www.reddit.com/user/UnfazedButDazed
5. https://github.com/polishdude20

[AdamMarciniak]

Hey Leon! Thanks for reaching out. Your project looks promising! I'm just wondering how is it implemented that makes it so fast? It looks like the top graph in your demo has a limited number of data points as well as your middle and bottom graph. As new data comes in, you slide the top data along thus keeping the same number of data points and you just remove every other datapoint in the middle graph to keep the same number of points? Because otherwise it seems to me you'd be redrawing each point on every refresh and with a huge amount of points that's not very fast. I'm expecting my data to be something on the order of 1.5 million datapoints or so spread out through multiple graphs. (16 graphs * 40 points per second * 30 mins) this is a worst case scenario though. If you look at my demo, you can see the top graph shows an overview of all time and I've got that having a constant amount of spread equally through the data. The bottom graphs need to be more detailed and so they need to have detail when zooming in and less detail when zoomed out in order to keep performance good. Thanks, Adam

…

On Wed, Dec 4, 2019 at 9:44 AM Leon Sorokin ***@***.***> wrote: hey Adam, i saw your comment on css-tricks.com: I’m currently using multiple canvases to model dune buggy data in realtime and it has to deal with many thousands of points. The CPU is constantly at 100% when using it and I’ve been trying to find ways of reducing the load. i have a project you might be interested in: https://github.com/leeoniya/uPlot particularly, this demo: https://leeoniya.github.io/uPlot/demos/stream-data.html cheers! Leon P.S. i found you via: 1. https://old.reddit.com/r/javascript/comments/e5z56e/making_an_audio_waveform_visualizer_with_vanilla/ 2. https://css-tricks.com/making-an-audio-waveform-visualizer-with-vanilla-javascript/ 3. http://adam.teaches.engineering 4. https://www.reddit.com/user/UnfazedButDazed 5. https://github.com/polishdude20 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#5>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEID5YOSJI7H3U27JFRS6PDQW7UALANCNFSM4JVMKVTQ> .

-- Adam Marciniak

[leeoniya]

I'm just wondering how is it implemented that makes it so fast?

the main thing that makes it faster than naive raw canvas .lineTo() calls is this simple optimization: leeoniya/uPlot#15. besides that, uPlot avoids doing any unnecessary mem allocations - it allocates nothing per datapoint (but this is compared to other charting libs, not raw Canvas).

Because otherwise it seems to me you'd be redrawing each point on every refresh and
with a huge amount of points that's not very fast.

yeah, 1.5M points is not gonna be great with the current .setData() api which invalidates & rebuilds the Path2D objects on each call. i opened leeoniya/uPlot#62 for an addData() api which can append to and reuse, translate and/or scale the internal Path2D objects instead. i actually think i found a chrome canvas bug when exploring this recently: https://bugs.chromium.org/p/chromium/issues/detail?id=1030804

i think with these optimizations, it should be easy to do 1.5M smoothly.

it would be interesting to try doing this via webGL (i've recently stumbled upon a micro webgl abstraction that looks interesting: https://github.com/doodlewind/beam). also https://www.pixijs.com/ might work, but it's not "micro" by any stretch of the imagination.

[AdamMarciniak]

So the way I have mine setup is I redraw all of the points to the canvas whenever there's a new point that comes in. It seems like this is the only way because if I want to append the point, I'd need to scale all the other data because it's live so it has to move to the left to make room. Rendering as columns looks promising but does this not have the same effect as just rendering less points? Adam

…

On Fri., Dec. 6, 2019, 1:30 p.m. Leon Sorokin, ***@***.***> wrote: I'm just wondering how is it implemented that makes it so fast? the main thing that makes it faster than naive raw canvas .lineTo() calls is this simple optimization: leeoniya/uPlot#15 <leeoniya/uPlot#15>. besides that, uPlot avoids doing any unnecessary mem allocations - it allocates nothing per datapoint (but this is compared to other charting libs, not raw Canvas). Because otherwise it seems to me you'd be redrawing each point on every refresh and with a huge amount of points that's not very fast. yeah, 1.5M points is not gonna be great with the current .setData() api which invalidates & rebuilds the Path2D objects on each call. i opened leeoniya/uPlot#62 <leeoniya/uPlot#62> for an addData() api which can append to and reuse, translate and/or scale the internal Path2D objects instead. i actually think i found a chrome canvas bug when exploring this recently: https://bugs.chromium.org/p/chromium/issues/detail?id=1030804 i think with these optimizations, it should be easy to do 1.5M smoothly. it would be interesting to try doing this via webGL (i've recently stumbled upon a micro webgl abstraction that looks interesting: https://github.com/doodlewind/beam). also https://www.pixijs.com/ might work, but it's not "micro" by any stretch of the imagination. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#5>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEID5YJC25B3ZMUPPOB34C3QXK76VANCNFSM4JVMKVTQ> .

[leeoniya]

take a look at the fiddle [1] in the chrome bug. that's probably gonna be the way to do it. basically you'd cache the Path2D object and append to it. then on each data append/redraw, re-apply a new matrix transform/translate on top of it. maybe via https://developer.mozilla.org/en-US/docs/Web/API/Path2D/addPath

the "fun" part will be figuring out when to do the invalidation & full rebuild so you don't end up growing the path indefinitely. unless you find an api to cheaply create a sub-path and trim/clip the original. i haven't played with it too much yet, but that seems to be the way to go, IMO. if you end up experimenting, i'd be interested in your findings.

it's probably possible to do this without Path2D and drawing/appending via ctx.beginPath() and applying the transforms to the Canvas directly. lots of things to try.

Rendering as columns looks promising but does this not have the same effect
as just rendering less points?

it's just a way to avoid the mem and perf overhead of allocating 1.5M objects/arrays. beyond init time, it likely has no impact as far as the actual drawing is concerned.

[1] https://jsfiddle.net/dt3u5fm8/