Plotly streaming performance

[pculbertson-bdai]

Hi Brent! I wanted to get your advice on a somewhat strange use case -- I'm building an app where I'd like to plot a relatively large amount of data that I'm streaming from the Python side (specifically, like dozens of line plots that will update in real-time).

I've done this before using OpenGL, but would love to do something similar using viser + plotly. However, I'm (understandbly) seeing that trying to stream this amount of data (following your design pattern in the Plotly example) is overwhelming the webapp (it's dropping to like 5-10 FPS).

I've seen people suggest doing this using, e.g., Dash with client-side updates, but was curious if you had any suggestions on how to do large-scale real-time plotting in viser. Totally happy to just open a separate OpenGL window if that's the simplest solution also. Thanks!!!

[brentyi]

Hi again Preston!

This sounds like a reasonable use case. It would be nice to take some steps toward real-time plotting capabilities—I've spent a ton of time staring at rqt_plot so I understand how useful it is 😛. But unfortunately yeah, we're not there yet.

I haven't used Dash personally but that sounds like it could be reasonable. There's visdom, which looks cool but I also haven't personally worked with. Rerun and Pangolin also do this kind of plotting quite well.

With Plotly specifically I did some quick experiments just now and the bottleneck seems almost entirely in plotly.js, so there's not much we can do to optimize. Other options I can think of are:

• Design our own real-time plotting system.
• Find some other Javascript-based plotting library that we can wrap that's faster.
• Improve support for displaying + updating images in viser, and just send plots as images.

The first two would be large undertakings. The third would be generally useful and could be implemented quite quickly; would that be good enough? I think we could expose a server.gui.add_image() that could be updated reasonably efficiently (similar to how we update the viewport background for nerf/gaussian rendering with server.scene.set_background_image()).

[pculbertson-bdai]

Hi Brent, thanks so much for the detailed answer (and sorry for my super slow response lol!).

I totally agree with the options you're outlining here - I think as a "quick fix" streaming plot images makes sense, and if I get a chance I can start prototyping and profiling something like server.gui.add_image (we ended up "back burnering" the plotting after your answer, since it doesn't seem "plug-and-play").

I think the first two definitely seem like huge undertakings that are probably not worth the effort - something like rerun's plotting would be cool, but it's also a very heavy dependency with some amount of feature overlap with viser, so it's probably best not to add as a dependency.

Will ping you again if I get a chance to code up something -- but really want to thank you again for the thoughtful answer!!

[brentyi]

Thanks for the update!! Yeah, the add_image() would also be generally useful + fast for me to throw together so I can also let you know if I get to it. I've been using the Plotly API for images occasionally but it's quite slow.

With #315 merged you could also draw the plots as lines in the 3D viewport...