[wishlist] add an option for coloring scatter plot points based on data

[yarikoptic]

E.g. by selecting one of the (ordered, e.g. numbers ;-)) attributes to define colors of the points, thus effectively visualizing a 3rd dimension within the 2d scatter plot. Here is an example of scatter plotting (homebrewed scripts) of two neuroimaging volumes:

where y coordinate was used to color the scatter plot, immediately (without brushing) providing a clue about location of extreme points

I know that it would interfer with brushing feature, but may be with a choice of colormaps to be used

[ChrisBeaumont]

This has been on our radar for a while (along with data-encoded point size). This is technically pretty easy to add, but unfortunately matplotlib plots with color coded points are an order of magnitude slower to render than monochromatic ones. So it's also a potential pain point from an interactivity perspective

[yarikoptic]

I know, mpl indeed becomes more sluggish, but still it might work for some cases with smallish datasets.

Another implementation idea which might come useful: may be it would be feasible to bring OpenGL into the game for such renderings. E.g. there is a bit elderly (underused) https://code.google.com/p/glumpy/ and more recentish Galry https://github.com/rossant/galry (see also paper presenting it : http://journal.frontiersin.org/Journal/10.3389/fninf.2013.00036/full). Just did their example/plot.py with 100000 points and colors to each (plot(x, y1, 'oy', color=np.random.uniform(0, 1, (n, 4)))) and works splendid... and here we are -- in discussion at rossant/galry#10 (comment) where I was made aware of Galry's reincarnation into https://github.com/vispy/vispy.

[rossant]

@ChrisBeaumont and I have been recently discussing about a possible Vispy backend for glue/matplotlib (see this issue). Short answer is: it's possible, we (vispy devs) know what to do, and it might be ready-ish by the end of the summer.

Just a remark about scatter plots: like Chris said, matplotlib's scatter is more flexible but much slower than plot(..., '.'). When using mplexporter (cc @jakevdp), it looks like the former uses path collection, whereas the latter uses markers. Path collections seem to be pretty slow in matplotlib.

In Vispy, it will be much faster, but I fear we'll also have a discrepancy between the two methods. This limitation comes from matplotlib's internals and pyplot interface. Yet, it is definitely possible to do a scatter plot with heterogeneous sizes/colors with optimal performance on the GPU (see this example for instance). Not sure how to solve this issue with mplexporter.

Alternatively, we could provide another plotting interface that doesn't use matplotlib/mplexporter at all.

[yarikoptic]

with that example I am running into (the same) RuntimeError: Shader compilation error in Fragment Shader 3 (<string>)

[ChrisBeaumont]

My envisioned vispy integration looks either like:

1. A true vispy backend for matplotlib (doesn't fully leverage smart memory allocation onto GPUs, but my own experiments still suggest it would be ~50x faster than MPL currently). Having this would let glue use vispy automatically, without any extra code.

2. A separate scatter plot widet in Glue that bypasses MPL entirely, and works with vispy directly.

[rossant]

with that example I am running into (the same) RuntimeError: Shader compilation error in Fragment Shader 3 ()

git pull and try again? I've added the #version 120 pragma, it might solve your problem.

[yarikoptic]

ok -- will do. Is that linux magazine article out?

did git pull on http://github.com/vispy/vispy but there is no new ( today) commits

[rossant]

@yarikoptic could you open a new issue on the linuxmag article repo? I wouldn't want to pollute glue's issue tracker too much ;)

[yarikoptic]

ah -- you meant to "pull" the linux mag one -- got that one fresh, seems indeed to plot now ;-) but, ok, I will add an issue (different) just to please you

[astrofrog]

Note that we now have a plugin called glue-vispy-viewers that implements 3D viewers based on VisPy, and allow e.g. color-coding of points by attributes, etc. We should add 2D viewers to that plugin too in the same way (glue-viz/glue-vispy-viewers#138).

@rossant @yarikoptic - is this something you are still interested in helping with?

[PennyQ]

@astrofrog just wanna let you know we got this wish-list mentioned on the group meeting today :)

[astrofrog]

This has been implemented using Matplotlib in #1358 and will be in v0.11, and I'll investigate OpenGL solutions for future releases.