3D API

[tbreloff]

I have decent support for 3D plotting already, but the internals are a little too hacky/messy for my tastes. I want to refactor a little:

• Better context-aware handling of input data (for example: matrix gets passed through for a surface plot, but not for a scatter) see: True heat maps #147 (comment)
• Arguments/handling of z-axis settings (subsumes 3D plotting #57)
• push!/append! and getindex/setindex! for 3D data

[GravityAssisted]

Do you know what all backends support 3D plots ? As far as I know, its only matplotlib, am I correct ?

[tbreloff]

Plotly, PlotlyJS, GR, and GLVisualize as well.

[pkofod]

(and soon PGFPlots https://github.com/pkofod/pgfplots_temp/blob/a9612f01032cb95922480ab13bdb922882ba27db/PGFPLots.ipynb , at least scatter atm, but surface plots need to be added to PGFPlots.jl first. They are certainly supported in PGFPlots: http://pgfplots.sourceforge.net/gallery.html )

[tbreloff]

I just merged (into dev) a pretty substantial overhaul to the internals of plot creation. As part of the overhaul:

• dictionaries that represent keyword arguments now use the typealias: typealias KW Dict{Symbol, Any}. you'll see this all over the place, so take note if you're working on backend code.
• separation of old createKWargsList into process_inputs and build_series_args (also in a new file)... more robust handling of corner cases (I hope!)
• z argument is "first class" now, so 3D should be just as natural as 2D in many cases
• replaced keyword splatting with dictionary passing in many places (and many more to come)
• overhaul to the way the group arg is processed

I plan to continue code cleanup, but maybe not for a couple weeks. Please, if you notice anything weird after all these changes, open an issue.

cc: @spencerlyon2 @pkofod @dlfivefifty

[pkofod]

Everything looks OK for the pgfplots branch.

[dlfivefifty]

glvisualize() is working.

I'm disappointed that surface(::Matrix,::Matrix,::Matrix) isn't working yet, to support

using PyPlot

r=linspace(0.,1.,100)
θ=linspace(-π,π,100)
x=Float64[r*cos(θ) for r in r, θ in θ]
y=Float64[r*sin(θ) for r in r, θ in θ]
z=Float64[exp((r*sin(θ))^2+(r*cos(θ))^2) for r in r, θ in θ]

surf(x,y,z)

[tbreloff]

It doesn't work? Are you on dev? I'll have to look later. It should be easy to support after the reorg from a couple days ago.

On Mar 19, 2016, at 4:36 PM, Sheehan Olver notifications@github.com wrote:

glvisualize() is working.

I'm disappointed that surface(::Matrix,::Matrix,::Matrix) isn't working yet, to support

using PyPlot

r=linspace(0.,1.,100)
θ=linspace(-π,π,100)
x=Float64[r_cos(θ) for r in r, θ in θ]
y=Float64[r_sin(θ) for r in r, θ in θ]
z=Float64[exp((r_sin(θ))^2+(r_cos(θ))^2) for r in r, θ in θ]

surf(x,y,z)
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub

[tbreloff]

Small change just pushed up to dev. Here's what I see for PyPlot:

and here's the similar call with Plots:

[dlfivefifty]

Beautiful!

Sent from my iPhone

On 20 Mar 2016, at 12:39, Tom Breloff notifications@github.com wrote:

Small change just pushed up to dev. Here's what I see for PyPlot:

and here's the similar call with Plots:

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub

[dlfivefifty]

The surface(::Matrix,::Matrix,::Matrix) variant doesn't work in glvisualize or plotlyjs yet:

using Plots; glvisualize()

r=linspace(0.,1.,100)
θ=linspace(-π,π,100)
x=Float64[r*cos(θ) for r in r, θ in θ]
y=Float64[r*sin(θ) for r in r, θ in θ]
z=Float64[exp((r*sin(θ))^2+(r*cos(θ))^2) for r in r, θ in θ]

surface(x,y,z)
ERROR: MethodError: `start` has no method matching start(::Plots.Surface{Array{Float64,2}})
 in map at abstractarray.jl:1157
 in _add_series at /Users/solver/.julia/v0.4/Plots/src/backends/glvisualize.jl:35
 in _add_series at /Users/solver/.julia/v0.4/Plots/src/plot.jl:224
 in plot! at /Users/solver/.julia/v0.4/Plots/src/plot.jl:102
 in plot at /Users/solver/.julia/v0.4/Plots/src/plot.jl:56
 in surface at /Users/solver/.julia/v0.4/Plots/src/Plots.jl:180

I'm happy to fix the GLVisualize case when I get the chance!

PS PlotlyJS 3d looks great, and feels much faster than PyPlot

[GravityAssisted]

Do you plan to support MathGL.jl? MathGL is very nice scientific plotting library and has 3d capability. MathGL website.

[tbreloff]

No immediate plans. It's been mentioned before, but are there any "killer features" that don't exist in other supported backends?

The best way is for someone to step up and submit a PR!

On Mar 21, 2016, at 12:31 AM, Nitin Arora notifications@github.com wrote:

Do you plan to support MathGL.jl? MathGL is very nice scientific plotting library and has 3d capability. MathGL website.

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub

[protogeezer]

Would this be an appropriate spot to start discussing potential 3D technologies that could be brought to Julia? I'm interested in contributing a native-Qt graphics layer - probably in conjunction with VTK 7.0. There are, however, options that need to be debated.

[tbreloff]

Would this be an appropriate spot

Are you suggesting building a new backend? That should be its own issue, and most likely doesn't belong in Plots.

I'm interested in contributing a native-Qt graphics layer

Cool! If you want to build some core, low-level functionality, it would be "easy" to tap into the Plots interface. What do you plan on offering beyond GLVisualize or anything else that exists?

I probably won't contribute directly to this effort, but happy to help answer questions best I can.

[protogeezer]

One one hand it would be great if what I end up doing makes it possible for you to extend the easy to use Plots package to more robust simple 3D plots. (at least through 1.4+ Matplotlib's 3D wasn't general - the z buffer algorithm they used periodically couldn't correctly chose the correct order to paint objects). On the other hand I do want to try to find an astute set of features to contribute to the overall Julia project. Once you get the gist for the information I'm seeking, feel free to switch to a better suited issue.

For example - how best to solicit the input others that have an interest in 3d or even faster or higher quality 2d capabilities. Does anyone really care if state-of-the-art graphics is even available? If wireframes, meshes and line plots are good enough, I may be barking up the wrong tree... If so, there would certainly not be much reason to think about going beyond VTK to ITK...

For my own purposes, a clean Julia interface to VTK would be nearly ideal. At some point, the only way to create complex graphics (a 3D model with 2D slices, e.g.) is to start building the scene directly in VTK. However, I still frequently use Mayavi too. If we reach a consensus on what you need to provide something like a Mayavi high-level 3d capability, I'd be happy to make that work.

One of the more sensitive issues I've been studying is the role to propose for Compose/Compose3D. I'm not sure that there would be much left of them once a full featured graphics framework is available. Qt and VTK (views) both offer similar functionality and appear a lot faster and in many ways more robust. In particular, once generating graphic file formats and layout of multiple plots is done using the framework(s), then there isn't much left for Compose.

My experiments with the CXX.jl package make me think it may be too slow to reach Dan Jones goal of Gadfly being fast enough to drive animation in realtime. Do you have any other experience with CXX.jl?

Reorganizing the various graphics packages away from a focus on Compose may also largely obviate Escher.jl: Qt offers a robust way to generate interactive browser content.

Tim Holy has a "Julia/VTK Proof of Concept" in his repository. He claims no or low overhead calls to C++. So, one tack could be to extend his LLVM code to generate Julia types to mimic the VTK or Qt classes - thus making it possible to use an OO coding style to traverse and call members of the classes.

Would something like that work for you? We'd have to experiment a bit to ascertain whether the VTK 2D capabilities are on par with the Qt 2D and basic 3D views.

The other issue I'm not real sure how to navigate is the dissonance between the julia runtime and "native" runtime functions - threads, the event loop, maybe even memory management. It would be fundamentally important to be able to use both the REPL and windows with (GUI) controls without the hangs that are possible now. And the frequent pauses for the GC are a problem when trying to do realtime animation. Those sorts of things ought to be resolvable with configuration options.

Your insights would be appreciated!

[tbreloff]

I have a lot to say about some of this. I've spent a ton of time doing Qt guis/visualizations, building real-time detached processes for gui loops. I like zeromq as a messaging layer, and shared memory for anything fixed-size. I think the gui-scene in julia is ripe for an overhaul, but it's not high enough a priority for me to take it on.

There are some really nice ideas in Compose/Compose3D, but I'm not sure its quite scalable to big simulations. Before you get too deep into any implementations, you should definitely see if GLVisualize meets your needs (and specifics on where it falls short). I think, if it doesn't already exist, it could be hooked up to Qt.

To be clear... The role of Plots is for the interface and niceties of common visualizations. Ideally I never have to write raw drawing/rendering commands, just translating to the "language" of the backend. Also ideally I can reuse existing abstractions when possible. For example, I'd rather not make my own mesh types... I'd prefer to leverage Simon's work (and collaborators). I have tried to contribute to backend packages when features are missing that belong there.

I'll try to write more another time. In the meantime, maybe you could create a wishlist of features, and a few examples of visualizations/workflows that you want to enable. Examples help a ton when making design decisions.

On Apr 19, 2016, at 7:13 PM, protogeezer notifications@github.com wrote:

One one hand it would be great if what I end up doing makes it possible for you to extend the easy to use Plots package to more robust simple 3D plots. (at least through 1.4+ Matplotlib's 3D wasn't general - the z buffer algorithm they used periodically couldn't correctly chose the correct order to paint objects). On the other hand I do want to try to find an astute set of features to contribute to the overall Julia project. Once you get the gist for the information I'm seeking, feel free to switch to a better suited issue.

For example - how best to solicit the input others that have an interest in 3d or even faster or higher quality 2d capabilities. Does anyone really care if state-of-the-art graphics is even available? If wireframes, meshes and line plots are good enough, I may be barking up the wrong tree... If so, there would certainly not be much reason to think about going beyond VTK to ITK...

For my own purposes, a clean Julia interface to VTK would be nearly ideal. At some point, the only way to create complex graphics (a 3D model with 2D slices, e.g.) is to start building the scene directly in VTK. However, I still frequently use Mayavi too. If we reach a consensus on what you need to provide something like a Mayavi high-level 3d capability, I'd be happy to make that work.

One of the more sensitive issues I've been studying is the role to propose for Compose/Compose3D. I'm not sure that there would be much left of them once a full featured graphics framework is available. Qt and VTK (views) both offer similar functionality and appear a lot faster and in many ways more robust. In particular, once generating graphic file formats and layout of multiple plots is done using the framework(s), then there isn't much left for Compose.

My experiments with the CXX.jl package make me think it may be too slow to reach Dan Jones goal of Gadfly being fast enough to drive animation in realtime. Do you have any other experience with CXX.jl?

Reorganizing the various graphics packages away from a focus on Compose may also largely obviate Escher.jl: Qt offers a robust way to generate interactive browser content.

Tim Holy has a "Julia/VTK Proof of Concept" in his repository. He claims no or low overhead calls to C++. So, one tack could be to extend his LLVM code to generate Julia types to mimic the VTK or Qt classes - thus making it possible to use an OO coding style to traverse and call members of the classes.

Would something like that work for you? We'd have to experiment a bit to ascertain whether the VTK 2D capabilities are on par with the Qt 2D and basic 3D views.

The other issue I'm not real sure how to navigate is the dissonance between the julia runtime and "native" runtime functions - threads, the event loop, maybe even memory management. It would be fundamentally important to be able to use both the REPL and windows with (GUI) controls without the hangs that are possible now. And the frequent pauses for the GC are a problem when trying to do realtime animation. Those sorts of things ought to be resolvable with configuration options.

Your insights would be appreciated!

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub

[protogeezer]

Hi Tom - I got distracted on other issues for the past week or so. I hope we can continue the conversation and evolve a design for a julia graphics framework that is likely to succeed.

The high level goal is proposing and ultimately implementing a cross-platform graphics (with the attendant runtime) framework that will put julia on par with other scientific packages - matlab, scilab, mathematica, etc. I don't believe GTK graphics and the LibUV runtime will make it in the long term. That is a major obstacle to having high performance graphics.

The strength of thinking in terms of Qt or Qt + VTK or VTK for 2D and 3D graphics is that they allow (in that order) increasing exposure of the underlying window/graphics layers in (especially) OS X and Windows (obviously Qt on Linux is a slam dunk). I'd actually prefer the latter since it seems to expose the most of the underlying graphics layer (and power), but Qt makes it a lot easier to envision a coherent runtime and graphics layer.

I'm particularly impressed with the new sophistication in VTK 7.0. Thus my interest in thoroughly vetting it. One of the decision points that I don't have a handle on is whether Qt's 2D and simple 3D graphics views are better for some graphics than that in VTK. Certainly ITK has no peer functionality in Qt.

You hit the nail on the head with the observation that the current graphics design (mostly Compose) is probably not going to scale. When I met with Dan J. last year he lamented that he was probably well on the way to writing the slowest graphics package on the planet. Thus the suggestion that a native Qt graphics framework might fit the bill. I can't tell yet whether sliding that in under Compose is feasible to meet the suggested performance and feature list Dan suggested.

The two subordinate wishes are (1) a Cmake based build system - something I'd need to make it possible to configure the build to take advantage of platform strengths, and do builds in parallel, and (2) a window-aware and probably platform aware runtime (i.e. no libuv).

It seems to me that VTK (and or Qt) is to the graphics layer as LLVM is to the core numeric capabilities. There is currently a lack of balance in that regard that I'd like to help fix.

[tbreloff]

I don't see anything actionable here, so I'm going to close. @SimonDanisch is going to blow us all away soon with his glvisualize backend :)