first pass at nteract support #562

chriddyp · 2016-09-02T21:13:40Z

No description provided.

chriddyp · 2016-09-02T21:14:51Z

Here's a first pass at offline interact support. Here's a comment in the commit for some details:

Display 2 versions of the figure:
 - The raw HTML is the "old-school" style - we just throw in a JS script
   inside the cell. This works for a lot of environments but some JS is
   stripped out in some rendering environments / platforms like
   GitHub, nteract, and eventually Plotly
 - The custom `mime-types` is the new-school way. nteract supports
   rendering plotly graphs this way (see https://github.com/nteract/nteract/pull/662)
   and others will hopefully follow suite.
 Both of these bundles will be saved as part of the notebook. The renderer
 will choose which one to render. So 'text/html' is like a fallback for
 any environment that doesn't have the 'application/json+plotly.v1'
 renderer available.
 The bummer is that we're injecting the figure JSON in here twice, so
 notebooks that are pretty big are about to get a lot bigger.
 The upside is that we're dropping any compatability for older notebook
 versions

chriddyp · 2016-09-02T21:15:30Z

Screenshot from nteract:

Associated nteract PR: nteract/nteract#662

chriddyp · 2016-09-02T21:16:48Z

Works across nteract and nbviewer, however opening these notebooks inside my existing jupyter notebook causes a validation error. It's harmless I think, but a pretty scary message to see.

chriddyp · 2016-09-02T21:17:28Z

CC @theengineear @rgbkrk @jackparmer @cldougl @charleyferrari @cldougl

theengineear · 2016-09-02T21:34:22Z

plotly/offline/offline.py

+    # Both of these bundles will be saved as part of the notebook. The renderer
+    # will choose which one to render. So 'text/html' is like a fallback for
+    # any environment that doesn't have the 'application/json+plotly.v1'
+    # renderer available.


Cool, yah, this makes sense to me. Agreed that notebooks getting bigger is a bummer... I'm not sure how much of an edge case that is.

rgbkrk · 2016-09-02T21:39:14Z

That's some hardcore validation. I'll bet if our key isn't application/json+ it would let it be.

@minrk, @fperez does this fall out of spec?

rgbkrk · 2016-09-02T21:48:15Z

Tried out application/vnd.plotly.v1+json in case it was just a hard check on format within application/json. No go. This is going to affect the custom mimetype for the widgets in JupyterLab too.

rgbkrk · 2016-09-02T22:14:30Z

Ah, here we go. Pretty explicit in the schema here if not the intention. If it's a custom mimetype it expects an array of strings or a string.

rgbkrk · 2016-09-02T22:20:55Z

I suppose we'd need the notebook schema to also have:

"patternProperties": {
  "^application/json\+[a-zA-Z0-9]+/[a-zA-Z0-9\\-\\+\\.]+$": {
    "type": "object"
  }
}

for JupyterLab/ipywidgets:

"patternProperties": {
  "^application/vnd\.[a-zA-Z0-9]+/[a-zA-Z0-9\\-\\+\\.]+$": {
    "type": "object"
  }
}

rgbkrk · 2016-09-02T22:25:44Z

Current proposal for registering custom mimetypes in nteract: nteract/nteract#670

fperez · 2016-09-02T22:43:30Z

Not quite sure where your thinking is right now, are you going in the direction of extending the nb schema?

rgbkrk · 2016-09-02T22:47:31Z

I'd like to pick the right future direction while not crippling usage of the classic notebook. There are quite a few options here, so I was outlining some thoughts on them.

fperez · 2016-09-02T22:52:12Z

Got it. I guess once this is a bit more fleshed out, would be good to run it by the Jupyter list for broader thoughts.

rgbkrk · 2016-09-02T22:59:20Z

Yeah, I'll make an outline in an email on the jupyter list. It's working quite nicely in nteract, and I'm working on vega support, soon bokeh as well.

rgbkrk · 2016-09-02T23:00:51Z

Beyond that though, it seems like the only way for us to do custom extensions is to make them be double encoded JSON in the current schema. Only the pure field application/json allows regular objects.

rgbkrk · 2016-09-03T15:28:15Z

Per resolution in jupyter/nbformat#43, the plotly mimetype should be application/vnd.plotly.v1+json.

rgbkrk · 2016-09-06T20:22:47Z

Since jupyter/nbformat#42 has been merged we can now send over the raw JSON on the next nbformat release.

However, since users of old versions of the notebook should allow this to still work, we can send the serialized JSON as a string directly, which was handled for backwards compatibility in nteract here: https://github.com/nteract/nteract/pull/675/files#diff-18d3d49b31452599ff46a67c571aa4f8R7

One other thing that still needs to be changed is that the mimetype should now be application/vnd.plotly.v1+json

theengineear · 2017-01-18T06:09:48Z

@chriddyp @rgbkrk just tried this and it seemed to work fine?

(this is just the current plotly release that I used)

Has something just changed in nteract?

chriddyp · 2017-01-18T15:14:28Z

@theengineear - online mode works fine, this would just be for offline support.
I'm all for supporting this PR. the only drawback is that we will be including the bundle and the Json twice: once for the interact minetype and another for Jupiter notebooks that don't support the mimetype enhancement yet. while we could detect which notebook type the author is using, we don't know which notebook type the viewer might be using, so we need to include both. big notebooks will go from 15MB to 30MB.

it doesn't seem like there is any getting around this, so it seems like it's something that we should just add. thoughts?

jackparmer · 2017-01-18T18:00:47Z

+1 from me. We can't really promote nteract too much without offline mode working. We'd just get too many support Q's.

theengineear · 2017-01-29T19:24:43Z

plotly/offline/offline.py

+    # The upside is that we're dropping any compatability for older notebook
+    # versions
+    display_bundle = {
+        'application/json+plotly.v1': {


@chriddyp I believe this needs to be updated now (as per #562 (comment)) to application/vnd.plotly.v1+json.

theengineear · 2017-01-31T18:37:38Z

^^ weird, sign_in seemed to fail all over the place in our tests... rebuilding...

theengineear · 2017-01-31T19:37:22Z

hrm... I think there's something off with how other tests are getting sign_in'd to. I.e., the setup for the tests is not sandboxed... which is problematic. I'm going to take the time to get this right.

We should have our tox (or CI?) setup always start the build off by deleting ~/.plotly/. Then, the more difficult change is to ensure that we use the PlotlyTestCase whenever we write a test that requires changing config/credentials in any way.

This will likely take a little while since a lot of our tests are still written in module-style (i.e., not class-based), so we need to first refact these tests so that we can properly subclass PlotlyTestCase 😿. Maw.

☕️ ☕️ ☕️ It's worth getting this correct now, we keep running into these issues.

Really, fewer of our tests should even be hitting a Plotly server, that that's a problem for another day...

Ideally, we'd just totally mock out the Python API and setup a suite of API tests inside the streambed repo. Again, definitely for another day, but it would greatly speed up our tests and completely decouple our tests from any specific plotly server.

theengineear · 2017-02-01T16:08:55Z

@chriddyp @rgbkrk I got tests passing and AFAIK offline mode is working as expected:

Anything else to be done here? I'm a little booked up on other tasks and don't want to let this linger much longer.

//cc @jackparmer

theengineear · 2017-02-01T16:09:52Z

^^ That's this nteract version

theengineear · 2017-02-02T21:31:05Z

Sorry to keep pinging, but are we all good with this @chriddyp? Anything else you wanted to add?

theengineear · 2017-02-02T21:31:23Z

It's got my 💃 :)

rgbkrk

Cool to see this moving forward!

rgbkrk · 2017-02-02T21:36:16Z

plotly/offline/offline.py

@@ -273,7 +273,7 @@ def iplot(figure_or_data, show_link=True, link_text='Export to plot.ly',
          validate=True, image=None, filename='plot_image', image_width=800,
          image_height=600):
    """
-    Draw plotly graphs inside an IPython notebook without
+    Draw plotly graphs inside an IPython or Jupter notebook without


s/Jupter/Jupyter/

rgbkrk · 2017-02-02T21:39:02Z

plotly/offline/offline.py

+    # - The raw HTML is the "old-school" style - we just throw in a JS script
+    #   inside the cell. This works for a lot of environments but some JS is
+    #   stripped out in some rendering environments / platforms like
+    #   GitHub, nteract, and eventually Plotly


JavaScript is not stripped out in nteract - the core differences are that:

require is commonjs require rather than requirejs' require in the desktop app

we don't have the same JavaScript API as the Jupyter notebook

Neither of those should be relied on for HTML outputs regardless (or at least some primitive feature detection and fallbacks need to happen).

^^ I ended up just 🔪-ing a lot of that comment out, I don't think it needs to get committed here.

rgbkrk · 2017-02-02T21:39:19Z

plotly/offline/offline.py

+    #   GitHub, nteract, and eventually Plotly
+    # - The custom `mime-types` is the new-school way. nteract supports
+    #   rendering plotly graphs this way (see https://github.com/nteract/nteract/pull/662)
+    #   and others will hopefully follow suite.


s/follow suite/follow suit/

1. no side effect files from running tests 2. don’t auto-open plot in browser, it auto-focuses and is incredibly distracting when multi-tasking.

Clean-slates each test. It doesn’t mean that the files aren’t dirty between tests, but because `session` takes precedence, this helps a lot.

chriddyp · 2017-02-03T00:06:00Z

looks great to me, this is awesome! 💃

Madhu94 · 2018-02-05T14:08:08Z

while we could detect which notebook type the author is using, we don't know which notebook type the viewer might be using, so we need to include both. big notebooks will go from 15MB to 30MB.

@chriddyp Did you find a way around this ? Or, do notebooks with multiple mime types just have to be larger ?

rgbkrk · 2018-02-05T16:30:31Z

They only have to be larger if you wish to support old versions of the notebooks.

Madhu94 · 2018-02-05T22:06:08Z

Thanks for the quick response @rgbkrk. Sorry, rephrasing my question, "Notebooks with custom mime-types which also have to be used with nbviewer just have to be larger? "

I guess that is a question to be asked in the nbconvert/nbviewer repo.

rgbkrk · 2018-02-05T22:41:23Z

Someone would have to add support to nbconvert and nbviewer, that's for sure.

Madhu94 · 2018-02-05T22:59:24Z

I'll take a shot.

Ipywidgets render in nbviewer by fetching the widget models and views from unpkg, and i think even this can now be configured. Maybe a similar strategy for fetching mime-renderers from somewhere..?

Anyway, I'll open a ticket for discussion. Thanks a lot. :)

chriddyp closed this Sep 2, 2016

chriddyp reopened this Sep 2, 2016

theengineear reviewed Sep 2, 2016
View reviewed changes

rgbkrk mentioned this pull request Sep 3, 2016

Relax pattern for JSON mime-types jupyter/nbformat#42

Closed

This was referenced Oct 23, 2016

Get embed_notebook() working inside nteract plotly/plotly.R#768

Closed

Add nteract support for JupyterPlot JuliaPlots/PlotlyJS.jl#90

Closed

Making plotly for R, Julia, & Python work in nteract nteract/nteract#1061

Closed

rgbkrk mentioned this pull request Nov 3, 2016

Plotly needs a jupyterlab extension to work with Jupyter Lab #600

Closed

rgbkrk mentioned this pull request Nov 21, 2016

Offline Plotly graphs not displaying nteract/nteract#1229

Closed

rgbkrk mentioned this pull request Dec 14, 2016

[WIP] feat: prototype a table component + mimetype for tabular data nteract/nteract#1280

Closed

theengineear reviewed Jan 29, 2017

View reviewed changes

rgbkrk approved these changes Feb 2, 2017

View reviewed changes

chriddyp and others added 13 commits February 2, 2017 17:04

first pass at nteract support

f1d223c

Make offline tests less annoying :)

c063b41

1. no side effect files from running tests 2. don’t auto-open plot in browser, it auto-focuses and is incredibly distracting when multi-tasking.

🔧 Cleanup test_get_figure.py.

463dfae

🔧 Cleanup test_get_requests.py.

1441b71

🔧 Cleanup test_grid.py.

ed6866f

🔧 Cleanup test_image.py.

4d66f1e

🔧 Cleanup test_meta.py.

71cb962

🔧 Cleanup test_plot.py.

0470c12

🔧 Cleanup test_stream.py.

8d1287c

🔧 Cleanup test_session.py.

c7d36f7

🔧 Cleanup test_file.py.

c586863

Load default config in PlotlyTestCase.

bf6d796

Clean-slates each test. It doesn’t mean that the files aren’t dirty between tests, but because `session` takes precedence, this helps a lot.

🔧 Clean up some comments.

f65724f

theengineear force-pushed the nteract branch from f15ae46 to f65724f Compare February 2, 2017 22:23

theengineear merged commit be3dab6 into master Feb 3, 2017

theengineear deleted the nteract branch February 3, 2017 00:15

DinoV mentioned this pull request Apr 10, 2017

Support plotly mime type (application/vnd.plotly.v1+json) fsprojects/IfSharp#142

Open

first pass at nteract support #562

first pass at nteract support #562

Conversation

chriddyp commented Sep 2, 2016

chriddyp commented Sep 2, 2016

chriddyp commented Sep 2, 2016

chriddyp commented Sep 2, 2016

chriddyp commented Sep 2, 2016

theengineear Sep 2, 2016

Choose a reason for hiding this comment

rgbkrk commented Sep 2, 2016

rgbkrk commented Sep 2, 2016

rgbkrk commented Sep 2, 2016

rgbkrk commented Sep 2, 2016 • edited Loading

rgbkrk commented Sep 2, 2016

fperez commented Sep 2, 2016

rgbkrk commented Sep 2, 2016

fperez commented Sep 2, 2016

rgbkrk commented Sep 2, 2016

rgbkrk commented Sep 2, 2016

rgbkrk commented Sep 3, 2016 • edited Loading

rgbkrk commented Sep 6, 2016

theengineear commented Jan 18, 2017

chriddyp commented Jan 18, 2017

jackparmer commented Jan 18, 2017 • edited Loading

theengineear Jan 29, 2017

Choose a reason for hiding this comment

theengineear commented Jan 31, 2017

theengineear commented Jan 31, 2017

theengineear commented Feb 1, 2017

theengineear commented Feb 1, 2017

theengineear commented Feb 2, 2017

theengineear commented Feb 2, 2017

rgbkrk left a comment

Choose a reason for hiding this comment

rgbkrk Feb 2, 2017

Choose a reason for hiding this comment

rgbkrk Feb 2, 2017

Choose a reason for hiding this comment

theengineear Feb 2, 2017

Choose a reason for hiding this comment

rgbkrk Feb 2, 2017

Choose a reason for hiding this comment

chriddyp commented Feb 3, 2017

Madhu94 commented Feb 5, 2018

rgbkrk commented Feb 5, 2018

Madhu94 commented Feb 5, 2018 • edited Loading

rgbkrk commented Feb 5, 2018

Madhu94 commented Feb 5, 2018

rgbkrk commented Sep 2, 2016 •

edited

Loading

rgbkrk commented Sep 3, 2016 •

edited

Loading

jackparmer commented Jan 18, 2017 •

edited

Loading

Madhu94 commented Feb 5, 2018 •

edited

Loading