Making widgets talk to each other!

[ramnathv]

I have been experimenting with the ability to use widgets as components and then allow them to communicate with each other.

Here are some examples

1. Life Expectancy Choropleth + Line
2. Datamaps + Chroniton
3. Datamaps + Morris + Sortable

The first example uses shiny, while the second does not.

In the case of Shiny, I am using Shiny.onInputChange to trigger changes and communicate back to the server from the client on mouseover. In the second example, I am directly manipulating the datamap from the callback code of the chroniton widget.

The more I think about this, I believe that something like a pubsub design pattern will be helpful here. htmlwidgets can provide basic pubsub infrastructure and widget authors can provide callbacks that allow users to broadcast data on certain events. In a similar vein, widget authors can also expose callbacks that allow a widget to listen to certain events and trigger a change.

By following some conventions regarding how these events are named, it would be possible for widgets to talk to each other using pubsub. Currently, this idea is still very hazy in my mind, but I wanted to put it out here so that it can trigger a broader discussion.

In the shiny case, things are easier, since Shiny.onInputChange provides a simple pubsub mechanism. I am thinking of a more general mechanism that will work with and without Shiny.

@jcheng5 I would really appreciate any thoughts you have on this, given your experience working with javascript libraries. @jjallaire @timelyportfolio @yihui Any thoughts/comments on this idea would be welcome, as I think it would be a very powerful way forward for htmlwidgets to create a componentized architecture.

[timelyportfolio]

I agree pubsub would most likely be the best pattern. While crossfilter is nice in dc.js I think it will prove too limited for multiple htmlwidgets and difficult to teach to users. Ideally, the same event information will get passed to Shiny.onInputChange in shiny mode as our pubsub engine when not in Shiny mode, or we could allow the pubsub to notify shiny.onInputChange. I have been monitoring the work on vega closely and experimenting with various pubsub libraries. I don't have a favorite.

[abresler]

This looks amazing I can't wait

---

Alex Bresler
abresler@asbcllc.com
​www.asbcllc.com​
917-455-0239​ (cell)​
On Mar 5, 2015 8:10 PM, "Ramnath Vaidyanathan" notifications@github.com
wrote:

I have been experimenting with the ability to use widgets as components
and then allow them to communicate with each other.

Here are some examples

1. Life Expectancy Choropleth + Line
   https://www.youtube.com/watch?v=fbmSHQrrJrI
2. Datamaps + Chroniton https://t.co/9KzVYkbErf

The first example uses shiny, while the second does not.

In the case of Shiny, I am using Shiny.onInputChange to trigger changes
and communicate back to the server from the client on mouseover. In the
second example, I am directly manipulating the datamap from the callback
code of the chroniton widget.

The more I think about this, I believe that something like a pubsub
design pattern will be helpful here. htmlwidgets can provide basic pubsub
infrastructure and widget authors can provide callbacks that allow users to
broadcast data on certain events. In a similar vein, widget authors can
also expose callbacks that allow a widget to listen to certain events and
trigger a change.

By following some conventions regarding how these events are named, it
would be possible for widgets to talk to each other using pubsub.
Currently, this idea is still very hazy in my mind, but I wanted to put it
out here so that it can trigger a broader discussion.

In the shiny case, things are easier, since Shiny.onInputChange provides
a simple pubsub mechanism. I am thinking of a more general mechanism that
will work with and without Shiny.

@jcheng5 https://github.com/jcheng5 I would really appreciate any
thoughts you have on this, given your experience working with javascript
libraries. @jjallaire https://github.com/jjallaire @timelyportfolio
https://github.com/timelyportfolio @yihui https://github.com/yihui
Any thoughts/comments on this idea would be welcome, as I think it would be
a very powerful way forward for htmlwidgets to create a componentized
architecture.

—
Reply to this email directly or view it on GitHub
#86.

[ramnathv]

Here is another example using the pubsubz architecture.

http://bl.ocks.org/ramnathv/raw/51f61a43f81910b868a5/

The idea is rather simple. The morris and dimple widgets listens to an event named elid_group and updates the chart accordingly. The datamaps widget broadcasts the state being hovered on to both these widgets to effect the change. It is very similar to what Shiny.onInputChange does. The question is can we write these events in such a way that both static and shiny contexts can take advantage of it.

[happyshows]

@ramnathv is there any updates on how htmlwidgets will communicate with each other in future? Also, could you share the code for the examples?

[ramnathv]

@happyshows this is a hard problem to solve in general. So it will take some time before we have more clarity on this. As for code, these are all widgets under development. So once I push them to github, I will also release the examples.

[happyshows]

@ramnathv thanks. In future will these code be under htmlwidgets or independent repo? I'm just trying to find a way to trace the progress.

[timelyportfolio]

Just circling back on this. Any thoughts on how to implement this?

[ramnathv]

The more I think of it, pubsub seems to be the answer. I will write up whatever I have discovered so far so that we can push this forward.

[jcheng5]

FWIW, I'm planning to look at this problem in earnest in October, probably in collaboration with @jjallaire and @hadley. I have a hard time imagining how the answer would not involve some kind of pubsub/event bus, but the semantics will need to be chosen very carefully.

[ramnathv]

@jcheng5 I would love to be a part of this discussion if possible.

[jcheng5]

@ramnathv Yeah, for sure.

[happyshows]

@ramnathv I'm also interested in clippyr (visual help guide) , is it on github under different name?

[ramnathv]

@happyshows I will put it up on github this weekend.

[happyshows]

@ramnathv , is clippyr on github now?

[timelyportfolio]

@jcheng5 & @ramnathv , it's October :) and I'm ready when you are for this discussion.

[jcheng5]

Just want to let you guys know that I am actively working on this. @jjallaire, @hadley, @wch, and I were all together in person last week and discussed this at length. I'll soon have a strawman writeup and prototype to serve as a starting point for further discussion.

[ramnathv]

Great! Looking forward to this @jcheng5 !

[timelyportfolio]

very happy to hear. let me know if I can help.

[jcheng5]

Sorry for the delay. I had to reboot this effort twice, I didn't realize how big the solution space was when I first started thinking about this and it took a while to get to an approach that was both practical and flexible.

I'm thinking about things in terms of a "near-term plan" that is actionable today and involves code we definitely can all agree needs to be written, but is modestly scaled in terms of ambition. Then we wait to see what kind of emergent behavior comes out of those improvements, and what opportunities we have to improve htmlwidgets to help out. Though below I also propose one possible direction that might be interesting to explore.

Near-term plan

The approach I'm advocating has two "prongs" today that address the needs of two different types of user. One is the R user who doesn't know JavaScript, and the other is the intermediate-to-expert JavaScript programmer.

For R users who don't know JavaScript:

We want to introduce a mechanism that will allow them to do things like linked brushing, linked identify, and dc.js/crossfilter-style filtering, without any effort on the users' part; just rendering three widgets that share a (new) "group" attribute value should link them together. Enabling this will take a significant amount of effort on widget authors' parts though, and we will have to provide both education and encouragement. The payoff is pretty huge if we can get this right.

That's the ultimate goal. The first step we want to take in this direction is to introduce a layer that different widgets in a page can use to speak to each other. I've built a prototype of such a layer in a tiny package I'm calling crosstalk. There's a NOTES.md document that outlines the capabilities, and the repo is here: https://github.com/rstudio/crosstalk

(BTW, for posterity, the link to the intro document is pinned to a specific commit; so if you're looking for the most up-to-date version of that file, be sure to go to the master branch first.)

I threw together a minimal d3 scatter plot package to explore these ideas without getting distracted by the complexity of most real-world widgets. The repo is at https://github.com/jcheng5/d3scatter and you can see an example of it in action with crosstalk here: http://rpubs.com/jcheng/crosstalk-demo

Crosstalk is designed not just for widget-to-widget communication, but also widget-to-Shiny and Shiny-to-widget communication as well. Try the examples on the d3scatter README. I think it'd also be very easy to extent Crosstalk to support other viz frameworks like RCloud, Bokeh.js, and Plotly.

I would love to get feedback on this approach over the next month. The high-level selection API mentioned in the NOTES.md still needs to be completed; I'm hoping to finish that next week.

For JS programmers:

The above approach is only going to be helpful for some common, standardized behaviors. It's not that helpful for totally ad-hoc interactions between widgets. For programmers proficient in both R and JavaScript, it can be challenging to get htmlwidgets to work together even with custom JavaScript code, without resorting to some ugly hacks. Primarily this is because we don't provide an easy way for htmlwidgets to have arbitrary methods on their objects; and even if we did, we don't have an easy way for regular JavaScript code to get ahold of widget objects. I have addressed the first part of that in this PR: #171

If we agree on that approach the second part of it (making it easy to get ahold of widget objects) will be straightforward. If these changes are noncontroversial they could be on the master branch before the end of next week.

Future plans

So non-JS-wielding R users will be able to do basic interactivity, and JS-savvy R users will be able to do anything they want (by writing custom JS code). I suspect there may be a fertile ground in between, where we can allow non-JS R users to do some declarative stuff in R that is way more flexible than just basic interactivity, but still keeps them from having to learn JS. I'm thinking maybe something akin to the Cocoa Target-Action pattern. This is pretty speculative though, and depends on the adoption of Crosstalk in the first place. So we'll wait to see what the reaction is to the first two prongs before we move forward with this more ambitious idea.

---

Feedback appreciated. If this isn't making sense to people I'm happy to vchat, or make a brief screencast or something, next week after the Thanksgiving holiday.

[jjallaire]

@hafen and @cpsievert this is a general purpose mechanism for widget-to-widget and shiny-to-widget communication that we are working on now. We'd love to make sure that there are no major impedance problems with rbokeh and plotly as getting those packages to work with this scheme would be a huge win. Let us know your thoughts.

[jjallaire]

@hafen and @cpsievert , The overall design is here:

https://github.com/rstudio/crosstalk/blob/master/NOTES.md

And here is a demo of what the R code currently looks like (with and without Shiny):

https://github.com/jcheng5/d3scatter/blob/master/README.md

[timelyportfolio]

@jcheng5, @ramnathv will group be the universal argument across all htmlwidgets to express which htmlwidgets will listen on the same "channel"? I only ask because I know that in rCharts we used group/groups as an argument similar to lattice. If we plan for group to be the argument, then we will need to change our rCharts2 API. Thanks.

[jcheng5]

Yeah leaflet has a similar problem with the word "group". We can change it.

[timelyportfolio]

What do other pubsub implementations use? Would channel be an option?

[timelyportfolio]

Another question out of ignorance, would this work with proxy mechanisms such as those in leaflet and I think soon in visNetwork? I would think yes, but thought I would bring it up just in case.

[jcheng5]

It really depends on the implementation to do it correctly, but I would imagine yes.

[jcheng5]

The "For JS programmers" section from #86 (comment) is now feature complete in #172.

[cpsievert]

Great stuff @jcheng5! So excited to work on this. I have some work in progress here and have some basic examples here.

Make sure you install the right branch before running examples:

devtools::install_github("ropensci/plotly@feature/events")

A few things/questions:

(1) For brush events, plotlyjs natively supports one mode: zoom. I have mentioned to the plotlyjs engineers (@etpinard, @alexcjohnson, @chriddyp, @cldougl, et. al.) that adding support for more modes (e.g., select box a la bokeh) would be a huge help, and there is intention on working on this.
(2) group has a different meaning in plot_ly(), so I used set instead (this can change, of course).
(3) Is there an elegant way to make an inline-block of htmlwidgets via htmltools?

[jackparmer]

++ @mdtusz ^^^

[etpinard]

@cpsievert data.points[i].xaxis and data.points[i].yaxis are indeed circular structures.

They refer to the points' corresponding cartesian axis objects which you shouldn't need for your use case.

I'd recommend doing something like:

data.points.forEach(function(pt) {
  delete pt.xaxis;
  delete pt.yaxis;
});
JSON.stringify(data);

[alexcjohnson]

@cpsievert @etpinard even better, pull out just the pieces you really want from data.points. Each point has the COMPLETE trace and fullData (after supplyDefaults) trace, the latter of which contains references to code in _module... wayyyy more than you want. I'd prefer:

JSON.stringify(data.points.map(function(pt) {
    return {
        curveNumber: pt.curveNumber,
        pointNumber: pt.pointNumber,
        x: pt.x,
        y: pt.y,
        key: pt.data.myKey[pt.pointNumber] // if you need something else as the key
    };
});

[cpsievert]

Thanks @etpinard @alexcjohnson. I agree that we should send as little data as possible, so I bootstrapped on @alexcjohnson suggestion in plotly/plotly.R@d180812

Here is a demo of handling plotly click events in shiny via crosstalk (source).

[jcheng5]

Nice work @cpsievert. The demo app code looks really clean.

You could simplify further by getting rid of the rv reactive expression; you can declare cv as a variable directly inside the server function, and use cv$get() from any reactive expression, observer, or output you wish. In fact, even if you decide to keep the rv reactive for future purposes, you should still hoist the cv <- ClientValue::new(...) declaration at the top level of the server function, and just put cv$get() inside the body of rv.

[jonathancallahan]

Just wanted to say that I'm very excited to see this work progress. I see mention of widget-widget and widget-shiny interactions but I didn't see any mention of "widget-R".

Will this work allow us to create widgets that interact with the RStudio environment/console?

I'm imagining an interactive widget that allows people to visually select some subset from a dataframe and that selection or that subset will somehow be communicated back to the RStudio environment. Alternatively, a widget be could used as a point-and-click interface to create an R expression that gets loaded into the console.

We are already creating a package that harnesses the dygraph and leaflet widgets to basically customize RStudio into a statistical analysis tool for non-programmer, Forest Service Personnel. It sure would be awesome if there were a mechanism by which scientists could click on a monitoring site in the leaflet widget and have their selection immediately available in the current R environment.

[jcheng5]

@jonathancallahan Yes, but you don't even need htmlwidgets for that--anything you can do in Shiny can be harnessed by the current R environment.

Try this (requires devtools::install_github("rstudio/shinygadgets"); and looks best in RStudio, although it'll work with any R):

library(shiny)
library(shinygadgets)
library(leaflet)

ui <- fillPage(
  leafletOutput("map", height = "100%")
)

server <- function(input, output, session) {
  output$map <- renderLeaflet({
    leaflet(quakes) %>% addTiles() %>% addCircleMarkers(layerId = as.character(1:nrow(quakes)))
  })

  observeEvent(input$map_marker_click, {
    stopApp(quakes[as.numeric(input$map_marker_click$id),])
  })
}

runGadget(ui, server)

[hafen]

Sorry I'm late to the game - been doing an insane amount of travel and catch-up when not traveling. This looks awesome and I'm really excited to dig into it. I'll be more involved at the beginning of the year.

[timelyportfolio]

@ramnathv, any progress on a new scaffold template? I'm almost thinking of working on this as my htmlwidget of the week. Let me know if I can help.

[timelyportfolio]

Or, I could convert one with a writeup to publicize the new direction for the year. Any particular htmlwidgets that provide an easy use case? Would knob be appropriate since it was a test for the first implementation?

[ramnathv]

@timelyportfolio I already have a local branch with the new scaffold template. I am on vacation currently, but will push it out over the weekend. I will be including an example as well.

[timelyportfolio]

Now that my widget per week commitment is over, I can play with crosstalk :) Here is d3scatter with parcoords http://bl.ocks.org/timelyportfolio/4c5718a7efe5c0abd363. Currently, I only have send not receive for parcoords. Thanks so much @jcheng5 for spearheading this.

[happyshows]

@timelyportfolio great achievement for 2015 :D Could you setup a voting app with multi choices for each widget and improve the highly rated ones later on?

[timelyportfolio]

@happyshows, thanks for all your usage/feedback on htmlwidgets. I had hoped to get a feel for popularity by Github stars, but unfortunately getting stars on Github for R projects seems nearly impossible. Unless otherwise motivated by feedback, ideas, use cases, I'll probably just pick my favorites.

[hafen]

@timelyportfolio congrats on getting through the year of htmlwidgets! I'd love to get all your missing widgets onto the htmlwidgets gallery. If we made the site more prominent and added a gallery, etc., then maybe it would become a place for things like github stars to become more effective.

[happyshows]

@timelyportfolio Maybe put a survey monkey link on your site to collect preference & feedback data instead? just my 2 cents.

[ThomasSiegmund]

Hi all,

I'm very happy about crosstalk. IMHO this is the next big step for the Shiny and htmlwidgets ecosystem.

I have now added support for crosstalk in D3TableFilter. There is a demo app combining a d3scatter and a d3tf widget. Selecting dots in the plot highlights corresponding rows in the table and vice versa.

I guess it will take a while and a few good apps to figure out all the semantics. The table may allow to filter out rows - what should happen to the dots in the graph? Should we have an additional crosstalk variable ("hidden") ?

Thanks a lot @jcheng5 !

[timelyportfolio]

@jcheng5, @ramnathv any updates on this?

[ramnathv]

@timelyportfolio mobservable has changed their API significantly. I am reworking things using the new version and will put it out here. There are still several outstanding issues in terms of the order in which observables get evaluated. Based on conversations with @jcheng5, it is bound to be much harder doing everything client side. But I think there is scope for an effective solution.

[timelyportfolio]

@ramnathv I wondered if the change to mobx would throw a kink into the solution, and I guess the answer is yes. As always, I am very interested and happy to help in any way possible.

[jcheng5]

I should probably write up a blog post or something about the current state of things--I've had the same conversation privately with a number of people. Basically there's several different directions things are growing, each of which is ideal for a particular kind of widget user (depending on how skilled they are in JS, how much time they want to spend writing custom code, and how much R runtime support [Shiny] is needed).

I had told @cpsievert and @hafen (and possibly @ramnathv) I was having second thoughts about the crosstalk approach, but subsequent conversations with @jjallaire have convinced me that it's an important piece of the story. I'm tentatively planning on investing further in crosstalk over the next few months, with an eye towards a CRAN release (or merging into htmlwidgets master?) and getting several more htmlwidget packages working well with it.

[timelyportfolio]

@jcheng5 + @ramnathv thanks so much for the updates. I'm just a little excited :) and anxious to test and experiment. I stopped the crosstalk earlier this year to make sure I wasn't headed in the wrong direction.

[jcheng5]

@ramnathv "The order in which observables get evaluated" sounds bad--I believe it shouldn't be a problem if you follow some simple (well, simple-ish) rules. This was one of the main topics of my 3 hour reactivity tutorial at ShinyDevCon in January. We should talk about this if this isn't something that mobservable considers a solved problem.

[ramnathv]

@jcheng5 the problem is not with how mobservable deals with things. it is more about syntactic sugar that i was trying to add that automatically creates observables underlying elements in the dom. so the order issue arises due to the dom, not because of the underlying library itself. I will create a set of simple examples to illustrate the issue.