[research] Decorate module output

[gogonzo]

We need to find an optimal way to allow app developer to "decorate" output of the module (including interactivity). In tmc modules are developed as follows.

• there is a template_xy function which generates a list of calls needed to produce an output
• template_xy is evaluated in qenv and plots use to be assigned to p binding (it is defined in the template)
• then p variable is extracted from qenv and passed to plot_with_settings

template_xy <- function(... <many arguments like dataname xvar yvar>) {
}

srv_xy <- function(...) {
  ...
  all_q <- reactive({
      my_calls <- template_xy(...)
      teal.code::eval_code(anl_q(), as.expression(my_calls))
  })

  plot_r <- reactive(all_q()[["p"]])
  plot_with_settings(plot_r)
}

We need to find a nice way to include any arbitrary code to change the p. No holds barred!
Issue should be addressed by POC

Definition of Done

• Research on how to decorate module output
• Followed by PoC(s) for coreDev and SME review -> [Discussion]: Discuss with SME team the concept of module decoration #1385
• Once decided, refine and create actionable issue for implementation

[gogonzo]

Discussion 1 summary

@kpagacz and I discussed the issue and decided to first focus on allowing app developers to provide arbitrary code to modify the module’s output. Interactivity is not yet included. We agreed almost immediately that decorators need to be provided through the tm_xyz argument and passed down by server_args to the srv_xyz. Discussion about the other ways seems irrelevant, but we will come back to this later. We considered three scenarios, though the list is not exhaustive.

1. Decorator passed as expression list

tm_xy <- function(
  ...
  decorator = expression(
    ggplot2::flip_axis(...) +
    ggplot2::ggtitle(...) +
    ggplot2::theme(...)
  )
)

Above call could be passed to the server and in the reactive which produces the plot call it would look something like this:

plot_call <- ...
if (length(decorator)) {
  plot_call <- substitute(p <- p + <decorator>) 
}

• ✅ simple api to evaluate inside of the module. Can be automatically generated from ggplot_args
• ✅ doesn't require knowing anything about p object created inside of the module. p doesn't have to be used as it will be automatically added to the final call
• ❌ with above it won't be possible to do more than just appending ggplot elements.

2. Decorator as a function

tm_xy <- function(
  ...,
  decorator = expression(
    function(plot, data) {
      plot + 
        ggplot2::flip_axis(...) +
        ggplot2::ggtitle(...) + 
        ggplot2::theme(...) +
        ggplot2::geom_points(data, aes(x = x, y = y))
     }
  )
)

Inside of the module above would be consumed in the following way. Because function is passed as expression it will be added to SRC for reproducibility.

substitute(
  p <- decorate(iris, p),
  list(
    decorate = eval(decorator)
  )
)

• ✅ doesn't need to know data object name nor p
• ❌ not elegant SRC output

3. teal_transform_module

If we consider that some decorations will depend on the app user input (limiting axes range, changing title etc.) then teal_transform_module could be a solution.

• ✅ standardised way of modifying objects for teal
• ❌ require to know the bindings names inside of the qenv like plot or ADSL

[gogonzo]

Discussion 2 summary

After short meeting, we agreed that in order to execute any custom call inside of the teal_module one of the following is needed:

1. App-developer can provide a function(plot) without knowing how plot is named inside the srv_xy. Inside of the srv_xy body, module-developer would call decorator(plot = p). Function can have more arguments, for example function(plot, data) for example to add ggplot elements dependent on some data.
2. App-developer can provide a call which can be executed inside of the module, but then app-developer should know the names of the objects as plot and data doesn't have to be the exact names inside of the teal_module. Then the module documentation should explicitly say that plot object is named p and data object is named ADSL. In most of the cases dataname is not fixed, so app-developer would have to know that if module receives iris then iris needs to be included in a decorate.

[gogonzo]

Conclusions from the last (general) meeting

1. ui and server is needed for decoration in some cases
2. J&J need only to provide a working decorate-call
3. If we implement (2), it shouldn't put the burden on app developer to know the names of internal objects. Objects used by app developer should be a standard convention for decorate api
4. If (3) will have a standard naming convention, then teal_module(s) could also use this standard convention to name objects - technically renaming names in the teal_module could have the same effect
5. Decorators based on data will be a rabbit hole and would require app developer to understand teal_module reactivity (to know what is expected shape of the final dataset which plot/table is based on)
6. We might need an access to the inputs also

Initial apps showing apps the problem

Initial app for plot decoration (simple case, one output)

library(teal)
identity_decorator <- list(
  ui = function(id) NULL,
  server = function(id, data) {
    data
  }
)

tm_decorated_plot <- function(label = "module", decorator = identity_decorator) {
  module(
    label = label,
    ui = function(id, decorator) {
      ns <- NS(id)
      div(
        selectInput(ns("dataname"), label = "select dataname", choices = NULL),
        selectInput(ns("x"), label = "select x", choices = NULL),
        selectInput(ns("y"), label = "select y", choices = NULL),
        decorator$ui(ns("decorate")),
        plotOutput(ns("plot")),
        verbatimTextOutput(ns("text"))
      )
    },
    server = function(id, data, decorator) {
      moduleServer(id, function(input, output, session) {
        observeEvent(data(), {
          updateSelectInput(inputId = "dataname", choices = teal.data::datanames(data()))
        })

        observeEvent(input$dataname, {
          updateSelectInput(inputId = "x", choices = colnames(data()[[input$dataname]]))
          updateSelectInput(inputId = "y", label = "select y", choices = colnames(data()[[input$dataname]]))
        })

        q1 <- reactive({
          req(input$dataname, input$x, input$y)
          data() |>
            within(
              {
                plot <- ggplot2::ggplot(dataname, ggplot2::aes(x = x, y = y)) +
                  ggplot2::geom_point()
              },
              dataname = as.name(input$dataname),
              x = as.name(input$x),
              y = as.name(input$y)
            )
        })

        q2 <- decorator$server("decorate", data = q1)

        plot_r <- reactive({
          req(q2())
          q2()[["plot"]]
        })

        output$plot <- renderPlot(plot_r())
        output$text <- renderText({
          teal.code::get_code(q2())
        })
      })
    },
    ui_args = list(decorator = decorator),
    server_args = list(decorator = decorator)
  )
}

app <- init(
  data = teal_data(iris = iris, mtcars = mtcars),
  modules = modules(
    tm_decorated_plot("1")
  )
)

runApp(app)

[shajoezhu]

Hi @gogonzo , I was wondering for teal_transform_module, is that a new package? or per module that could be a transform module?