column filters for large tables

[vnijs]

It seems that using column filters can slow down (server side) table rendering when the data has more than a few 100K rows. Are factor levels and min/max already determined in R? If not, is it possible to push these characteristics to DT when using server-side processing?

Is this related to #50 ?

[yihui]

Could you provide a minimal reproducible example?

[vnijs]

Minimal example below. DT seems to handle bigger datasets pretty well but notice the difference in render-time if dat$vs is setup as a factor or character. Although you probably don't want a variable with that many levels in your data, R's stringtofactor conversion might lead to datasets with factors like this (e.g., customer ID).

library(dplyr)
library(shiny)
library(DT)

nrobs <- 40000
dat <- sample_n(mtcars, nrobs, replace = TRUE)
dat$vs <- as.factor(paste0("factor", rep(1:nrobs)))

## Notice the difference if you (do not) comment out the next line
## If you set nrobs to 400,000 or more it will take a loooong time before
## the table is rendered
# dat$vs <- as.character(dat$vs)

shinyApp(
  ui = fluidPage(
    fluidRow(DT::dataTableOutput("tbl")),
    tags$button(id = "stop", type = "button",
                class = "btn btn-danger action-button shiny-bound-input",
                onclick = "window.close();", "Stop")
  ),
  server = function(input, output, session) {
    widget <- DT::datatable(dat, selection = "none",
      rownames = FALSE, style = "bootstrap",
      filter = list(position = "top")
    )
    output$tbl <- DT::renderDataTable(widget)
    observeEvent(input$stop, {stopApp("Stopped viewdata")})
  }
)

[yihui]

Yeah you probably do not want factors with such a large number of levels in practice, but I have fixed the issue anyway. Thanks!

[vnijs]

Works great! Thanks @yihui