action-transfer.Rmd

# Uploads and downloads {#action-transfer}

```{r, include = FALSE}
source("common.R")
```

Transferring files to and from the user is a common feature of apps. It's most commonly used to upload data for analysis, or download the results as a dataset or as a report. This chapter shows the UI and server components you'll need to transfer files in and out of your app. 

```{r setup}
library(shiny)
```

## Upload

We'll start by discussing file uploads, showing you the basic UI and server components, and then showing how they fit together in a simple app.

### UI

The UI needed to support file uploads is simple: just add `fileInput()` to your UI:

```{r}
ui <- fluidPage(
  fileInput("file", "Upload a file")
)
```

Like most other UI components, there are only two required arguments: `id` and `label`. The `width`, `buttonLabel` and `placeholder` arguments allow you to tweak the appearance in other ways. I won't discuss them further here, but you can read more about them in `?fileInput`.

### Server

Handling `fileInput()` on the server is a little more complicated than other inputs. Most inputs use simple vectors, but `input$file` returns a data frame with four columns:

* `name`: the original file name on the user's computer.

* `size`: the file size, in bytes. By default, the user can only upload files 
  up to 5 MB. You can increase this limit by setting the `shiny.maxRequestSize`
  option prior to starting Shiny. For example, to allow up to 10 MB run
  `options(shiny.maxRequestSize = 10 * 1024^2)`.

* `type`: the "MIME type"[^mime-type] of the file. This is a formal 
  specification of the file type that is usually derived from the extension. 
  It is rarely needed in Shiny apps.

* `datapath`: the path to where the data has been uploaded on the server.
  Treat this path as ephemeral: if the user uploads more files, this file 
  may be deleted. The data is always saved to a temporary directory and given 
  a temporary name.

[^mime-type]: MIME type is short for "**m**ulti-purpose **i**nternet **m**ail **e**xtensions type". As you might guess from the name, it was original designed for email systems, but now it's used widely across many internet tools. A MIME type looks like `type/subtype`. Some common examples are `text/csv`, `text/html`, `image/png`, `application/pdf`, `application/vnd.ms-excel` (excel file).

I think the easiest way to get to understand this data structure is to make a simple app. Run the following code and upload a few files to get a sense of what data Shiny is providing.

```{r}
ui <- fluidPage(
  fileInput("upload", NULL, buttonLabel = "Upload...", multiple = TRUE),
  tableOutput("files")
)
server <- function(input, output, session) {
  output$files <- renderTable(input$upload)
}
```

Note my use of the `label` and `buttonLabel` arguments to mildly customise the appearance, and use of `multiple = TRUE` to allow the user to upload multiple files. 

### Uploading data

If the user is uploading a dataset, there are three details that you need to be aware of:

* `input$file` is initialised to  `NULL` on page load, so you'll need
  `req(input$file)` to make sure your code waits until the first file is 
  uploaded.
  
* The `accept` argument allows you to limit the possible inputs. The easiest 
  way is to supply a character vector of file extensions, like 
  `accept = ".csv"`. But the `accept` argument is only a suggestion to 
  the browser, and is not always enforced, so it's good practice to also
  validate it (e.g. Section \@ref(validate)) the yourself. The easiest way 
  to get the file extension in R is `tools::file_ext()`, but note that it 
  strips the leading `.`.

Putting all these ideas together gives us the following app where you can upload a `.csv` file and see the first `n` rows:

```{r}
ui <- fluidPage(
  fileInput("file", NULL, accept = c(".csv", ".tsv")),
  numericInput("n", "Rows", value = 5, min = 1, step = 1),
  tableOutput("head")
)

server <- function(input, output, session) {
  data <- reactive({
    req(input$file)
    
    ext <- tools::file_ext(input$file$name)
    switch(ext,
      csv = vroom::vroom(input$file$datapath, delim = ","),
      tsv = vroom::vroom(input$file$datapath, delim = "\t"),
      validate("Invalid file; Please upload a .csv or .tsv file")
    )
  })
  
  output$head <- renderTable({
    head(data(), input$n)
  })
}
```

Note that since `multiple = FALSE` (the default), `input$file` will be a single row data frame, and `input$file$name` and `input$file$datapath` will be a length-1 character vectors. 

## Download

Next, we'll look at file downloads, showing you the basic UI and server components, then seeing how you might use them to allow the user to download data or reports.

### Basics

Again, the UI is straightforward: use either `downloadButton(id)` or `downloadLink(id)` to give the user something to click to download a file: 

```{r}
ui <- fluidPage(
  downloadButton("download1"),
  downloadLink("download2")
)
```

You can customise the appearance using the `class` argument by using one of `"btn-primary"`, `"btn-success"`, `"btn-info"`, `"btn-warning"`, or `"btn-danger"`. You can also change the size with `"btn-lg"`, `"btn-sm"`, `"btn-xs"`. Finally, you can make buttons span the entire width of the element they are embedded within using `"btn-block"`. See the detail of the underlying CSS classes at <http://bootstrapdocs.com/v3.3.6/docs/css/#buttons>. You can also add a custom icon with the `icon` argument.

Unlike other outputs, `downloadButton()` is not paired with a render function. Instead, you use `downloadHandler()`, which looks something like this:

```{r, eval = FALSE}
output$download <- downloadHandler(
  filename = function() {
    paste0(input$dataset, ".csv")
  },
  content = function(file) {
    write.csv(data(), file)
  }
)
```

`downloadHandler()` has two arguments, both functions:

*   `filename` should be a function with no arguments that returns a file
    name (as a string). The job of this function is to create the name that will
    shown to the user in the download dialog box. 

*   `content` should be a function with one argument, `file`, which is the path
    to save the file. The job of this function is to save the file in a place 
    that Shiny knows about, so it can then send it to the user.

Next we'll put these pieces together to show how to transfer data files or reports to the user.

### Downloading data

The following app shows off the basics of data download by allowing you to download any dataset in the datasets package as a tab separated file[^tsv-csv] file. I recommend using `.tsv` (tab separated value) instead of `.csv` (comma separated values) because many European countries use commas to separate the whole and fractional parts of a number (e.g. `1,23` vs `1.23`). This means they can't use commas to separate fields and instead use semi-colons. You can avoid this complexity by using tab separated files.

```{r}
ui <- fluidPage(
  selectInput("dataset", "Pick a dataset", ls("package:datasets")),
  tableOutput("preview"),
  downloadButton("download", "Download .tsv")
)

server <- function(input, output, session) {
  data <- reactive({
    out <- get(input$dataset, "package:datasets")
    if (!is.data.frame(out)) {
      validate(paste0("'", input$dataset, "' is not a data frame"))
    }
    out
  })
  
  output$preview <- renderTable({
    head(data())
  })
    
  output$download <- downloadHandler(
    filename = function() {
      paste0(input$dataset, ".tsv")
    },
    content = function(file) {
      vroom::vroom_write(data(), file)
    }
  )
}
```

Note the use of `validate()` to only allow the user to download datasets that are data frames. A better approach would be to pre-filter the list, but this lets you see another application of `validate()`.

### Downloading reports

As well downloading data, you may want the users of your app to download a report, that summarises the result of interactive exploration in the Shiny app. This is quite a lot of extra work, because you also need to display the same information in a different format, but it is very useful for high-stakes apps.

One powerful way to generate such a report is with a parameterised RMarkdown document, <https://bookdown.org/yihui/rmarkdown/parameterized-reports.html>. A parameterised RMarkdown file has a `params` field in the YAML metadata:

```yaml
title: My Document
output: html_document
params:
  year: 2018
  region: Europe
  printcode: TRUE
  data: file.csv
```

And inside the document, you can refer to these values using `params$year`, `params$region` etc. 

The values in the YAML metadata are basically defaults; you'll generally override them by providing the `params` argument in a call to `rmarkdown::render()`. This makes it easy to generate many different reports from the same `.Rmd`. 

Here's a simple example adapted from <https://shiny.rstudio.com/articles/generating-reports.html>, which describes this technique in more detail. The key idea is to call `rmarkdown::render()` from the `content` argument of `downloadHander()`. If you want to produce other output formats, just change the output format in the `.Rmd`, and make sure to update the extension.

```{r}
ui <- fluidPage(
  sliderInput("n", "Number of points", 1, 100, 50),
  downloadButton("report", "Generate report")
)

server <- function(input, output, session) {
  output$report <- downloadHandler(
    filename = "report.html",
    content = function(file) {
      params <- list(n = input$n)

      rmarkdown::render("report.Rmd", 
        output_file = file,
        params = params,
        envir = new.env(parent = globalenv())
      )
    }
  )
}
```

There are a few other tricks worth knowing about:

*   If the report takes some time to generate, use one of the techniques from
    Chapter \@ref(action-feedback) to let the user know that your app is 
    working.

*   In many deployment scenarios, you won't be able to write to the working 
    directory, which RMarkdown will attempt to do. You can work around this by 
    copying the report to a temporary directory when your app starts (i.e. 
    outside of `server()`):

    ```{r}
    report_path <- tempfile(fileext = ".Rmd")
    file.copy("report.Rmd", report_path, overwrite = TRUE)
    ```

    Then replace `"report.Rmd"` with `report_path` in the call to 
    `rmarkdown::render()`.

*   By default, RMarkdown will render the report in the current process, which
    means that it will inherit many settings from the Shiny app (like loaded 
    packages, options, etc). For greater robustness, I recommend running 
    `render()` in a separate R session using the callr package:
    
    ```{r, eval = FALSE}
    render_report <- function(input, output, params) {
      rmarkdown::render(input,
        output_file = output,
        params = params,
        envir = new.env(parent = globalenv())
      )
    }
    
    server <- function(input, output) {
      output$report <- downloadHandler(
        filename = "report.html",
        content = function(file) {
          params <- list(n = input$slider)
          callr::r(
            render_report,
            list(input = report_path, output = file, params = params)
          )
        }
      )
    }
    ```

You can see all these pieces put together in `rmarkdown-report/`.

## Case study

To finish up, we'll work through a small case study where we upload a file (with user supplied separator), preview it, perform some optional transformations using the [janitor package](http://sfirke.github.io/janitor), by Sam Firke, and then let the user download it as a `.tsv`. 

To make it easier to understand how to use the app, I've used `sidebarLayout()` to divide the app into three main steps:

1.  Uploading and parsing the file:

    ```{r}
    ui_upload <- sidebarLayout(
      sidebarPanel(
        fileInput("file", "Data", buttonLabel = "Upload..."),
        textInput("delim", "Delimiter (leave blank to guess)", ""),
        numericInput("skip", "Rows to skip", 0, min = 0),
        numericInput("rows", "Rows to preview", 10, min = 1)
      ),
      mainPanel(
        h3("Raw data"),
        tableOutput("preview1")
      )
    )
    ```

2.  Cleaning the file.

    ```{r}
    ui_clean <- sidebarLayout(
      sidebarPanel(
        checkboxInput("snake", "Rename columns to snake case?"),
        checkboxInput("constant", "Remove constant columns?"),
        checkboxInput("empty", "Remove empty cols?")
      ),
      mainPanel(
        h3("Cleaner data"),
        tableOutput("preview2")
      )
    )
    ```

3.  Downloading the file.

    ```{r}
    ui_download <- fluidRow(
      column(width = 12, downloadButton("download", class = "btn-block"))
    )
    ```

Which get assembled into a single `fluidPage()`:

```{r}
ui <- fluidPage(
  ui_upload,
  ui_clean,
  ui_download
)
```

This same organisation makes it easier to understand the app:

```{r}
server <- function(input, output, session) {
  # Upload ---------------------------------------------------------------
  raw <- reactive({
    req(input$file)
    delim <- if (input$delim == "") NULL else input$delim
    vroom::vroom(input$file$datapath, delim = delim, skip = input$skip)
  })
  output$preview1 <- renderTable(head(raw(), input$rows))
  
  # Clean ----------------------------------------------------------------
  tidied <- reactive({
    out <- raw()
    if (input$snake) {
      names(out) <- janitor::make_clean_names(names(out))
    }
    if (input$empty) {
      out <- janitor::remove_empty(out, "cols")
    }
    if (input$constant) {
      out <- janitor::remove_constant(out)
    }
    
    out
  })
  output$preview2 <- renderTable(head(tidied(), input$rows))
  
  # Download -------------------------------------------------------------
  output$download <- downloadHandler(
    filename = function() {
      paste0(tools::file_path_sans_ext(input$file$name), ".tsv")
    },
    content = function(file) {
      vroom::vroom_write(tidied(), file)
    }
  )
}
```

### Exercises