Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for writing multiple files to zip/tar #203

Closed
JiaxiangBU opened this issue Mar 21, 2019 · 7 comments
Closed

Add support for writing multiple files to zip/tar #203

JiaxiangBU opened this issue Mar 21, 2019 · 7 comments
Assignees
@schochastics
Labels
enhancement

Comments

@JiaxiangBU
Copy link

Here are two reproducible examples.

  1. I find it cannot export a compressed file in the format zip.
  2. I find a list of data.frame exporting is supported in the format xlsx, instead of tar. In the future, any chance to add this feature. I am not sure, but in my daily work, it is often to compress a couple of files in a zip file.
library(rio)
suppressMessages(library(tidyverse))
safely(export)(mtcars, "mtcars.tsv.tar")
#> $result
#> [1] "mtcars.tsv.tar"
#> 
#> $error
#> NULL
safely(export)(mtcars, "mtcars.tsv.zip")
#> Warning in system2(zip, args, invisible = TRUE): '"zip"' not found
#> $result
#> NULL
#> 
#> $error
#> <simpleError in compress_out(cfile = cfile, filename = file, type = compress): File compression failed for mtcars.tsv.zip!>

Created on 2019-03-21 by the reprex package (v0.2.1)

Session info 
devtools::session_info()
#> - Session info ----------------------------------------------------------
#>  setting  value                                              
#>  version  R version 3.5.3 (2019-03-11)                       
#>  os       Windows 7 x64 SP 1                                 
#>  system   x86_64, mingw32                                    
#>  ui       RTerm                                              
#>  language (EN)                                               
#>  collate  Chinese (Simplified)_People's Republic of China.936
#>  ctype    Chinese (Simplified)_People's Republic of China.936
#>  tz       Asia/Taipei                                        
#>  date     2019-03-21                                         
#> 
#> - Packages --------------------------------------------------------------
#>  package     * version date       lib source        
#>  assertthat    0.2.0   2017-04-11 [1] CRAN (R 3.5.2)
#>  backports     1.1.3   2018-12-14 [1] CRAN (R 3.5.2)
#>  broom         0.5.1   2018-12-05 [1] CRAN (R 3.5.2)
#>  callr         3.1.1   2018-12-21 [1] CRAN (R 3.5.2)
#>  cellranger    1.1.0   2016-07-27 [1] CRAN (R 3.5.2)
#>  cli           1.0.1   2018-09-25 [1] CRAN (R 3.5.2)
#>  colorspace    1.3-2   2016-12-14 [1] CRAN (R 3.5.2)
#>  crayon        1.3.4   2017-09-16 [1] CRAN (R 3.5.2)
#>  curl          3.2     2018-03-28 [1] CRAN (R 3.5.2)
#>  data.table    1.11.8  2018-09-30 [1] CRAN (R 3.5.2)
#>  desc          1.2.0   2018-05-01 [1] CRAN (R 3.5.2)
#>  devtools      2.0.1   2018-10-26 [1] CRAN (R 3.5.2)
#>  digest        0.6.18  2018-10-10 [1] CRAN (R 3.5.2)
#>  dplyr       * 0.8.0.1 2019-02-15 [1] CRAN (R 3.5.2)
#>  evaluate      0.12    2018-10-09 [1] CRAN (R 3.5.2)
#>  forcats     * 0.3.0   2018-02-19 [1] CRAN (R 3.5.2)
#>  foreign       0.8-71  2018-07-20 [2] CRAN (R 3.5.3)
#>  fs            1.2.6   2018-08-23 [1] CRAN (R 3.5.2)
#>  generics      0.0.2   2018-11-29 [1] CRAN (R 3.5.2)
#>  ggplot2     * 3.1.0   2018-10-25 [1] CRAN (R 3.5.2)
#>  glue          1.3.0   2018-07-17 [1] CRAN (R 3.5.2)
#>  gtable        0.2.0   2016-02-26 [1] CRAN (R 3.5.2)
#>  haven         2.0.0   2018-11-22 [1] CRAN (R 3.5.2)
#>  highr         0.7     2018-06-09 [1] CRAN (R 3.5.2)
#>  hms           0.4.2   2018-03-10 [1] CRAN (R 3.5.2)
#>  htmltools     0.3.6   2017-04-28 [1] CRAN (R 3.5.2)
#>  httr          1.4.0   2018-12-11 [1] CRAN (R 3.5.2)
#>  jsonlite      1.6     2018-12-07 [1] CRAN (R 3.5.2)
#>  knitr         1.21    2018-12-10 [1] CRAN (R 3.5.2)
#>  lattice       0.20-38 2018-11-04 [2] CRAN (R 3.5.3)
#>  lazyeval      0.2.1   2017-10-29 [1] CRAN (R 3.5.2)
#>  lubridate     1.7.4   2018-04-11 [1] CRAN (R 3.5.2)
#>  magrittr      1.5     2014-11-22 [1] CRAN (R 3.5.2)
#>  memoise       1.1.0   2017-04-21 [1] CRAN (R 3.5.2)
#>  modelr        0.1.2   2018-05-11 [1] CRAN (R 3.5.2)
#>  munsell       0.5.0   2018-06-12 [1] CRAN (R 3.5.2)
#>  nlme          3.1-137 2018-04-07 [2] CRAN (R 3.5.3)
#>  openxlsx      4.1.0   2018-05-26 [1] CRAN (R 3.5.2)
#>  pillar        1.3.1   2018-12-15 [1] CRAN (R 3.5.2)
#>  pkgbuild      1.0.2   2018-10-16 [1] CRAN (R 3.5.2)
#>  pkgconfig     2.0.2   2018-08-16 [1] CRAN (R 3.5.2)
#>  pkgload       1.0.2   2018-10-29 [1] CRAN (R 3.5.2)
#>  plyr          1.8.4   2016-06-08 [1] CRAN (R 3.5.2)
#>  prettyunits   1.0.2   2015-07-13 [1] CRAN (R 3.5.2)
#>  processx      3.2.1   2018-12-05 [1] CRAN (R 3.5.2)
#>  ps            1.3.0   2018-12-21 [1] CRAN (R 3.5.2)
#>  purrr       * 0.2.5   2018-05-29 [1] CRAN (R 3.5.2)
#>  R6            2.3.0   2018-10-04 [1] CRAN (R 3.5.2)
#>  Rcpp          1.0.0   2018-11-07 [1] CRAN (R 3.5.2)
#>  readr       * 1.3.1   2018-12-21 [1] CRAN (R 3.5.2)
#>  readxl        1.2.0   2018-12-19 [1] CRAN (R 3.5.2)
#>  remotes       2.0.2   2018-10-30 [1] CRAN (R 3.5.2)
#>  rio         * 0.5.16  2018-11-26 [1] CRAN (R 3.5.3)
#>  rlang         0.3.1   2019-01-08 [1] CRAN (R 3.5.2)
#>  rmarkdown     1.11    2018-12-08 [1] CRAN (R 3.5.3)
#>  rprojroot     1.3-2   2018-01-03 [1] CRAN (R 3.5.2)
#>  rvest         0.3.2   2016-06-17 [1] CRAN (R 3.5.2)
#>  scales        1.0.0   2018-08-09 [1] CRAN (R 3.5.2)
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 3.5.2)
#>  stringi       1.2.4   2018-07-20 [1] CRAN (R 3.5.2)
#>  stringr     * 1.3.1   2018-05-10 [1] CRAN (R 3.5.2)
#>  testthat      2.0.1   2018-10-13 [1] CRAN (R 3.5.2)
#>  tibble      * 2.0.1   2019-01-12 [1] CRAN (R 3.5.2)
#>  tidyr       * 0.8.2   2018-10-28 [1] CRAN (R 3.5.2)
#>  tidyselect    0.2.5   2018-10-11 [1] CRAN (R 3.5.2)
#>  tidyverse   * 1.2.1   2017-11-14 [1] CRAN (R 3.5.2)
#>  usethis       1.4.0   2018-08-14 [1] CRAN (R 3.5.2)
#>  withr         2.1.2   2018-03-15 [1] CRAN (R 3.5.2)
#>  xfun          0.4     2018-10-23 [1] CRAN (R 3.5.2)
#>  xml2          1.2.0   2018-01-24 [1] CRAN (R 3.5.2)
#>  yaml          2.2.0   2018-07-25 [1] CRAN (R 3.5.2)
#>  zip           1.0.0   2017-04-25 [1] CRAN (R 3.5.2)
#> 
#> [1] C:/Users/lijiaxiang/Documents/R/win-library/3.5
#> [2] C:/Program Files/R/R-3.5.3/library
library(rio)
suppressMessages(library(tidyverse))
list(data_mtcars = mtcars,
     data_iris   = iris
     ) %>% 
    safely(export)(file = 'data_export.tar')
#> $result
#> NULL
#> 
#> $error
#> <simpleError in get_ext(file): 'file' has no extension>

Created on 2019-03-21 by the reprex package (v0.2.1)

Session info 
devtools::session_info()
#> - Session info ----------------------------------------------------------
#>  setting  value                                              
#>  version  R version 3.5.3 (2019-03-11)                       
#>  os       Windows 7 x64 SP 1                                 
#>  system   x86_64, mingw32                                    
#>  ui       RTerm                                              
#>  language (EN)                                               
#>  collate  Chinese (Simplified)_People's Republic of China.936
#>  ctype    Chinese (Simplified)_People's Republic of China.936
#>  tz       Asia/Taipei                                        
#>  date     2019-03-21                                         
#> 
#> - Packages --------------------------------------------------------------
#>  package     * version date       lib source        
#>  assertthat    0.2.0   2017-04-11 [1] CRAN (R 3.5.2)
#>  backports     1.1.3   2018-12-14 [1] CRAN (R 3.5.2)
#>  broom         0.5.1   2018-12-05 [1] CRAN (R 3.5.2)
#>  callr         3.1.1   2018-12-21 [1] CRAN (R 3.5.2)
#>  cellranger    1.1.0   2016-07-27 [1] CRAN (R 3.5.2)
#>  cli           1.0.1   2018-09-25 [1] CRAN (R 3.5.2)
#>  colorspace    1.3-2   2016-12-14 [1] CRAN (R 3.5.2)
#>  crayon        1.3.4   2017-09-16 [1] CRAN (R 3.5.2)
#>  curl          3.2     2018-03-28 [1] CRAN (R 3.5.2)
#>  data.table    1.11.8  2018-09-30 [1] CRAN (R 3.5.2)
#>  desc          1.2.0   2018-05-01 [1] CRAN (R 3.5.2)
#>  devtools      2.0.1   2018-10-26 [1] CRAN (R 3.5.2)
#>  digest        0.6.18  2018-10-10 [1] CRAN (R 3.5.2)
#>  dplyr       * 0.8.0.1 2019-02-15 [1] CRAN (R 3.5.2)
#>  evaluate      0.12    2018-10-09 [1] CRAN (R 3.5.2)
#>  forcats     * 0.3.0   2018-02-19 [1] CRAN (R 3.5.2)
#>  foreign       0.8-71  2018-07-20 [2] CRAN (R 3.5.3)
#>  fs            1.2.6   2018-08-23 [1] CRAN (R 3.5.2)
#>  generics      0.0.2   2018-11-29 [1] CRAN (R 3.5.2)
#>  ggplot2     * 3.1.0   2018-10-25 [1] CRAN (R 3.5.2)
#>  glue          1.3.0   2018-07-17 [1] CRAN (R 3.5.2)
#>  gtable        0.2.0   2016-02-26 [1] CRAN (R 3.5.2)
#>  haven         2.0.0   2018-11-22 [1] CRAN (R 3.5.2)
#>  highr         0.7     2018-06-09 [1] CRAN (R 3.5.2)
#>  hms           0.4.2   2018-03-10 [1] CRAN (R 3.5.2)
#>  htmltools     0.3.6   2017-04-28 [1] CRAN (R 3.5.2)
#>  httr          1.4.0   2018-12-11 [1] CRAN (R 3.5.2)
#>  jsonlite      1.6     2018-12-07 [1] CRAN (R 3.5.2)
#>  knitr         1.21    2018-12-10 [1] CRAN (R 3.5.2)
#>  lattice       0.20-38 2018-11-04 [2] CRAN (R 3.5.3)
#>  lazyeval      0.2.1   2017-10-29 [1] CRAN (R 3.5.2)
#>  lubridate     1.7.4   2018-04-11 [1] CRAN (R 3.5.2)
#>  magrittr      1.5     2014-11-22 [1] CRAN (R 3.5.2)
#>  memoise       1.1.0   2017-04-21 [1] CRAN (R 3.5.2)
#>  modelr        0.1.2   2018-05-11 [1] CRAN (R 3.5.2)
#>  munsell       0.5.0   2018-06-12 [1] CRAN (R 3.5.2)
#>  nlme          3.1-137 2018-04-07 [2] CRAN (R 3.5.3)
#>  openxlsx      4.1.0   2018-05-26 [1] CRAN (R 3.5.2)
#>  pillar        1.3.1   2018-12-15 [1] CRAN (R 3.5.2)
#>  pkgbuild      1.0.2   2018-10-16 [1] CRAN (R 3.5.2)
#>  pkgconfig     2.0.2   2018-08-16 [1] CRAN (R 3.5.2)
#>  pkgload       1.0.2   2018-10-29 [1] CRAN (R 3.5.2)
#>  plyr          1.8.4   2016-06-08 [1] CRAN (R 3.5.2)
#>  prettyunits   1.0.2   2015-07-13 [1] CRAN (R 3.5.2)
#>  processx      3.2.1   2018-12-05 [1] CRAN (R 3.5.2)
#>  ps            1.3.0   2018-12-21 [1] CRAN (R 3.5.2)
#>  purrr       * 0.2.5   2018-05-29 [1] CRAN (R 3.5.2)
#>  R6            2.3.0   2018-10-04 [1] CRAN (R 3.5.2)
#>  Rcpp          1.0.0   2018-11-07 [1] CRAN (R 3.5.2)
#>  readr       * 1.3.1   2018-12-21 [1] CRAN (R 3.5.2)
#>  readxl        1.2.0   2018-12-19 [1] CRAN (R 3.5.2)
#>  remotes       2.0.2   2018-10-30 [1] CRAN (R 3.5.2)
#>  rio         * 0.5.16  2018-11-26 [1] CRAN (R 3.5.3)
#>  rlang         0.3.1   2019-01-08 [1] CRAN (R 3.5.2)
#>  rmarkdown     1.11    2018-12-08 [1] CRAN (R 3.5.3)
#>  rprojroot     1.3-2   2018-01-03 [1] CRAN (R 3.5.2)
#>  rvest         0.3.2   2016-06-17 [1] CRAN (R 3.5.2)
#>  scales        1.0.0   2018-08-09 [1] CRAN (R 3.5.2)
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 3.5.2)
#>  stringi       1.2.4   2018-07-20 [1] CRAN (R 3.5.2)
#>  stringr     * 1.3.1   2018-05-10 [1] CRAN (R 3.5.2)
#>  testthat      2.0.1   2018-10-13 [1] CRAN (R 3.5.2)
#>  tibble      * 2.0.1   2019-01-12 [1] CRAN (R 3.5.2)
#>  tidyr       * 0.8.2   2018-10-28 [1] CRAN (R 3.5.2)
#>  tidyselect    0.2.5   2018-10-11 [1] CRAN (R 3.5.2)
#>  tidyverse   * 1.2.1   2017-11-14 [1] CRAN (R 3.5.2)
#>  usethis       1.4.0   2018-08-14 [1] CRAN (R 3.5.2)
#>  withr         2.1.2   2018-03-15 [1] CRAN (R 3.5.2)
#>  xfun          0.4     2018-10-23 [1] CRAN (R 3.5.2)
#>  xml2          1.2.0   2018-01-24 [1] CRAN (R 3.5.2)
#>  yaml          2.2.0   2018-07-25 [1] CRAN (R 3.5.2)
#>  zip           1.0.0   2017-04-25 [1] CRAN (R 3.5.2)
#> 
#> [1] C:/Users/lijiaxiang/Documents/R/win-library/3.5
#> [2] C:/Program Files/R/R-3.5.3/library
@JiaxiangBU JiaxiangBU changed the title Bug Report/ Feature Request: Bug Report/ Feature Request: zip problem. Mar 21, 2019
@leeper
Copy link
Contributor

leeper commented Dec 24, 2019

The problem with exporting to .zip is not reproducible for me. I would guess you do not have a zip tool installed on your system. See ? utils::zip for more information on that.

The second issue about writing multiple files to a zip or tar is a good idea but it's not currently supported. I'll add it to the wishlist.

@leeper leeper changed the title Bug Report/ Feature Request: zip problem. Add support for writing multiple files to zip/tar Dec 24, 2019
@JiaxiangBU
Copy link
Author

Thanks for your reply.

@leeper
Copy link
Contributor

leeper commented Dec 24, 2019

I'm going to reopen as a reminder to work on writing multiple files to zip/tar. Thanks again.

@leeper leeper reopened this Dec 24, 2019
@schochastics
Copy link
Member

Brainstorming ideas on how to implement this:

export_list(list(
    mtcars1 = mtcars[1:10, ],
    mtcars2 = mtcars[11:20, ],
    mtcars3 = mtcars[21:32, ]
), file = c("file1.csv", "file2.csv", "file3.csv"), compress = "data.zip")

Adds a new parameter. My first that ws to make this TRUE/FALSE, but we need to get the zip file name somehow (or we default to something like data.zip, which i do not think is a good idea.)

Could, if needed, be extended to also write to directories:
i.e.:

  • compress="" -> do nothing else than create the 3 files
  • compress="data" -> create, if needed, the folder data and move files there
  • compress="data.zip" -> write all files to zip

AFAIS this does not require too much changes in the code.

Doesn't add a new parameter. Instead, the zip file name is appended to the file list.
I think this is confusing though.

export_list(list(
    mtcars1 = mtcars[1:10, ],
    mtcars2 = mtcars[11:20, ],
    mtcars3 = mtcars[21:32, ]
), file = c("file1.csv", "file2.csv", "file3.csv","data.zip"))

I think this would also not require too many changes

@schochastics schochastics self-assigned this Sep 6, 2023
@chainsawriot
Copy link
Collaborator

@schochastics I like the first idea better (adding a new parameter). Referencing the relevant functions (e.g. utils::tar, utils::zip) and using the tidyverse-style naming, I think the parameter should be called archive_file.

@schochastics
Copy link
Member

@chainsawriot ok sounds good. Should archive_file support writing to a directory or compressed files only?

@chainsawriot
Copy link
Collaborator

I think directory is also a good idea. If it is the case, the argument is not always a file. Maybe just archive

archive = NULL # just dump all files to the current directory
archive = "~/directory" #dump to this directory
archive = "~/all_my_illegal_bibcoin_transactions.zip"`
archive = "./I_think_windows_users_dont_use.tar"`

schochastics added a commit to schochastics/rio that referenced this issue Sep 6, 2023
@schochastics
added archiving to export_list (gesistsa#203)
c9d01a5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@schochastics schochastics

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@chainsawriot @leeper @JiaxiangBU @schochastics