pluck is slowed down by list2

[mgirlich]

Checking the performance of pluck() with profvis() one can see that most of the time is spend in list2(). In the benchmarks below one can see that pluck() resp. map() used as an extractor could be ~ 2.5 times faster:

devtools::load_all("~/GitHub/purrr/")
#> Loading purrr

pluck2 <- function(.x, index, .default = NULL) {
  .Call(
    pluck_impl,
    x = .x,
    index = index,
    missing = .default,
    strict = FALSE
  )
}

index <- list("basic_information", "master_id")
bench_it <- function(n) {
  x <- rep(repurrrsive::discog, n)
  
  bench::mark(
    map = map(x, index),
    pluck = map(x, pluck, "basic_information", "master_id"),
    pluck2 = map(x, pluck2, index = index)
  )
}

bench_it(1)
#> # A tibble: 3 x 6
#>   expression      min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 map           561µs    603µs     1522.   21.96KB     27.7
#> 2 pluck         509µs    547µs     1645.   24.51KB     27.3
#> 3 pluck2        225µs    238µs     3784.    3.78MB     21.7
bench_it(10)
#> # A tibble: 3 x 6
#>   expression      min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 map          5.63ms   5.95ms      151.    12.2KB     37.1
#> 2 pluck        5.06ms   5.28ms      179.    12.2KB     38.6
#> 3 pluck2       2.09ms   2.26ms      398.    12.2KB     25.9
bench_it(100)
#> Warning: Some expressions had a GC in every iteration; so filtering is disabled.
#> # A tibble: 3 x 6
#>   expression      min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 map          60.2ms   65.1ms      14.6     121KB     18.3
#> 2 pluck        54.8ms   56.3ms      16.7     121KB     16.7
#> 3 pluck2       21.3ms   22.6ms      40.9     121KB     15.6
bench_it(1000)
#> Warning: Some expressions had a GC in every iteration; so filtering is disabled.
#> # A tibble: 3 x 6
#>   expression      min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 map           691ms    691ms      1.45    1.18MB     17.4
#> 2 pluck         600ms    600ms      1.67    1.18MB     16.7
#> 3 pluck2        240ms    244ms      4.11    1.18MB     15.1

^{Created on 2020-03-03 by the reprex package (v0.3.0)}

Session info

devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 3.6.1 (2019-07-05)
#>  os       macOS Catalina 10.15.3      
#>  system   x86_64, darwin15.6.0        
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_US.UTF-8                 
#>  ctype    en_US.UTF-8                 
#>  tz       UTC                         
#>  date     2020-03-03                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  !  package     * version    date       lib source                       
#>     assertthat    0.2.1      2019-03-21 [1] CRAN (R 3.6.0)               
#>     backports     1.1.5      2019-10-02 [1] CRAN (R 3.6.0)               
#>     bench         1.1.1      2020-01-13 [1] CRAN (R 3.6.1)               
#>     callr         3.4.2      2020-02-12 [1] CRAN (R 3.6.1)               
#>     cli           2.0.2      2020-02-28 [1] CRAN (R 3.6.0)               
#>     crayon        1.3.4      2017-09-16 [1] CRAN (R 3.6.0)               
#>     desc          1.2.0      2018-05-01 [1] CRAN (R 3.6.0)               
#>     devtools      2.2.2      2020-02-17 [1] CRAN (R 3.6.1)               
#>     digest        0.6.25     2020-02-23 [1] CRAN (R 3.6.0)               
#>     ellipsis      0.3.0      2019-09-20 [1] CRAN (R 3.6.0)               
#>     evaluate      0.14       2019-05-28 [1] CRAN (R 3.6.0)               
#>     fansi         0.4.1      2020-01-08 [1] CRAN (R 3.6.1)               
#>     fs            1.3.1      2019-05-06 [1] CRAN (R 3.6.0)               
#>     glue          1.3.1      2019-03-12 [1] CRAN (R 3.6.0)               
#>     highr         0.8        2019-03-20 [1] CRAN (R 3.6.0)               
#>     htmltools     0.4.0      2019-10-04 [1] CRAN (R 3.6.0)               
#>     knitr         1.28       2020-02-06 [1] CRAN (R 3.6.1)               
#>     magrittr      1.5        2014-11-22 [1] CRAN (R 3.6.0)               
#>     memoise       1.1.0      2017-04-21 [1] CRAN (R 3.6.0)               
#>     pillar        1.4.3.9000 2020-01-07 [1] Github (r-lib/pillar@8f5918c)
#>     pkgbuild      1.0.6      2019-10-09 [1] CRAN (R 3.6.0)               
#>     pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 3.6.0)               
#>     pkgload       1.0.2      2018-10-29 [1] CRAN (R 3.6.0)               
#>     prettyunits   1.1.1      2020-01-24 [1] CRAN (R 3.6.1)               
#>     processx      3.4.2      2020-02-09 [1] CRAN (R 3.6.1)               
#>     profmem       0.5.0      2018-01-30 [1] CRAN (R 3.6.0)               
#>     ps            1.3.2      2020-02-13 [1] CRAN (R 3.6.1)               
#>  VP purrr       * 0.3.3.9000 2019-10-18 [?] CRAN (R 3.6.0)               
#>     R6            2.4.1      2019-11-12 [1] CRAN (R 3.6.0)               
#>     Rcpp          1.0.3      2019-11-08 [1] CRAN (R 3.6.0)               
#>     remotes       2.1.1      2020-02-15 [1] CRAN (R 3.6.0)               
#>     repurrrsive   1.0.0      2019-07-15 [1] CRAN (R 3.6.0)               
#>     rlang         0.4.5      2020-03-01 [1] CRAN (R 3.6.1)               
#>     rmarkdown     2.1        2020-01-20 [1] CRAN (R 3.6.0)               
#>     rprojroot     1.3-2      2018-01-03 [1] CRAN (R 3.6.0)               
#>     rstudioapi    0.11       2020-02-07 [1] CRAN (R 3.6.0)               
#>     sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 3.6.0)               
#>     stringi       1.4.6      2020-02-17 [1] CRAN (R 3.6.1)               
#>     stringr       1.4.0      2019-02-10 [1] CRAN (R 3.6.0)               
#>     testthat    * 2.3.2      2020-03-02 [1] CRAN (R 3.6.1)               
#>     tibble        2.1.3      2019-06-06 [1] CRAN (R 3.6.0)               
#>     usethis       1.5.1      2019-07-04 [1] CRAN (R 3.6.0)               
#>     utf8          1.1.4      2018-05-24 [1] CRAN (R 3.6.0)               
#>     vctrs         0.2.3      2020-02-20 [1] CRAN (R 3.6.0)               
#>     withr         2.1.2      2018-03-15 [1] CRAN (R 3.6.0)               
#>     xfun          0.12       2020-01-13 [1] CRAN (R 3.6.1)               
#>     yaml          2.2.1      2020-02-01 [1] CRAN (R 3.6.0)               
#> 
#> [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library
#> 
#>  V ── Loaded and on-disk version mismatch.
#>  P ── Loaded and on-disk path mismatch.

[lionel-]

I don't think there's anything to do about this since dynamic dots are part of the interface of that function.

[mgirlich]

Would it be possible to call pluck_impl() directly from map() when used to subset by an index? Or would it be an option to export an extractor function that takes a list instead of dynamic dots?

As much as I appreciate the idea of dynamic dots I think the overhead is quite big in this case.

[lionel-]

The overhead is really quite small. Your n = 1 case is actually an n = 155, since you're mapping pluck() over a list of size 155.

I'll look into an early exit in list2() when there is no dynamic dots feature used in .... Transferring to rlang.