Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reprex skipping some statements (these should cause an error) #438

Closed
abalter opened this issue Mar 19, 2023 · 2 comments
Closed

Reprex skipping some statements (these should cause an error) #438

abalter opened this issue Mar 19, 2023 · 2 comments
Labels
reprex needs a minimal reproducible example

Comments

@abalter
Copy link

abalter commented Mar 19, 2023

The statements where I try to pivot a lazy table that is very wide give me a c-stack error. I'm trying to show this in a reprex. However, those statements are not running in the reprex---or at least the error does not print.

I inserted a statement after them that would throw an error and it does.

reprex

library(tidyverse)
library(dbplyr)
#> 
#> Attaching package: 'dbplyr'
#> The following objects are masked from 'package:dplyr':
#> 
#>     ident, sql
library(RSQLite)
library(arrow)
#> 
#> Attaching package: 'arrow'
#> The following object is masked from 'package:lubridate':
#> 
#>     duration
#> The following object is masked from 'package:utils':
#> 
#>     timestamp

Nids = 10
Nyears = 10
Ndates = 12*Nyears
start_year = 2010

"*******  Create Very Wide Table  *********"
#> [1] "*******  Create Very Wide Table  *********"
tb_wide =
  crossing(
    id = str_glue("id_{1:Nids}"),
    year = start_year:(start_year+Nyears-1),
    month = month.abb %>% tolower()
  ) %>%
  tibble() %>%
  mutate(value = rnorm(Nids*Ndates)) %>%
  unite(month, year, col="date", sep="_") %>%
  pivot_wider(names_from=date, values_from = value)

tb_wide %>% nrow()
#> [1] 10
tb_wide %>% ncol()
#> [1] 121
colnames(tb_wide)
#>   [1] "id"       "apr_2010" "aug_2010" "dec_2010" "feb_2010" "jan_2010"
#>   [7] "jul_2010" "jun_2010" "mar_2010" "may_2010" "nov_2010" "oct_2010"
#>  [13] "sep_2010" "apr_2011" "aug_2011" "dec_2011" "feb_2011" "jan_2011"
#>  [19] "jul_2011" "jun_2011" "mar_2011" "may_2011" "nov_2011" "oct_2011"
#>  [25] "sep_2011" "apr_2012" "aug_2012" "dec_2012" "feb_2012" "jan_2012"
#>  [31] "jul_2012" "jun_2012" "mar_2012" "may_2012" "nov_2012" "oct_2012"
#>  [37] "sep_2012" "apr_2013" "aug_2013" "dec_2013" "feb_2013" "jan_2013"
#>  [43] "jul_2013" "jun_2013" "mar_2013" "may_2013" "nov_2013" "oct_2013"
#>  [49] "sep_2013" "apr_2014" "aug_2014" "dec_2014" "feb_2014" "jan_2014"
#>  [55] "jul_2014" "jun_2014" "mar_2014" "may_2014" "nov_2014" "oct_2014"
#>  [61] "sep_2014" "apr_2015" "aug_2015" "dec_2015" "feb_2015" "jan_2015"
#>  [67] "jul_2015" "jun_2015" "mar_2015" "may_2015" "nov_2015" "oct_2015"
#>  [73] "sep_2015" "apr_2016" "aug_2016" "dec_2016" "feb_2016" "jan_2016"
#>  [79] "jul_2016" "jun_2016" "mar_2016" "may_2016" "nov_2016" "oct_2016"
#>  [85] "sep_2016" "apr_2017" "aug_2017" "dec_2017" "feb_2017" "jan_2017"
#>  [91] "jul_2017" "jun_2017" "mar_2017" "may_2017" "nov_2017" "oct_2017"
#>  [97] "sep_2017" "apr_2018" "aug_2018" "dec_2018" "feb_2018" "jan_2018"
#> [103] "jul_2018" "jun_2018" "mar_2018" "may_2018" "nov_2018" "oct_2018"
#> [109] "sep_2018" "apr_2019" "aug_2019" "dec_2019" "feb_2019" "jan_2019"
#> [115] "jul_2019" "jun_2019" "mar_2019" "may_2019" "nov_2019" "oct_2019"
#> [121] "sep_2019"

"******  Create lazy arrow table  *****"
#> [1] "******  Create lazy arrow table  *****"
readr::write_csv(tb_wide, "tb_wide.csv")
tb_arrow = read_csv_arrow("tb_wide.csv", as_data_frame = F)
tb_arrow %>% nrow()
#> [1] 10
tb_arrow %>% ncol()
#> [1] 121

"******  Create memdb table  ******"
#> [1] "******  Create memdb table  ******"
tb_mdb = memdb_frame(tb_wide)
tb_arrow %>% nrow()
#> [1] 10
tb_arrow %>% ncol()
#> [1] 121

"******  Create in-memory sqlite table  ******"
#> [1] "******  Create in-memory sqlite table  ******"
con = dbConnect(RSQLite::SQLite(),":memory")
copy_to(con, tb_wide, "tb_wide")
dbListTables(con)
#> [1] "sqlite_stat1" "sqlite_stat4" "tb_wide"
tb_db = tbl(con, "tb_wide")
tb_arrow %>% nrow()
#> [1] 10
tb_arrow %>% ncol()
#> [1] 121


"******  Try Pivoting  ******"
#> [1] "******  Try Pivoting  ******"
tb_long =
  tb_wide %>%
  pivot_longer(cols=-id, names_to="date", values_to = "value")

dim(tb_long)
#> [1] 1200    3
colnames(tb_long)
#> [1] "id"    "date"  "value"

tb_arrow %>%
  to_duckdb() %>%
  pivot_longer(cols=-id, names_to="date", values_to = "value")

tb_mdb %>%
  pivot_longer(cols=-id, names_to="date", values_to = "value")

tb_db %>%
  pivot_longer(cols=-id, names_to="date", values_to = "value")


print(nonexistent_variable)
#> Error in print(nonexistent_variable): object 'nonexistent_variable' not found

sessionInfo()
#> R version 4.2.2 (2022-10-31)
#> Platform: x86_64-conda-linux-gnu (64-bit)
#> Running under: Ubuntu 22.04.1 LTS
#> 
#> Matrix products: default
#> BLAS/LAPACK: /home/users/balter/micromamba/lib/libopenblasp-r0.3.21.so
#> 
#> locale:
#>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
#>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
#>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
#>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
#>  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> other attached packages:
#>  [1] arrow_10.0.1    RSQLite_2.3.0   dbplyr_2.3.1    lubridate_1.9.2
#>  [5] forcats_1.0.0   stringr_1.5.0   dplyr_1.1.0     purrr_1.0.1    
#>  [9] readr_2.1.4     tidyr_1.3.0     tibble_3.2.0    ggplot2_3.4.1  
#> [13] tidyverse_2.0.0
#> 
#> loaded via a namespace (and not attached):
#>  [1] styler_1.9.1      tidyselect_1.2.0  xfun_0.37         colorspace_2.1-0 
#>  [5] vctrs_0.5.2       generics_0.1.3    htmltools_0.5.4   yaml_2.3.7       
#>  [9] utf8_1.2.3        blob_1.2.3        rlang_1.0.6       R.oo_1.25.0      
#> [13] pillar_1.8.1      glue_1.6.2        withr_2.5.0       DBI_1.1.3        
#> [17] R.utils_2.12.2    bit64_4.0.5       R.cache_0.16.0    lifecycle_1.0.3  
#> [21] munsell_0.5.0     gtable_0.3.1      R.methodsS3_1.8.2 evaluate_0.20    
#> [25] memoise_2.0.1     knitr_1.42        tzdb_0.3.0        fastmap_1.1.1    
#> [29] parallel_4.2.2    fansi_1.0.4       scales_1.2.1      cachem_1.0.7     
#> [33] vroom_1.6.1       fs_1.6.1          bit_4.0.5         hms_1.1.2        
#> [37] digest_0.6.31     stringi_1.7.12    duckdb_0.7.1-1    grid_4.2.2       
#> [41] cli_3.6.0         tools_4.2.2       magrittr_2.0.3    crayon_1.5.2     
#> [45] pkgconfig_2.0.3   ellipsis_0.3.2    reprex_2.0.2      timechange_0.2.0 
#> [49] assertthat_0.2.1  rmarkdown_2.20    rstudioapi_0.14   R6_2.5.1         
#> [53] compiler_4.2.2

Created on 2023-03-19 with reprex v2.0.2

code

library(tidyverse)
library(dbplyr)
library(RSQLite)
library(arrow)

Nids = 10
Nyears = 10
Ndates = 12*Nyears
start_year = 2010

"*******  Create Very Wide Table  *********"
tb_wide =
  crossing(
    id = str_glue("id_{1:Nids}"),
    year = start_year:(start_year+Nyears-1),
    month = month.abb %>% tolower()
  ) %>%
  tibble() %>%
  mutate(value = rnorm(Nids*Ndates)) %>%
  unite(month, year, col="date", sep="_") %>%
  pivot_wider(names_from=date, values_from = value)

tb_wide %>% nrow()
tb_wide %>% ncol()
colnames(tb_wide)

"******  Create lazy arrow table  *****"
readr::write_csv(tb_wide, "tb_wide.csv")
tb_arrow = read_csv_arrow("tb_wide.csv", as_data_frame = F)
tb_arrow %>% nrow()
tb_arrow %>% ncol()

"******  Create memdb table  ******"
tb_mdb = memdb_frame(tb_wide)
tb_arrow %>% nrow()
tb_arrow %>% ncol()

"******  Create in-memory sqlite table  ******"
con = dbConnect(RSQLite::SQLite(),":memory")
copy_to(con, tb_wide, "tb_wide")
dbListTables(con)
tb_db = tbl(con, "tb_wide")
tb_arrow %>% nrow()
tb_arrow %>% ncol()


"******  Try Pivoting  ******"
tb_long =
  tb_wide %>%
  pivot_longer(cols=-id, names_to="date", values_to = "value")

dim(tb_long)
colnames(tb_long)

tb_arrow %>%
  to_duckdb() %>%
  pivot_longer(cols=-id, names_to="date", values_to = "value")

tb_mdb %>%
  pivot_longer(cols=-id, names_to="date", values_to = "value")

tb_db %>%
  pivot_longer(cols=-id, names_to="date", values_to = "value")


print(nonexistent_variable)

sessionInfo()
@hadley
Copy link
Member

hadley commented Nov 22, 2023

Thanks for providing some code that illustrates the problem — it's a great start 😄 However, your code seems to do more than required to illustrate the problem, and the extra complexity is making it hard for me to understand. Can you please try and simplify your example some more? The more minimal you can make the reprex, the faster I can identify the problem and fix it.

@hadley hadley added the reprex needs a minimal reproducible example label Nov 22, 2023
@jennybc
Copy link
Member

jennybc commented Jan 10, 2024

Duplicate of #388, which has been recognized as r-lib/evaluate#103.

@jennybc jennybc closed this as completed Jan 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
reprex needs a minimal reproducible example
Projects
None yet
Development

No branches or pull requests

3 participants