data.table::update.dev.pkg() warning or error #4403

drag05 · 2020-04-27T16:32:23Z

data.table v 1.12.8:

applying

data.table::update.dev.pkg()

gives the following message in R v. 4.0.0, R Studio v 1. 2.5042 both with Admin rights:

R data.table package has been updated to NA (1.12.8)
Warning message:
In file.copy(savedcopy, lib, recursive = TRUE) :
  problem copying C:\R\R-4.0.0\library\00LOCK\data.table\libs\x64\datatable.dll to C:\R\R-4.0.0\library\data.table\libs\x64\datatable.dll: Permission denied

> sessionInfo()
R version 4.0.0 (2020-04-24)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.0.0    tools_4.0.0       data.table_1.12.8

The text was updated successfully, but these errors were encountered:

jangorecki · 2020-04-27T16:36:52Z

It is actually the same problem as I have seen on CRAN windows checks today. I was hoping it is a local CRAN issue related to new 4.0.0 setup, it seems it is not.
Thanks for reporting. @drag05 could you just confirm you are using rtools 4.0?
Problem is probably related to installing from source on Windows.

tlapak · 2020-04-27T17:48:38Z

I have the same issue on Windows 10 using R 3.6.2. update.dev.pkg() downloads the binaries from gitlab, no? I don't have rtools installed on that machine. Specifying a lib, which I put in my documents folder, let me run the "update" successfully.

Edit: Actually using R 3.6.2

> sessionInfo()
R version 3.6.2 (2019-12-12)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 17763)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.1252 
[2] LC_CTYPE=English_United Kingdom.1252   
[3] LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C                           
[5] LC_TIME=English_United Kingdom.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_3.6.2

Using

install.packages("data.table", type="source", repos="https://Rdatatable.gitlab.io/data.table")

works smoothly.

mattdowle · 2020-04-27T17:52:51Z

We've seen this before on CRAN checks several times; the workaround there is that we ping Uwe and he reruns. CRAN checks are run in parallel. I don't think data.table is reinstalled by CRAN each time but sometimes it is, and when it does, if another package that uses data.table is running its checks at the same time, the install fails because the .dll is in use by that other package. Prior discussions with Uwe and others, if I remember correctly, are essentially that this is a Windows feature and there is no known way to alleviate it (other than perhaps adding to the error message a hint to the user to close all other R sessions and perhaps reboot too). Windows locks its dll's. Unlike Linux and MacOS which have a mechanism to allow several versions of the same .so to be in use.

I don't think it is R 4.0.0 per se. It's just that the event of R 4.0.0 release triggers people to update who perhaps do not update very often.

Could the people who are having troubles ensure that all R sessions are shutdown, and then try upgrading/installing data.table again? If you have a .RProfile, and that .RProfile loads data.table, then you have to turn that off otherwise data.table loads on startup and the lock is opened on that dll. This applies to every R package with a .dll as far as I know.

jangorecki · 2020-04-27T17:53:02Z

I am able to reproduce problem. update.dev.pkg() seems to be broken in R 4.0.0 because R holds the data.table package (because function is in the package) more strictly than before.

To install latest master please use this for now

install.packages("data.table", repos="https://Rdatatable.gitlab.io/data.table")

Just for the record, building from source on windows for R 4.0.0

## download and install
# https://cran.r-project.org/bin/windows/base/R-4.0.0-win.exe
# https://cran.r-project.org/bin/windows/Rtools/rtools40-x86_64.exe
set PATH=C:\rtools40\usr\bin;%PATH%
R                 ## ensure R 4.0.0
Sys.which("make") ## ensure rtools40 path

# install 1.12.8
install.packages("data.table", type="source", repos="https://cran.r-project.org")
# install devel
install.packages("data.table", type="source", repos="https://Rdatatable.gitlab.io/data.table")

drag05 · 2020-04-27T18:56:36Z

@mattdowle
R 4.0.0 asks for all previously R v < 4 installed packages to be re-installed. My issue popped up while I was reading about fcase() in News: the data.table had already been installed anew as requested ...

drag05 · 2020-04-27T19:02:37Z

@jangorecki

I have tried twice and got this every time:

> install.packages("data.table", repos="https://Rdatatable.gitlab.io/data.table")
trying URL 'https://Rdatatable.gitlab.io/data.table/bin/windows/contrib/4.0/data.table_1.12.9.zip'
Content type 'application/zip' length 2675878 bytes (2.6 MB)
downloaded 960 KB

Error in read.dcf(file.path(pkgname, "DESCRIPTION"), c("Package", "Type")) : 
  cannot open the connection
In addition: Warning messages:
1: In download.file(url, destfile, method, mode = "wb", ...) :
  downloaded length 983040 != reported length 2675878
2: In unzip(zipname, exdir = dest) : error 1 in extracting from zip file
3: In read.dcf(file.path(pkgname, "DESCRIPTION"), c("Package", "Type")) :
  cannot open compressed file 'data.table/DESCRIPTION', probable reason 'No such file or directory'

and

> sessionInfo()
R version 4.0.0 (2020-04-24)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.0.0 tools_4.0.0

also

> Sys.which('make')
                              make 
"C:\\rtools40\\usr\\bin\\make.exe"

jangorecki · 2020-04-27T19:16:13Z

@drag05 install.packages warnings seems like issue with your connection. It doesn't download complete zip archive. AFAIK it is not related to the issue reported in this thread, and should be possible to "fix" by a retry (or more retries, maybe router restarting can help as well?).
Original issue has been addressed in PR #4404.

drag05 · 2020-04-27T19:21:16Z

@jangorecki

Thank you for the tip! I was inside my VPN. Now it works both ways - it seems:


> install.packages("data.table", repos="https://Rdatatable.gitlab.io/data.table")
trying URL 'https://Rdatatable.gitlab.io/data.table/bin/windows/contrib/4.0/data.table_1.12.9.zip'
Content type 'application/zip' length 2675878 bytes (2.6 MB)
downloaded 2.6 MB

package ‘data.table’ successfully unpacked and MD5 sums checked

The downloaded binary packages are in
        C:\Users\Dragos\AppData\Local\Temp\Rtmp2VNd5T\downloaded_packages

and

> data.table::update.dev.pkg()
R data.table package is up-to-date at dd7609e83132e19c9be80d71c73e8a8f95e19b27 (1.12.9)

So, I guess - no VPN allowed which may not be that good ...

drag05 · 2020-04-27T19:38:31Z

Thank you for your prompt replies! I wonder if I should open a new issue involving VPN.
Please advise!

jangorecki · 2020-04-27T19:40:46Z

I will re-open this issue, as I was able to reproduce the problem on windows. The issue was not just connection, but windows that locks dll, unloading namespace before reinstalling seems to remove the lock.

drag05 · 2020-04-28T02:23:54Z

In my case the data.table wasn't loaded at any time, nor other R sessions were open.
It could be that being inside a VPN triggered another issue.

jangorecki · 2020-04-28T10:34:48Z

when you call data.table:: it automatically loads package namespace, do not confuse with attaching a namespace.

drag05 · 2020-04-28T13:38:23Z

@jangorecki
Now I see what you meant; if I understand you correctly, the functions after the data.table:: should unloadNamespace('data.table') before doing everything else.

jangorecki · 2020-04-28T14:54:15Z

not functions, but this particular function, because it attempts to upgrade this namespace

drag05 · 2020-04-28T15:38:20Z

I have miswritten. I meant it could happen to other namespaces too. Couldn't it?

jangorecki · 2020-04-28T16:19:42Z

It is related only to reinstalling pkg. Using :: in general is fine, but when you upgrade pkg it is best to have its namespace unloaded. Locks during upgrades on Windows happens quite commonly.

jangorecki self-assigned this Apr 27, 2020

jangorecki added a commit that referenced this issue Apr 27, 2020

unload namespace before reinstalling, #4403

8b1aa6a

jangorecki mentioned this issue Apr 27, 2020

unload namespace before reinstalling, #4403 #4404

Merged

jangorecki added the platform-specific label Apr 27, 2020

drag05 closed this as completed Apr 27, 2020

jangorecki reopened this Apr 27, 2020

jangorecki added this to the 1.12.9 milestone May 26, 2020

jangorecki added the install label May 26, 2020

mattdowle closed this as completed in #4404 Jun 2, 2020

mattdowle pushed a commit that referenced this issue Jun 2, 2020

unload namespace before reinstalling, #4403 (#4404)

09fb6c3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

data.table::update.dev.pkg() warning or error #4403

data.table::update.dev.pkg() warning or error #4403

drag05 commented Apr 27, 2020 •

edited

Loading

jangorecki commented Apr 27, 2020 •

edited

Loading

tlapak commented Apr 27, 2020 •

edited

Loading

mattdowle commented Apr 27, 2020 •

edited

Loading

jangorecki commented Apr 27, 2020 •

edited

Loading

drag05 commented Apr 27, 2020

drag05 commented Apr 27, 2020 •

edited

Loading

jangorecki commented Apr 27, 2020 •

edited

Loading

drag05 commented Apr 27, 2020

drag05 commented Apr 27, 2020

jangorecki commented Apr 27, 2020

drag05 commented Apr 28, 2020 •

edited

Loading

jangorecki commented Apr 28, 2020

drag05 commented Apr 28, 2020

jangorecki commented Apr 28, 2020

drag05 commented Apr 28, 2020 •

edited

Loading

jangorecki commented Apr 28, 2020 •

edited

Loading

data.table::update.dev.pkg() warning or error #4403

data.table::update.dev.pkg() warning or error #4403

Comments

drag05 commented Apr 27, 2020 • edited Loading

jangorecki commented Apr 27, 2020 • edited Loading

tlapak commented Apr 27, 2020 • edited Loading

mattdowle commented Apr 27, 2020 • edited Loading

jangorecki commented Apr 27, 2020 • edited Loading

drag05 commented Apr 27, 2020

drag05 commented Apr 27, 2020 • edited Loading

jangorecki commented Apr 27, 2020 • edited Loading

drag05 commented Apr 27, 2020

drag05 commented Apr 27, 2020

jangorecki commented Apr 27, 2020

drag05 commented Apr 28, 2020 • edited Loading

jangorecki commented Apr 28, 2020

drag05 commented Apr 28, 2020

jangorecki commented Apr 28, 2020

drag05 commented Apr 28, 2020 • edited Loading

jangorecki commented Apr 28, 2020 • edited Loading

drag05 commented Apr 27, 2020 •

edited

Loading

jangorecki commented Apr 27, 2020 •

edited

Loading

tlapak commented Apr 27, 2020 •

edited

Loading

mattdowle commented Apr 27, 2020 •

edited

Loading

jangorecki commented Apr 27, 2020 •

edited

Loading

drag05 commented Apr 27, 2020 •

edited

Loading

jangorecki commented Apr 27, 2020 •

edited

Loading

drag05 commented Apr 28, 2020 •

edited

Loading

drag05 commented Apr 28, 2020 •

edited

Loading

jangorecki commented Apr 28, 2020 •

edited

Loading