Empty cache directories are not deleted

[lorenzwalthert]

(On macoS as their directories are reported as zero size):

du -h /Users/ripley/Library/Caches/org.R-project.R/R
0B /Users/ripley/Library/Caches/org.R-project.R/R/dipsaus/file_locks
0B /Users/ripley/Library/Caches/org.R-project.R/R/dipsaus
0B /Users/ripley/Library/Caches/org.R-project.R/R/R.cache/styler/1.9.1
0B /Users/ripley/Library/Caches/org.R-project.R/R/R.cache/styler
0B /Users/ripley/Library/Caches/org.R-project.R/R/renv/sandbox/R-4.4-r84396/aarch64-apple-darwin22.4.0/d88109f6
0B /Users/ripley/Library/Caches/org.R-project.R/R/renv/sandbox/R-4.4-r84396/aarch64-apple-darwin22.4.0
0B /Users/ripley/Library/Caches/org.R-project.R/R/renv/sandbox/R-4.4-r84396
0B /Users/ripley/Library/Caches/org.R-project.R/R/renv/sandbox
0B /Users/ripley/Library/Caches/org.R-project.R/R/renv

The CRAN policy says

"packages may store user-specific data, configuration and cache files in their respective user directories obtained from tools::R_user_dir(), provided that by default sizes are kept as small as possible and the contents are actively managed (including removing outdated material)."

so these empty dirs should have have been removed. For renv this seems to be from some user package, so if this could be encouraged in documentarian that would be good.

For dipsaus and styler, please correct before 2023-06-02 to safely retain your packages on CRAN.

[lorenzwalthert]

CRAN: And now we have a styler update leaving 63 files in .../R/R.cache/styler/1.10.0.

[lorenzwalthert]

Thanks for the update. Can you detail how you used {styler} in order to end up with this result? Because I can’t reproduce this running R CMD Check.

If you invoked styler in other ways than R CMD Check, e.g. interactively, cache files are written by default but should be removed after 6 days in compliance with the CRAN policy. That was even the case before version 1.10.0.

What’s new with this release is that, the empty, version dependent folder (e.g. /Users/ripley/Library/Caches/org.R-project.R/R/R.cache/styler/1.10.0) as well as it’s parent folder are deleted with styler:::.onLoad() if empty. I tested this behaviour on my local machine and it works.

The R.cache cache root (/Users/ripley/Library/Caches/org.R-project.R/R/R.cache) can’t be deleted by {styler} as it always contains a README.md managed by {R.cache} and I don’t think it’s {styler}’s responsibility to manage it anyways.

Lorenz Walthert

[lorenzwalthert]

CRAN reply:

Thanks for the update. Can you detail how you used {styler} in order to end up with this result? Because I can’t reproduce this running R CMD Check.
If you invoked styler in other ways than R CMD Check, e.g. interactively, cache files are written by default but should be removed after 6 days in compliance with the CRAN policy. That was even the case before version 1.10.0.

I did not. The update script ran R CMD check, and also checked those depending on styler (in alphabetical order)

I see this also on my Linux server. In both cases the file times are not all close together so not from a single R session:

ls -ltr /Users/ripley/Library/Caches/org.R-project.R/R/R.cache/styler/1.10.0
...
-rw-r--r-- 1 ripley staff 0 24 May 19:09 7d5f1f62dde25ce5bcb022b4d28b5729.Rcache
...
-rw-r--r-- 1 ripley staff 0 24 May 19:42 561d783db56caa8566745e0c9a7f5111.Rcache

and all are empty files. styler itself was checked at around 19:41.

I have just re-checked styler: that added

-rw-r--r-- 1 ripley staff 0 25 May 09:02 b9b163f367a2d53ad88c5691bea3a492.Rcache
-rw-r--r-- 1 ripley staff 0 25 May 09:02 b0771a393e3ba417df6a6d65d3c74f37.Rcache
-rw-r--r-- 1 ripley staff 0 25 May 09:02 3831a51402d5444c4ee8426245192456.Rcache
-rw-r--r-- 1 ripley staff 0 25 May 09:02 d22b74f22448d4e56fe67829318bc735.Rcache
-rw-r--r-- 1 ripley staff 0 25 May 09:02 3a2b53148a7be7ae63e1935c60772808.Rcache
-rw-r--r-- 1 ripley staff 0 25 May 09:02 87522266053e05b44ca6799cc61e0ab2.Rcache
-rw-r--r-- 1 ripley staff 0 25 May 09:02 b40f33213c9736fcfa4a29dd870c4bef.Rcache
-rw-r--r-- 1 ripley staff 0 25 May 09:02 2ccadc2fd68c4bd71413bdf00c3fb15d.Rcache
-rw-r--r-- 1 ripley staff 0 25 May 09:02 07daeeb8cefcc9361fd35d76c77c21a1.Rcache
-rw-r--r-- 1 ripley staff 0 25 May 09:02 04f963a3a88003e40804015236d8d94b.Rcache
-rw-r--r-- 1 ripley staff 0 25 May 09:02 23332ee2e3962b205cddaa505471bb53.Rcache
-rw-r--r-- 1 ripley staff 0 25 May 09:02 203efad44c58df246e2d0b0d395c9472.Rcache

but did not remove yesterday's files.

I am not sure how you can claim

"but should be removed after 6 days in compliance with the CRAN policy"

6 days is not in the CRAN policy, and last time I reported files that had not be accessed in a month. If I only run styler once a month, the files will remain for that month.

What’s new with this release is that, the empty, version dependent folder (e.g. /Users/ripley/Library/Caches/org.R-project.R/R/R.cache/styler/1.10.0) as well as it’s parent folder are deleted with styler:::.onLoad() if empty. I tested this behaviour on my local machine and it works.
It is not empty ... which is my main point.

The R.cache cache root (/Users/ripley/Library/Caches/org.R-project.R/R/R.cache) can’t be deleted by {styler} as it always contains a README.md managed by {R.cache} and I don’t think it’s {styler}’s responsibility to manage it anyways.

That is not what I have been reporting.

"The time of the volunteers is CRAN’s most precious resource"

[lorenzwalthert]

Summary

• styler 1.9.1 was leaving an empty cache directory behind (with no cache files), which was my intended behaviour but a problem for the CRAN maintainers.
• to fix that, styler 1.10.0 was released to delete a cache directory when empty (Delete cache directories if empty #1119).
• running R cmd check on styler 1.10.0 or recurse dependencies caused the creation of cache files (and folders) on CRAN volunteer's machine, which was unexpected. I can’t reproduce the problem locally when running R CMD Check. Can be due to styler update but also due to an update of a dependency.

cc: @krlmlr

[lorenzwalthert]

More results from my investigation:

There is a test that clears the styler cache (test-zzz.R, line 9) that should clear the cache, if there is any. If the test environment is not CRAN, we also check if the directory is empty and error if not. These tests passed CI always. I activated the cache and styled a file to populate the cache. Then:

• Running test-zzz.R with testthat::test() deletes everything under /Users/lorenz/Library/Caches/org.R-project.R/R/R.cache/styler/ and leaves an empty /Users/lorenz/Library/Caches/org.R-project.R/R/R.cache/styler.
• running rcmdcheck::rcmdcheck() with —as-cran or without, the check does not clear the directory, but also does not add anything.
  After I changed line 10 in to R.cache::clearCache(R.cache::getCachePath("styler"), recursive = TRUE, prompt = FALSE) to force the removal, it still did not work. Maybe there is a permission issue?

Revdepcheck also said there was one for {shinymeta}:

* installing *source* package ‘shinymeta’ ...
** package ‘shinymeta’ successfully unpacked and MD5 sums checked
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
Error: .onLoad failed in loadNamespace() for 'styler', details:
  call: mkdirs.default(path, mustWork = TRUE)
  error: Failed to create directory (tried 5 times), most likely because of lack of file permissions (directory '/Users/lorenz/Library/Caches/org.R-project.R/R/R.cache/styler' exists but nothing beyond): /Users/lorenz/Library/Caches/org.R-project.R/R/R.cache/styler/1.10.0.9000
Execution halted
ERROR: lazy loading failed for package ‘shinymeta’
* removing ‘/Users/lorenz/git/styler/revdep/checks.noindex/shinymeta/new/shinymeta.Rcheck/shinymeta’

This seem not to exist on CRAN and maybe that means revdepcheck does not give us reliable results.

That's all I have for now.

[lorenzwalthert]

After installing the most recent R version and all packges required in my local debugging environment, I am able to reproduce the CRAN behaviour. The cache was not deactivated in all vignettes and my suspiction is that in previous R versions, the enviornment from testing was used, where the cache was deactivated explicitly. So this problem was not related to the {styler} update, but the R update on the CRAN infrastructure. The caches should be deactivated in all vignettes now as introduced in #1124 and the package was submitted to CRAN again.