feat: add ignore_cache to fetch_args_list

[dshemetov]

Checklist

Please:

• Make sure this PR is against "dev", not "main" (unless this is a release
  PR).
• Request a review from one of the current epidatr main reviewers:
  brookslogan, dshemetov, nmdefries, dsweber2.
• Makes sure to bump the version number in DESCRIPTION. Always increment
  the patch version number (the third number), unless you are making a
  release PR from dev to main, in which case increment the minor version
  number (the second number).
• Describe changes made in NEWS.md, making sure breaking changes
  (backwards-incompatible changes to the documented interface) are noted.
  Collect the changes under the next release number (e.g. if you are on
  1.7.2, then write your changes under the 1.8 heading).

Change explanations for reviewer

• Adds the ignore_cache argument to fetch_args_list() to make it easier to temporarily ignore the cache
• Adds some documentation to the is_cachable property getter for clarity
• Refactors a few of the internals of fetch functions
  • Disassemble cache_epidata_call into pieces, move most of the pieces over to fetch
  • Remove fetch_tbl and simplify fetch logic
  • Consolidate cache utility functions in cache.R (check_is_cachable moved from utils.R)

Magic GitHub syntax to mark associated Issue(s) as resolved when this is merged into the default branch

• Resolves Refactor idea: the fetch call chain is too complex #200

[brookslogan]

I'm getting some tests failing when running locally where check_is_cachable gives unexpected results. There was also a CHECK warning above from not having @importFrom tibble tibble that seems like it still applies but might not be caught in tests.

[brookslogan]

I have some minor suggestions that probably don't relate to the refactor itself but also include some similar cleanup, but I did not spot why my local tests might be failing; looked good to me but seems that I missed something (hence just commenting). Here's the output if helpful:

ℹ Testing epidatr
! epidatr cache is being used (set env var EPIDATR_USE_CACHE=FALSE if not intended).
ℹ The cache directory is ~/.cache/R/epidatr.
ℹ The cache will be cleared after 14 days and will be pruned if it exceeds 4096 MB.
ℹ The log of cache transactions is stored at ~/.cache/R/epidatr/logfile.txt.
✔ | F W  S  OK | Context
✔ |          1 | auth
✖ | 1       42 | cache
──────────────────────────────────────────────────────────────────────────────────────────────
Failure (test-cache.R:128:7): check_is_cachable
check_is_cachable(epidata_call, fetch_args) is not FALSE

`actual`:   TRUE 
`expected`: FALSE
Backtrace:
    ▆
 1. └─epidatr (local) check_fun(...) at test-cache.R:142:3
 2.   └─testthat::expect_false(check_is_cachable(epidata_call, fetch_args)) at test-cache.R:128:7
──────────────────────────────────────────────────────────────────────────────────────────────
✔ |         19 | check
✔ |          3 | covidcast [1.2s]
✔ |      1  74 | endpoints
✔ |         12 | epidatacall
✔ |          6 | epirange
✔ |         42 | model
✔ |      1   0 | request
✔ |      1  28 | utils

══ Results ═══════════════════════════════════════════════════════════════════════════════════
Duration: 3.2 s

── Skipped tests (3) ─────────────────────────────────────────────────────────────────────────
• empty test (2): test-endpoints.R:1:1, test-utils.R:24:1
• This site is down. (1): test-request.R:2:3

── Failed tests ──────────────────────────────────────────────────────────────────────────────
Failure (test-cache.R:128:7): check_is_cachable
check_is_cachable(epidata_call, fetch_args) is not FALSE

`actual`:   TRUE 
`expected`: FALSE
Backtrace:
    ▆
 1. └─epidatr (local) check_fun(...) at test-cache.R:142:3
 2.   └─testthat::expect_false(check_is_cachable(epidata_call, fetch_args)) at test-cache.R:128:7

[ FAIL 1 | WARN 0 | SKIP 3 | PASS 227 ]

[dshemetov]

Fixed a bunch of issues:

• test caught a bug (if fields is not null, then we don't cache; lost that check in the refactor), fixed that
• rewrote a few other tests
• improved tests so they actually clear and disable the cache after every test using withr::defer
• fixed incorrect parsing of only_supports_classic endpoints
• addressed other minor changes

Thanks for the review!

[brookslogan]

Sorry, somehow thought this was already merged. Taking a look now. Quick first not: I think running tests deleted my actual cache instead of the test cache somehow? And then I get output like:

Error in file(file, ifelse(append, "a", "w")) : 
  cannot open the connection
In addition: Warning messages:
1: Using cached results with `as_of` within the past week (or the future!). This will likely result in an
invalid cache. Consider
ℹ disabling the cache for this session with `disable_cache` or permanently with environmental variable
  `EPIDATR_USE_CACHE=FALSE`
ℹ setting `EPIDATR_CACHE_MAX_AGE_DAYS=1` to e.g. `3/24` (3 hours).

from trying the pub_covidcast examples + applying an as_of = Sys.Date() - 7 (should have been 8 I guess to not get this much spam).

Though some other attempts are easier to interpret:

Error in file(file, ifelse(append, "a", "w")) : 
  cannot open the connection
In addition: Warning message:
In file(file, ifelse(append, "a", "w")) :
  cannot open file '/home/<username>/.cache/R/epidatr/logfile.txt': No such file or directory

[brookslogan]

Not sure how, but one of these clear_cache() calls seems to have cleared my real cache rather than the test cache when I ran devtools::test().

[dshemetov]

Hm, thanks for reminding me, this happened to me too... Current guess: if you have a cache activated before tests starts, subsequent set_cache calls just reuse that cache, so then a later clear_cache destroys it. Will check.

[brookslogan]

Nice catches. I've noted a couple minor issues + request (maybe in some separate Issue/PR) to improve messaging when a cache has been cleared unexpectedly and we still try to use it.

[dshemetov]

Also, I'm now thinking that ignore_cache is kinda weird... what doesn't seem natural is that if there is a cache and we ignore_cache, then we never update the cache. What's probably better is something like refresh_cache, which will fetch from API and overwrite what's in the cache currently, if the cache is on.