Review caching

[sandorkertesz]

Is your feature request related to a problem? Please describe.

The current caching is documented at https://earthkit-data.readthedocs.io/en/latest/guide/caching.html

The problem is that the cache currently also serves as temporary space. So when caching is turned off there is nowhere to download files from URLs or services.

The task is to review the caching policies and see how we could use separate concepts for the cache and the temporary space.

Related issues:

• Can not retrieve data from the CDS when the cache is switched off #204
• Load settings from environment variables and/or switch off auto_save_settings by default #210

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

Organisation

ECMWF

[sandorkertesz]

Changes so far:

• the "off" cache-policy now uses a temporary directory so all retrievals are possible to use
• the default cache-policy was changed to "off" from "user"
• the cache/temporary directory path can be queried as cache.directory()

See:
https://earthkit-data.readthedocs.io/en/feature-cache_and_tmp/examples/cache.html
https://earthkit-data.readthedocs.io/en/feature-cache_and_tmp/guide/caching.html