Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix[next]: use current working directory as default cache folder root #1744

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

egparedes
Copy link
Contributor

@egparedes egparedes commented Nov 19, 2024

Change the root folder of the gt4py cache directory from the system temp folder to the current working directory, which is more visible and also avoids polluting shared filesystems in hpc clusters.

@egparedes egparedes changed the title fix[next]: change default cache folder in the config fix[next]: use current working directory as default cache folder root Nov 20, 2024
@egparedes egparedes marked this pull request as ready for review November 20, 2024 04:49
Copy link
Contributor

@tehrengruber tehrengruber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was always fine with the existing behavior as every project needs to configure this anyway and it shipped around still existing caching issues. Just a couple of notes / observations I thought are worth mentioning:

  • DEBUG=True now means we will easily use out-dated cache entries indefinitely since the cache is not cleared anymore on reboot like in tmp and our cache key does not contain e.g. the GT4Py version.
  • The caching behavior is now path dependent, if you run a driver script from a different path you will recompile completely.

@havogt
Copy link
Contributor

havogt commented Nov 20, 2024

* `DEBUG=True` now means we will easily use out-dated cache entries indefinitely since the cache is not cleared anymore on reboot like in `tmp` and our cache key does not contain e.g. the GT4Py version.

I see this as an improvement over the previous behavior. Now it is consistent, before it was depending on the stability of your computer.

@egparedes
Copy link
Contributor Author

GT4Py cartesian, and I believe also DaCe, have always used the current directory as the root for the cache folder, so gt4py.next using the global temp was actually a compatibility-breaking behavior in some way. cPython has a slightly different behavior (it places the bytecode cache folder where the sources are defined) but also doesn't use the global temp.

We should definitely open an issue about gt4py not using its version (or even the python version) for the caching key.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants