-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CircleCI caching: allow ftime to be ceil(ftime_req) in Base.stale_cachefile #47433
Conversation
😬 |
Truncating to microseconds was one thing, but truncating to whole seconds seems way too coarse, in my opinion. |
I think it'd prefer it if we instead implemented #45541 In other words, we would first check mtimes, and if the mtime check failed, instead of immediately considering the cachefile stale, we would check the hashes of the source files. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See comments above.
Thanks for the review, I agree that this is hacky. But I might argue that this is less coarse that existing logic:
( ftime != floor(ftime_req) ) && # Issue #13606, PR #13613: compensate for Docker images rounding mtimes seems much more prone to produce a false positive. |
Having said the above your suggested approach sounds much better. do you have a rough idea of how much of a slowdown it can incur in practice? |
I think this is fine since it is an exact check and not a range check like the microsecond check that was mentioned. |
Dismissing as stale per the conversation above
…hefile It appears that [caching functionalities](https://circleci.com/docs/caching/) provided by CircleCi, a leading CI/CD provider, can truncate timestamps to full seconds, resulting in re-compilations as below: ``` Rejecting stale cache file /root/.julia/compiled/v1.8/ComponentArrays/cYHSD_3rQji.ji (mtime 1.6673960929277816e9) because file /root/.julia/packages/ComponentArrays/YyD7i/src/ComponentArrays.jl ``` This PR relaxes the `is_stale` check to be robust against rounding-to-second timestamp mutations. I can provide a minimal CircleCI configuration file to reproduce if this is helpful.
…hefile (#47433) * CircleCI caching: allow ftime to be ceil(ftime_req) in Base.stale_cachefile It appears that [caching functionalities](https://circleci.com/docs/caching/) provided by CircleCi, a leading CI/CD provider, can truncate timestamps to full seconds, resulting in re-compilations as below: ``` Rejecting stale cache file /root/.julia/compiled/v1.8/ComponentArrays/cYHSD_3rQji.ji (mtime 1.6673960929277816e9) because file /root/.julia/packages/ComponentArrays/YyD7i/src/ComponentArrays.jl ``` This PR relaxes the `is_stale` check to be robust against rounding-to-second timestamp mutations. I can provide a minimal CircleCI configuration file to reproduce if this is helpful. (cherry picked from commit bf92e83)
…hefile (#47433) * CircleCI caching: allow ftime to be ceil(ftime_req) in Base.stale_cachefile It appears that [caching functionalities](https://circleci.com/docs/caching/) provided by CircleCi, a leading CI/CD provider, can truncate timestamps to full seconds, resulting in re-compilations as below: ``` Rejecting stale cache file /root/.julia/compiled/v1.8/ComponentArrays/cYHSD_3rQji.ji (mtime 1.6673960929277816e9) because file /root/.julia/packages/ComponentArrays/YyD7i/src/ComponentArrays.jl ``` This PR relaxes the `is_stale` check to be robust against rounding-to-second timestamp mutations. I can provide a minimal CircleCI configuration file to reproduce if this is helpful. (cherry picked from commit bf92e83)
…hefile (#47433) * CircleCI caching: allow ftime to be ceil(ftime_req) in Base.stale_cachefile It appears that [caching functionalities](https://circleci.com/docs/caching/) provided by CircleCi, a leading CI/CD provider, can truncate timestamps to full seconds, resulting in re-compilations as below: ``` Rejecting stale cache file /root/.julia/compiled/v1.8/ComponentArrays/cYHSD_3rQji.ji (mtime 1.6673960929277816e9) because file /root/.julia/packages/ComponentArrays/YyD7i/src/ComponentArrays.jl ``` This PR relaxes the `is_stale` check to be robust against rounding-to-second timestamp mutations. I can provide a minimal CircleCI configuration file to reproduce if this is helpful. (cherry picked from commit bf92e83)
…hefile (#47433) * CircleCI caching: allow ftime to be ceil(ftime_req) in Base.stale_cachefile It appears that [caching functionalities](https://circleci.com/docs/caching/) provided by CircleCi, a leading CI/CD provider, can truncate timestamps to full seconds, resulting in re-compilations as below: ``` Rejecting stale cache file /root/.julia/compiled/v1.8/ComponentArrays/cYHSD_3rQji.ji (mtime 1.6673960929277816e9) because file /root/.julia/packages/ComponentArrays/YyD7i/src/ComponentArrays.jl ``` This PR relaxes the `is_stale` check to be robust against rounding-to-second timestamp mutations. I can provide a minimal CircleCI configuration file to reproduce if this is helpful. (cherry picked from commit bf92e83)
…hefile (#47433) * CircleCI caching: allow ftime to be ceil(ftime_req) in Base.stale_cachefile It appears that [caching functionalities](https://circleci.com/docs/caching/) provided by CircleCi, a leading CI/CD provider, can truncate timestamps to full seconds, resulting in re-compilations as below: ``` Rejecting stale cache file /root/.julia/compiled/v1.8/ComponentArrays/cYHSD_3rQji.ji (mtime 1.6673960929277816e9) because file /root/.julia/packages/ComponentArrays/YyD7i/src/ComponentArrays.jl ``` This PR relaxes the `is_stale` check to be robust against rounding-to-second timestamp mutations. I can provide a minimal CircleCI configuration file to reproduce if this is helpful. (cherry picked from commit bf92e83)
…hefile (#47433) * CircleCI caching: allow ftime to be ceil(ftime_req) in Base.stale_cachefile It appears that [caching functionalities](https://circleci.com/docs/caching/) provided by CircleCi, a leading CI/CD provider, can truncate timestamps to full seconds, resulting in re-compilations as below: ``` Rejecting stale cache file /root/.julia/compiled/v1.8/ComponentArrays/cYHSD_3rQji.ji (mtime 1.6673960929277816e9) because file /root/.julia/packages/ComponentArrays/YyD7i/src/ComponentArrays.jl ``` This PR relaxes the `is_stale` check to be robust against rounding-to-second timestamp mutations. I can provide a minimal CircleCI configuration file to reproduce if this is helpful. (cherry picked from commit bf92e83)
…hefile (#47433) * CircleCI caching: allow ftime to be ceil(ftime_req) in Base.stale_cachefile It appears that [caching functionalities](https://circleci.com/docs/caching/) provided by CircleCi, a leading CI/CD provider, can truncate timestamps to full seconds, resulting in re-compilations as below: ``` Rejecting stale cache file /root/.julia/compiled/v1.8/ComponentArrays/cYHSD_3rQji.ji (mtime 1.6673960929277816e9) because file /root/.julia/packages/ComponentArrays/YyD7i/src/ComponentArrays.jl ``` This PR relaxes the `is_stale` check to be robust against rounding-to-second timestamp mutations. I can provide a minimal CircleCI configuration file to reproduce if this is helpful. (cherry picked from commit bf92e83)
It appears that caching functionalities provided by CircleCi (a leading CI/CD provider) can truncate timestamps to full seconds, resulting in re-compilations as below:
This PR relaxes the
is_stale
check to be robust against truncating-to-seconds timestamp mutations.I can provide a minimal CircleCI configuration file to reproduce if this is helpful.