Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

provide Base.crc32c checksum and use it in checking cache staleness #18127

Closed
wants to merge 11 commits into from
Closed

provide Base.crc32c checksum and use it in checking cache staleness #18127

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
c64720b
add Adler's hardware-accelerated crc32c code to libsupport
stevengj Aug 18, 2016
cd75ee4
use file checksum in addition to timestamp in checking cache stalenes…
stevengj Aug 18, 2016
e8258b7
crc32c tests
stevengj Aug 18, 2016
d1ec565
don't use hardware crc with MS compilers (until we can translate it t…
stevengj Aug 19, 2016
2f562ec
note license in LICENSE.md and add_license_to_files.jl
stevengj Aug 19, 2016
ae00f03
test for new checksum cache-invalidation behavior
stevengj Aug 19, 2016
c4e5e73
enable hardware CRC in 32-bit mode
stevengj Aug 19, 2016
462a1bb
sufficient for the time to change, not the date; presumably all files…
stevengj Aug 19, 2016
cc19c64
get rid of stray refs to __x86_64__
stevengj Aug 19, 2016
a222ab0
silence implicit-cast warning for C++
stevengj Aug 19, 2016
4e0b762
clang doesn't seem to like the crc instructions on 32-bit x86, even w…
stevengj Aug 19, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions base/loading.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@ end
# to synchronize multiple tasks trying to import/using something
const package_locks = Dict{Symbol,Condition}()

cache_checksum(path) = crc32c(read(path))
cache_checksum(path) = isfile(path) ? crc32c(read(path)) : 0x00000000

# used to optionally track dependencies when requiring a module:
const _require_dependencies = Tuple{String,Float64,UInt32}[]
Expand Down Expand Up @@ -553,7 +553,8 @@ function stale_cachefile(modpath, cachefile)
return true # cache file was compiled from a different path
end
for (f,ftime,checksum) in files
# Issue #13606: compensate for Docker images rounding mtimes
# mtime check of ftime and floor(ftime) is due to issue #13606:
# compensate for Docker images rounding mtimes
if mtime(f) ∉ (ftime, floor(ftime)) &&
cache_checksum(f) != checksum
return true
Expand Down
15 changes: 12 additions & 3 deletions test/compile.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,16 +159,23 @@ try
""")

Base.compilecache("FooBar")
sleep(2)
@test isfile(joinpath(dir, "FooBar.ji"))

sleep(2) # wait to make sure file modification date changes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you might need a bit more than 2 seconds to ensure the date changes

(not serious)

touch(FooBar_file)
insert!(Base.LOAD_CACHE_PATH, 1, dir2)
Base.recompile_stale(:FooBar, joinpath(dir, "FooBar.ji"))
sleep(2)
# just touching the file should not invalidate the old cachefile (#17845)
@test !isfile(joinpath(dir2, "FooBar.ji"))
@test !Base.stale_cachefile(FooBar_file, joinpath(dir, "FooBar.ji"))

open(FooBar_file, "a") do io
println(io) # invalidate checksum
end
Base.recompile_stale(:FooBar, joinpath(dir, "FooBar.ji"))
@test isfile(joinpath(dir2, "FooBar.ji"))
@test Base.stale_cachefile(FooBar_file, joinpath(dir, "FooBar.ji"))
@test !Base.stale_cachefile(FooBar_file, joinpath(dir2, "FooBar.ji"))
@test Base.stale_cachefile(FooBar_file, joinpath(dir, "FooBar.ji"))

write(FooBar_file,
"""
Expand All @@ -177,6 +184,8 @@ try
error("break me")
end
""")
@test Base.stale_cachefile(FooBar_file, joinpath(dir, "FooBar.ji"))
@test Base.stale_cachefile(FooBar_file, joinpath(dir2, "FooBar.ji"))

try
redirected_stderr()
Expand Down