Reset timestamps to 1970-01-01 during packaging for repeatable builds #5344
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bjaglin
pushed a commit
to bjaglin/sbt-memoize
that referenced
this pull request
May 26, 2020
We need a nightly containing sbt/sbt#5344 if we want children projects to be insensitive to repackaging.
bollinger
reviewed
Oct 12, 2020
Timestamp 0 is the intended behavior of sbt 1.4.0 to make the build more repeatable. |
This was referenced Nov 16, 2020
|
commenting and saying "1970" so it comes up in search results :) (since 1970 is the date all the files get) 1970 1970 1970 1970 !!! |
eed3si9n
changed the title
|
+1 to the previous comments and the linked play issue playframework/playframework#10572, this change breaks etag computation in play. while the suggested fix in that thread (of setting an environment variable) does work, it's frustrating that there's no way to do it directly from sbt. it might be nicer if there was an sbt setting for this, also, since the new code always provides |
|
@dontgitit I've created a new issue for this - #6235 |
eed3si9n
added a commit
to eed3si9n/sbt
that referenced
this pull request
Jan 3, 2021
Fixes sbt#6235 In sbt 1.4.0 (sbt#5344) we started wiping out the timestamps in JAR to make the builds more repeatable. This had an unintended consequence of breaking Play's last-modified response header (playframework/playframework#10572). This adds an opt-out from the default: ```scala Compile / packageBin / packageOptions += Package.keepTimestamps ``` ### before ``` $ ll example total 32 -rw-r--r-- 1 eed3si9n wheel 901 Jan 1 1970 Greeting.class -rw-r--r-- 1 eed3si9n wheel 3079 Jan 1 1970 Hello$.class -rw-r--r-- 1 eed3si9n wheel 738 Jan 1 1970 Hello$delayedInit$body.class -rw-r--r-- 1 eed3si9n wheel 875 Jan 1 1970 Hello.class ``` ### after ``` $ ll target/scala-2.13/hello_2.13-0.1.0-SNAPSHOT/example total 32 -rwxr-xr-x 1 eed3si9n wheel 901 Jan 3 12:20 Greeting.class* -rwxr-xr-x 1 eed3si9n wheel 3079 Jan 3 12:20 Hello$.class* -rwxr-xr-x 1 eed3si9n wheel 738 Jan 3 12:20 Hello$delayedInit$body.class* -rwxr-xr-x 1 eed3si9n wheel 875 Jan 3 12:20 Hello.class* ```
This was referenced Jan 8, 2021
eed3si9n
added a commit
to eed3si9n/sbt
that referenced
this pull request
Jan 25, 2021
…mmitDate Fixes sbt#6235 In sbt 1.4.0 (sbt#5344) we started wiping out the timestamps in JAR to make the builds more repeatable. This had an unintended consequence of breaking Play's last-modified response header (playframework/playframework#10572). This adds an opt-out from the default: ```scala ThisBuild / packageOptions += Package.keepTimestamps ``` Before ------ ``` $ ll example total 32 -rw-r--r-- 1 eed3si9n wheel 901 Jan 1 1970 Greeting.class -rw-r--r-- 1 eed3si9n wheel 3079 Jan 1 1970 Hello$.class -rw-r--r-- 1 eed3si9n wheel 738 Jan 1 1970 Hello$delayedInit$body.class -rw-r--r-- 1 eed3si9n wheel 875 Jan 1 1970 Hello.class ``` After ----- ``` $ ll target/scala-2.13/hello_2.13-0.1.0-SNAPSHOT/example total 32 -rwxr-xr-x 1 eed3si9n wheel 901 Jan 3 12:20 Greeting.class* -rwxr-xr-x 1 eed3si9n wheel 3079 Jan 3 12:20 Hello$.class* -rwxr-xr-x 1 eed3si9n wheel 738 Jan 3 12:20 Hello$delayedInit$body.class* -rwxr-xr-x 1 eed3si9n wheel 875 Jan 3 12:20 Hello.class* ```
eed3si9n
added a commit
to eed3si9n/sbt
that referenced
this pull request
Jan 25, 2021
Fixes sbt#6235 In sbt 1.4.0 (sbt#5344) we started wiping out the timestamps in JAR to make the builds more repeatable. This had an unintended consequence of breaking Play's last-modified response header (playframework/playframework#10572). This adds a global setting called `packageTimestamp`, which is initialized as follows: ```scala packageTimestamp :== Package.defaultTimestamp, ``` Here the `Package.defaultTimestamp` would pick either the value from the `SOURCE_DATE_EPOCH` environment variable or 2010-01-01. To opt out of this default, the user can use: ```scala ThisBuild / packageTimestamp := Package.keepTimestamps // or ThisBuild / packageTimestamp := Package.gitCommitDateTimestamp ``` Before (sbt 1.4.6) ------------------ ``` $ ll example total 32 -rw-r--r-- 1 eed3si9n wheel 901 Jan 1 1970 Greeting.class -rw-r--r-- 1 eed3si9n wheel 3079 Jan 1 1970 Hello$.class -rw-r--r-- 1 eed3si9n wheel 738 Jan 1 1970 Hello$delayedInit$body.class -rw-r--r-- 1 eed3si9n wheel 875 Jan 1 1970 Hello.class ``` After (using Package.gitCommitDateTimestamp) -------------------------------------------- ``` $ unzip -v target/scala-2.13/root_2.13-0.1.0-SNAPSHOT.jar Archive: target/scala-2.13/root_2.13-0.1.0-SNAPSHOT.jar Length Method Size Cmpr Date Time CRC-32 Name -------- ------ ------- ---- ---------- ----- -------- ---- 288 Defl:N 136 53% 01-25-2021 03:09 888682a9 META-INF/MANIFEST.MF 0 Stored 0 0% 01-25-2021 03:09 00000000 example/ 901 Defl:N 601 33% 01-25-2021 03:09 3543f377 example/Greeting.class 3079 Defl:N 1279 59% 01-25-2021 03:09 848b4386 example/Hello$.class 738 Defl:N 464 37% 01-25-2021 03:09 571f4288 example/Hello$delayedInit$body.class 875 Defl:N 594 32% 01-25-2021 03:09 ad295259 example/Hello.class -------- ------- --- ------- 5881 3074 48% 6 files ```
This was referenced May 8, 2022
7 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Ref sbt/io#279
This is a resend of #5233
original description
This is a continuation of the work in sbt/io#58 , which I decided to get back into partly because Maven has been adding features to the core plugins, and it would be good to be on par :).
As discussed in sbt/io#58, we should be careful about the risk of breaking incremental builds - I haven't found any such issues yet though. Note that this PR only affects the timestamps of entries in jars: timestamps of classes on the filesystem and the filesystem timestamp of the jar itself are not affected.
Discussing this with @eed3si9n, there could be special-purposes build configurations that (for example in order to avoid deleting/creating many files) compile classes directly into a jar rather than first to individual files. I haven't seen this configuration yet, but I suspect it wouldn't use
Defaults.packageTask, but a customPackage/IOinvocation. Any custom build configuration that requires non-static timestamps can passNoneas thetimeparameter to keep the original behavior.This PR requires sbt/io#279