Preserving app file mtimes at build time

[pclalv]

As of lifecycle#80, the lifecycle ensures that all app
files have the same predetermined mtime at build time, regardless of
the app file's actual mtime. It would be nice to have an option to
preserve app file mtimes at buildtime in order to support build steps
that use mtime to validate the cache and/or its contents.

Detailed use-case

I have a Rails app with a Sprockets asset pipeline. In
order to build images for the Rails app we run several it through
several buildpacks, including a Ruby buildpack and a Sprockets asset
compilation buildpack.

The most basic Sprockets asset compilation buildpack essentially just
needs to compile static assets from source files and write those
compiled assets to a known place. It can compile every asset from
scratch every time and generate the correct results.

However, compiling every asset from scratch every time is not
efficient. We would like to enable our Sprockets buildpack to be able
to use a cache and thereby only compile assets that need to be
compiled. Sprockets is built with support for caching; as with GNU
make, a source file's mtime is the underlying mechanism for
determining whether a cached compiled asset is valid. Because every
source file has the same mtime on every build, a Sprockets buildpack
will compile an asset from a source file exactly once and never
compile it again, due to the source file mtime being the same every
every time.

[ekcasey]

@pclalv When exporting image layers we want to normalize all file times so that we can reuse existing blobs and avoid expensively re-uploading layers to the registry when the layer file content has not changed.

Fortunately, if the issue you are experiencing is caused by source file mtimes in the app dir during build, this is not incompatible with normalized times at export. It appears that pack is unnecessarily normalizing mtimes when copying the app from the filesystem into the build container. pack could preserve the mtimes during app copy allowing them to be set to real values while your buildpack runs. The exporter should still normalize all date times when creating the final image.

[zmackie]

@ekcasey Am I right in thinking this is an issue in pack and not the lifecycle?

[ekcasey]

@zmackie I believe so, I'll move it over.

[pclalv]

Ah yeah, my mistake. While we invoke the lifecycle directly in our remote builds, I usually test our buildpacks locally with pack, and I mistakenly assumed that lifecycle builds and pack builds would be equivalent in this regard. It would be nice to smooth out this difference, though.

[ameyer-pivotal]

Hi @pclalv, thanks for reporting this! We're in agreement with @ekcasey in that we should be able to preserve source file mod times during the build with no negative side effects (they will still get normalized before exporting the final image). This will resolve your issue and also #184.

[ameyer-pivotal]

Hi @pclalv, we've got PR #505 up for this. Feel free to take a look at that branch and try it out.