Asset fingerprinting and SRI

[RealOrangeOne]

It'd be nice if there were a way of accessing a fingerprint or hash of static files, for a more reliable way of caching assets.

I've given this a bit of thought, and probably need some input.

Ideally, it'd just be an additional function, say get_static, which returned a structure which contained the path, fingerprinted URL, hash, things like that. Because this requires the static files to exist before running, it would also serve as a link checker. However, What do we do about live reloading? When editing a static file, we'd either need to keep track of the linked resources per page, and only reload the linked pages, or just reload the whole site. (The latter is so much easier!)

I see no reason why this couldnt work for any static file in the site, eg images. It wouldnt play well with images after they've been processed, but static images should work fine!

[Keats]

The current timestamp is not a very good solution I agree.
I would probably just change the cachebust attribute of get_url to hash it though rather than having a new function

[RealOrangeOne]

That would work for fingerprinting the URL, but wouldnt generate the required SRI data. I could change the return type to an object, but not sure I want to make a breaking change to something so fundamental to existing sites. Hence the new function.

If you're happy for me to add a breaking change, I can do that too. Or, I could just remove the cachebust argument to get_url, so people have to use get_static when they want their cache busted

[Keats]

Do you have a link for SRI?

[RealOrangeOne]

https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity is a pretty good one. Or the spec at https://w3c.github.io/webappsec-subresource-integrity/, which is incredibly dry!

[dancek]

Is there still interest for this? I'd like hash-based cache busting. SRI is nice in theory too, but practically only useful for external resources (which zola couldn't hash during compilation).

[tcurdt]

IMO this is still relevant.

[Keats]

I would like a PR for that.

[dancek]

I'll try to make a PR within the next couple of days.

[mattly]

You know, I got all excited because I saw some activity on this ticket and punted on doing some work to my hugo site because I thought, yay, they're tackling this finally! Hugo's template system is just as insane as golang, and migrating my fledgling site would be a worthwhile yak shave.

But the PR that closed this issue does not provide the desired feature. It changes the cache busting mechanism. There's still no way for me to fill out the value of an integrity='{{ get_file_hash("styles.css")}}' attribute.

[Keats]

{{ get_file_hash("styles.css")}} if that's all that's needed then it's very easy to add! Does it need to be a specific hash (I'm thinking sha256) or anything is fine?

[dancek]

The SRI spec says SHA-256, -384 and -512 MUST be supported by browsers IIRC. But SHA-384 is generally recommended and used for some reason. I think it'd make sense to let the user choose between those three hashes.

[dancek]

I can make another PR for this. I didn't include it in the original one because I wasn't whether anyone would use it (and whether the project would want it).

[Keats]

Let's start with just SHA-384, unless they are all coming from the same crate.

[dancek]

They're all from the sha-2 crate which we already depend on.

[dancek]

While working on SRI support I realized I basically broke all cache-busting in my previous PR, as now get_url expects to find the file in content_path when it's actually in static/. Oops, sorry for that!

I've got get_file_hash implemented with the same issue. I'll send a PR to show what I'm up to, but I'll fix the static/ path once I'm sure what the best way to do it is.

[dancek]

get_file_hash has been merged and will be in the next release.

[Absolucy]

get_file_hash can't be used for SRI: it returns hex, the integrity attribute expects base64.

[Keats]

There is a base64_encode filter, would {{ get_file_hash(..) | base64_encode }} work?

[Absolucy]

get_file_hash returns hex tho, so you'd get the base64 of the hex?

[SkypLabs]

get_file_hash returns hex tho, so you'd get the base64 of the hex?

Correct, so get_file_hash cannot be used for SRI. This section in the documentation should be updated accordingly.

I think the current implementation of get_file_hash can still be useful in some situations so I would be in favour of adding another function like get_file_b64_hash for SRI. If you agree with me, I'm willing to work on the related PR.

[Absolucy]

Yeah, I agree.

[Keats]

Just make it an argument of the get_file_hash function?

[SkypLabs]

Just make it an argument of the get_file_hash function?

Why not!

Given that I don't see any scenario in which returning the raw binary value of the hash could be exploitable in Markdown / HTML, I suggest that the additional parameter, when set to true, makes the function return directly the base64-encoded binary value of the hash.

Something like:

// Returns 'fa6bad40f388d8f12525971a9d60a05797ee69b34bdb929b08a3c8ce1dc1c39116088286bf62e43b2435394f501964b1'.
get_file_hash(path="js/app.js", sha_type=384)

// Returns '+mutQPOI2PElJZcanWCgV5fuabNL25KbCKPIzh3Bw5EWCIKGv2LkOyQ1OU9QGWSx'.
get_file_hash(path="js/app.js", sha_type=384, base64=true)

What do you think?

[Keats]

Yep sounds good

[SkypLabs]

I'm working on the PR but there is something I don't get about the can_get_file_hash_* unit tests.

zola/components/templates/src/global_fns/mod.rs

Line 502 in 49b6087

create_file(&dir.join("app.css"), "// Hello world!")

According to this piece of code, the test functions compute the hash values of a file named app.css and containing the string // Hello world!. The problem is that when I create a file having the same name and content, I get different hash values.

For example with SHA256:

$ cat app.css
// Hello world!
$ sha256sum app.css
72d204026937df55cf1eec5fc77d8dbc323b7103b37b788b9a03e9357eddca5a  app.css

And according to the related unit test:

zola/components/templates/src/global_fns/mod.rs

Line 818 in 49b6087

"572e691dc68c3fcd653ae463261bdb38f35dc6f01715d9ce68799319dd158840"

Am I missing something here?

[Absolucy]

@SkypLabs

❯ printf '// Hello world!' | sha256sum -
572e691dc68c3fcd653ae463261bdb38f35dc6f01715d9ce68799319dd158840  -

❯ printf "// Hello world\!\n" | sha256sum -
72d204026937df55cf1eec5fc77d8dbc323b7103b37b788b9a03e9357eddca5a  -

it's just the newline that changes the hash

[SkypLabs]

I actually thought about it but strangely, when I added a new line with Vim, it didn't get the right hash value.

Thanks anyway. I know the issue comes from me now.

[SkypLabs]

PR created: #1339.

I'm updating the documentation.