Implement etag support for category files

[bdraco]

Since I couldn't come with a solution that avoided checking the files, we can at least prevent re-downloading them if they haven't changed which means refreshing the mobile app has to download ~160 bytes per file instead of the whole file which still makes a big difference.

Retains the max-age=0 and forces revalidation.

Fixes #1653

2020-11-16 05:12:31 DEBUG (MainThread) [custom_components.hacs] Serving simple-thermostat/simple-thermostat.js from /config/www/community/simple-thermostat/simple-thermostat.js with etag 0x6cda4805 (not-modified)
2020-11-16 05:12:31 DEBUG (MainThread) [custom_components.hacs] Serving lovelace-fold-entity-row/fold-entity-row.js from /config/www/community/lovelace-fold-entity-row/fold-entity-row.js with etag 0xd548b9d3 (not-modified)
2020-11-16 05:12:31 DEBUG (MainThread) [custom_components.hacs] Serving lovelace-card-tools/card-tools.js from /config/www/community/lovelace-card-tools/card-tools.js with etag 0x444c2ffc (not-modified)
2020-11-16 05:12:31 DEBUG (MainThread) [custom_components.hacs] Serving lovelace-slider-entity-row/slider-entity-row.js from /config/www/community/lovelace-slider-entity-row/slider-entity-row.js with etag 0x1c2fe06c (not-modified)
2020-11-16 05:12:31 DEBUG (MainThread) [custom_components.hacs] Serving garbage-collection-card/garbage-collection-card.js from /config/www/community/garbage-collection-card/garbage-collection-card.js with etag 0x3fd15b8d (not-modified)
2020-11-16 05:12:31 DEBUG (MainThread) [custom_components.hacs] Serving mini-graph-card/mini-graph-card-bundle.js from /config/www/community/mini-graph-card/mini-graph-card-bundle.js with etag 0x9d487a51 (not-modified)
2020-11-16 05:12:31 DEBUG (MainThread) [custom_components.hacs] Serving search-card/search-card.js from /config/www/community/search-card/search-card.js with etag 0xb87350ef (not-modified)
2020-11-16 05:12:31 DEBUG (MainThread) [custom_components.hacs] Serving battery-state-card/battery-state-card.js from /config/www/community/battery-state-card/battery-state-card.js with etag 0xd0b41886 (not-modified)
2020-11-16 05:12:31 DEBUG (MainThread) [custom_components.hacs] Serving lovelace-card-templater/lovelace-card-templater.js from /config/www/community/lovelace-card-templater/lovelace-card-templater.js with etag 0xf30f0e11 (not-modified)

[bdraco]

Had an opportunity to production test this since lovelace card tools updated and verified its working as expected

[quthla]

This massively slows down loading of the frontend by more than 5 secs when served by an Raspberry Pi 3.

Is the computation of the etag so heavy? Maybe just stick with sending 304 when received last modified equals to local last modified? Had to revert this for now and load time is again down to 1 sec.

[bdraco]

This massively slows down loading of the frontend by more than 5 secs when served by an Raspberry Pi 3.

Is the computation of the etag so heavy? Maybe just stick with sending 304 when received last modified equals to local last modified? Had to revert this for now and load time is again down to 1 sec.

Is your system cpu bound? On my Rpi4, the time to hash the file is less than 1ms

How long does it take to calculate the fnvhash of a 1MiB string on your system?

How long does it take with the built in python hash function? We could probably switch to this instead since etag don't have strong format requirements and it would happen in c code instead which should be much faster if it's the hash performance.

[quthla]

Can you post a gist with the benchmarks you need so I can run it on my Pi?

And you don't like the idea of just relying on last modified header?

[bdraco]

@quthla Its probably easier to try this PR: #1744

[quthla]

Much better now but why are you enclosing the hash in quotes?

[bdraco]

Can you post a gist with the benchmarks you need so I can run it on my Pi?

And you don't like the idea of just relying on last modified header?

If-Modified-Since has trouble with going back in time when restoring from backups which makes hashing the file contents generally preferred.

[bdraco]

Much better now but why are you enclosing the hash in quotes?

Didn't work for me in Firefox until I did, and it appears it's what mozilla wants: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag

[quthla]

It seems nginx won't cache the item if max age is set to 0.

[bdraco]

It seems nginx won't cache the item if max age is set to 0.

That behavior is existing. Please consider sending a PR to address.

[quthla]

https://trac.nginx.org/nginx/ticket/1182

There's a discussion about that behavior. Maybe worth setting the header mentioned?