Cloudfront cache not cleared after uploading new db-dump.tar.gz

[alex]

Current Behavior

When a new db-dump.tar.gz, cloudfront may continue to cache the old one, leading to pulls of stale data.

Expected Behavior

Once new data is uploaded, cloudfront's cache should be cleared so users get the most recent dump.

Steps To Reproduce

The following Python script consistently shows it for me.

>>> import requests
>>> session = requests.session()
>>> response = session.head("https://cloudfront-static.crates.io/db-dump.tar.gz")
>>> response.raise_for_status()
>>> response.headers
{'Content-Type': 'application/gzip', 'Content-Length': '306589988', 'Connection': 'keep-alive', 'Date': 'Mon, 19 Jun 2023 18:16:39 GMT', 'Last-Modified': 'Mon, 19 Jun 2023 02:08:04 GMT', 'ETag': '"a89773f47d7ee8e190b5e9bff90dae4b"', 'x-amz-server-side-encryption': 'AES256', 'x-amz-version-id': 'YTA5l_UzT5W9RKf.ktEG5XNiwbYWoN9A', 'Accept-Ranges': 'bytes', 'Server': 'AmazonS3', 'X-Cache': 'Hit from cloudfront', 'Via': '1.1 b3303b43ad528209e1d81e70b2fc5964.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'ORD58-P4', 'X-Amz-Cf-Id': 'QXB3D97htgNJ8y7_ZfkLekm6tD_f7iwRx7c-eQE0DxTY_6YLLD_RjA==', 'Age': '70235'}

Note the X-Cache headers indicating a CF hit combined with the Last-Modified header which shows that it's yesterdays data.

Environment

No response

Anything else?

No response

[alex]

Thanks! I guess the next cache clear will be tomorrow, when the dump is re-dumped, right?

[Turbo87]

once it is deployed, yes :)

[alex]

Ah, of course. I got used to CD :-)

…

On Thu, Jun 22, 2023 at 11:02 AM Tobias Bieniek ***@***.***> wrote: once it is deployed, yes :) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

-- All that is necessary for evil to succeed is for good people to do nothing.

[Turbo87]

I got used to CD

yeah, we use it for the staging env, but promoting that to production is still a manual step due to database migrations and stuff like that.

[enricozb]

Not trying to bump a closed issue thread, but I'm also not sure if this is related. I'm being served a stale version of the dump when I don't provide a Accept-Encoding header:

Setting Accept-Encoding to gzip, deflate, br grabs me the new data, but I'm actually not sure if that's just related to how cloudfront is caching the request.

Is there any way around cloudfront's cache or should I just expect to possibly have ~48-hour stale data?

[LawnGnome]

That's weird. @jdno, do you have any insight here?

[jdno]

Hm that's weird. As far as I understand our configuration and CloudFront's documentation, this shouldn't happen. I can't reproduce it with today's database dump, but it's probably a good idea to create a new issue for this so that we can track this separately.

[LawnGnome]

Opened as rust-lang/simpleinfra#601.