Skip to content

Conversation

@Mark-Simulacrum
Copy link
Member

#381 will also need to be merged, and this is based on #384.

@QuietMisdreavus
Copy link
Contributor

What was the motivation for removing the static_files handler? It's still useful for those who aren't using S3, who are just setting up (since the base docs.rs pages depend on an older version of the rustdoc stylesheets, which at the moment need to be downloaded from an existing host of docs.rs).

src/db/file.rs Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it would be useful to run the database query anyway, and instead of switching on content = 'in-s3', switch on whether the row exists in the table? I know it makes no difference for the production service, but it feels more resilient for anyone who hosts their own version of it and transitions to S3 after being online for a while.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer not to, as it does add some amount of overhead and to an extent makes the code more complex. move-to-s3 should be the preferred route for migrating local installations to S3.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, i'm just thinking about the period between saving S3 credentials into the environment and the moment cratesfyi database move-to-s3 finishes. Any calls to the website in that time will fail because we won't be looking at the database for the files.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true. move-to-s3 is no longer an operation that can run in the background. Since it's likely to never be used again outside local installations though that seems fine -- and if in the future there is a desire to make that happen the code change isn't really that bad.

@Mark-Simulacrum
Copy link
Member Author

Mark-Simulacrum commented Jul 25, 2019

I primarily removed the static files handler since independent of using S3 or the DB, it seems odd to store some files on disk and some on S3. There should be no reason to differentiate, IMO; in particular I always get annoyed when standing up a new instance as the new files need to basically be copied from production which isn't great. I can re-add it back and configure it on the staging server if you'd like, but I'd rather move us towards solely using S3/DB storage.

@QuietMisdreavus
Copy link
Contributor

I would be more willing to ditch the static_files handler if i hadn't had trouble using cratesfyi database add-directory to manually add files to the database. The last time i tried (when i was trying to load settings.js files for previous versions of rustdoc) it exited successfully but didn't actually add the files to the database. I don't know if i just wasn't using it correctly, but in the absence of knowing that would work i would rather keep static_files around.

@Mark-Simulacrum
Copy link
Member Author

I've dropped the commit removing static_files. I realized that prod can just keep those files in S3, as before, since we check the database/S3 prior to looking at static files.

Copy link
Contributor

@QuietMisdreavus QuietMisdreavus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good! Thanks so much.

@QuietMisdreavus QuietMisdreavus merged commit 13e1ce9 into rust-lang:master Jul 25, 2019
@Mark-Simulacrum Mark-Simulacrum deleted the migration-fixes branch July 25, 2019 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants