[For devops] S3 bucket to save question files

[bqle]

OHQ would like an S3 bucket that allow us to store files that users attach to a question.
We would like to be able to:

• From backend, save files to the bucket like how penn-mobile does their images
• Delete files from the S3 on request
• No file updating is necessary (only upload & delete)
• From frontend, retrieve the file given the file url

[joyliu-q]

TLDR: For this issue (to store files users attach), I don't see any problems with it. We can create the bucket and get your credentials set up, and you can store the files you want in it :D

Considerations

• The uploaded file size should be capped.
• Feel free to organize the S3 bucket however you want, but it may be good idea to create a subfolder dedicated to storing user questions instead of putting them in root. Just in case you want to use the bucket for other things in the future.

Implementation

After investigating the current set-up, different products appears to have S3 buckets manually created & configured for them, along with other S3 buckets.

What that means is that the quick & easy solution is to manually create an S3 bucket for you guys, but it may be time for us to improve the organization of the S3 buckets and switch over to managed instead of manual.

Question for y'all (@bqle)

How fast do you guys need this to be set-up? If you want this to be done soon in order to start working on this project, I can do the manual S3 creation asap and switch over later. If this is just a consideration for a project starting next semester, maybe we can figure out cleaning up the S3 buckets & access first.

[bqle]

Cheers, thank you Joy! We will probably implement file size cap but probably through backend logic And thank you for suggesting a subfolder; we will likely follow your suggestion.
To answer your last question, I think we would like to work on this feature over the summer, so could you do the manual S3 set up on this one if that won't cause extra hassle for the switch over next sem?

[joyliu-q]

Hey Ben! I created a manual bucket for you called ohq, but there are a few differences from this bucket compared to other products.

Other buckets are publicly accessible because the things stored on there (e.g. user profile pictures, general assets/icons) are not very sensitive, but I imagine in this case, storing the user questions in a publicly accessible bucket may be a safety concern.

• What are your thoughts on this? At the end of the day, user questions are viewable by professors and TAs on the history regardless, and honestly I don't think students will mind the files they upload being in a public space. At the same time, this seems potentially questionable.

Accessing S3 bucket

List/read/write access: I created a new IAM account called ohq, attached a respective S3 policy, and populated our secrets manager with its corresponding credentials. Now, OHQ in production should be able to access S3 through the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY env variables.

Regardless of public access, you should still be able to use those credentials from Vault to do what you want. Constance & Kevin should both have Vault access and they can get you to add those secrets to your local .env file (and let your team know so they can update theirs as well!). <-- Feel free to redirect them to this thread.

Let me know if you have trouble accessing after getting the access key creds set-up. Right now I made the bucket is not public but anyone with appropriate permissions can grant public access to objects, similar to the set-up of penn-mobile-portal. To integrate this with your product, feel free to reference how portal is doing it (which is the most similar to your use case), as well as other products like courses & clubs.