File Upload Support

[glenn-sorrentino]

A top request is to allow file uploads. To enable this feature while protecting ourselves legally and from incurring increased costs due to cloud storage, this should only be enabled for the Personal Server.

Acceptance Criteria

1. Users may access files settings through a dedicated tab enabled only for Personal Server.
2. The feature is disabled if the user has not uploaded a PGP key.
3. Toggling on the feature reveals more settings, including setting size limits and uploaded a file-specific PGP key.
4. Size limits may be set using a numerical value and unit type chosen from a <select> menu. "100" and "GB" for example.
5. A user must toggle on the file-specific PGP option.

Design

[brassy-endomorph]

Can you make a mockup for what submission of a file looks like? I'm assuming just a file picker with the normal message

[brassy-endomorph]

Also for simplicity can we have file upload sizes restricted to megabytes only for now since kilobytes is useless and gigabytes is more than we want?

[glenn-sorrentino]

Can you make a mockup for what submission of a file looks like? I'm assuming just a file picker with the normal message

Yeah for now just use the native HTML component. We can fine tune it once working.

[glenn-sorrentino]

Also for simplicity can we have file upload sizes restricted to megabytes only for now since kilobytes is useless and gigabytes is more than we want?

I suspect the most likely file sizes will be MB and GB, so let's initially include those. We're not going to be turning this on for our main Hush Line instance. At least not right away.

[brassy-endomorph]

We're not going to be turning this on for our main Hush Line instance

So we need feature flags?

I suspect the most likely file sizes will be MB and GB

If this is a tip line and not a leak line, can I recommend against GB? Also multi-GB uploads to S3 are more complicated, so I'd honestly say capping at at 500 MB is reasonable, and even as low as 100 MB (hardcoded I mean) would not be overly restrictive either.

[glenn-sorrentino]

We're not going to be turning this on for our main Hush Line instance

So we need feature flags?

I suspect the most likely file sizes will be MB and GB

If this is a tip line and not a leak line, can I recommend against GB? Also multi-GB uploads to S3 are more complicated, so I'd honestly say capping at at 500 MB is reasonable, and even as low as 100 MB (hardcoded I mean) would not be overly restrictive either.

I think we should add a toggle in Settings for the admin to decide if files should be enabled or not. If all of the env variables are present for their S3 or whatever, the toggle appears.

[glenn-sorrentino]

We're not going to be turning this on for our main Hush Line instance

So we need feature flags?

I suspect the most likely file sizes will be MB and GB

If this is a tip line and not a leak line, can I recommend against GB? Also multi-GB uploads to S3 are more complicated, so I'd honestly say capping at at 500 MB is reasonable, and even as low as 100 MB (hardcoded I mean) would not be overly restrictive either.

500 MB sounds good.

[brassy-endomorph]

I think we should add a toggle in Settings for the admin to decide if files should be enabled or not. If all of the env variables are present for their S3 or whatever, the toggle appears.

We really will need to do #637 after this because some of these configs are going to be very annoying to deal with otherwise.

Also this leads to another question because right now the options I have working are:

1. S3 backend for uploading files
2. Local FS backend for uploading files
3. Error otherwise

What you're proposing sounds like

1. S3 backend for uploading files
2. Optionally no backend at all which disables things in the UI / app itself

If we go with this option, it forces dev to use S3 for dev/testing, otherwise we have to add an option 3 to that (FS backend) which changes your original statement from "Valid S3 config implies admins can enable file uploads" to "Any valid configured backend implies admins can enable file uploads." Which is probably what we want because it makes typical dev/test a little more lightweight.

[brassy-endomorph]

In discussions about this, it wasn't mentioned that we support rPi physical devices that run Hushline, so we 100% have to have a driver for the FS and not just S3.

IMO it should be like this:

• some features require a storage driver
• if any such feature is enabled (via env var flag) but no storage driver is enabled, allow endpoints to 503 and log messages for the sys admin. perhaps flash messages for admins when they log in too. or have some other in-app notification strategy
• storage driver is set via env var and can be file-system, s3 or none (default).
• related configs are only pulled and parsed based off the driver's env var