AWS S3 Backend uses "ENDPOINT_URL" not "PUBLIC_URL" setting

[SatrosDev]

All documentation lists "PUBLIC_URL" as the required setting, however the S3 backend is using "ENDPOINT_URL". This causes the S3 backend to not work unless you include an "ENDPOINT_URL" setting.

[meeb]

These are different settings. PUBLIC_URL is used to test assets uploaded to the bucket, so the front-end hostname. ENDPOINT_URL is used to specify a different endpoint to interact and upload assets to and is used for when you need to override the endpoint to upload to a different non-AWS but S3 compatible service (e.g Backblaze or Cloudflare's R2).

You would make API requests and upload to https://some-region.blah.service.com as the ENDPOINT_URL but the PUBLIC_URL would be https://example.com/.

The default for the S3 backend should not require ENDPOINT_URL be set, the default should just be whatever boto3 defaults to (which is AWS). If you're experiencing an issue where publishing doesn't work when you do not specify an ENDPOINT_URL that's a bug.

What happens when you don't set ENDPOINT_URL with the S3 backend and run a distill-publish?

[SatrosDev]

Sorry, I dug into the issue a little more and figured out the real problem. Currently if you don't have "ENDPOINT_URL" set you will get an error related to having an invalid endpoint when trying to publish. However even if you set it you will get an error when list_remote_files() is called (though this is probably because the endpoint I was passing isn't a real endpoint because I confused it with public_url).

When creating the boto3 client, the endpoint setting is being passed along to boto3, but if there is no setting it will pass an empty string to boto3 which is not valid. If you pass None instead it works as expected.

In the amazon_s3.py authenticate function
endpoint_url = self.options.get('ENDPOINT_URL', '')
should most likely be:
endpoint_url = self.options.get('ENDPOINT_URL', None)

[meeb]

Oh, yes you are correct. I actually noticed this when I merged the PR but obviously forgot to actually commit that change:

#92 (comment)

[SatrosDev]

Thank you!

[meeb]

Fixed in the above, will be merged into the next release. You can just set 'ENDPOINT_URL': None in your DISTILL_PUBLISH options for now as a work-around. I expect you know that just mentioning for anyone else who finds this issue.