Skip to content

Amazon S3, Cloudfront & Rails Assets

Jump to bottom
Dylan Fisher edited this page Jun 8, 2022 · 60 revisions

Amazon S3

Create a new bucket

US East (N. Virginia) region by default

Permissions:

  • Block public access to the bucket (the default Amazon recommendation). An origin access identity (OAI) will be used to allow viewing all S3 uploads via the Cloudfront host.

Add a project tag to help with billing organization.

Set CORS policy

[
    {
        "AllowedHeaders": [
            "Authorization",
            "x-amz-date",
            "x-amz-content-sha256",
            "content-type"
        ],
        "AllowedMethods": [
            "GET",
            "POST",
            "PUT"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [
            "ETag"
        ],
        "MaxAgeSeconds": 3000
    },
    {
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "MaxAgeSeconds": 3000
    }
]

Create a new user with programmatic access

Add an inline policy that gives full S3 permissions to this user for a single bucket:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "s3:ListAllMyBuckets",
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Action": "s3:*",
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::my-bucket-name",
                "arn:aws:s3:::my-bucket-name/*"
            ]
        }
    ]
}

Shrine Configuration

Forest will configure Shrine to use the following subdirectories of your S3 bucket to store media files:

  • media used to store uploaded MediaItem attachments
  • cache used to store temporarily cached direct S3 file uploads

Configure a Lifecyle rule on your S3 bucket to automatically clear old files in the cache directory (see Clearing Cache for more information). Use the following options in your Lifecyle rule:

  • Lifecycle rule name: Expire shrine cache
  • Filter type, prefix: cache/
  • Lifecycle rule actions:
    • Check Expire current versions of objects
    • Expire current version of object: after 7 days from creation
    • Check "Delete expired delete markers or incomplete multipart uploads" 7 days from creation

Image Handler

Deploy a new Serverless Image Handler using the AWS CloudFormation Template.

  • First create an S3 bucket for your media my-project-bucket
  • CorsEnabled: Yes
  • CorsOrigin: * for testing, otherwise specify the domain name. # TODO: can we specify multiple domains?
  • SourceBuckets: my-project-bucket
  • DeployDemoUI: No
  • LogRetentionPeriod: 1 # TODO: how important is the ability to retain these logs?
  • AutoWebP: false

ActiveJob

In your Rails app, specify an ActiveJob queue adapter as appropriate, and add the appropriate gem to your Gemfile.

For simple sites that can risk losing jobs and don't need dedicated worker dynos: config.active_job.queue_adapter = :sucker_punch

For more complex sites with dedicated worker dynos: config.active_job.queue_adapter = :sidekiq

And in your Procfile: worker: bundle exec sidekiq -c 5 -q default -q mailers

You'll also need to configure Redis to use Sidekiq. If using Heroku, just install the Heroku Redis add-on and Sidekiq will pick up the Redis instance via the REDIS_URL ENV variable set by Heroku.

You may need to fix TLS SSL issues related to Sidekiq on Heroku. See this for more info https://stackoverflow.com/a/65845804/1986068.

CloudFront

Create new distribution for Heroku

Origin domain name my-new-app.herokuapp.com

Origin ID Heroku

Origin Protocol Policy Match Viewer

Allowed HTTP methods: GET, HEAD, OPTIONS

Cache key and origin requests:

  • Cache policy: CachingOptimized policy
  • Origin request policy: CORS-CustomOrigin

Create a custom domain name for the distribution

  • Add a CNAME record for the custom domain name that points to the Cloudfront distribution. e.g. CNAME media.mywebsite.com -> abcde12345.cloudfront.net
  • TODO: troubleshoot and/or verify this works

Create an origin for the S3 bucket

  • S3 bucket access: Choose Yes use OAI (bucket can restrict access to only CloudFront) and press the "Create new OAI" button, and check the update bucket policy to allow access.

Create two behaviors for the S3 bucket

  • Path Pattern: media/*
  • Path Pattern: forest/*

Cache key and origin requests for both behaviors:

  • Cache policy: CachingOptimized policy
  • Origin request policy: CORS-S3Origin

Rails assets

Add rack-cors gem to gemfile:

gem 'rack-cors', require: 'rack/cors'

# application.rb

config.middleware.insert_before 0, Rack::Cors do
  allow do
    origins '*'
    resource '*', headers: :any, methods: [:get, :post, :options]
  end
end

Configure Rack to gzip static content

# application.rb

config.middleware.insert_after ActionDispatch::Static, Rack::Deflater

Add the following to your Rails credentials:

s3:
  bucket: 'my-bucket-name'
  region: 'us-east-1'
  access_key_id: 'YOUR_KEY_HERE'
  secret_access_key: 'YOUR_SECRET_HERE'
asset_host: https://abc123.cloudfront.net
host_origin: https://my-app.herokuapp.com
image_handler_host: https://def456.cloudfront.net

Simple Email Service (SES)

When setting up SES you must request production access. Use the following email template to help avoid additional support request correspondence with Amazon.

How often we plan to send email: this is based on user transactions – when a user creates an account, when they verify their email address, when they request a reset password email.

How we manage our recipient list: users are stored in our database and removed when a user destroys their account.

How we manage bounces: we monitor bounces within our web application and flag users with high bounce rates for review by admins. Additionally, we use Google reCAPTCHA to avoid user signups by spam account.

We do not provide a method to unsubscribe from emails because all mail sent from our SES account is transactional and directly related to the user's own interactions with their profile.

If your request is not thorough enough, you may receive the following email asking for additional clarification:

Thank you for submitting your request to increase your sending limits. We are unable to grant your request at this time because we do not have enough information about your use case.

If you can provide additional information about how you plan to use Amazon SES, we may be able to grant your request. In your response, include as much detail as you can about your email-sending processes and procedures.

For example, tell us how often you send email, how you maintain your recipient lists, and how you manage bounces, complaints, and unsubscribe requests. It is also helpful to provide examples of the email you plan to send so we can ensure that you are sending high-quality content.

You can provide this information by replying to the correspondence, in the console link below. Our team provides an initial response to your request within 24 hours. If we’re able to do so, we'll grant your request within this 24-hour period. However, if we need to obtain additional information from you, it might take longer to resolve your request.

Thank you for contacting Amazon Web Services.

Clone this wiki locally