Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Persist rendered picture variants #1882

Merged
merged 22 commits into from
Jul 17, 2020
Merged

Conversation

tvdeyen
Copy link
Member

@tvdeyen tvdeyen commented Jun 19, 2020

What is this pull request for?

Alchemy has image processing build in thanks to the wonderful Dragonfly gem.

The image processing though, happens on-the-fly, not up-front like in image attachment libraries like paperclip. This gives lots of flexibility and enables us to provide image cropping right in Alchemy's UI.

But it this has performance implications. Using Rack::Cache helps, but is not ideal for larger sites. Also larger sites will most likely use a cloud file storage like AWS S3 to store images anyway. Since Dragonfly has support for storing the master images on S3, it does not have built in support for persisting the result of a rendered image.

This PR introduces (besides a lot of refactoring regarding processing images) a new active record model Alchemy::PictureThumb that holds the signature to an image transformation process (a serialized Dragonfly::Job) and a uid to the resulting rendered image. This uid is then returned as the picture.url.

So, the first time a picture gets rendered it processes and encodes the image and stores the result in the configured Dragonfly::Store (default Dragonfly::FileDataStore). For normal installation this means the thumbnail is served from the public/pictures folder and usually picked up by your Webserver before it even hits the Ruby process.

For installations that use S3 as its datastore the image thumbnail will be stored in the S3 bucket and the public bucket URL returned as picture.url, so next time the picture url is hit it loads the image directly from S3.*

*) the latter needs the yet to be released alchemy-dragonfly-s3 gem.

Checklist

  • I have followed Pull Request guidelines
  • I have added a detailed description into each commit message
  • I have added tests to cover this change

@tvdeyen
Copy link
Member Author

tvdeyen commented Jun 19, 2020

TODO

  • Check why thumbnails get processed twice via Dragonfly.
DRAGONFLY: shell command: convert /var/folders/_l/jqwgn8pn6719dp9q3drb04dm0000gn/T/dragonfly20200619-17947-1ohq0cp.jpg -quality 85 /var/folders/_l/jqwgn8pn6719dp9q3drb04dm0000gn/T/dragonfly20200619-17947-4aypuv.jpg
DRAGONFLY: shell command: convert /var/folders/_l/jqwgn8pn6719dp9q3drb04dm0000gn/T/dragonfly20200619-17947-134vn17.jpg -quality 85 /var/folders/_l/jqwgn8pn6719dp9q3drb04dm0000gn/T/dragonfly20200619-17947-h6gawa.jpg

@tvdeyen tvdeyen self-assigned this Jun 19, 2020
@tvdeyen tvdeyen added this to the 5.1 milestone Jun 24, 2020
@tvdeyen tvdeyen force-pushed the remote-images branch 5 times, most recently from cd464cb to db7c83d Compare June 30, 2020 16:37
@tvdeyen tvdeyen force-pushed the remote-images branch 4 times, most recently from 1105b6b to 57315f6 Compare July 17, 2020 06:51
@tvdeyen tvdeyen force-pushed the remote-images branch 2 times, most recently from 3909333 to 5d29b87 Compare July 17, 2020 11:01
@tvdeyen
Copy link
Member Author

tvdeyen commented Jul 17, 2020

@mamhoff extracted several PRs

tvdeyen added 12 commits July 17, 2020 14:32
The Picture::Url module included transformations to create a variant out of an image. Let's seperate that in it's own class so we can extract url generation into a class.
We only need them in the PictureVariant
Meant as extension point for alternative URL representations of picture variants. For example images hosted on S3 would return the `remote_url` in its url class.
The default of the url format in dragonfly is `:name` what is an alias to `:basename.:ext`. We used to pass `Picture#name` (without the suffix) into the ragonfly `url` method. Since we now extracted the url generation out of the picture class it might be possible to pass a dragonfly attachment (generated by PictureVariant) into the url class. Leading to duplicated extensions in the url, because dragonflys name is basename+ext
In order to be able to cache images on remote storages we need to lazy process the image variant during URL generating.
We need those calculations in the picture model, but not in the picture variant class.
w/o this change we cannot test presence of validations
A persisted reference to a renderd picture
tvdeyen added 10 commits July 17, 2020 14:46
Whenver we generate an URL for a processed image (resized, cropped or encoded in a dfferent format) we persist a thumb and return its public uid as an url for performance reasons. Next time we hit the same image it will be served by the webserver instead.
Instead of repeating this rescue logi let's rescue that where we call the chain of things.
Dragonfly sets it to the image_file name anyway. Setting it to a staitc value even if the image_file might have another filename is not reflecting actual behavior.
With the Attachment.url_class it is a extension point for alternativ url classes (ie. for remote URLs from S3)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants