Wrong image rotation because of EXIF data when using Thumbor filters

[marco910]

Describe the bug
When resizing images using thumbor filters in the URL, some images get flipped to the left side in some browsers (see additional context). This only happens to images that were processed by the serverless image handler, not to the original image which is served directly from the S3 bucket.
I checked the images where the issue occurs and saw that all those images have an orientation value set in their EXIF metadata (e.g. 6 when they are wrongly flipped to the left side instead of portrait).
On my serverless image handler deployment is the AUTO_WEBP option enabled, which automatically returns WebP images when the browser supports it. When adding the format(jpg) thumbor filter to the request, a JPG is returned and the image rotated correctly (even with the EXIF data inside it).
When stripping the EXIF data using the strip_exif thumbor filter, the image is also rotated correctly no matter of it's a JPG or WebP.

Expected behavior
Images should always be rotated correctly no matter if an orientation is set in the EXIF data or the output format is a WebP, JPG, PNG or similar.

Please complete the following information about the solution:

• Version: v6.2.5
• Region: eu-central-1
• Was the solution modified from the version published on this repository? => No
• If the answer to the previous question was yes, are the changes available on GitHub? => No, because not modified
• Have you checked your service quotas for the sevices this solution uses? => Yes
• Were there any errors in the CloudWatch Logs? => No errors in the log

Additional context
The images are only flipped to the left side in Chromium browser. In for example Safari everything looks fine.

I'm not sure if this is a browser issue, serverless image handler issue, or WebP issue.

[simonkrol]

Hi @marco910,

Can you provide an example of a request (you can redact the distribution URL and filename if you'd like) that sees this happen?

If you download the image and open it locally through Mac preview or a similar Windows tool, does the image show properly? My intuition is that you're running into a similar issue as #495, where some browsers don't support rotational metadata for WebP images. So even though the metadata is being maintained in the result, the browser doesn't know what to do with it.

One reason Safari likely works is that it doesn't support WebP images, so SIH is returning the image in the same format as it exists in the S3 bucket.

Let me know,
Simon

[marco910]

@simonkrol

Do you mean a screenshot from an example request or the request headers?

When downloading both the original image (which is display correctly in the browser) and the flipped image to my Mac and opening both in the Preview app, both images are rotated correctly.
So, this is maybe related to the missing support for rotational metadata support on WebP. Is there any information about when to expect that for Chrome?

It was also my though that it works in Safari because of the missing WebP support. Both Firefox and Chrome (which support WebP) are having the issue.

[simonkrol]

Hi @marco910,
If using a base64 encoded request, I'd be interested in the Javascript object that was encoded into base64. If using Thumbor, it would be the path part of the url (Eg: filters:abc()/filters:def()/imageKey.jpg)

It appears that the issue is tracked in chrome under https://bugs.chromium.org/p/webp/issues/detail?id=445 , though it has been closed as wontfix, so we likely won't see support anytime soon.

Unfortunately a fix on our end likely falls outside the scope of the solution, and could introduce a behaviour change for other customers. If your use case allows it, you could consider disabling AutoWebP, or include filters:strip_exif() (Thumbor) or edits: {rotate: null}} (Base64 encoded) in your requests. This inclusion instructs Sharp to manually rotate the image to match what the orientation value indicates, then remove that value from the image entirely.

Let me know if that works for you,
Simon

[marco910]

Hi @simonkrol

We're only using Thumbor filters vie the URL. Here are a few examples:

• default requests (with flipped image): https://img.mydomain.com/500x0/subfolder/image.jpg
• as JPG (correct orientation): https://img.mydomain.com/500x0/filters:format(jpg)/subfolder/image.jpg
• without EXIF (correct orientation): https://img.mydomain.com/500x0/filters:strip_exif()/subfolder/image.jpg

We experienced the issue very rare and only with portrait images taken with smartphones. After exporting the images for example in Photoshop, the EXIF data get's removed and the image will get returned correctly from SIH.

I totally understand that you can not make any changes to the current solution. That's not what I want either because it obviously a browser issue.
Unfortunately, we can not use filters:strip_exif() on all requests because this would also remove other metadata that we need.

We agreed with internally and without client to prevent using portrait images or stripping EXIF data before using them through SIH. This prevents the flipped images in some browsers.

Thanks for your help and the awesome AWS solution you built with the SIH!

[simonkrol]

Hi @marco910,

It might be worth trying out strip_exif() and seeing if the metadata you need sticks around. Contrary to the naming, it should only affect the Orientation exif tag.

Thanks for using SIH :)
Simon

[simonkrol]

Hi @marco910 ,
My apologies for the ping(and the post on a closed issue). We're looking to gain some insight into how our most active Github customers are using the Solution and how we can better support those use cases. Would you be interested in having a half hour call with me some time over the next few weeks? Let me know here if you're interested and I'll get the process underway.

Thank you,
Simon

[marco910]

Hi @simonkrol ,

Thanks for reaching out. I would really appreciate giving you some insights and feedback about the solution. I'll send you a message on LinkedIn and then we can schedule a call.

Marco