Inspect EXIF data for rotation - Image orientation/rotation is wrong for some images

[Haprog]

I tried compressing a photo taken with my Samsung Galaxy S8 (in portrait orientation) but the image shows up in wrong orientation (rotated 90 degrees) in https://squoosh.app/ and there also didn't seem to be any option to rotate the image.

The photo shows in correct orientation if I view it e.g. in image preview of macOS Finder. If I save the compressed image from squoosh then it will show up in wrong orientation also in other apps.

I suspect the issue is that squoosh doesn't take into account the EXIF Orientation information in the photo.

Here's the photo I had the issue with:

(Ps. looks like the image also appears in wrong orientation in this GitHub issue using Chrome at least, but if I click on it so that it opens in a new tab then it is shown in the correct orientation. It should be shown so that the text is oriented horizontally [easily readable].)

If I check the EXIF data of the image (with e.g. exiftool) I can see it has the correct orientation info:

$ exiftool test.jpg | grep Orientation
Orientation                     : Rotate 90 CW

Or checking with identify (from ImageMagick):

$ identify -verbose test.jpg | grep Orientation
  Orientation: RightTop
    exif:Orientation: 6
    exif:thumbnail:Orientation: 6

[kosamari]

@Haprog Thanks for creating the issue! definitely heard the same from attendees at ChromeDevSummit. We'll discuss with the team after a little break & travel back home.

[Haprog]

This might be useful for implementing a fix: https://github.com/exif-js/exif-js

[Haprog]

I just found some info about a kind of related issue in Chrome: https://stackoverflow.com/q/42401203/1362634

Though that's probably not relevant to implementing a fix for this.

[jakearchibald]

I'm working on a rotate option right now. The second phase of this would be to read exif data to set the rotation automatically. Thanks for filling this!

[jakearchibald]

We need to watch out for browsers that apply rotation automatically (Safari) and how we might detect this. I guess we could create a 2x1 JPEG with the orientation exif, and measure what size it appears on the page.

[Haprog]

That doesn't necessarily matter. When the user selects an image you could just immediately remove the EXIF data (or at least the Orientation property) and apply the rotation specified by it to the binary data and display that image. Then all browsers should display it correctly. At least this is not a concern for the modified/compressed images, right?

But if you want to be strict about being able to display the original image without any modifications, then I guess you could do something like you suggested (have a small test image with the Orientation property set and check the size) and if needed rotate the original image via e.g. CSS transformation?

[Haprog]

Or maybe there could be a separate polyfill for image-orientation: from-image; CSS. https://developer.mozilla.org/en-US/docs/Web/CSS/image-orientation ?

But you would anyway need to handle the case when/if removing the EXIF data when creating processed images.

[jakearchibald]

you could just immediately remove the EXIF data (or at least the Orientation property) and apply the rotation specified by it to the binary data and display that image

That might be a lot harder/slower than simply passing the image to the browser's decoder.

Or maybe there could be a separate polyfill for image-orientation: from-image;

That would be harder still. There isn't really a hook into style calculation and rendering. Not even sure if we can do this with the layout/paint APIs.

[surma]

Personally, I’d prefer to keep it simple. I’d vote for ignoring EXIF data completely and just offering a “Rotate by 90 degree” button.

[Haprog]

I agree that it would already improve the situation a lot if there just was a manual rotation tool. But the best UX would be to use the Exif rotation info properly.

But there are 2 problems.

1. Does the browser display the selected original image in correct orientation?
2. When the user saves the compressed image, is the rotation of the output image as expected (the same as what you saw in browser)?

If we would ignore issue 1 for now (some browsers may show it in correct orientation and some in incorrect orientation?) we probably need to test and at least make sure that the compressed image is always in expected orientation so that the user can see if the image need to be rotated or not. Just thinking about if there can be a situation that the original image is shown correctly (because browser rotates it) but compressed image may be in different orientation (if the process strips Exif data)?

[Haprog]

Btw I just tested my image (the one above) by loading it into https://squoosh.app/ in these browsers.

Tested on macOS Mojave 10.14.1

• Chome 70.0.3538.102 (64-bit)
• Chrome Canary 72.0.3614.0 (64-bit)
• Firefox 63.0.3 (64-bit)
• Firefox Beta 64.0b10 (64-bit)
• Firefox Nightly 65.0a1 (2018-11-18) (64-bit)
• Safari 12.0.1 (14606.2.104.1.1)
• Safari Technology Preview Release 70 (Safari 12.1, WebKit 14607.1.13)
• Opera 56.0.3051.104

Tested on Android 8.0.0 (Samsung Galaxy S8):

• Android Chome 70.0.3538.80
• Android Firefox 63.0.2

All of these browsers consistently show the image in wrong orientation (all the same way) when loaded into Squoosh. I also tested loading this GitHub issue in all of these browsers and it was exactly the same. All browsers tested show the image in the same way. In the <img> tag in an HTML document the image is always shown wrongly oriented. And in all of the browsers when opening the image directly in a new tab, only then the image is shown in correct orientation.

[Haprog]

Also tested in a virtual machine running Internet Explorer 11.0.9600.19155 (update versions: 11.0.90, KB4462949) on Win 8.1. The result was the same when viewing GitHub issue but the image in a separate tab also shows in wrong orientation. I was unable to test on https://squoosh.app/ on IE 11 since it seems to be unusable on IE 11. Drag & drop doesn't work (just opens the dragged image in the tab instead of selecting it into Squoosh) and clicking on "select an image" doesn't do anything on IE 11.

When opening https://squoosh.app/ on IE 11 with JS console open, there's a JS error "SCRIPT1028: Expected identifier, string or number\n File: squoosh.app, Line: 1, Column: 26111"

[jakearchibald]

@Haprog

When the user saves the compressed image, is the rotation of the output image as expected (the same as what you saw in browser)?

Our JPEG output doesn't include exif data, so the pixels are stored in the order they appear.

it seems to be unusable on IE 11

We're only targeting modern browsers.

[Haprog]

Now I also tested on Edge running on Win 10 in a virtual machine.
Edge 42.17134.1.0 (EdgeHTML 17.17134) seems to work the same as all the other "modern" browsers (and works with Squoosh) except it also shows the wrong orientation when image is opened in new tab (similar to IE11).

So it would seem that there is no need to check the browser behaviour regarding orientation of original image in <img> since all modern browsers seem to show it in the same way atm.

Giving a manual rotation option is good, but even better would be to just automatically apply the rotation specified by Exif Orientation.

[jakearchibald]

I've got a PR open for a "rotate" option #322 (https://deploy-preview-322--squoosh.netlify.com/).

[jakearchibald]

This has landed. But I'd still like to consider reading exif data and auto-applying the "correct" rotation.

[OliverJAsh]

We're hitting the same issue as this over at Unsplash.

The best solution seems to be along the lines of:

1. Read orientation from Exif, e.g. exif-js or https://stackoverflow.com/a/40867559/5932012
2. Apply orientation to image, e.g. https://stackoverflow.com/a/40867559/5932012

We need to watch out for browsers that apply rotation automatically (Safari) and how we might detect this.

And browsers with support for image-orientation: from-image, which is only Firefox at the time of writing. Although it's probably safe to assume that won't be the case, as long we don't specify that CSS!

@jakearchibald Have you got any ideas how to detect whether the image is already correctly rotated?

[tracker1]

Orientation is a bane of my existence it feels like at times... Especially since a lot of mobile camera apps lock the phone orientation and set the photo relative. I'm pretty sure it will become more common as time goes on.

[jakearchibald]

@OliverJAsh

Have you got any ideas how to detect whether the image is already correctly rotated?

The problem case is iOS Safari. It seems to be the only browser that applies EXIF orientation data to an <img>. Test: https://output.jsbin.com/wamevelupa/.

I guess you'd need to feature-detect it by creating a 1x2 JPEG with a 90 deg rotation via EXIF, and see what the width is when it'd decoded.

[OliverJAsh]

Good idea. We can keep it small with a tiny image, e.g. 1x2 px.

I had a go at doing this in https://stackblitz.com/edit/feature-detect-img-orientation?file=index.ts.

Here's how I created the test File:

1. draw a 1x2 black pixel in an editor
2. save as JPEG
3. use Exif editor to adjust the orientation
4. convert resulting JPEG to base 64 string
5. convert base 64 string to Blob via b64-to-blob

Is there a better way to do this? I'm sure it could be smaller.

Note in my testing iOS Safari only respects the orientation when the image element has been appended to the DOM, so we have to make sure the feature detection also does that.

I'm wondering if it would be so wrong to do a user agent check here. I guess the risk is other browsers could adopt this "feature" in the future.

[jakearchibald]

Note in my testing iOS Safari only respects the orientation when the image element has been appended to the DOM, so we have to make sure the feature detection also does that.

Also, it seems like img.naturalWidth and img.naturalHeight refer to the no-EXIF dimensions.

Of course this doesn't help for 180 deg rotations, or square images.

[jakearchibald]

Might be worth passing it to a JPEG minifier (but retaining EXIF of course). It might be tricky to minify since JPEG includes EXIF by including a whole TIFF (but without the image data).

[glenn-jocher]

+1
Same issue in MacOS. Vertical images show up horizontal in VOTT. No way to correct.

[surma]

No way to correct.

I mean, there is a rotate button.

[glenn-jocher]

Issue seems to be with HEIC to jpg export in Apple ecosystem. Seems like exif tag may be applied twice. Solution is to export to jpg without exif.

[hq5544]

+1
This issue seems occurs on all my browsers, include chrome safari and firefox.
While on iphone images show the right orientation, android phone is wrong.
It feels so weird.

[cgibsonmm]

Is there any progress on this? It feels like this has been going on forever. I really hate building sites that give users the ability to upload images if there is no way to render them to the page correctly

[jakearchibald]

Have you seen the rotate button?

[mh-alahdadian]

wow!
Still there is no good solution?

[surma]

Still there is no good solution?

I can only re-state @jakearchibald’s comment: Have you seen the rotate button?

Maybe someone can re-state what the problem is. Everything that has been described in this thread so far is solved by this very rotate button as far as I can tell.

[Haprog]

I haven't tried squoosh in a while but the rotate button seems like a great workaround to the issue, however ideally the image should automatically appear in the "original/correct" orientation since the information is in the file.

I'd like to keep this issue open until it works automatically (though the rotate button is still a nice additional feature to have even if this is fixed).

[felquis]

In my testing, the Orientation meta alone was not enough to tell me how to proper display the image, the EXIF of one vendor say the image was taken at position 8, but in another vendor the same rotation returned a 6, and as you rotate the smartphone, it won't tell if it is mirrored or not as well. :/ hard to track

[Haprog]

Btw the image-orientation: from-image; CSS has now been implemented in Chromium. It will be in the next stable Chrome 81 if that helps. https://caniuse.com/#feat=css-image-orientation

[Haprog]

the EXIF of one vendor say the image was taken at position 8, but in another vendor the same rotation returned a 6

@felquis If the source data is wrong, then it's a problem with one of the vendors if they incorrectly set the Orientation meta data.

If the image is originally saved in such a way that it is mirrored and there is Orientation meta saved, then the Orientation meta should also tell you if it's mirrored or not (see the ASCII art letter F examples at http://sylvana.net/jpegcrop/exif_orientation.html), but again if the vendor doesn't set it correctly it can be a problem in which case we can't automatically fix it.

The best we can do is assume the data is correct and interpret it by the spec.

[OliverJAsh]

Also, it seems like img.naturalWidth and img.naturalHeight refer to the no-EXIF dimensions.

In iOS Safari, these properties actually seem to refer to the Exif dimensions, and it looks like they are considering standarding this behaviour here: whatwg/html#4495 (comment)

[kocheick]

Not working with Firefox or Epihany (latest version as of today 6/1/20), however chrome (latest version) is fine.
I'm using elementaryOS.

on the left side Firefox, right side is Chrome