Add support for EXIF orientation

[ronaldbarendse]

Most, if not all, image viewers already support automatically rotating the displayed image based on EXIF orientation and even browsers are now doing this by widely supporting image-orientation: from-image. Users now expect all software to properly display and process images (which isn't a crazy expectation).

ImageSharp does include an AutoOrientProcessor and AutoOrient() extension method, but that needs to be called before doing any mutations (otherwise those might use the 'wrong' orientation, e.g. swapping width/height) and resets the orientation to 0 degrees (top-left).

ImageSharp.Web doesn't include a web processor for this and only has an OnProcessedAsync callback, which get called after all mutations are applied (which is too late). It's possible to create your own processor and always add the command using OnParseCommandsAsync. Because this command needs to be the first and the commands are stored as an IDictionary<string, string>, you have to create a copy, remove all existing commands, add the auto-orient command and re-add the commands. Implementing your own IRequestParser could be an alternative, possibly wrapping/decorating an existing implementation, but would then add a cross-cutting concern.

Ideally, correctly supporting this in ImageSharp could be done by:

• Adding a new bool AutoOrient { get; set; } = true; property to the ImageSharp Configuration class;
• When enabled, rotate/flip the image before mutating and revert back when done (without altering the EXIF orientation value), so the mutations are executed on the correctly oriented image data;
• Optionally, this setting could be an enum that also allows you to reset/remove the EXIF orientation value (and not revert back after mutating, which is what the current AutoOrientProcessor does).

And for ImageSharp.Web, we should ensure the image is also mutated when ProcessingBehavior.All is used and no commands are available (but only when required, so setting enabled and EXIF orientation value preset and not 0 degrees). Otherwise, it can still stream the original image stream.

[JimBobSquarePants]

Users now expect all software to properly display and process images (which isn't a crazy expectation).

No they don’t. They expect the correct (dictated) orientation to be rotated to what their naïve expectations are by magic.

Adding a new bool AutoOrient { get; set; } = true; property to the ImageSharp Configuration class;

If we were to add support it would be implemented purely as a processor instance with explicit instruction per request. It's really important we do not introduce unexpected behavior on a global level.

And for ImageSharp.Web, we should ensure the image is also mutated when ProcessingBehavior.All is used and no commands are available (but only when required, so setting enabled and EXIF orientation value preset and not 0 degrees). Otherwise, it can still stream the original image stream.

This makes no sense. If there are no commands and ProcessingBehavior.All is used then the input stream is simply copied across to the cache. That preserves the default behavior for the image. We should never alter that input without explicit instruction.

[ronaldbarendse]

They expect the correct (dictated) orientation to be rotated to what their naïve expectations are by magic.

Naïve expectations and magic or not, I've seen a lot of people having issues after uploading photos (primarily from their phone, which includes the orientation by default) and ImageSharp/ImageSharp.Web resizing/cropping it, as it defaults to using the actual image data and keeps the EXIF orientation metadata. Resizing an image with an EXIF orientation of 90/270 degrees to a width of 100 will actually result in a height of 100 when displayed in an EXIF orientation aware viewer (like your browser):

It's really important we do not introduce unexpected behavior on a global level

My suggestion is to add this to the Configuration class, which can be overridden in multiple ways (e.g. per method call/mutation, only for the ImageSharp.Web middleware, etc.). I agree that enabling this by default would require a new major version, but it could already be included in a new minor version if disabled (to keep existing behavior).

We should never alter that input without explicit instruction.

The new setting could include multiple ways to deal with the EXIF orientation, so instead of a boolean, we could have:

public enum AutoOrientMode
{
    // Do not auto-orient the image.
    Disabled,
    // Auto-orient image data based on the EXIF orientation flag (before doing mutations).
    KeepFlag,
    // Auto-orient image data based on the EXIF orientation flag (before doing the first mutation) and remove the tag.
    RemoveFlag
}

In the case this is set to RemoveFlag (naming could be better/different), ImageSharp should actually rotate/flip the image and remove the EXIF rotation flag. This behavior is very similar to the current AutoOrient() command (that sets the orientation to 0 degrees, instead of removing the flag).

And thinking about this again, I see your point: then you should indeed explicitly call this command 🙃 Just thinking out loud here and trying to explain the reasoning I had...

---

To summarize: this feature should be opt-in (for now) to not introduce unexpected behavior and adding support for EXIF orientation could be done by either:

1. Rotating/flipping the image data before doing mutations and restore after (will require more processing);
2. Make all processors orientation-aware, e.g. swapping width/height when resizing (will require rewriting the affected processors).

Because this is then only required when doing mutations, the existing ProcessingBehavior.All can indeed be as-is. What do you think would be the best way to make ImageSharp orientation-aware?

[ronaldbarendse]

Looks like there's already a question about the current behavior on the ImageSharp discussions: SixLabors/ImageSharp#1339.

Maybe this discussion should actually be moved to the ImageSharp repository, as it's not specific for ImageSharp.Web.

[JimBobSquarePants]

No. Do not bring a discussion there. Orientation and Size are two entirely separate metrics and should not be confused. ImageSharp resizes an image correctly by size. That’s an industry standard.

The OP in that discussion was incorrect

[sannyjacobsson]

Any update on this? I have the same issue in one of my projects.

[JimBobSquarePants]

No. My decision on the matter is final.

[sannyjacobsson]

Ok. Just to be clear. It will be up to each and every developer then to add some type of custom processing steps?

[JimBobSquarePants]

Before you continue I would like to remind you that.

A. You are consuming this software under the Apache 2.0 license. Which means AS-IS with no obligation of any support.
B. We follow a code of conduct here.

I’m being very clear here. I’m not introducing ANY changes that conflate size and orientation.

I am open, however, to breaking changes (Since we’re releasing a major version soon) that allow easy insertion of commands via OnParseCommandsAsync. If someone wants to discuss then PR those changes may happen.

[ronaldbarendse]

The ResizeWebProcessor will now respect the EXIF orientation by default (as part of v2) 🎉

• Respect EXIF orientation when resizing #217
• Apply updates to XY and Anchor positions based on EXIF metadata #221

Also awesome to see this discussion identified the issue regarding inserting/ordering commands, which got fixed in #203 👍🏻