Camera shows flipped image

[yankedev]

When capacitor uses the pwa-elements camera plugin (in a browser) the picture is flipped horizontally.
This seems a common behaviour on browser camera:
https://christianheilmann.com/2013/07/19/flipping-the-image-when-accessing-the-laptop-camera-with-getusermedia/

In the above link it is recommended to flip it using CSS classes:
video {
-webkit-transform: scaleX(-1);
transform: scaleX(-1);
}

Note: Yesterday I opened this bug in the capacitor repo (ionic-team/capacitor#997), I will now close it.

[yankedev]

It is probably better to have an configuration option (i.e. mirrored=true) to activate this functionality.
In some cases (for example when taking a picture of text), it is better to NOT mirror the image.

[mlynch]

What device does this happen on? Since this doesn't apply to all devices it seems we need a more nuanced solution

[UlisesCeca]

The same is also happening to me aside of other things:

• When I use the back camera the image is mirrored.
• Like the OP, when taking the picture it goes to horizontal.
• If I switch from front camera to back camera and then go back to front camera, the button to take the picture doesn't work.

My device is a Sony Xperia XZ1 but I tried to do it on other 2 devices and the behavior was the same

[eastjie]

I have added the following package to my web application:
"@capacitor/cli": "^1.1.0",
"@capacitor/core": "^1.1.0",
"@ionic/pwa-elements": "^1.3.0"

The image preview will not be flipped, but the video has still problem. I thinks, the style should be deleted "transform: scaleX(-1)" for element 'video' , but how can I change the style for it, because it's in the Shadow Root Tree "shadow-root".

How can man changes those styles for camera, if I use it in third web application without Ionic?

Thanks very much.

[voltairebiton]

also have issue with my samsung a7. if i try camera on a ionic pwa web app, the front camera is inverted and after taking the picture, the result is rotated to 90 degrees. Same thing with the rear camera but its rotating to -90deg. However if i go on to build it as an android app, the camera is working fine.

[pablogravielseo]

Same issue here from what @UlisesCeca and @voltairebiton said, I've tested with Samsung Galaxy s9.

[voltairebiton]

my temporary fix for now was to override pwa-camera.entry.js and comment out this code
if (typeof ImageCapture === 'undefined') { to override ImageCapture. just that i need to do it everytime i npm install.

[Monomachus]

From what I've researched today it's related to EXIF of the image, and I found the JS code that allows detecting the orientation: https://jsfiddle.net/wunderbart/dtwkfjpg/
Plus here are some images with different orientations: https://github.com/recurser/exif-orientation-examples

Maybe someone could patch existing code to show the image in the correct orientation

[moxival]

Is this going to be fixed at some point, the camera is kind of not usable at the moment

• The rear camera on Pixel is mirrored as well (you can literally have a stroke while trying to align the camera with something)
• When taking the picture it goes to horizontal
• If I switch from front camera to back camera and then go back to front camera, the button to take the picture doesn't work

[ilianiv]

The rear camera on iOS 12 is mirrored, too

[costleya]

As @eastjie noted, removing the inline transform on the video element that gets added by default fixes the issue (at least for me). It also coincidentally clears up the strange issue with rotating the camera twice. However, removing this means mucking with the HTML inside 2 shadow doms, and detecting change.

It seems this option was only intended for the front-facing camera (however, this option is never changed anyway - this.facing is never reassigned....) Should "facing logic" be removed, and have mirror functionality be a boolean flag, as others have suggested?

[jcruzrg]

Any solution? Thanks

[antoninbeaufort]

I can confirm that the problem for Android is here even in Ionic 5.0.0 and Capacitor 1.5.0, in my case if I take a picture from rear camera in portrait mode, my picture as DataUrl is rotated by 90° left when rendered in img.

And when I take a picture from Chrome on my computer as selfie, the picture is mirrored horizontally.
My smartphone is a SM-A320FL (2017).

EDIT : It's strange because on my Xiaomi Redmi Note 5 the picture is not rotated.

[twrayden]

For anyone having trouble with this, this is what I've done to fix it (temporarily):

Plugins.Camera.getPhoto({
  resultType: CameraResultType.Uri,
  direction: CameraDirection.Front
}).then(image => {
  console.log(image);
});
setTimeout(() => {
  const video = document
    .querySelector("pwa-camera-modal-instance")
    .shadowRoot.querySelector("pwa-camera")
    .shadowRoot.querySelector(".camera-wrapper .camera-video video");
  video.style.transform = null;
}, 100);

It's not pretty, but it works.

[rufogongora]

For anyone having trouble with this, this is what I've done to fix it (temporarily):

Plugins.Camera.getPhoto({
  resultType: CameraResultType.Uri,
  direction: CameraDirection.Front
}).then(image => {
  console.log(image);
});
setTimeout(() => {
  const video = document
    .querySelector("pwa-camera-modal-instance")
    .shadowRoot.querySelector("pwa-camera")
    .shadowRoot.querySelector(".camera-wrapper .camera-video video");
  video.style.transform = null;
}, 100);

It's not pretty, but it works.

I did the exact same thing yesterday, and while it works to flip the preview window, the image captured still gets flipped by 90 degrees

[Odisseuss]

Same issue here, in the browser image gets flipped horizontally but on android thing get a bit weird. With the front camera, the preview looks ok, but when the picture is taken the image is flipped horizontally and zoomed in/cropped. With the back camera, the preview is flipped horizontally, but when the picture is taken, it isn't flipped anymore but it is zoomed in/cropped and rotated 90 degrees anti-clockwise.

I see no one got this zooming in problem, is it an error on my part?

EDIT: Photo looks zoomed in just in the preview, when displayed it is fine.

[antoninbeaufort]

I have a starter for a workaround for rotated images :
We can use the exif.Orientation property of the image and then rotate the base64/dataURL with a canvas :

// set proper canvas dimensions before transform & export
if (4 < srcOrientation && srcOrientation < 9) {
  canvas.width = height;
  canvas.height = width;
} else {
  canvas.width = width;
  canvas.height = height;
}

// transform context before drawing image
switch (srcOrientation) {
  case 2: ctx.transform(-1, 0, 0, 1, width, 0); break;
  case 3: ctx.transform(-1, 0, 0, -1, width, height); break;
  case 4: ctx.transform(1, 0, 0, -1, 0, height); break;
  case 5: ctx.transform(0, 1, 1, 0, 0, 0); break;
  case 6: ctx.transform(0, 1, -1, 0, height, 0); break;
  case 7: ctx.transform(0, -1, -1, 0, height, width); break;
  case 8: ctx.transform(0, -1, 1, 0, 0, width); break;
  default: break;
}

Source for the rotate image function by exif : https://stackoverflow.com/a/40867559/9681418

[Monomachus]

@antoninbeaufort
I thought they already had this incorporated into this repo, well on a project I've done about ~6 months ago - I've used the following

public static getOrientation(file, callback) {
    const reader = new FileReader();

    reader.onload = (event) => {
        const view = new DataView((event.target as any).result as ArrayBuffer);

        if (view.getUint16(0, false) !== 0xFFD8) { return callback(-2); }

        const length = view.byteLength;
        let offset = 2;

        while (offset < length) {
            const marker = view.getUint16(offset, false);
            offset += 2;

            if (marker === 0xFFE1) {
                if (view.getUint32(offset += 2, false) !== 0x45786966) {
                    return callback(-1);
                }
                const little = view.getUint16(offset += 6, false) === 0x4949;
                offset += view.getUint32(offset + 4, little);
                const tags = view.getUint16(offset, little);
                offset += 2;

                for (let i = 0; i < tags; i++) {
                    if (view.getUint16(offset + (i * 12), little) === 0x0112) {
                        return callback(view.getUint16(offset + (i * 12) + 8, little));
                    }
                }
            } else if ((marker & 0xFF00) !== 0xFF00) { break; } else { offset += view.getUint16(offset, false); }
        }
        return callback(-1);
    };

    reader.readAsArrayBuffer(file.slice(0, 64 * 1024));
}

public static getOrientationFromBase64(srcBase64, callback) {

    const file = this.dataURLtoFile(srcBase64, 'some-random-name.jpg');

    const reader = new FileReader();

    reader.onload = (event) => {
        const view = new DataView((event.target as any).result);

        if (view.getUint16(0, false) !== 0xFFD8) { return callback(-2); }

        const length = view.byteLength;
        let offset = 2;

        while (offset < length) {
            const marker = view.getUint16(offset, false);
            offset += 2;

            if (marker === 0xFFE1) {
                if (view.getUint32(offset += 2, false) !== 0x45786966) {
                    return callback(-1);
                }
                const little = view.getUint16(offset += 6, false) === 0x4949;
                offset += view.getUint32(offset + 4, little);
                const tags = view.getUint16(offset, little);
                offset += 2;

                for (let i = 0; i < tags; i++) {
                    if (view.getUint16(offset + (i * 12), little) === 0x0112) {
                        return callback(view.getUint16(offset + (i * 12) + 8, little));
                    }
                }
            } else if ((marker & 0xFF00) !== 0xFF00) { break; } else { offset += view.getUint16(offset, false); }
        }
        return callback(-1);
    };

    reader.readAsArrayBuffer(file.slice(0, 64 * 1024));
}

public static resizeAndResetOrientation(srcBase64, srcOrientation, desiredWidth: number, callback) {
    const img = new Image();

    img.onload = () => {
        let divideFactor = 1;
        if (img.width > desiredWidth) {
            divideFactor = desiredWidth / img.width;
        }

        const width = img.width * divideFactor,
            height = img.height * divideFactor,
            canvas = document.createElement('canvas'),
            ctx = canvas.getContext('2d');

        // set proper canvas dimensions before transform & export
        if (4 < srcOrientation && srcOrientation < 9) {
            canvas.width = height;
            canvas.height = width;
        } else {
            canvas.width = width;
            canvas.height = height;
        }

        // transform context before drawing image
        switch (srcOrientation) {
            case 2: ctx.transform(-1, 0, 0, 1, width, 0); break;
            case 3: ctx.transform(-1, 0, 0, -1, width, height); break;
            case 4: ctx.transform(1, 0, 0, -1, 0, height); break;
            case 5: ctx.transform(0, 1, 1, 0, 0, 0); break;
            case 6: ctx.transform(0, 1, -1, 0, height, 0); break;
            case 7: ctx.transform(0, -1, -1, 0, height, width); break;
            case 8: ctx.transform(0, -1, 1, 0, 0, width); break;
            default: break;
        }

        // draw image
        ctx.drawImage(img, 0, 0, img.width * divideFactor, img.height * divideFactor);

        // export base64
        callback(canvas.toDataURL('image/jpeg'));
    };

    img.src = srcBase64;
}

And then using it like this...

onFileChosen(event: Event) {
    const pickedFile = (event.target as HTMLInputElement).files[0];
    if (!pickedFile) {
      return;
    }

    let originalOrientation = 1;

    ImageHelper.getOrientation(pickedFile, (orientation) => {
      console.log('Orientation', orientation);
      originalOrientation = orientation;

      const fr = new FileReader();
      fr.onload = () => {
        const dataUrl = fr.result.toString();
        this.manipulateImage(dataUrl, originalOrientation, pickedFile.name);
      };
      fr.readAsDataURL(pickedFile);
    });

}

private manipulateImage(dataUrl: string, originalOrientation: number, imageName?: string) {
    ImageHelper.resizeAndResetOrientation(dataUrl, originalOrientation, NEEDED_WIDTH, (fixedDataUrl) => {
      this.selectedImage = fixedDataUrl;
      const transformedFile = ImageHelper.dataURLtoFile(fixedDataUrl, ImageHelper.getImageName(imageName, 'some-random-file.jpg'));
      this.imagePick.emit(transformedFile);
    });
}

Full gist here:
https://gist.github.com/Monomachus/261c067206440f059f53a78c647d870c

[antoninbeaufort]

@Monomachus it's look like they didn't, now the exif is available directly in the result of Camera.getPhoto function, so it make it easier but I think that developers shouldn't have to create their own functions to rotate automatically, this should be included in Capacitor.

[rufogongora]

My solution was to get away from this plugin and use the native HTML5 camera button, so far this has worked perfectly with the android devices I've tested and iPhone as well, here's the snippet:

<input type="file" onChange={handleImageUpload} accept="image/*" capture/>

You can try it on your phone on this sample page:
http://anssiko.github.io/html-media-capture/

[Monomachus]

@Monomachus it's look like they didn't, now the exif is available directly in the result of Camera.getPhoto function

Actually what I found then was that sometimes the exIf they give you is not correct (for example when you choose a for from camera roll)

[edumartins7]

For anyone having trouble with this, this is what I've done to fix it (temporarily):

Plugins.Camera.getPhoto({
  resultType: CameraResultType.Uri,
  direction: CameraDirection.Front
}).then(image => {
  console.log(image);
});
setTimeout(() => {
  const video = document
    .querySelector("pwa-camera-modal-instance")
    .shadowRoot.querySelector("pwa-camera")
    .shadowRoot.querySelector(".camera-wrapper .camera-video video");
  video.style.transform = null;
}, 100);

It's not pretty, but it works.

Works and saved my day, thank you.

Had better results with setInterval, because if the user decided to discard the photo instead of keeping it, it would go back to the camera and it would be flipped again.

[vinybnb]

2 years but ionic team still hasn't fixed this big problem. When we work with text recognition, the camera image is flipped so we can not see the text correctly. It's terrible.

[antoninbeaufort]

@RavenVn Have you tried the alternative offered by @rufogongora?

[vinybnb]

@antoninbeaufort I need to develop on mobile browser as well as mobile app. The camera is working fine in native app but flipped on mobile browser. We can use other library (eg. camera.js) to have a normal camera but I need the same code base. And for the @rufogongora solution, sorry I just see it's the file input not the camera button any where. Thank you

[antoninbeaufort]

@RavenVn the file input with "capture" attribute shows the camera on devices that support it (a majority of mobiles devices), you're welcome.

[mlynch]

As you can all probably tell, this problem is really complex. Can someone sum up the exact issues and what we should change in this? I'm seeing a whole bunch of device-specific issues above that are going to be very difficult to code around.

Should we just remove the transform as a first step?

[mlynch]

Alright, the transform has been removed in 3.x.

If you could all try that and let us know how that works for you.

As for some of these issues, we're literally just accessing the normal getUserMedia APIs. Apart from the horizontal flip, we're not doing anything that would cause an image to be rotated just 90 degrees.

Unfortunately, if a certain device has issues in their implementation, we're not going to be able to easily fix that unless there's a specific known fix we can apply. In those cases, we could use help and PRs to code around certain broken implementations.

[mlynch]

Was able to reproduce the 90 degree turn on an Android emulator, will look into it. Any tips welcome

[mlynch]

So, if we load in an exif library and then apply a counter rotation for the orientation exif field, that fixes it in the preview.

But what would you want us to do with the image we send back? Send it back raw w/ the exif data, or transform it to be upright and send it back?

[mlynch]

Okay I think what I'm going to do is rotate the image, make it upright, and return that back. The exif data would still indicate the original direction, but really I don't see the point in sending back a non-upright photo.

[jahnog]

Okay I think what I'm going to do is rotate the image, make it upright, and return that back. The exif data would still indicate the original direction, but really I don't see the point in sending back a non-upright photo.

I don't know if this is the best option. In my case, I'm capturing an image and send it to a back end server to process it. If the image is rotated but the EXIF is not changed, then the server process, that already rotates the image according to its EXIF data, will wrongly rotate it.

If your are adding a rotation capability, could it be made available to the developer? so the developer can choose to rotate it or not? either as an additional parameter to the capture method or as utility function, etc.

[mlynch]

Yea, great point. Maybe we just return the image as it was and leave the exif and orientation fix part of the developer/user's job, but we can at least fix it up for the preview

[mlynch]

Alright well I'm confused af. For a while the images I was getting back from the emulator were rotated and had EXIF orientation of 6. However, they sudddenly stopped coming back rotated but kept the exif orientation of 6.

I just published a pre-release under the @next tag. If someone on an Android that was having issues could give it a try and let me know if it fixed the issue for you, that would be awesome. This release also has a bunch of other small fixes (like keeping the camera pointed in the direction you had it after cancelling).

npm install @ionic/pwa-elements@next

[mlynch]

Alright, well I did my best. 3.0.1 should have a number of tweaks and improvements, including hopefully addressing this one.

However, I've made the decision that users will need to detect and handle exif metadata themselves. This is out of the scope of this plugin because, once the image is returned to you, giving you back anything but the actual image itself, however oriented it is, doesn't seem right to me.

Here's a function modified from others I've seen here that detects orientation of an image blob:

private getOrientation(file): Promise<number> {
    return new Promise(resolve => {
      const reader = new FileReader();

      reader.onload = (event) => {
        const view = new DataView((event.target as any).result as ArrayBuffer);

        if (view.getUint16(0, false) !== 0xFFD8) { return resolve(-2); }

        const length = view.byteLength;
        let offset = 2;

        while (offset < length) {
          const marker = view.getUint16(offset, false);
          offset += 2;

          if (marker === 0xFFE1) {
            if (view.getUint32(offset += 2, false) !== 0x45786966) {
              return resolve(-1);
            }
            const little = view.getUint16(offset += 6, false) === 0x4949;
            offset += view.getUint32(offset + 4, little);
            const tags = view.getUint16(offset, little);
            offset += 2;

            for (let i = 0; i < tags; i++) {
              if (view.getUint16(offset + (i * 12), little) === 0x0112) {
                return resolve(view.getUint16(offset + (i * 12) + 8, little));
              }
            }
          } else if ((marker & 0xFF00) !== 0xFF00) { break; } else { offset += view.getUint16(offset, false); }
        }
        return resolve(-1);
      };

      reader.readAsArrayBuffer(file.slice(0, 64 * 1024));
    });
 }

[LauraRamirez22]

2 years but ionic team still hasn't fixed this big problem. When we work with text recognition, the camera image is flipped so we can not see the text correctly. It's terrible.

Could you resolve this issue? @RavenVn