🐛 In landscape mode, enabling frameProcessor causes preview image stretching and incorrect orientation

[boiboif]

What's happening?

In landscape mode

Preview without frameProcessor:

Preview with frameProcessor enabled:

Reproduceable Code

const frameProcessor = useFrameProcessor(
    frame => {
      'worklet';
    },
    [handleDetectedFaces],
  );

<Camera
  ref={camera}
  style={StyleSheet.absoluteFill}
  device={device}
  isActive={true}
  photo
  // frameProcessor={frameProcessor}
/>

Relevant log output

> Configure project :react-native-vision-camera
[VisionCamera] Thank you for using VisionCamera ??
[VisionCamera] If you enjoy using VisionCamera, please consider sponsoring this project: https://github.com/sponsors/mrousavy
[VisionCamera] node_modules found at C:\Users\87332\Desktop\tc\healthybd-app\node_modules
[VisionCamera] VisionCamera_enableFrameProcessors is set to true!
[VisionCamera] react-native-worklets-core found, Frame Processors are enabled!
[VisionCamera] VisionCamera_enableCodeScanner is set to false!

Camera Device

{
  "formats": [],
  "sensorOrientation": "landscape-left",
  "hardwareLevel": "full",
  "maxZoom": 10,
  "minZoom": 1,
  "maxExposure": 24,
  "supportsLowLightBoost": false,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": false,
  "isMultiCam": false,
  "minFocusDistance": 10,
  "minExposure": -24,
  "name": "0 (BACK) androidx.camera.camera2",
  "hasFlash": true,
  "hasTorch": true,
  "position": "back",
  "id": "0"
}

Device

redmi k50

VisionCamera Version

4.5.3

Can you reproduce this issue in the VisionCamera Example app?

No, I cannot reproduce the issue in the Example app

Additional information

• I am using Expo
• I have enabled Frame Processors (react-native-worklets-core)
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[maintenance-hans]

Guten Tag, Hans here! 🍻

I see you have provided some information about ze issue, but to help mrousavy resolve it effectively, we need to gather more logs. Please provide logs from adb logcat while reproducing ze issue, as they can help diagnose what’s happening during runtime.

Also, it might be worth considering sponsoring ze project if you’d like more immediate attention on this issue: Sponsor mrousavy.

Let’s make sure ze issue is clear so we can get to ze bottom of it together!

Note: If you think I made a mistake, please ping @mrousavy to take a look.

[meleffendi]

I'm having the same issue on 4.5.2 on OnePlus5.

In my case, when the phone is in portrait mode, the preview shows correct. If I rotate the phone to landscape, I get the incorrect orientation and stretching.

Were you able to fix it?

[boiboif]

I'm having the same issue on 4.5.2 on OnePlus5.我在 OnePlus5 的 4.5.2 上遇到了同样的问题。

In my case, when the phone is in portrait mode, the preview shows correct. If I rotate the phone to landscape, I get the incorrect orientation and stretching.就我而言，当手机处于纵向模式时，预览显示正确。如果我将手机旋转到横向，我会得到不正确的方向和拉伸。

Were you able to fix it?你能修好它吗？

My application is a landscape-oriented app. Currently, my solution is to force the camera screen to portrait orientation.

const portraitHoc = (Component: React.ComponentType<any>) => {
  return (props: any) => {
    const [hasLocked, setHasLocked] = React.useState(false);
    const previousOri = React.useRef<OrientationType>();

    React.useEffect(() => {
      let timer: NodeJS.Timeout;
      locker.getOrientation(o => {
        previousOri.current = o;
        locker.lockToPortrait();
        timer = setInterval(() => {
          locker.getOrientation(o => {
            if (o === OrientationType.PORTRAIT) {
              setHasLocked(true);
              clearInterval(timer);
            }
          });
        }, 100);
      });

      return () => {
        clearInterval(timer);
        if (previousOri.current === OrientationType['LANDSCAPE-RIGHT']) {
          locker.lockToLandscapeRight();
        } else if (previousOri.current === OrientationType['LANDSCAPE-LEFT']) {
          locker.lockToLandscapeLeft();
        } else {
          locker.lockToLandscape();
        }
      };
    }, []);

    return hasLocked ? <Component {...props} /> : null;
  };
};

export default portraitHoc(Facedetect);

[dalisalvador]

This problem can be reproduced using the example app.
When using frameProcessor, the Frame.orientation is fixed to the native orientation of the device. What I don't understand is why this has to also lock the Preview.

Related to #3051 (comment)