Image size issue

[Ahmedlag]

Hello,

I'm facing an issue setting captured image size in iOS & Android

I'm using capacitor 6 and plugin version 6.2.13

When i set the image size :

return CameraPreview.capture({
  quality: 66,
  width: 600,
  height: 1000
});

The captured image will never have the resolution i've chosen ! The only thing i can control is the quality

Any solution to this ?

[riderx]

That an issue I'm currently trying to solve.
The size should be a size available on the device, if not, we try to send the one who matches the ratio. If we don't have any, we return the biggest.
I think I will disable the width and height and let it only be chosen in the configuration, that will be less misleading

[Ahmedlag]

@riderx thank you

i've worked on something but i'm not sure if it is the right approach

(For iOS)
I've added a function called resizeImage that will resize the image before converting it to a jpeg

func resizeImage(image: UIImage, targetSize: CGSize) -> UIImage {
    let size = image.size

    let widthRatio  = targetSize.width  / size.width
    let heightRatio = targetSize.height / size.height

    var newSize: CGSize
    if widthRatio > heightRatio {
        newSize = CGSize(width: size.width * heightRatio, height: size.height * heightRatio)
    } else {
        newSize = CGSize(width: size.width * widthRatio, height: size.height * widthRatio)
    }

    let rect = CGRect(origin: .zero, size: newSize)

    UIGraphicsBeginImageContextWithOptions(newSize, false, 1.0)
    image.draw(in: rect)
    let newImage = UIGraphicsGetImageFromCurrentImageContext()
    UIGraphicsEndImageContext()

    return newImage!
}

And i call the method once the image is captured

@objc func capture(_ call: CAPPluginCall) {
    DispatchQueue.main.async {

        let quality: Int = call.getInt("quality", 85)
        let width: Int = call.getInt("width", 650)
        let height: Int = call.getInt("height", 1000)


        self.cameraController.captureImage { (image, error) in

            guard let image = image else {
                print(error ?? "Image capture error")
                guard let error = error else {
                    call.reject("Image capture error")
                    return
                }
                call.reject(error.localizedDescription)
                return
            }
            
            let resizedImage = self.resizeImage(image: image, targetSize: CGSize(width: width, height: height))
            
            let imageData: Data?
            if self.cameraController.currentCameraPosition == .front {
                let flippedImage = resizedImage.withHorizontallyFlippedOrientation()
                imageData = flippedImage.jpegData(compressionQuality: CGFloat(quality/100))
            } else {
                imageData = resizedImage.jpegData(compressionQuality: CGFloat(quality/100))
            }

            if self.storeToFile == false {
                let imageBase64 = imageData?.base64EncodedString()
                call.resolve(["value": imageBase64!])
            } else {
                do {
                    let fileUrl=self.getTempFilePath()
                    try imageData?.write(to: fileUrl)
                    call.resolve(["value": fileUrl.absoluteString])
                } catch {
                    call.reject("error writing image to file")
                }
            }
        }
    }
}

[riderx]

This will break the aspect ratio of the image right?

[Ahmedlag]

@riderx "resizeImage" doesnt break the aspect ratio of the image, it ensures that the aspect ratio is maintained by calculating the correct scaling factor based on the target size.

[riderx]

if the image is 19:9 and we want 4:3 where does the pixel go then ?
Or we break the pixel size and force the ratio or the image will have empty space?
Maybe can you share image output to explain if i miss something?

[eggbeard]

A common use case to want a preview at or around screen resolution and then to capture an image at a much higher resolution (but often the same aspect ratio). I have a use case where I have a screen of 890x534px. I want to take pictures at a 4:3 aspect ratio. I can easily calculate the viewport for positioning a 4:3 preview. left: 89 and a 712x534 preview. The plugin need not do that.

Now I need to capture at 1440x1080, but the max supported by my device is 8000x6000

Using the preview (video) size is not close to what I want, and I don't want to request larger video since this uses more resources and can be slow and jerky. I also don't want max resolution since this is huge and slow to resize. So capture must be able to supply dimensions to meet that need. There are a lot of the same problems as are faced by the web api MediaTrackConstraints. So if you specify width and height and this is not available this should error as being over constrained. If instead we specify an aspect ratio and a width for example, it's more likely that we can deliver a capture that closely matches the request.

It would be much easier to supply arguments of how big should the capture be, if the available sizes could be interrogated by the consuming code. Then the 'matching code' in getOptimalPictureSizeForPreview doesn't have to guess what the user means

[riderx]

@WcaleNieWolny than something you can look up for

[WcaleNieWolny]

@eggbeard @riderx you want me to add a function like getSupportedPictureSizes?

[WcaleNieWolny]

Is implementing a function that returns following output sufficient to resolve this issue?

{
  "supportedPictureSizes": [
    {
      "facing": "Back",
      "supportedPictureSizes": [
        {
          "width": 4032,
          "height": 3024
        },
        {
          "width": 4000,
          "height": 3000
        },
        {
          "width": 4032,
          "height": 2268
        },
        {
          "width": 3840,
          "height": 2160
        },
        {
          "width": 4000,
          "height": 2000
        },
        {
          "width": 3264,
          "height": 2448
        },
        {
          "width": 3264,
          "height": 1836
        },
        {
          "width": 2560,
          "height": 1920
        },
        {
          "width": 2688,
          "height": 1512
        },
        {
          "width": 1920,
          "height": 1920
        },
        {
          "width": 2560,
          "height": 1280
        },
        {
          "width": 2048,
          "height": 1536
        },
        {
          "width": 1920,
          "height": 1440
        },
        {
          "width": 1920,
          "height": 1080
        },
        {
          "width": 4624,
          "height": 3472
        }
      ]
    },
    {
      "facing": "Front",
      "supportedPictureSizes": [
        {
          "width": 4032,
          "height": 3024
        },
        {
          "width": 4000,
          "height": 3000
        },
        {
          "width": 4032,
          "height": 2268
        },
        {
          "width": 3840,
          "height": 2160
        },
        {
          "width": 4000,
          "height": 2000
        },
        {
          "width": 3264,
          "height": 2448
        },
        {
          "width": 3264,
          "height": 1836
        },
        {
          "width": 2560,
          "height": 1920
        },
        {
          "width": 2688,
          "height": 1512
        },
        {
          "width": 1920,
          "height": 1920
        },
        {
          "width": 2560,
          "height": 1280
        },
        {
          "width": 2048,
          "height": 1536
        },
        {
          "width": 1920,
          "height": 1440
        },
        {
          "width": 1920,
          "height": 1080
        },
        {
          "width": 4208,
          "height": 3072
        }
      ]
    }
  ]
}

[WcaleNieWolny]

If getSupportedPictureSizes doesn't solve this issue, please reopen it

[alexcroox]

@riderx I agree with @WcaleNieWolny, we used getSupportedPictureSizes() to ensure we passed the correct width/height to CameraPreview.capture() but it ignores it on iOS and the resulting image comes out as 1080x1920 whatever width/height we provide

[alexcroox]

I might be looking in the wrong place but on iOS capture() there doesn't seem to be any code relating to width/height

https://github.com/Cap-go/camera-preview/blob/main/ios/Plugin/Plugin.swift#L266-L304