[Feature]-[508]:Camera widget bugs

[osloma]

Is there an existing issue for this?

• I have searched the existing issues

Summary

We can see that when Appsmith is run on an Android, pictures are not having all the quality than the camera on the phone can capture. Also, using iPhone, pictures taken are only a completely black image where nothing can be seen

Why should this be worked on?

The use case for this is to capture pictures of documentation (like bills, receipts, etc.) that should be in a resolution that can be read easily.
In our case, we are storing the pictures in Cloudinary but pictures have reduced size (around 350x350 pixels) that make them very hard to read their content (letters cannot be read easily). We have tried to change the resolution of the camera on the phones with no result

So far we are finding:

• On Android, it works but we get low resolution pictures
• On iPhone, pictures are completely black

The goal will be to be able to select the resolution required on the picture before it's taken. It could be done either from the widget (more preferred way) or from the running camera on the application.

Thank you so much!

[Nikhil-Nandagopal]

@osloma thank you for the information. We'll debug this and let you know what is going wrong

[dilippitchika]

@vivekverma2312 can you ask someone to test on an android phone?

It is broken in iphone for sure.

[dilippitchika]

Stats

Stat	Values
Reach	127
Effort (months)	0.5

[dilippitchika]

Found a set of issues
Apple devices -

1. [Blocker] Iphone safari and chrome unable to capture image in camera in iOS 14.4
2. [Blocker] In Ipad the camera goes black after capturing the image
   

Android devices -

1. [Blocker] The back camera feed seems to be mirrored.
2. [Blocker] When I tap the full screen, the tooltip stays open and I am unable to see any other button
3. [Blocker] When I turn on/off the video, the icon jumps a few pixel to the top right and back.
4. [Blocker] In full screen mode I’m not able to switch the camera (the dropdown doesn’t open)
5. [Blocker] Weird noise after taking a picture
6. [Blocker] Low resolution of the image leading to hazy images for app developers

Enhancements

1. [Enhancement] Switching between back and front camera is a bit inconvenient. My general reflex is flip with one press but here I have to click a dropdown and then select the camera
2. [Enhancement] When I take a photo, a check and a dismiss icon appears, which was fine but the reset of the actions were still visible like the turn on/off video, switch camera which IMO is not relevant when I try to save/discard the photo.
3. [Enhancement] I feel like the controls should not be a part of the camera view port. As the controls are white in color, if I point at a light background; it’s really hard to locate the controls. This seems to be fine in the full screen mode.
4. [Enhancement] All the buttons are very closed together, it always felt like I’m gonna press something else
5. [Enhancement] The button to take a picture should be a bit bigger so that the press area is more prominent

Slack discussion thread - https://theappsmith.slack.com/archives/C02JV8G1MP0/p1653567793725609
Library issue -
mozmorris/react-webcam#57

[dilippitchika]

@techbhavin please start triaging all the blockers mentioned first and leave a comment on if all these blockers can be fixed or not. If you need any help with devices, do reach out to me or @vivekverma2312

Here's the sample app - https://app.appsmith.com/app/untitled-application-5/page1-628f70677901344ba8d2e7cb

[techbhavin]

@dilippitchika updated noted of blocker issue

Apple Devices:

1. Not Reproduce
2. Not Fixed - camera library issue

Android Device :

1. Working as expected
2. Fixed
3. Fixed
4. Not Possible - In a full-screen mode popover is not working open issue in Popover Elements (Tooltip, Popover, Overlay) do not open with Portals #3248
5. Not able to reproduce. (Look like it is the device-specific issue)
6. Fixed

[dilippitchika]

Regarding

Android Device :
can we change the tooltip position to the right? or do we have any approach?
Why is the tooltip staying open, is there a reason for it? Tooltip should only open when we have clicked on it right?

Low resolution of the image leading to hazy images for app developers -- Please talk to ashit once, he can show this to you

[dilippitchika]

Scope:

1. Fix all the items which are marked as "we can fix" by Bhavin
2. Check the rest of the blockers with QA team's help if they are reoccurring

@techbhavin @vivekverma2312

[techbhavin]

Why is the tooltip staying open, is there a reason for it? The tooltip should only open when we have clicked on it right?

here is the library issue tooltip working fine with a click but there is an issue while touching.
@dilippitchika

[chandannkumar]

@techbhavin Couple of observations here while Testing camera widget on Android Device:

1. When we switch the camera and click to Take Photo/Video, camera widgets breaks until browser refresh.
   https://images.zenhubusercontent.com/275993885/3d34f410-2975-4f23-8cb7-4b584a1e5443/screenrecorder_2022_06_23_13_37_45_389_0_.mp4

2. Tooltip blocks the camera/Recorder click button when we exit fullscreen mode
   
   

3. Camera names in the dropdown are weird, names must be simple
   

[dilippitchika]

@techbhavin what is the suggestion regarding tooltip and fullscreen library? I thought there was a discussion around it.

[techbhavin]

@dilippitchika

about the tooltip and fullscreen

• For the tooltip,it is standard behavior with a Mobile Device. here tooltip is visible with long touch and when we touch again on anywhere on the screen it will hide again. At the moment we don’t have any alternative solutions for this.

• In full-screen mode, we can’t switch the camera because the popover is not visible. so instead of adding a popover to change the camera can we add a standard round icon for the swap camera? like below-attached image

[dilippitchika]

I'll answer 2 first, if we can detect which is front camera and which is back then yes we can add this icon. The problem I currently think is we don't have a way to do that, please correct me if I am wrong @techbhavin.

You may think it may not matter but the problem is if you have 3 cameras on your laptop. Maybe we can still use an icon here but it should be a different camera icon which changes the camera. This way we can get rid of tooltip completely.

Regarding fullscreen, can we just remove that functionality for now?

[techbhavin]

Agree with your point @dilippitchika,

Yes, we can remove fullscreen but can we do a similar like webcamera to show list of available camera

and Currently, we have a tooltip on capture and a fullscreen icon so can we remove this tooltip?

[dilippitchika]

but can we do a similar like webcamera to show list of available camera

Are you referring to how it shows the settings? If so, then we can proceed from a product standpoint, but i thought there is a tech limitation.

and Currently, we have a tooltip on capture and a fullscreen icon so can we remove this tooltip

Yes we can remove this for now

[dilippitchika]

@vivekverma2312 @chandannkumar let's discuss this issue once on a call, i believe this needs a small discussion to align and decide if we need more issues created from this issue.

[vivekverma2312]

@techbhavin On ios it's almost impossible to use camera widget

1. Controls are not visible on the widget (In background actually it exists)
2. Image click makes the widget black
3. Changing from camera to video also makes the widget black
4. Tapping on Fullscreen does nothing

ios:15.5

RPReplay_Final1656396008.MP4

[chandannkumar]

@techbhavin A Observation related to picture quality
cc: @dilippitchika

• Adding two photo which shows the Pixel and details of image taken from Phone's camera and Appsmith camera widget

1. DP Image (camera widget) :
   
   *Picture downloaded via image widget

2. Phone camera(original):
   

[chandannkumar]

Moving this issue to in progress due to blockers and callouts. There is also a merge conflict in this PR

[techbhavin]

As per our discussion,

• I have to put the camera rotate icon instead of the camera menu (list of the camera). but there is one more functionality that is camera turns off. so when we will put the camera rotate icon then how to manage camera turn-off functionality.
• Now one more dought is what should we do for the audio menu that is shown in video mode of the camera widget and also the same issue will happen with audio mute functionality like as camera turn-off.

@dilippitchika

[techbhavin]

• Fixed the camera unable to capture images on ios devices.
• Fixed the camera goes black after capturing the image in ios devices.
• Fixed the changing from camera to video and also makes the widget black in ios devices.
• Removed the tooltip so it can not cause tooltip-related issues.
• Give the mirror functionality in the camera widget.
• Fixed the getting weird noise after taking the picture.
• Improved the picture quality of the camera.
• Fixed the camera widget break when we switch the camera or take pictures after switching the camera.

@chandannkumar @vivekverma2312

[dilippitchika]

@techbhavin we can add another icon if needed, which turns on/off camera and one which changes cameras.

[techbhavin]

• Fixed the show video/audio input menus in full-screen mode.
• Fixed to show the control panel in the full-screen mode of the safari browser.

@chandannkumar @vivekverma2312

[chandannkumar]

Observation:
While recording a video from from front camera as soon as i switch camera to rear, the video stops recording and saves automatically but the recording timer is still continues on camera widget.

https://images.zenhubusercontent.com/275993885/276d845f-802b-4325-b048-1cd48faa2122/screenrecorder_2022_07_26_16_33_41_679_0_.mp4

@techbhavin Not sure about the video recording stops automatically on switching camera is a bug or an expected behavior

[techbhavin]

• Fix the camera switching issue when recording video

@chandannkumar

[vivekverma2312]

@techbhavin On ios when my mic is disabled and I record a video the sound is getting recorded.

[techbhavin]

For now, we are removing the video/audio mute buttons for ios devices because of an issue with the Safari browser. whenever we get the solution for that at that time we fix that.

@chandannkumar @vivekverma2312

cc: @dilippitchika @somangshu