Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a cross-platform API for camera access #2266

Merged
merged 22 commits into from
Jan 17, 2024

Conversation

freakboy3742
Copy link
Member

@freakboy3742 freakboy3742 commented Dec 8, 2023

Adds an API to capture images with a device camera.

Includes an example app, plus an implementation (with 100% test coverage) for iOS and macOS.

It's worth noting that the sample app won't do anything on the simulator - because the iOS simulator doesn't fully implement the camera. Trying to take a photo on the iOS simulator will log a warning that the camera isn't available. You need to deploy to an actual device to see this working.

The test coverage involves extensive mocking of APIs that can't be invoked. Most of the actual camera APIs have been mocked; this is required to prevent iOS from crashing on x86_64, and is needed on macOS because there's no guarantee that the test machine will have cameras (and in CI, it's almost guaranteed to not have cameras).

Fixes #2279.

PR Checklist:

  • All new features have been tested
  • All new features have been documented
  • I have read the CONTRIBUTING.md file
  • I will abide by the code of conduct

core/src/toga/hardware/camera.py Outdated Show resolved Hide resolved
core/src/toga/hardware/camera.py Outdated Show resolved Hide resolved
@mhsmith
Copy link
Member

mhsmith commented Jan 15, 2024

iOS CI is currently crashing because it doesn't have the camera permission. This should be fixed once permission support is merged into Briefcase. But it's worth looking into whether whether we could turn the crash into a more useful message, because right now it looks like this:

tests/test_window.py::test_select_folder_dialog[initial_directory4-True-result4] SKIPPED (Select Folder dialog not implemented on iOS) [ 18%]
tests/test_window.py::test_select_folder_dialog[initial_directory5-True-None] SKIPPED (Select Folder dialog not implemented on iOS) [ 19%]
Application has crashed!
========================
Python runtime error: Source type 1 not available
Child process terminated with signal 15: Terminated

Test suite didn't report a result.

@freakboy3742
Copy link
Member Author

freakboy3742 commented Jan 17, 2024

It turns out the problem wasn't the permission (at least, not directly). The iOS simulator doesn't have a working camera. On the M1 simulator, you can display the camera panel, but it logs errors; but on x86_64, trying to configure the camera panel raises a fatal error.

The fix is to take the same approach as on macOS - mock most of the actual camera APIs. This involves a lot less dark magick dealing with the TCC database, and also means the test suite can run on physical devices.

In the process of diagnosing the problem, I've also been able to add protection against omitting the camera permission. The testbed app defines this permission; but if an iOS app tries to use the Camera API and hasn't defined the camera permission, it will raise a RuntimeException.

@mhsmith mhsmith merged commit 126e7e8 into beeware:main Jan 17, 2024
69 checks passed
@freakboy3742 freakboy3742 deleted the ios-camera branch January 17, 2024 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add an API for camera access
2 participants