Functional WebRTC in WebkitGTK on Linux! #8426
Replies: 2 comments 30 replies
-
This issue is also blocking me from rendering a camera feed through My tauri project is also powered by a Some related threads: |
Beta Was this translation helpful? Give feedback.
-
|
P.S. @campbellcole If you ever get it work I'd love to know about it! |
Beta Was this translation helpful? Give feedback.
-
It felt like that for a year now :/ I'm pretty sure we won't see it in 2.44 at least (won't be released for another 2-3 months anyway). |
Beta Was this translation helpful? Give feedback.
-
|
That not great but it at least puts me out of misery. If anyone is looking for an alternative in the meantime, I've found success using # 1. pipe live camera preview to /dev/video0
gphoto2 --stdout --capture-movie | ffmpeg -i - -vcodec mjpeg -pix_fmt yuv420p -q:v 2 -threads 0 -f v4l2 /dev/video0
# 2. serve video
ustreamer --format=MJPEG<!-- 3. Display video in frontend -->
<img src='http://localhost:8080/stream' /> |
Beta Was this translation helpful? Give feedback.
-
IT WORKS!!@thenbe good news! I have just successfully opened a webcam using WebRTC in Tauri!!! This was a bit of a doozy so bear with me. Step 1: Get the code and modify dependenciesFirst, I started by cloning each repository and ensuring the work tree was the v2 work tree (branch depends on the repo):
We need to be on the v2 branches because the Then, make a demo app (I used pnpm): In each repository, ensure the dependencies point to each other by
Step 2: Update Nix flake:The previous flake uses {
description = "";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
flake-utils.url = "github:numtide/flake-utils";
rust-overlay.url = "github:oxalica/rust-overlay";
};
outputs = { self, nixpkgs, flake-utils, rust-overlay }:
flake-utils.lib.eachDefaultSystem(system:
let
pkgs = import nixpkgs {
inherit system;
overlays = [ (import rust-overlay) ];
};
common = with pkgs; [
(webkitgtk_4_1.overrideAttrs (final: prev: {
cmakeFlags = prev.cmakeFlags ++ [
"-DENABLE_MEDIA_STREAM=ON"
"-DENABLE_WEB_RTC=ON"
];
buildInputs = prev.buildInputs ++ [ openssl ];
}))
gst_all_1.gstreamer
gst_all_1.gst-plugins-good
gst_all_1.gst-plugins-base
gst_all_1.gst-plugins-bad
gtk3
glib
dbus
openssl_3
librsvg
libsoup_3
];
libraries = with pkgs; [
cairo
pango
gdk-pixbuf
zlib
harfbuzz
atk
stdenv.cc.cc.lib
libGL
libxkbcommon
fontconfig
freetype
wayland
] ++ common;
packages = with pkgs; [
(rust-bin.fromRustupToolchainFile ./rust-toolchain.toml)
nodejs_20
nodePackages_latest.pnpm
pkg-config
curl
wget
cargo-expand
mold
clang
] ++ common;
in
with pkgs;
{
devShells.default = mkShell rec {
nativeBuildInputs = packages;
buildInputs = libraries;
# THESE ARE DISABLED BECAUSE WE MUST USE X11 TO GET WEBRTC WORKING, SEE THE "CAVEATS" SECTION
#
# will not render correctly on wayland without this (see https://github.com/tauri-apps/tauri/issues/7354)
# XDG_DATA_DIRS = "${gsettings-desktop-schemas}/share/gsettings-schemas/${gsettings-desktop-schemas.name}:${gtk3}/share/gsettings-schemas/${gtk3.name}";
# GIO_MODULE_DIR = "${glib-networking}/lib/gio/modules";
GST_PLUGIN_PATH = "${gst_all_1.gst-plugins-good}/lib/gstreamer-1.0:${gst_all_1.gst-plugins-base}/lib/gstreamer-1.0:${gst_all_1.gst-plugins-bad}/lib/gstreamer-1.0";
WEBKIT_DISABLE_COMPOSITING_MODE = "1";
GDK_BACKEND = "x11";
LD_LIBRARY_PATH = lib.makeLibraryPath buildInputs;
RUST_BACKTRACE = "full";
};
}
);
}Step 3: Modifying WRY (part 1)I enabled every possible setting I could find that was related to WebRTC: diff --git a/src/webkitgtk/mod.rs b/src/webkitgtk/mod.rs
index ba9b4a1..436133f 100644
--- a/src/webkitgtk/mod.rs
+++ b/src/webkitgtk/mod.rs
@@ -370,6 +405,20 @@ impl InnerWebView {
settings
.set_enable_back_forward_navigation_gestures(attributes.back_forward_navigation_gestures);
+ // Enable WebRTC (requires custom build of webkitgtk)
+ println!("HELLO THERE: We are enabling WebRTC for debugging!");
+ settings.set_enable_webrtc(true);
+ settings.set_enable_media_stream(true);
+ settings.set_enable_mediasource(true);
+ settings.set_enable_media(true);
+ settings.set_enable_media_capabilities(true);
+ settings.set_enable_encrypted_media(true);
+ // settings.set_enable_mock_capture_devices(true);
+ settings.set_media_playback_requires_user_gesture(false);
+ settings.set_media_playback_allows_inline(true);
+ settings.set_media_content_types_requiring_hardware_support(None);
+ // settings.set_disable_web_security(true);I haven't yet distilled which of these are actually necessary, I will do that after I post this. Step 4: Modifying WRY (part 2)Believe it or not, we are almost done. However, if you try to use There are a few potential hacks to fix this, but none are up to Tauri's security standards, so don't expect it to work like this once support is officially added. We need to register a signal handler for this signal. The page linked above has a demo handler that pops up a window to ask for permission. This is also possible in Rust (this diff leaves out the necessary imports, easy to figure them out though): diff --git a/src/webkitgtk/mod.rs b/src/webkitgtk/mod.rs
index ba9b4a1..f1c04f4 100644
--- a/src/webkitgtk/mod.rs
+++ b/src/webkitgtk/mod.rs
@@ -363,6 +364,39 @@ impl InnerWebView {
context.set_use_system_appearance_for_scrollbars(false);
}
+ // prompt for permission from user
+ webview.connect_permission_request(move |_, request| {
+ let dialog = MessageDialog::new(
+ None::<>k::Window>,
+ DialogFlags::MODAL,
+ gtk::MessageType::Question,
+ gtk::ButtonsType::YesNo,
+ "Allow permissions request?",
+ );
+
+ dialog.show();
+
+ let res = dialog.run();
+
+ match res {
+ ResponseType::Yes => {
+ request.allow();
+ }
+ _ => {
+ request.deny();
+ }
+ }
+
+ // this might happen automatically on drop but might as well do it here
+ // it's unsafe because we still have the dialog object so don't use it
+ // after this line
+ unsafe {
+ dialog.destroy();
+ }
+
+ true
+ });
+Alternatively, you can simply allow every request: diff --git a/src/webkitgtk/mod.rs b/src/webkitgtk/mod.rs
index ba9b4a1..f1c04f4 100644
--- a/src/webkitgtk/mod.rs
+++ b/src/webkitgtk/mod.rs
@@ -363,6 +364,39 @@ impl InnerWebView {
context.set_use_system_appearance_for_scrollbars(false);
}
+ // allow all permission requests for debugging
+ webview.connect_permission_request(move |_, request| {
+ request.allow();
+ true
+ });Step 5: Try it out!!We are now ready to use a webcam in the Tauri app. I did this using the import { useEffect, useState } from "react";
import "./App.css";
import Webcam from "react-webcam";
function App() {
const [selectedDevice, setSelectedDevice] = useState<string | null>(null);
const [devices, setDevices] = useState<MediaDeviceInfo[]>([]);
useEffect(() => {
navigator.mediaDevices
.enumerateDevices()
.then((devices) => {
setDevices(devices.filter((device) => device.kind === "videoinput"));
})
.catch((e) => {
alert(JSON.stringify(e));
});
}, []);
return (
<div className="container">
<ul>
{devices.map((device) => {
const selected = selectedDevice === device.deviceId;
return (
<li
key={device.deviceId}
style={(selected && { color: "green" }) || {}}
onClick={() => setSelectedDevice(device.deviceId)}
>
{device.label}
</li>
);
})}
</ul>
{selectedDevice && (
<Webcam audio={false} videoConstraints={{ deviceId: selectedDevice }} />
)}
</div>
);
}
export default App;Once this page is loaded, click the device you want to use from the list, then it will prompt for permission (if you went that route) or just immediately display your webcam on screen.
Viola! CaveatsWaylandI was only able to get this working using X11 as the GDK backend. When attempting to use Wayland, the following error is produced many, many times (I think once per frame): This could be specific to my webcam so it may be worth trying to use Wayland yourself. To do this, comment out the Non-NixOS EnvironmentsThis obviously only works because we've compiled webkitgtk ourselves, but I think it's important to mention that getting this to work on any non-Nix operating systems could be pretty challenging. Hypothetically, we could just build the app with Tauri v2 and update the v4l2loopback DevicesI don't know if it's my setup or intrinsic to gstreamer's WebRTC implementation, but v4l2loopback devices didn't appear in the list for me. In the past, I've had issues getting certain programs to recognize those devices unless something is actively writing to it, so that could be the issue (I will update this section if I figure it out). The strange thing is that we absolutely have the v4l2 plugins installed for gstreamer because we've installed all but the |
Beta Was this translation helpful? Give feedback.
-
|
Hey @thenbe and @ajaypillay After having similar issues in my own application with the modifications shared above, I've cloned and built the example repo shared by Ajay to ensure that it's not a problem in my code, and the same issues persist. Without making any modifications, the screen share media stream is obtained successfully, but its If I replace However, If I replace I'm running:
I would appreciate some troubleshooting steps or possible solutions! :) |
Beta Was this translation helpful? Give feedback.
-
|
@ninjadev64 - My specs are:
I'm not too sure how to address that issue beyond suggesting trying out a downgraded GStreamer version. |
Beta Was this translation helpful? Give feedback.
-
|
NixOS/nixpkgs#265771 - how does this relate to it, can webrtc support be enabled just by uh enabling experimental features for webkitgtk (and does Tauri V2) support WebRTC for Linux? |
Beta Was this translation helpful? Give feedback.
-
|
+1 |
Beta Was this translation helpful? Give feedback.
-
I have the same question. I have a one way stream and i want to know if its possible to use with Tarui V2 and webkitgtk? |
Beta Was this translation helpful? Give feedback.
-
EDIT: I GOT IT TO WORK! See my reply #8426 (comment)
Old attempt (doesn't work)
Hi all, I am making this discussion because tauri-apps/wry#85 is locked. I have managed to get the experimental WebRTC support to work partially. I believe the only remaining issue is enabling it in the WebView's settings (https://webkitgtk.org/reference/webkit2gtk/2.38.6/method.Settings.set_enable_webrtc.html), though I did do some
ripgreping of the source code and it seems this may be automatically enabled when-DENABLE_WEB_RTC=ON.Nonetheless, I am able to execute
navigator.mediaDevices.enumerateDevices():Before recompiling webkitgtk, this was an empty array.
This output seems to indicate some sort of failure given how different it is from the output I get when running this in Firefox, but I will continue working on this and see if I can figure it out. I get a
NotAllowedErrorwhen I attempt to actually open a device, but that's a lot further than I got before starting this.For anyone interested in trying this, you'll have to compile WebkitGTK yourself and that takes a while. I used Nix to accomplish this. If you want to try this on Nix(OS), then you'll need the following config (I used a flake.nix, and the following dependencies are on top of everything else required for a Tauri app, I've left that stuff out for brevity):
I wish I could report a functional webcam in a Tauri app, but we're not quite there yet. Very excited about this regardless. Thanks for the Tauri team's hard work, looking forward to the day this isn't an experimental feature 😄
Beta Was this translation helpful? Give feedback.
All reactions