diff --git a/html/background/retro.webp b/html/background/retro.webp new file mode 100644 index 0000000000..09df77afa2 Binary files /dev/null and b/html/background/retro.webp differ diff --git a/html/demos.html b/html/demos.html index a4ad9dd377..fc9f1a57e2 100644 --- a/html/demos.html +++ b/html/demos.html @@ -110,6 +110,14 @@

Janus WebRTC Server: Demo Tests

Canvas Capture A variant of the Echo Test demo, that shows how to use a canvas element as a WebRTC media source. + + Web Audio Processing + A variant of the Echo Test demo, that shows how to use Web Audio to process audio before sending it to Janus. + + + Virtual Background + A variant of the Echo Test demo, that shows how to use something like MediaPipe to add a virtual background before sending video to Janus. + VP9-SVC Video Room A variant of the Video Room demo, that allows you to test the VP9 SVC layer selection, if available. diff --git a/html/navbar.html b/html/navbar.html index f1998e2adb..471857a75e 100644 --- a/html/navbar.html +++ b/html/navbar.html @@ -32,6 +32,8 @@
  • End-to-end Encryption
  • Multichannel Opus (surround)
  • Canvas Capture
    • +
  • Web Audio Processing
    • +
  • Virtual Background
  • VP9-SVC Video Room
  • Admin/Monitor
    • diff --git a/html/virtualbg.html b/html/virtualbg.html new file mode 100644 index 0000000000..30eab185f6 --- /dev/null +++ b/html/virtualbg.html @@ -0,0 +1,125 @@ + + + + + + +Janus WebRTC Server (multistream): Virtual Background + + + + + + + + + + + + + + + + + + +Fork me on GitHub + + + +
    +
    +
    +
    +

    Plugin Demo: Virtual Background + +

    +
    +
    +
    +
    +

    Demo details

    +

    This is a variant of the Echo Test and Canvas demos meant to + showcase how you can use libraries like + MediaPipe + to add a virtual background to your webcam capture before sending it + to Janus: everything is exactly the same in term of available controls, + features, and the like, with the substantial difference that we'll + play a bit with what we'll send on the video stream.

    +

    More precisely, the demo captures the webcam feed via a + getUserMedia call, and then will make use of MediaPipe + to segment the video and replace the background by drawing on a + canvas element. The canvas element is then used + as the actual source of media for our PeerConnection, which means the + video we get back from the EchoTest plugin should reflect the + tweaks we've made on the stream.

    +

    Notice that this is a very naive implementation, which is heavily + based on the example + from the MediaPipe Selfie Segmentation documentation.

    +

    Press the Start button above to launch the demo.

    +
    +
    +
    +
    +
    +
    +
    +
    +
    +

    Local Stream +
    + + +
    + + +
    +
    +

    +
    +
    + + +
    +
    +
    +
    +
    +
    +
    +

    Remote Stream

    +
    +
    +
    +
    +
    +
    +
    +
    + +
    +
    +
    +
    + + + diff --git a/html/virtualbg.js b/html/virtualbg.js new file mode 100644 index 0000000000..af4cf718f0 --- /dev/null +++ b/html/virtualbg.js @@ -0,0 +1,685 @@ +// We make use of this 'server' variable to provide the address of the +// REST Janus API. By default, in this example we assume that Janus is +// co-located with the web server hosting the HTML pages but listening +// on a different port (8088, the default for HTTP in Janus), which is +// why we make use of the 'window.location.hostname' base address. Since +// Janus can also do HTTPS, and considering we don't really want to make +// use of HTTP for Janus if your demos are served on HTTPS, we also rely +// on the 'window.location.protocol' prefix to build the variable, in +// particular to also change the port used to contact Janus (8088 for +// HTTP and 8089 for HTTPS, if enabled). +// In case you place Janus behind an Apache frontend (as we did on the +// online demos at http://janus.conf.meetecho.com) you can just use a +// relative path for the variable, e.g.: +// +// var server = "/janus"; +// +// which will take care of this on its own. +// +// +// If you want to use the WebSockets frontend to Janus, instead, you'll +// have to pass a different kind of address, e.g.: +// +// var server = "ws://" + window.location.hostname + ":8188"; +// +// Of course this assumes that support for WebSockets has been built in +// when compiling the server. WebSockets support has not been tested +// as much as the REST API, so handle with care! +// +// +// If you have multiple options available, and want to let the library +// autodetect the best way to contact your server (or pool of servers), +// you can also pass an array of servers, e.g., to provide alternative +// means of access (e.g., try WebSockets first and, if that fails, fall +// back to plain HTTP) or just have failover servers: +// +// var server = [ +// "ws://" + window.location.hostname + ":8188", +// "/janus" +// ]; +// +// This will tell the library to try connecting to each of the servers +// in the presented order. The first working server will be used for +// the whole session. +// +var server = null; +if(window.location.protocol === 'http:') + server = "http://" + window.location.hostname + ":8088/janus"; +else + server = "https://" + window.location.hostname + ":8089/janus"; + +var janus = null; +var echotest = null; +var opaqueId = "canvas-"+Janus.randomString(12); + +var localTracks = {}, localVideos = 0, + remoteTracks = {}, remoteVideos = 0; +var bitrateTimer = null; +var spinner = null; + +var audioenabled = false; +var videoenabled = false; + +var doSimulcast = (getQueryStringValue("simulcast") === "yes" || getQueryStringValue("simulcast") === "true"); +var doSvc = getQueryStringValue("svc"); +if(doSvc === "") + doSvc = null; +var acodec = (getQueryStringValue("acodec") !== "" ? getQueryStringValue("acodec") : null); +var vcodec = (getQueryStringValue("vcodec") !== "" ? getQueryStringValue("vcodec") : null); +var vprofile = (getQueryStringValue("vprofile") !== "" ? getQueryStringValue("vprofile") : null); +var doDtx = (getQueryStringValue("dtx") === "yes" || getQueryStringValue("dtx") === "true"); +var doOpusred = (getQueryStringValue("opusred") === "yes" || getQueryStringValue("opusred") === "true"); +var simulcastStarted = false; + +// Canvas object +var canvas = null;; +var context = null; +var canvasStream = null; +var width = doSimulcast ? 1280 : 640, + height = doSimulcast ? 720 : 360; + +// We use this image as our virtual background +const image = new Image(); +image.src = './background/retro.webp'; + +$(document).ready(function() { + canvas = document.getElementById('canvas'); + context = canvas.getContext('2d'); + // Initialize the library (all console debuggers enabled) + Janus.init({debug: "all", callback: function() { + // Use a button to start the demo + $('#start').one('click', function() { + $(this).attr('disabled', true).unbind('click'); + // Make sure the browser supports WebRTC + if(!Janus.isWebrtcSupported()) { + bootbox.alert("No WebRTC support... "); + return; + } + // Create session + janus = new Janus( + { + server: server, + // No "iceServers" is provided, meaning janus.js will use a default STUN server + // Here are some examples of how an iceServers field may look like to support TURN + // iceServers: [{urls: "turn:yourturnserver.com:3478", username: "janususer", credential: "januspwd"}], + // iceServers: [{urls: "turn:yourturnserver.com:443?transport=tcp", username: "janususer", credential: "januspwd"}], + // iceServers: [{urls: "turns:yourturnserver.com:443?transport=tcp", username: "janususer", credential: "januspwd"}], + // Should the Janus API require authentication, you can specify either the API secret or user token here too + // token: "mytoken", + // or + // apisecret: "serversecret", + success: function() { + // Attach to EchoTest plugin + janus.attach( + { + plugin: "janus.plugin.echotest", + opaqueId: opaqueId, + success: function(pluginHandle) { + $('#details').remove(); + echotest = pluginHandle; + Janus.log("Plugin attached! (" + echotest.getPlugin() + ", id=" + echotest.getId() + ")"); + // We're connected to the plugin, create and populate the canvas element + createCanvas(); + $('#start').removeAttr('disabled').html("Stop") + .click(function() { + $(this).attr('disabled', true); + if(bitrateTimer) + clearInterval(bitrateTimer); + bitrateTimer = null; + janus.destroy(); + }); + }, + error: function(error) { + console.error(" -- Error attaching plugin...", error); + bootbox.alert("Error attaching plugin... " + error); + }, + consentDialog: function(on) { + Janus.debug("Consent dialog should be " + (on ? "on" : "off") + " now"); + if(on) { + // Darken screen and show hint + $.blockUI({ + message: '
    ', + css: { + border: 'none', + padding: '15px', + backgroundColor: 'transparent', + color: '#aaa', + top: '10px', + left: (navigator.mozGetUserMedia ? '-100px' : '300px') + } }); + } else { + // Restore screen + $.unblockUI(); + } + }, + iceState: function(state) { + Janus.log("ICE state changed to " + state); + }, + mediaState: function(medium, on, mid) { + Janus.log("Janus " + (on ? "started" : "stopped") + " receiving our " + medium + " (mid=" + mid + ")"); + }, + webrtcState: function(on) { + Janus.log("Janus says our WebRTC PeerConnection is " + (on ? "up" : "down") + " now"); + $("#videoleft").parent().unblock(); + }, + slowLink: function(uplink, lost, mid) { + Janus.warn("Janus reports problems " + (uplink ? "sending" : "receiving") + + " packets on mid " + mid + " (" + lost + " lost packets)"); + }, + onmessage: function(msg, jsep) { + Janus.debug(" ::: Got a message :::", msg); + if(jsep) { + Janus.debug("Handling SDP as well...", jsep); + echotest.handleRemoteJsep({ jsep: jsep }); + } + var result = msg["result"]; + if(result) { + if(result === "done") { + // The plugin closed the echo test + bootbox.alert("The Echo Test is over"); + if(spinner) + spinner.stop(); + spinner = null; + $('video').remove(); + $('#waitingvideo').remove(); + $('#peervideo').remove(); + $('#toggleaudio').attr('disabled', true); + $('#togglevideo').attr('disabled', true); + $('#bitrate').attr('disabled', true); + $('#curbitrate').hide(); + $('#curres').hide(); + return; + } + // Any loss? + var status = result["status"]; + if(status === "slow_link") { + toastr.warning("Janus apparently missed many packets we sent, maybe we should reduce the bitrate", "Packet loss?", {timeOut: 2000}); + } + } + // Is simulcast in place? + var substream = msg["substream"]; + var temporal = msg["temporal"]; + if((substream !== null && substream !== undefined) || (temporal !== null && temporal !== undefined)) { + if(!simulcastStarted) { + simulcastStarted = true; + addSimulcastButtons(msg["videocodec"] === "vp8"); + } + // We just received notice that there's been a switch, update the buttons + updateSimulcastButtons(substream, temporal); + } + }, + onlocaltrack: function(track, on) { + // We use the track ID as name of the element, but it may contain invalid characters + var trackId = track.id.replace(/[{}]/g, ""); + if(!on) { + // Track removed, get rid of the stream and the rendering + var stream = localTracks[trackId]; + if(stream) { + try { + var tracks = stream.getTracks(); + for(var i in tracks) { + var mst = tracks[i]; + if(mst) + mst.stop(); + } + } catch(e) {} + } + if(track.kind === "video") { + $('#myvideo' + trackId).remove(); + localVideos--; + if(localVideos === 0) { + // No video, at least for now: show a placeholder + if($('#videoleft .no-video-container').length === 0) { + $('#videoleft').append( + '
    ' + + '' + + 'No webcam available' + + '
    '); + } + } + } + delete localTracks[trackId]; + return; + } + // If we're here, a new track was added + var stream = localTracks[trackId]; + if(stream) { + // We've been here already + return; + } + if($('#videoleft video').length === 0) { + $('#videos').removeClass('hide').show(); + } + if(track.kind === "audio") { + // We ignore local audio tracks, they'd generate echo anyway + if(localVideos === 0) { + // No video, at least for now: show a placeholder + if($('#videoleft .no-video-container').length === 0) { + $('#videoleft').append( + '
    ' + + '' + + 'No webcam available' + + '
    '); + } + } + } else { + // New video track: create a stream out of it + localVideos++; + $('#videoleft .no-video-container').remove(); + stream = new MediaStream(); + stream.addTrack(track.clone()); + localTracks[trackId] = stream; + Janus.log("Created local stream:", stream); + $('#videoleft').append('