turn: add TURN server #163

netaskd · 2019-12-11T17:02:37Z

adds TURN server to the infrastructure.

saghul

Thanks a lot for getting this effort started! ❤️ Left some comments.

prosody/Dockerfile

turn.yml

turn/rootfs/defaults/docker-entrypoint.sh

web/rootfs/defaults/config.js

prosody/rootfs/defaults/conf.d/jitsi-meet.cfg.lua

saghul

Did another review round!

README.md

turn.yml

turn/rootfs/defaults/docker-entrypoint.sh

turn/Dockerfile

saghul · 2019-12-12T21:45:57Z

prosody/rootfs/defaults/conf.d/jitsi-meet.cfg.lua

+turncredentials_secret = "{{ .Env.TURN_SECRET | default "keepthissecret" }}";
+turncredentials = {
+ { type = "{{ .Env.TURN_TYPE | default "turns" }}",
+ host = "{{ .Env.TURN_HOST | default .Env.DOCKER_HOST_ADDRESS }}",


Shouldn't the default be TURN_PUBLIC_IP ?

prosody doesn't know about TURN_PUBLIC_IP in case TURN_PUBLIC_IP is not set from env file (got dynamically)

We can detect it in the init file and re-export it, perhaps?

I'll try to explain my view on it. TURN_HOST is an announced FQDN for reach cluster of turn. It has use inside prosody for provide XEP-0215.
TURN_PUBLIC_IP is a exactly external IP address of turn instance.
When we use TURN_HOST="turn.example.com", conditionally, A record can be turn.example.com TURN_PUBLIC_IP1, TURN_PUBLIC_IP2, TURN_PUBLIC_IP3 and client (that got TURN_HOST via XEP-0215) goes to this scope.

On the other hand, if it's just a single setup. TURN_HOST can be DOCKER_HOST_ADDRESS (if not set) and TURN_PUBLIC_IP is dynamically set.

I'm not completely sure of the latest comment.. I guess, we should remove default set for "TURN_HOST" and make it force needed, because if TURN_PUBLIC_IP and DOCKER_HOST_ADDRESS is different, setup will broken. WDYT?

Not sure I follow you here. I do understand that you could set a hostname. But if you don't, the IP that should be there is the publicc IP of the TURN server, right?

On hindsight DOCKER_HOST_ADDRESS was not a great name. I'll probably rename it to JVB_PUBLIC_IP in the future, which is a better name.

So, to me, it makes sense that the default is the TURN public IP, and not anything else.

I suggest to use only TURN_HOST for spreading hostname via XEP-0215. In this case we are suppose that:
a) the turn instance only one, and we exactly know how to reach it from outside.
b) the turn instance is multiple, but behind proxy. so, proxy knows how to reach each of them.

The TURN_HOST is mandatory variable, otherwise do not enable turn functions.

@saghul WDYT?

web/rootfs/etc/cont-init.d/10-config

turn/Dockerfile

env.example

prosody/Dockerfile

env.example

jkroepke · 2020-04-21T21:50:36Z

Can I help here? I have free time...

sliard · 2020-04-22T14:51:11Z

For information I finish a version that integrate a turn server into web container.
Why insert coturn inside web container and not create a new one ? It's for use the same SSH certificate.
https://github.com/sliard/docker-jitsi-meet/tree/upgrade_add_turn

It's not fair to create a new MR because it's base on this one at the begining.
I have one last problem, it works only with network_mode host into docker compose.

…osody

ccoenen · 2020-04-23T15:58:54Z

One thing that's currently missing is documentation: which ports one would need to open additionally. This should be added here:

https://github.com/jitsi/docker-jitsi-meet#external-ports

netaskd · 2020-04-23T16:52:35Z

One thing that's currently missing is documentation: which ports one would need to open additionally. This should be added here:

https://github.com/jitsi/docker-jitsi-meet#external-ports

added in 14d717c

holtgrewe · 2020-05-11T20:00:30Z

Hi, any news on this?

mihaiplesa · 2020-05-13T06:53:52Z

Having this in will be great.

holtgrewe · 2020-05-19T07:09:03Z

ping?

luixxiul · 2020-05-20T13:57:42Z

README.md

@@ -47,6 +47,9 @@ If you want to use jibri too, first configure host as described in JItsi BRoadca
 and then run Docker Compose as follows: ``docker-compose -f docker-compose.yml -f jibri.yml up -d``
 or to use jigasi too: ``docker-compose -f docker-compose.yml -f jigasi.yml -f jibri.yml up -d``

+If you want to enable TURN server, configure it and run Docker Compose as


This change needs to be applied to https://github.com/jitsi/handbook/blob/master/docs/devops-guide/docker.md (cf: #601)

Is there anything I can do to support his PR towards being merged? I'd really like to vacate Zoom with all of its problems for Jitsi Meet...

netaskd · 2020-05-25T21:53:40Z

As I see we have only one unresolved conversation around it. So, if @saghul have a couple of minutes to take a look this, I beleive we will fastly resolve and merge it.

holtgrewe · 2020-06-02T07:41:09Z

@saghul could you have a look at this?

Only you can save mankind from Zoom meetings :)

dsantos21 · 2020-06-17T16:57:49Z

Hi! Although I'm not really versed on all needed to make this work on docker, I could test it in my on premises installation. Would it help? We are using docker, but setting up TURN without docker. Yes... Our JMS doesn't currently have a TURN server... :(

citosid · 2020-06-23T18:19:37Z

Thank you so much for this work! I look forward to see this! Jitsi community really rocks!

holtgrewe · 2020-06-25T07:30:34Z

Hi, is there anything that could be done here to move this forward?

goacid · 2020-06-25T10:34:10Z

prosody/rootfs/defaults/conf.d/jitsi-meet.cfg.lua

+{{ if or (.Env.TURN_ENABLE_P2P | default "0" | toBool) (.Env.TURN_ENABLE_JVB | default "0" | toBool) }}
+turncredentials_secret = "{{ .Env.TURN_SECRET | default "keepthissecret" }}";
+turncredentials = {
+ { type = "{{ .Env.TURN_TYPE | default "turns" }}",
+ host = "{{ .Env.TURN_HOST | default "8.8.8.8" }}",
+ port = {{ .Env.TURN_PORT | default "3478" }},
+ transport = "{{ .Env.TURN_TRANSPORT | default "tcp" }}"
+ }
+}
+{{ end }}
+


The credential must be set in the prosody.cfg.lua files and not in the jitsi-meet.cfg.lua.
In jitsi-meet.cfg.lua you have to add the "turncredantials" in the module_enabled section of the meet.jitsi Virtualhost.

nestorconde · 2020-06-28T11:34:51Z

I've worked on this from the latest docker-jitsi-meet adding all the changes of the PR except for two:

It does not contain the changes to the README of the original PR since that page content has been moved somewhere else
It changes the name of the turn docker container (from jitsi/turn which wasn't found to instrumentisto/coturn).

I attach the patch which is mostly the same of the PR except for these two changes. I'd also be able to provide the terraform script that created the machines I'm testing in (in Digital Ocean). turn.diff

Problems

Jitsi works but I'm having some problems to test whether the turn server is working.

I can't block the port 10000/udp with ufw. I've tried to block ports with ufw but it doesn't seem to have any effect perhaps due to this?.

The other thing I've tried is to checking chrome://webrtc-internals (since I haven't been able to block 10000 this is not as useful as it could be). When I test with several participants I see in the some of the tabs "iceServers: [stun:meet-jit-si-turnrelay.jitsi.net:443]", but this is the same I see in a server where I have not applied the patch

@netaskd Any way I could go on from here in order to provide help to advance this pull request?

First option I can think of is to try other ways to block port 10000. Do other iptables management utilities have the same problem as ufw with respect to docker?, any advice on how to do block 10000/udp without stumbling on this problem?

holtgrewe · 2020-06-29T08:00:49Z

First option I can think of is to try other ways to block port 10000. Do other iptables management utilities have the same problem as ufw with respect to docker?, any advice on how to do block 10000/udp without stumbling on this problem?

Hi, does this help?

https://stackoverflow.com/questions/49549834/ufw-firewall-is-not-working-on-ubuntu-in-digitalocean/49563279#49563279

baccenfutter · 2020-06-29T08:24:38Z

Docker generally doesn't play well out-of-the-box with firewall configuration tools such as UFW, shorewall, etc. Such tools generate IPTABLES rules based on some "easier" configuration format, while Docker itself leverages IPTABLES to "magically" configure NAT for the containers. If both tools are not aware of each other, they will overwrite or shadow each-other's IPTABLE rules.

Both UFW and Shorwall have builtin support for Docker in their latest versions which is disabled by default. Enabling this feature will make UFW "docker-aware" and preserve Dockers IPTABLE rules when run.

However, it is still rather complicated to block incoming connections to Docker via IPTABLE rules, due to the fact that NAT is processed before the FILTER rules, meaning that packets to Docker-configured ports are redirected into the corresponding container before the packets run through the FILTER chain of the host - therefor never hitting the rule.

See: https://2.bp.blogspot.com/-skGmyfTzXGI/U84PI9EbjkI/AAAAAAAAD6Q/kjjfG17_yo8/s1600/Linux-Iptables-firewall-schema.png

Solution

The correct way to prevent access into a Docker container is to explicitly state 127.0.0.1 in the --port argument when running a container, such as

docker run --port 127.0.0.1:10000:10000 ...

This will only allow access to port 10000 when the connection is coming from the Docker host itself and will deny access when coming from elsewhere.

parruc · 2020-06-30T07:00:00Z

I also need this. How can I help the development? What's still open?

nestorconde · 2020-06-30T11:20:02Z

@baccenfutter What I did was modify the docker-compose.yml at the root of the project by changing

jvb
   ...
    ports:
-      '${JVB_PORT}:${JVB_PORT}/udp'
+      '127.0.0.1:${JVB_PORT}:${JVB_PORT}/udp'

Is that equivalent to what you suggested? (I don't know enough about docker and docker-compose to evaluate whether it is).

In any case after doing that I was having problems with the audio/video from the videoconference which only worked momentarily and then stopped working. Perhaps there's some unexpected side effect to that way of restricting the port?, if that's not the case then my patch is not working when 10000/udp is blocked.

@holtgrewe I did not try that because it seemed to me that I'd be changing more things than strictly needed (it would affect potentially other things). In order to evaluate whether the patch is working I need a test that isolates the changes so I wanted to only block 10000/udp and leave everything else the same.

@netaskd @saghul There's a lot of interest in this PR and some people would like to help but we don't have the information on how to do it. As far as I understand there's just this change pending review but the turncredentials code in netaskd branch have changed since the time the review was requested. Could you provide some guidance if helping is possible? If it isn't it would be useful to know that too ;)

jitsi#163 Fix prosody configuration Fix web configuration Fix name : turn => coturn Update README.md

vkruoso · 2020-07-19T15:54:34Z

turn/rootfs/defaults/docker-entrypoint.sh

+--no-multicast-peers \
+--realm=${TURN_REALM:-realm} \
+--listening-ip=$(hostname -i) \
+--external-ip=${TURN_PUBLIC_IP} \


As per https://hub.docker.com/r/instrumentisto/coturn they recommend using the detect-external-ip tool to automatically get the external IP. I did not check how that works, but I'm not sure as well to add a dependency on icanhazip.com to get the TURN server up a good thing.

Actually, the script detect-external-ip do the same ;)
See https://github.com/instrumentisto/coturn-docker-image/blob/master/rootfs/usr/local/bin/detect-external-ip.sh

Yeah, should have checked it before. Awesome. I've been testing this and works great. My use case is on kubernetes and Im having some issues with networking, but that is another story.

This has been adopted for the upstream coturn image, too, despite the strategy it uses is a little different.

https://github.com/coturn/coturn/blob/master/docker/coturn/rootfs/usr/local/bin/detect-external-ip.sh

See previous comment https://github.com/jitsi/docker-jitsi-meet/pull/163/files#r802157850

pandel · 2020-08-20T20:37:42Z

Read through all of this but didn't get it... will this also support adding an external TURN server without starting an additional coturn docker container which isn't needed in that case?

hermanbanken · 2020-09-11T18:38:58Z

What is the status of this PR, what discussion is still pending?
I'm also working on integrating Jitsi, and we really need to support TURN for corporate VPNs.

The setup we are using is to split traffic with Envoy & also terminate TLS there. That way both Jitsi's NGINX/web container & CoTURN don't need to do TLS offloading. Example envoy.yaml configuration:

# docker run -v $PWD:/etc/ssl/private -v $PWD:/data envoyproxy/envoy:v1.14.4 /usr/local/bin/envoy -c /data/envoy.yaml --mode validate
#
admin:
  address:
    socket_address: { address: 0.0.0.0, port_value: 9901 }

static_resources:
  listeners:
  # Main listener
  - address:
      socket_address: { address: 0.0.0.0, port_value: 4430 }
    # TLS Inspector parses the SNI & ALPN settings
    listener_filters:
    - name: "envoy.filters.listener.tls_inspector"
      typed_config: {}
    # One filter chain per SNI & ALPN
    filter_chains:
      # shard-1 HTTP
      - filter_chain_match: # shard-1 http
          server_names: ["shard-1.eu.example.org"]
          application_protocols: ["http/1.1", "h2"]
        transport_socket: &tmpl_transport_socket
          name: envoy.transport_sockets.tls
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
            common_tls_context:
              tls_certificates:
              - certificate_chain: { filename: "/etc/ssl/private/tls.crt" }
                private_key: { filename: "/etc/ssl/private/tls.key" }
        filters: &tmpl_http_filters
          - name: envoy.filters.network.http_connection_manager
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
              stat_prefix: ingress_http
              route_config:
                virtual_hosts:
                - name: default
                  domains: "*"
                  routes:
                  - match: { prefix: "/" }
                    route: { cluster: service_shard1_nginx }
              http_filters:
                - name: envoy.filters.http.router
      # shard-1 TURN
      - filter_chain_match:
          server_names: ["shard-1.eu.example.org"]
        transport_socket: *tmpl_transport_socket
        filters:
          - name: envoy.filters.network.tcp_proxy
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
              stat_prefix: tcp
              cluster: service_shard1_turn
      # fallback HTTP (redirects to Application Level router)
      - filter_chain_match:
          server_names: ["*.example.org"]
        transport_socket: *tmpl_transport_socket
        filters:
          - name: envoy.filters.network.http_connection_manager
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
              stat_prefix: ingress_http
              route_config:
                virtual_hosts:
                - name: default
                  domains: "*"
                  routes:
                  - match: { prefix: "/" }
                    route: { cluster: service_fallback }
              http_filters:
                - name: envoy.filters.http.router
      # fallback for health check
      - filter_chain_match:
        transport_socket: *tmpl_transport_socket
        filters:
          - name: envoy.filters.network.http_connection_manager
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
              stat_prefix: direct_http
              route_config:
                virtual_hosts:
                - name: default
                  domains: "*"
                  routes:
                  - match: { prefix: "/" }
                    direct_response:
                      status: 200
                      body:
                        inline_string: "Fallback"
              http_filters:
                - name: envoy.filters.http.router
  clusters:
  - name: service_fallback
    connect_timeout: 0.25s
    type: strict_dns
    lb_policy: round_robin
    load_assignment:
      cluster_name: service_fallback
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: nginx-internal
                port_value: 80
  - name: service_shard1_nginx
    connect_timeout: 0.25s
    type: strict_dns
    lb_policy: round_robin
    load_assignment:
      cluster_name: service_shard1_nginx
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: nginx.shard-1.svc.cluster.local
                port_value: 80
  - name: service_shard1_turn
    connect_timeout: 0.25s
    type: strict_dns
    lb_policy: round_robin
    load_assignment:
      cluster_name: service_shard1_turn
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: turn.shard-1.svc.cluster.local
                port_value: 5349

I'm also working on an Envoy xDS API endpoint that serves this configuration dynamically. That would ease adding shards drastically in our Kubernetes setup.

nestorconde · 2020-10-02T08:16:11Z

I've documented in the community my tests with a simple firewall and:

The setup described in this PR
Multiplexing (for turn to work when its ports are blocked by the firewall) based on the docs. My exact setup for multiplexing can be seen here

As far as I can see under with this setup for the videoconferences to work it's enough if the firewall allows 80, 443 (both outbound and inbound) and 10000 outbound.

I know this PR does not include the multiplexing bit but I think this contributes to the conversation and the multiplexing part is an interesting direction for turn in docker-jitsi-meet to pursue

hamletmun · 2022-01-07T00:06:31Z

turn/rootfs/defaults/docker-entrypoint.sh

+# run coturn server with API auth method enabled.
+turnserver -n ${TURN_ADMIN_OPTIONS} \
+--verbose \
+--prod \


coturn/coturn@520e172

--no-software-attribute Production mode: hide the software version.

That's a good spot!

almereyda · 2022-02-08T23:56:04Z

turn/Dockerfile

@@ -0,0 +1,13 @@
+ARG VERSION
+FROM instrumentisto/coturn:${VERSION:-latest}


This has been deprecated: instrumentisto/coturn-docker-image@0a4a7f0

It is suggested to switch to the upstream image coturn/coturn (🔗).

VidarHUN · 2022-09-27T20:18:35Z

Hi Folks,

This PR is still alive or somebody working on it? What kind of job is left to do?

Also, I don't understand this PR. As I can see the main goal of this PR is to make it happen to configure an external TURN server for Jitsi and you have created a new image which will be a coTURN server for testing or if somebody wants to use it. Am I right? If it's true then I would like to help you to finish this, because from the commits and because of my lack of knowledge with the configuration I can't see another way to configure Jitsi containers to use TURN servers.

If we can configure Jitsi to use TURN server, then there is a way to somehow force the relay through TURN with p2p connections? Can I just put a zero or false value into an env variable and force it?

turn: add TURN server

0d1baf8

saghul requested changes Dec 11, 2019

View reviewed changes

netaskd added 2 commits December 12, 2019 18:42

turn: the fist review changes

854753f

turn: add forgotten variable TURN_ADMIN_ENABLE

6c6df2c

saghul requested changes Dec 12, 2019

View reviewed changes

turn: the second review

656d48b

saghul mentioned this pull request Dec 17, 2019

Add STUN/TURN support #147

Closed

sapkra reviewed Dec 19, 2019

View reviewed changes

turn/Dockerfile Show resolved Hide resolved

mjattiot reviewed Feb 4, 2020

View reviewed changes

env.example Show resolved Hide resolved

travis-leith mentioned this pull request Feb 9, 2020

Is self hosted and ecrypted jitsi blockable by an ISP? #186

Closed

Craeckie suggested changes Mar 18, 2020

View reviewed changes

env.example Outdated Show resolved Hide resolved

sliard reviewed Mar 28, 2020

View reviewed changes

prosody/Dockerfile Show resolved Hide resolved

saghul mentioned this pull request Mar 30, 2020

Undocumented installation steps for using STUN/TURN with shared secrets #330

Open

mtyurt reviewed Apr 18, 2020

View reviewed changes

env.example Outdated Show resolved Hide resolved

netaskd added 3 commits April 23, 2020 15:06

base: add curl as necessary for getting mod_turncredentials.lua in pr…

5753213

…osody

misc: fix typo in a turn variable

cb8552a

misc: update default port min/max for turn service

d983578

netaskd force-pushed the pr_turn branch 2 times, most recently from 0cca664 to e68067d Compare April 23, 2020 16:42

vorburger and others added 2 commits April 23, 2020 19:48

doc: add tip re. ports to open on firewall to README

e0e0d81

turn: add ports that need to open additionally

14d717c

netaskd force-pushed the pr_turn branch from 51883d9 to 14d717c Compare April 23, 2020 16:51

netaskd requested a review from saghul May 5, 2020 20:57

lab-at-nohl mentioned this pull request May 17, 2020

Add .env variables for external TURN with shared secrets #595

Closed

luixxiul reviewed May 20, 2020

View reviewed changes

goacid reviewed Jun 25, 2020

View reviewed changes

goacid added a commit to goacid/docker-jitsi-meet that referenced this pull request Jul 3, 2020

Based on PR163 from netaskd

34b79ff

jitsi#163 Fix prosody configuration Fix web configuration Fix name : turn => coturn Update README.md

goacid mentioned this pull request Jul 3, 2020

Coturn #667

Open

vkruoso reviewed Jul 19, 2020

View reviewed changes

sapkra added the enhancement New feature or request label Oct 8, 2020

madmath03 mentioned this pull request Nov 23, 2020

✨ Add mod_turncredentials to prosody #837

Closed

pierreozoux mentioned this pull request May 25, 2021

Adds turn config. #1063

Closed

hamletmun reviewed Jan 7, 2022

View reviewed changes

almereyda reviewed Feb 8, 2022

View reviewed changes

		@@ -0,0 +1,13 @@
		ARG VERSION
		FROM instrumentisto/coturn:${VERSION:-latest}

turn: add TURN server #163

Are you sure you want to change the base?

turn: add TURN server #163

Conversation

netaskd commented Dec 11, 2019

saghul left a comment

Choose a reason for hiding this comment

saghul left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

netaskd Apr 23, 2020 • edited Loading

Choose a reason for hiding this comment

jkroepke commented Apr 21, 2020

sliard commented Apr 22, 2020 • edited Loading

ccoenen commented Apr 23, 2020

netaskd commented Apr 23, 2020

holtgrewe commented May 11, 2020

mihaiplesa commented May 13, 2020

holtgrewe commented May 19, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

netaskd commented May 25, 2020

holtgrewe commented Jun 2, 2020

dsantos21 commented Jun 17, 2020

citosid commented Jun 23, 2020

holtgrewe commented Jun 25, 2020

Choose a reason for hiding this comment

nestorconde commented Jun 28, 2020

holtgrewe commented Jun 29, 2020

baccenfutter commented Jun 29, 2020 • edited Loading

Solution

parruc commented Jun 30, 2020

nestorconde commented Jun 30, 2020

Choose a reason for hiding this comment

netaskd Jul 19, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

almereyda Feb 8, 2022 • edited Loading

Choose a reason for hiding this comment

pandel commented Aug 20, 2020

hermanbanken commented Sep 11, 2020 • edited Loading

nestorconde commented Oct 2, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

VidarHUN commented Sep 27, 2022

netaskd Apr 23, 2020 •

edited

Loading

sliard commented Apr 22, 2020 •

edited

Loading

baccenfutter commented Jun 29, 2020 •

edited

Loading

netaskd Jul 19, 2020 •

edited

Loading

almereyda Feb 8, 2022 •

edited

Loading

hermanbanken commented Sep 11, 2020 •

edited

Loading

nestorconde commented Oct 2, 2020 •

edited

Loading