Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make usable behind reverse proxy (like https://meet.jit.si/) #22

Open
nicolaspernoud opened this issue Oct 23, 2018 · 37 comments
Open

Make usable behind reverse proxy (like https://meet.jit.si/) #22

nicolaspernoud opened this issue Oct 23, 2018 · 37 comments
Labels
enhancement New feature or request

Comments

@nicolaspernoud
Copy link

https://meet.jit.si/ is usable behind a corporate reverse proxy (only TCP ports 80 and 443 allowed).
It would be amazing if the self hosted dockerised version would be as well...

Thanks.

Best regards and kudos for the fantastic work...

@damencho
Copy link
Member

This is covered in the TODO section of the README: what we need is a 'TURN server.' and a second public address, not sure how this is handled with docker.

@saghul
Copy link
Member

saghul commented Oct 23, 2018

@damencho We could also use that nginx module to proxy to a different upstream based on ALPN, right? That would be to do TURN on 443.

@damencho
Copy link
Member

Maybe, we need to test it, whether it works :)

@nicolaspernoud
Copy link
Author

Do you have a example of nginx configuration with alpn ?

@damencho
Copy link
Member

Nope, there is nothing we had tried, here is some stuff that I was reading:
https://superuser.com/questions/1135208/can-nginx-serve-ssh-and-https-at-the-same-time-on-the-same-port
http://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html#ssl_preread
The idea is nginx to detect https web traffic and serve the web (or bosh), but the rest to be considered as media traffic and forwarded to coturn tcp port, or the other way around, detect coturn traffic and forward it and the rest consider to be served by nginx. This way we can use port 443 for both serving the web and media.

@alexcustos
Copy link
Contributor

It seems I ran into the same issue. It's described well here jitsi/jitsi-meet#2805. I tried to use Turncredentials module to point my Coturn server, but Prosody crashes immediately after login, even with this module disabled. It's enough to have it in modules path.

So I would like to know if there is any way to enable TURN server for this Docker Compose setup?

Here's the relevant part of the log file
jitsi_prosody | mod_bosh                                  info  New BOSH session, assigned it sid 'c9307ab7-ad92-4493-992c-32cc6443b207'
jitsi_prosody | boshc9307ab7-ad92-4493-992c-32cc6443b207  info  Authenticated as hcisowftdjhb22or@guest.meet.jitsi
jitsi_prosody | mod_bosh                                  error Traceback[bosh]: /usr/lib/prosody/util/stanza.lua:62: invalid attribute value: expected string, got number
jitsi_prosody | stack traceback:
jitsi_prosody |         [C]: in function 'error'
jitsi_prosody |         /usr/lib/prosody/util/stanza.lua:62: in function 'check_text'
jitsi_prosody |         /usr/lib/prosody/util/stanza.lua:75: in function 'check_attr'
jitsi_prosody |         /usr/lib/prosody/util/stanza.lua:87: in function 'new_stanza'
jitsi_prosody |         /usr/lib/prosody/util/stanza.lua:109: in function 'tag'
jitsi_prosody |         /modules/mod_turncredentials.lua:28: in function '?'
jitsi_prosody |         /usr/lib/prosody/util/events.lua:79: in function </usr/lib/prosody/util/events.lua:75>
jitsi_prosody |         (...tail calls...)
jitsi_prosody |         /usr/lib/prosody/util/events.lua:79: in function </usr/lib/prosody/util/events.lua:75>
jitsi_prosody |         (...tail calls...)
jitsi_prosody |         /usr/lib/prosody/core/stanza_router.lua:180: in function 'core_post_stanza'
jitsi_prosody |         /usr/lib/prosody/core/stanza_router.lua:127: in function 'dispatch_stanza'
jitsi_prosody |         /usr/lib/prosody/modules/mod_bosh.lua:305: in function 'func'
jitsi_prosody |         /usr/lib/prosody/util/async.lua:127: in function </usr/lib/prosody/util/async.lua:125>
jitsi_prosody | stack traceback:
jitsi_prosody |         /usr/lib/prosody/util/async.lua:211: in function 'run'
jitsi_prosody |         /usr/lib/prosody/modules/mod_bosh.lua:447: in function 'cb_handlestanza'
jitsi_prosody |         /usr/lib/prosody/util/xmppstream.lua:182: in function </usr/lib/prosody/util/xmppstream.lua:162>
jitsi_prosody |         [C]: in function 'parse'
jitsi_prosody |         /usr/lib/prosody/util/xmppstream.lua:282: in function 'feed'
jitsi_prosody |         /usr/lib/prosody/modules/mod_bosh.lua:133: in function '?'
jitsi_prosody |         /usr/lib/prosody/util/events.lua:79: in function </usr/lib/prosody/util/events.lua:75>
jitsi_prosody |         (...tail calls...)
jitsi_prosody |         /usr/lib/prosody/net/http/server.lua:228: in function </usr/lib/prosody/net/http/server.lua:176>
jitsi_prosody |         [C]: in function 'xpcall'
jitsi_prosody |         /usr/lib/prosody/net/http/server.lua:108: in function 'process_next'
jitsi_prosody |         /usr/lib/prosody/net/http/server.lua:124: in function 'success_cb'
jitsi_prosody |         /usr/lib/prosody/net/http/parser.lua:177: in function 'feed'
jitsi_prosody |         /usr/lib/prosody/net/http/server.lua:155: in function </usr/lib/prosody/net/http/server.lua:154>
jitsi_prosody |         (...tail calls...)
jitsi_prosody |         /usr/lib/prosody/net/server_select.lua:915: in function </usr/lib/prosody/net/server_select.lua:899>
jitsi_prosody |         [C]: in function 'xpcall'
jitsi_prosody |         /usr/bin/prosody:80: in function 'loop'
jitsi_prosody |         /usr/bin/prosody:90: in main chunk
jitsi_prosody |         [C]: in ?
jitsi_prosody | boshc9307ab7-ad92-4493-992c-32cc6443b207  info  BOSH client disconnected: session close

@saghul
Copy link
Member

saghul commented Jan 24, 2019

So I would like to know if there is any way to enable TURN server for this Docker Compose setup?

Not at the moment, sorry.

@madmath03
Copy link
Contributor

It seems I have managed to fix this with #147, but that seemed far simpler than what I expected when reading earlier exchange regarding NGinx.

My P2P connections are using our dockerized Coturn server behind a reverse proxy, and I can successfully see Coturn receiving connections and am able to call other people on different networks.

In terms of configuration, I'm using something similar to this:

  # Frontend
  web:
    #image: jitsi/web
    image: monogramm/jitsi-web:dev
    # ...
    environment:
      - JVB_ENABLE_STUN_TURN=1
      - JVB_ENABLE_P2P_STUN_TURN=1
      # ...

  # XMPP server
  prosody:
    #image: jitsi/prosody
    image: monogramm/jitsi-prosody:dev
    # ...
    environment:
      - GLOBAL_MODULES=turncredentials
      - GLOBAL_CONFIG=turncredentials_secret = "${COTURN_TURN_SECRET}";\nturncredentials_host = "turn.${DOMAIN}";\nturncredentials_port = 5349;
      # ...

  # Coturn
  coturn:
    image: monogramm/docker-coturn:4
    container_name: coturn
    restart: always
    network_mode: "host"
    tty: true
    expose:
      - '3478'
      - '5349'
      - '3479'
      - '5350'
    environment:
      - LISTENING_PORT=3478
      - TLS_LISTENING_PORT=5349
      - ALT_LISTENING_PORT=3479
      - ALT_TLS_LISTENING_PORT=5350
      - LISTEN_IPS=${COTURN_LISTEN_IPS}
      - RELAY_IP=${COTURN_RELAY_IP}
      - STATIC_AUTH_SECRET=${COTURN_TURN_SECRET}
      - TLS_CERT=/ssl/coturn/live/${DOMAIN}/fullchain.pem
      - TLS_KEY=/ssl/coturn/live/${DOMAIN}/privkey.pem
      - REALM=turn.${DOMAIN}
      - VERBOSE=1
      - DEBUG=0
      - PROD=1
    volumes:
      - /srv/coturn/data:/srv
      - /etc/letsencrypt/live/${DOMAIN}:/ssl/coturn/live/${DOMAIN}:ro
      - /etc/letsencrypt/archive/${DOMAIN}:/ssl/coturn/archive/${DOMAIN}:ro
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro

What do you guys think ?

@damencho
Copy link
Member

@madmath03 I haven't looked in details but just wanted to give an idea, something I'm trying to test these days.

First, we use one other version of mod_turncredentials.lua which allows multiple entries configured. Like stun, turn and turns. https://github.com/otalk/mod_turncredentials/blob/master/mod_turncredentials.lua
With the above configured we can configure p2p to use this stun/turn server, rather than google's stun ...
The module needs few modifications to be used with prosody 0.11 (the types or ports for the server needed to be changed, cannot remmber right now).

And my idea is running coturn on some port, and advertise coturn turns to the main domain and port 443 and use nginx's preread and coturn ALPN to forward traffic from nginx to coturn. Here is some info I found on the subject:
coturn/coturn#259
https://superuser.com/questions/1135208/can-nginx-serve-ssh-and-https-at-the-same-time-on-the-same-port
http://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html#ssl_preread
https://github.com/coturn/coturn/wiki/turnserver#alpn

The above idea is something I want to add to jitsi-meet default install and add the mod_turncredentials to come with jitsi-meet-prosody pcakage, but few other changes are needed. I have it in my todo to test it and move some stuff ... but cannot give any ETA, as it is not my primary focus right now.

@madmath03
Copy link
Contributor

Thank you @damencho for your answer.

Being able to set multiple STUN / TURN entries definitely sounds nice. Modifying my PR to use another version of mod_turncredentials.lua is fairly easy, just need to have this other version either publicly accessible or in the repository. Maybe I can give it a try if you do not mind.

The idea to use port 443 and use NGinx preread + Coturn ALPN seems really nice too but rather complicated... I do not think I would be able to help with that.

I think the PR I sent is not incompatible with your objective, as you just need to replace mod_turncredentials and add the NGinx / Coturn config, and it might give a working solution for those who need a TURN server with Jitsi Meet.

@damencho
Copy link
Member

Yep, I agree. I'm not so familiar with docker, so my goal is debian packages. But we were discussing this with Saul at some point and will try to add it. As a started next week I can try pushing the mod_turncredntials to jitsi-meet repo so we have it there, need to do some more testing with it, though to make sure it works for the trunk we use and 0.11

@netaskd
Copy link
Contributor

netaskd commented Dec 11, 2019

Hi there,
I've done some tests using nginx ssl preread + coturn + mod_turncredentials.lua (prosody 0.11.3) on k8s.
Nginx ssl preread works good only with SNI. The idea with ALPN is good, but browsers do not send any stun/turn ALPN headers (coturn client it does). So, overall, the idea to use 443 port for the web and the turn traffic is working! @damencho thanks for the hint!
For this purpose, I've modified suggested lua module and now it works correctly with prosody 0.11.X: https://github.com/netaskd/mod_turncredentials/blob/master/mod_turncredentials.lua
@damencho could you please review it and add to the jitsi-meet plugins repo (or I can sent PR).
In the nearest future I'll add PR with turn support for docker-jitsi-meet repo.
Cheers!

@damencho
Copy link
Member

damencho commented Dec 11, 2019

I have worked also on the same and had good progress using ALPN. Browsers though send ALPN for the http traffic, this is how I did it. I send http traffice to the part serving meet and rest to cotrurn. I have plans to create a PR for that next week. I had also included the mod_turncredentials. I will compare your version with mine before creating the PR.
Thank you for the heads up.
I will be creating the PR in jitsi-meet, to install and configure by default coturn when installing with nginx, which also be the default choice and we are dropping jetty support.

@damencho
Copy link
Member

Here is the changes I was working on jitsi/jitsi-meet#4959

@pgnd
Copy link

pgnd commented Mar 29, 2020

is there a known-to-work, thoroughly documented example of setup & config for this^ jitsi-on-docker-behind-nginx-proxy anywhere yet? even if it's a WIP wiki doc ...

@madmath03 's config above is the closest I've found. so far ...

@saghul
Copy link
Member

saghul commented Mar 29, 2020

Not yet, we'll get to it soon.

@shoeberto
Copy link

+1 for instructions on reverse-proxying behind nginx. I tried the naive approach (quick start guide + setting up nginx reverse proxying rules) without much luck. Would be nice to have a turn-key solution.

@rolllo
Copy link

rolllo commented Apr 15, 2020

Hello All, I am also searching for a configuration of this
jitsi-on-docker-behind-nginx-proxy thing or
docker-compose-jitsi-with-traefik2.0 thing.
It would be so helpful to be able to integrate jitsi in a configurable reverse-proxy environment. Thank you!

@jussivesa
Copy link

Any updates on this?

@osscombat
Copy link

+1 for instructions on reverse-proxying behind nginx. I tried the naive approach (quick start guide + setting up nginx reverse proxying rules) without much luck. Would be nice to have a turn-key solution.

It is working fine right now, actually. Just proxypass to http 8000 and NAT the 10000/udp ports.

The only problem for me now is how to restrict the landing page from the public, at the same time allowing an access to the rooms, i.e.
forbid jitsi.domain.com
and allow
jitsi.domain.com/Chatroom

@shoeberto
Copy link

+1 for instructions on reverse-proxying behind nginx. I tried the naive approach (quick start guide + setting up nginx reverse proxying rules) without much luck. Would be nice to have a turn-key solution.

It is working fine right now, actually. Just proxypass to http 8000 and NAT the 10000/udp ports.

The only problem for me now is how to restrict the landing page from the public, at the same time allowing an access to the rooms, i.e.
forbid jitsi.domain.com
and allow
jitsi.domain.com/Chatroom

I've tried this, but continually get 502 Bad Gateway with "connection reset by peer" errors in the nginx logs when I try to hit my subdomain. Can you paste your nginx configuration, as well as any relevant docker-compose.yml/.env changes that you had to make?

@osscombat
Copy link

My reverse-proxy config for jitsi domain:

server {
    server_name jitsi.domain.com;
    set $upstream 192.168.0.100;

    location / {
    ssi on;
    proxy_pass http://$upstream:8000;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header Host $http_host;
    }
}

Here the 192.168.0.100 is the local address for docker host

In the .env file I edited only

DOCKER_HOST_ADDRESS=YOUR_PUBLIC_IP

all the other settings are the same as in example.env

To make audio & video working, you have to NAT the 10000/udp port from YOUR_PUBLIC_IP into 192.168.0.100 docker host using your firewall.

@shoeberto
Copy link

I'm still having issues getting nginx to talk to the container when the Docker daemon is running on localhost (502 codes, no matter the config). At this point I'm going to assume it's something silly that I'm missing. I think an authoritative guide would at least help point to issues that I'm not considering right now.

@jwillaz
Copy link

jwillaz commented Apr 18, 2020

After much frustration, I finally discovered the configuration that got things working for me. Here were my steps:

  1. As @osscombat mentioned, you definitely need to set up that proxy_pass in your Nginx config:
server {
    server_name jitsi.domain.com;
    set $upstream 192.168.0.100;

    location / {
    ssi on;
    proxy_pass http://$upstream:8000;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header Host $http_host;
    }
}
  1. On your router/firewall, make sure you have ports 80 (TCP), 443 (TCP), 4443 (TCP) and 10000 (UDP) forwarded to the LOCAL IP of your docker host (e.g. 192.168.x.x)
  2. I also set DOCKER_HOST_ADDRESS=192.168.x.x (my host server's local IP) in the .env file
  3. In the docker-compose.yml file, there seems to be a problem with the prosody configuration. In the example, it shows the following:
prosody:
image: jitsi/prosody
restart: ${RESTART_POLICY}
expose:
- '5222'
- '5347'
- '5280'

This needed to be changed to the following:

prosody:
image: jitsi/prosody
restart: ${RESTART_POLICY}
ports:
- 5222:5222
- 5347:5347
- 5280:5280
  1. I also needed to add this to each of the component configurations in docker-compose.yml (use your host local IP) to fix unknown host errors:
    extra_hosts:
      - "xmpp.meet.jitsi:192.168.x.x"

I hope this helps. Good luck!

@osscombat
Copy link

@jwillaz
I assume your proxy is not trusted in your setup, and you enable LE on your docker host to do the HTTPS job?

@nicolaspernoud
Copy link
Author

Just to remind a thing : the whole point of this issue was to be able to deploy jitsi without needing opening ports aside 443 : like meet.jit.si !
NAT and PAT of 9000, 4443 are not the point...

@jwillaz
Copy link

jwillaz commented Apr 19, 2020

@nicolaspernoud By documenting the tweaks that enable functionality for our given setups, I believe this provides good information for someone more well-versed to implement actual fixes. I agree with you - ideally I would not open anything beyond port 443, but for now it's either do this or don't use the tool at all.

@osscombat Yes, I am using the 'linuxserver/letsencrypt' image for nginx, and I'm mapping those certs to the Jitsi web container in my docker-compose.

@shoeberto
Copy link

Just to remind a thing : the whole point of this issue was to be able to deploy jitsi without needing opening ports aside 443 : like meet.jit.si !
NAT and PAT of 9000, 4443 are not the point...

In support of this, I would like to illustrate my setup, which I assume is very common for hobbyist deployments:

  • Server is a small VPS running Linux.
  • The VPS is also the docker host. Deployed containers expose their ports only on the loopback/localhost device.
  • The VPS runs nginx, with 80 and 443 already exposed to the world.
  • nginx is configured to listen to requests on subdomains that are pointed to the host, and performs reverse proxying between port 80/443 and whatever the appropriate container port is.
  • All subdomains are set up with LetsEncrypt, so traffic between clients and nginx is properly encrypted.

Since I'm trying to deploy Jitsi alongside other services that are on the machine, being able to reverse proxy requests to the subdomain to my container (including redirecting ports) is pretty critical. Obviously if the machine was just running the single Jitsi service, exposing the ports directly would make this task fairly trivial (minus the bit about self-managing certs).

I also am running a small number of other services successfully using this config, including LE certs and everything, so it's a bit frustrating to not understand why nginx reverse-proxy'd requests are resulting in 502s given that this is the only container where I'm having this issue.

@djpask
Copy link

djpask commented Apr 22, 2020

@jwillaz
Mine nginx reverse-proxied dockerized Jitsi-meet setup didn't work. But your advice to set DOCKER_HOST_ADDRESS=local_ip_address did the trick.
I noticed that in your nginx configuration you used your docker host address: you could also use your internat container hostname without exposing any port, provided that you add your jitsi meet docker network to your nginx container using

docker network connect docker-jitsi-meet_meet.jitsi nginx

Here is my nginx config (I'm using a linuxserver/letsencrypt nginx container)

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name meet.*;
    include /config/nginx/ssl.conf;
    client_max_body_size 0;    
    
location / {
        ssi on;
        include /config/nginx/proxy.conf;
        proxy_buffering off;
        proxy_pass http://docker-jitsi-meet_web_1;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header Host $http_host;
    }

}

For anyone seeing this post in times to come, I followed the https://github.com/jitsi/docker-jitsi-meet readme, but set DOCKER_HOST_ADDRESS=my_docker_host_local_ip and modified my nginx reverse proxy config as above. It worked.

@lvidaguren
Copy link

After much frustration, I finally discovered the configuration that got things working for me. Here were my steps:

  1. As @osscombat mentioned, you definitely need to set up that proxy_pass in your Nginx config:
server {
    server_name jitsi.domain.com;
    set $upstream 192.168.0.100;

    location / {
    ssi on;
    proxy_pass http://$upstream:8000;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header Host $http_host;
    }
}
  1. On your router/firewall, make sure you have ports 80 (TCP), 443 (TCP), 4443 (TCP) and 10000 (UDP) forwarded to the LOCAL IP of your docker host (e.g. 192.168.x.x)
  2. I also set DOCKER_HOST_ADDRESS=192.168.x.x (my host server's local IP) in the .env file
  3. In the docker-compose.yml file, there seems to be a problem with the prosody configuration. In the example, it shows the following:
prosody:
image: jitsi/prosody
restart: ${RESTART_POLICY}
expose:
- '5222'
- '5347'
- '5280'

This needed to be changed to the following:

prosody:
image: jitsi/prosody
restart: ${RESTART_POLICY}
ports:
- 5222:5222
- 5347:5347
- 5280:5280
  1. I also needed to add this to each of the component configurations in docker-compose.yml (use your host local IP) to fix unknown host errors:
    extra_hosts:
      - "xmpp.meet.jitsi:192.168.x.x"

I hope this helps. Good luck!

This one worked for me! I was having problems because of the corrupted and not deleted ~/.jitsi-meet-cfg folder.
I've also changed the
"xmpp.meet.jitsi:192.168.x.x"
for
"xmpp.meet.jitsi:${DOCKER_HOST_ADDRESS}"
on the components, since it's the same address and it's a usable environment variable

@osscombat
Copy link

I wonder, is it really necessary to
ssi on;
on the internet-facing reverse proxy in the above scenarios?

@pgnd
Copy link

pgnd commented May 16, 2020

@lvidaguren

when I make the change you suggested, to

prosody:
image: jitsi/prosody
restart: ${RESTART_POLICY}
ports:
- 5222:5222
- 5347:5347
- 5280:5280

And try a docker-compose up/down, I get

ERROR: The Compose file './docker-compose.yml' is invalid because:
services.prosody.expose is invalid: should be of the format 'PORT[/PROTOCOL]'
services.prosody.expose is invalid: should be of the format 'PORT[/PROTOCOL]'
services.prosody.expose is invalid: should be of the format 'PORT[/PROTOCOL]'

Do you see same? Did you make add'l changes?

@pruje
Copy link
Contributor

pruje commented May 18, 2020

@pgnd did you put the right indentation and quotes?
you should have:

ports:
  - '5222:5222'
  - '5347:5347'
  - '5280:5280'

@lvidaguren
Copy link

lvidaguren commented May 19, 2020

Hi @pgnd , sorry about the delay in the answer. Here is the docker-compose.yml diff, these were all the changes

docker-compose.yml.diff.gz

@lvidaguren
Copy link

@pruje The quotes didn't work for me in ports, Im using Docker version 19.03.6 and docker-compose docker-compose version 1.25.5, build 8a1c60f6 on Ubuntu 18.04

@ben-tvpp
Copy link

ben-tvpp commented May 26, 2020

I am a bit late to the party and sorry if I am getting the wrong end of the stick.

I use an Nginx reverse proxy and have got it working with lots of docker recipes. I just add

networks:
    default:
      external:
        name: nginx-proxy

To the end of the docker file and something like

        - VIRTUAL_HOST="jitsi.domain.tv"
        - LETSENCRYPT_HOST="jitsi.domain.tv"
        - LETSENCRYPT_EMAIL="bene@domain.com"

to the service that is web-facing. That's all it seems to take. the proxy I am using is at https://github.com/funkytwig/nginx-proxy. I've tried a few things (like adding nginx-proxy external network and changing all the networks to nginx-proxy) but can't get it working.

Sorry if I am barking up the wrong tree here but what have I missed.

@almereyda
Copy link
Contributor

To supplement the TURN configuration of @madmath03, there is nowadays an official documentation on how this should be configured:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests