Add worker_rlimit_nofile and worker_connections support to nginx config

README.md

@@ -149,6 +149,28 @@ ENV NGINX_WORKER_PROCESSES auto
 COPY ./app /app
 ```
 
+## Custom Nginx maximum connections per worker
+
+By default, Nginx will start with a maximum limit of 1024 connections per worker.
+
+If you want to set a different number you can use the environment variable `NGINX_WORKER_CONNECTIONS`, e.g:
+
+```Dockerfile
+ENV NGINX_WORKER_CONNECTIONS 2048
+```
+
+It cannot exceed the current limit on the maximum number of open files. See how to configure it in the next section.
+
+## Custom Nginx maximum open files
+
+The number connections per Nginx worker cannot exceed the limit on the maximum number of open files.
+
+You can change the limit of open files with the environment variable `NGINX_WORKER_OPEN_FILES`, e.g.:
+
+```Dockerfile
+ENV NGINX_WORKER_OPEN_FILES 2048
+```
+
 ## What's new
 
 <!-- 
@@ -164,6 +186,10 @@ To achieve that, the Python 3.6 version now uses a copy of the latest Nginx imag
 In the official Python image, there's a Stretch version only for Python 3.6. So, that's the only one that can be merged with the current Nginx image. That's why, in this image, only Python 3.6 supports multi-arch.
 
 -->
+* 2018-06-22: Make uWSGI require an app to run, instead of going in "full dynamic mode" while there was an error. Supervisord doesn't terminate itself but tries to restart uWSGI and shows the errors. Uses `need-app` as suggested by [luckydonald](https://github.com/luckydonald) in [this comment](https://github.com/tiangolo/uwsgi-nginx-flask-docker/issues/3#issuecomment-321991279).
+
+* 2018-06-22: Correctly handled graceful shutdown of uWSGI and Nginx. Thanks to [desaintmartin](https://github.com/desaintmartin) in [this PR](https://github.com/tiangolo/uwsgi-nginx-docker/pull/30).
+
 * 2018-02-04: It's now possible to set the number of Nginx worker processes with the environment variable `NGINX_WORKER_PROCESSES`. Thanks to [naktinis](https://github.com/naktinis) in [this PR](https://github.com/tiangolo/uwsgi-nginx-docker/pull/22).
 
 * 2018-01-14: There are now two Alpine based versions, `python2.7-alpine3.7` and `python3.6-alpine3.7`.

python2.7-alpine3.7/entrypoint.sh

@@ -15,6 +15,17 @@ USE_NGINX_WORKER_PROCESSES=${NGINX_WORKER_PROCESSES:-1}
 # Modify the number of worker processes in Nginx config
 sed -i "/worker_processes\s/c\worker_processes ${USE_NGINX_WORKER_PROCESSES};" /etc/nginx/nginx.conf
 
+# Set the max number of connections per worker for Nginx, if requested
+# Cannot exceed worker_rlimit_nofile, see NGINX_WORKER_OPEN_FILES below
+if [[ -v NGINX_WORKER_CONNECTIONS ]] ; then
+    sed -i "/worker_connections\s/c\    worker_connections ${NGINX_WORKER_CONNECTIONS};" /etc/nginx/nginx.conf
+fi
+
+# Set the max number of open file descriptors for Nginx workers, if requested
+if [[ -v NGINX_WORKER_OPEN_FILES ]] ; then
+    echo "worker_rlimit_nofile ${NGINX_WORKER_OPEN_FILES};" >> /etc/nginx/nginx.conf
+fi
+
 # Get the listen port for Nginx, default to 80
 USE_LISTEN_PORT=${LISTEN_PORT:-80}
 # Modify Nginx config for listen port

python2.7-alpine3.7/supervisord.ini

@@ -2,7 +2,7 @@
 nodaemon=true
 
 [program:uwsgi]
-command=/usr/sbin/uwsgi --ini /etc/uwsgi/uwsgi.ini --die-on-term --plugin python
+command=/usr/sbin/uwsgi --ini /etc/uwsgi/uwsgi.ini --die-on-term --need-app --plugin python
 stdout_logfile=/dev/stdout
 stdout_logfile_maxbytes=0
 stderr_logfile=/dev/stderr
@@ -14,3 +14,5 @@ stdout_logfile=/dev/stdout
 stdout_logfile_maxbytes=0
 stderr_logfile=/dev/stderr
 stderr_logfile_maxbytes=0
+# Graceful stop, see http://nginx.org/en/docs/control.html
+stopsignal=QUIT

python2.7-alpine3.7/uwsgi.ini

@@ -2,3 +2,5 @@
 socket = /tmp/uwsgi.sock
 chown-socket = nginx:nginx
 chmod-socket = 664
+# Graceful shutdown on SIGTERM, see https://github.com/unbit/uwsgi/issues/849#issuecomment-118869386
+hook-master-start = unix_signal:15 gracefully_kill_them_all

python2.7/Dockerfile

@@ -1,22 +1,106 @@
 FROM python:2.7
 
-MAINTAINER Sebastian Ramirez <tiangolo@gmail.com>
+LABEL maintainer="Sebastian Ramirez <tiangolo@gmail.com>"
 # Install uWSGI
 RUN pip install uwsgi
 
 # Standard set up Nginx
-ENV NGINX_VERSION 1.9.11-1~jessie
+ENV NGINX_VERSION 1.13.12-1~stretch
+ENV NJS_VERSION   1.13.12.0.2.0-1~stretch
 
-RUN apt-key adv --keyserver hkp://pgp.mit.edu:80 --recv-keys 573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62 \
-	&& echo "deb http://nginx.org/packages/mainline/debian/ jessie nginx" >> /etc/apt/sources.list \
+RUN set -x \
 	&& apt-get update \
-	&& apt-get install -y ca-certificates nginx=${NGINX_VERSION} gettext-base \
-	&& rm -rf /var/lib/apt/lists/*
+	&& apt-get install --no-install-recommends --no-install-suggests -y gnupg1 apt-transport-https ca-certificates \
+	&& \
+	NGINX_GPGKEY=573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62; \
+	found=''; \
+	for server in \
+		ha.pool.sks-keyservers.net \
+		hkp://keyserver.ubuntu.com:80 \
+		hkp://p80.pool.sks-keyservers.net:80 \
+		pgp.mit.edu \
+	; do \
+		echo "Fetching GPG key $NGINX_GPGKEY from $server"; \
+		apt-key adv --keyserver "$server" --keyserver-options timeout=10 --recv-keys "$NGINX_GPGKEY" && found=yes && break; \
+	done; \
+	test -z "$found" && echo >&2 "error: failed to fetch GPG key $NGINX_GPGKEY" && exit 1; \
+	apt-get remove --purge --auto-remove -y gnupg1 && rm -rf /var/lib/apt/lists/* \
+	&& dpkgArch="$(dpkg --print-architecture)" \
+	&& nginxPackages=" \
+		nginx=${NGINX_VERSION} \
+		nginx-module-xslt=${NGINX_VERSION} \
+		nginx-module-geoip=${NGINX_VERSION} \
+		nginx-module-image-filter=${NGINX_VERSION} \
+		nginx-module-njs=${NJS_VERSION} \
+	" \
+	&& case "$dpkgArch" in \
+		amd64|i386) \
+# arches officialy built by upstream
+			echo "deb https://nginx.org/packages/mainline/debian/ stretch nginx" >> /etc/apt/sources.list.d/nginx.list \
+			&& apt-get update \
+			;; \
+		*) \
+# we're on an architecture upstream doesn't officially build for
+# let's build binaries from the published source packages
+			echo "deb-src https://nginx.org/packages/mainline/debian/ stretch nginx" >> /etc/apt/sources.list.d/nginx.list \
+			\
+# new directory for storing sources and .deb files
+			&& tempDir="$(mktemp -d)" \
+			&& chmod 777 "$tempDir" \
+# (777 to ensure APT's "_apt" user can access it too)
+			\
+# save list of currently-installed packages so build dependencies can be cleanly removed later
+			&& savedAptMark="$(apt-mark showmanual)" \
+			\
+# build .deb files from upstream's source packages (which are verified by apt-get)
+			&& apt-get update \
+			&& apt-get build-dep -y $nginxPackages \
+			&& ( \
+				cd "$tempDir" \
+				&& DEB_BUILD_OPTIONS="nocheck parallel=$(nproc)" \
+					apt-get source --compile $nginxPackages \
+			) \
+# we don't remove APT lists here because they get re-downloaded and removed later
+			\
+# reset apt-mark's "manual" list so that "purge --auto-remove" will remove all build dependencies
+# (which is done after we install the built packages so we don't have to redownload any overlapping dependencies)
+			&& apt-mark showmanual | xargs apt-mark auto > /dev/null \
+			&& { [ -z "$savedAptMark" ] || apt-mark manual $savedAptMark; } \
+			\
+# create a temporary local APT repo to install from (so that dependency resolution can be handled by APT, as it should be)
+			&& ls -lAFh "$tempDir" \
+			&& ( cd "$tempDir" && dpkg-scanpackages . > Packages ) \
+			&& grep '^Package: ' "$tempDir/Packages" \
+			&& echo "deb [ trusted=yes ] file://$tempDir ./" > /etc/apt/sources.list.d/temp.list \
+# work around the following APT issue by using "Acquire::GzipIndexes=false" (overriding "/etc/apt/apt.conf.d/docker-gzip-indexes")
+#   Could not open file /var/lib/apt/lists/partial/_tmp_tmp.ODWljpQfkE_._Packages - open (13: Permission denied)
+#   ...
+#   E: Failed to fetch store:/var/lib/apt/lists/partial/_tmp_tmp.ODWljpQfkE_._Packages  Could not open file /var/lib/apt/lists/partial/_tmp_tmp.ODWljpQfkE_._Packages - open (13: Permission denied)
+			&& apt-get -o Acquire::GzipIndexes=false update \
+			;; \
+	esac \
+	\
+	&& apt-get install --no-install-recommends --no-install-suggests -y \
+						$nginxPackages \
+						gettext-base \
+	&& rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/nginx.list \
+	\
+# if we have leftovers from building, let's purge them (including extra, unnecessary build deps)
+	&& if [ -n "$tempDir" ]; then \
+		apt-get purge -y --auto-remove \
+		&& rm -rf "$tempDir" /etc/apt/sources.list.d/temp.list; \
+	fi
+
 # forward request and error logs to docker log collector
 RUN ln -sf /dev/stdout /var/log/nginx/access.log \
 	&& ln -sf /dev/stderr /var/log/nginx/error.log
-EXPOSE 80 443
-# Finished setting up Nginx
+EXPOSE 80
+# Removed the section that breaks pip installations
+# && apt-get remove --purge --auto-remove -y apt-transport-https ca-certificates 
+# Standard set up Nginx finished
+
+# Expose 443, in case of LTS / HTTPS
+EXPOSE 443
 
 # Make NGINX run on the foreground
 RUN echo "daemon off;" >> /etc/nginx/nginx.conf

python2.7/entrypoint.sh

@@ -11,6 +11,17 @@ USE_NGINX_WORKER_PROCESSES=${NGINX_WORKER_PROCESSES:-1}
 # Modify the number of worker processes in Nginx config
 sed -i "/worker_processes\s/c\worker_processes ${USE_NGINX_WORKER_PROCESSES};" /etc/nginx/nginx.conf
 
+# Set the max number of connections per worker for Nginx, if requested
+# Cannot exceed worker_rlimit_nofile, see NGINX_WORKER_OPEN_FILES below
+if [[ -v NGINX_WORKER_CONNECTIONS ]] ; then
+    sed -i "/worker_connections\s/c\    worker_connections ${NGINX_WORKER_CONNECTIONS};" /etc/nginx/nginx.conf
+fi
+
+# Set the max number of open file descriptors for Nginx workers, if requested
+if [[ -v NGINX_WORKER_OPEN_FILES ]] ; then
+    echo "worker_rlimit_nofile ${NGINX_WORKER_OPEN_FILES};" >> /etc/nginx/nginx.conf
+fi
+
 # Get the listen port for Nginx, default to 80
 USE_LISTEN_PORT=${LISTEN_PORT:-80}
 # Modify Nignx config for listen port

python2.7/supervisord.conf

@@ -2,7 +2,7 @@
 nodaemon=true
 
 [program:uwsgi]
-command=/usr/local/bin/uwsgi --ini /etc/uwsgi/uwsgi.ini --die-on-term
+command=/usr/local/bin/uwsgi --ini /etc/uwsgi/uwsgi.ini --die-on-term --need-app
 stdout_logfile=/dev/stdout
 stdout_logfile_maxbytes=0
 stderr_logfile=/dev/stderr
@@ -14,3 +14,5 @@ stdout_logfile=/dev/stdout
 stdout_logfile_maxbytes=0
 stderr_logfile=/dev/stderr
 stderr_logfile_maxbytes=0
+# Graceful stop, see http://nginx.org/en/docs/control.html
+stopsignal=QUIT

python2.7/uwsgi.ini

@@ -2,3 +2,5 @@
 socket = /tmp/uwsgi.sock
 chown-socket = nginx:nginx
 chmod-socket = 664
+# Graceful shutdown on SIGTERM, see https://github.com/unbit/uwsgi/issues/849#issuecomment-118869386
+hook-master-start = unix_signal:15 gracefully_kill_them_all

python3.5/entrypoint.sh

@@ -11,6 +11,17 @@ USE_NGINX_WORKER_PROCESSES=${NGINX_WORKER_PROCESSES:-1}
 # Modify the number of worker processes in Nginx config
 sed -i "/worker_processes\s/c\worker_processes ${USE_NGINX_WORKER_PROCESSES};" /etc/nginx/nginx.conf
 
+# Set the max number of connections per worker for Nginx, if requested
+# Cannot exceed worker_rlimit_nofile, see NGINX_WORKER_OPEN_FILES below
+if [[ -v NGINX_WORKER_CONNECTIONS ]] ; then
+    sed -i "/worker_connections\s/c\    worker_connections ${NGINX_WORKER_CONNECTIONS};" /etc/nginx/nginx.conf
+fi
+
+# Set the max number of open file descriptors for Nginx workers, if requested
+if [[ -v NGINX_WORKER_OPEN_FILES ]] ; then
+    echo "worker_rlimit_nofile ${NGINX_WORKER_OPEN_FILES};" >> /etc/nginx/nginx.conf
+fi
+
 # Get the listen port for Nginx, default to 80
 USE_LISTEN_PORT=${LISTEN_PORT:-80}
 # Modify Nignx config for listen port

python3.5/supervisord.conf

@@ -2,7 +2,7 @@
 nodaemon=true
 
 [program:uwsgi]
-command=/usr/local/bin/uwsgi --ini /etc/uwsgi/uwsgi.ini --die-on-term
+command=/usr/local/bin/uwsgi --ini /etc/uwsgi/uwsgi.ini --die-on-term --need-app
 stdout_logfile=/dev/stdout
 stdout_logfile_maxbytes=0
 stderr_logfile=/dev/stderr
@@ -14,3 +14,5 @@ stdout_logfile=/dev/stdout
 stdout_logfile_maxbytes=0
 stderr_logfile=/dev/stderr
 stderr_logfile_maxbytes=0
+# Graceful stop, see http://nginx.org/en/docs/control.html
+stopsignal=QUIT

python3.5/uwsgi.ini

@@ -2,3 +2,5 @@
 socket = /tmp/uwsgi.sock
 chown-socket = nginx:nginx
 chmod-socket = 664
+# Graceful shutdown on SIGTERM, see https://github.com/unbit/uwsgi/issues/849#issuecomment-118869386
+hook-master-start = unix_signal:15 gracefully_kill_them_all

python3.6-alpine3.7/entrypoint.sh

@@ -15,6 +15,17 @@ USE_NGINX_WORKER_PROCESSES=${NGINX_WORKER_PROCESSES:-1}
 # Modify the number of worker processes in Nginx config
 sed -i "/worker_processes\s/c\worker_processes ${USE_NGINX_WORKER_PROCESSES};" /etc/nginx/nginx.conf
 
+# Set the max number of connections per worker for Nginx, if requested
+# Cannot exceed worker_rlimit_nofile, see NGINX_WORKER_OPEN_FILES below
+if [[ -v NGINX_WORKER_CONNECTIONS ]] ; then
+    sed -i "/worker_connections\s/c\    worker_connections ${NGINX_WORKER_CONNECTIONS};" /etc/nginx/nginx.conf
+fi
+
+# Set the max number of open file descriptors for Nginx workers, if requested
+if [[ -v NGINX_WORKER_OPEN_FILES ]] ; then
+    echo "worker_rlimit_nofile ${NGINX_WORKER_OPEN_FILES};" >> /etc/nginx/nginx.conf
+fi
+
 # Get the listen port for Nginx, default to 80
 USE_LISTEN_PORT=${LISTEN_PORT:-80}
 # Modify Nignx config for listen port

python3.6-alpine3.7/supervisord.ini

@@ -2,7 +2,7 @@
 nodaemon=true
 
 [program:uwsgi]
-command=/usr/sbin/uwsgi --ini /etc/uwsgi/uwsgi.ini --die-on-term --plugin python3
+command=/usr/sbin/uwsgi --ini /etc/uwsgi/uwsgi.ini --die-on-term --need-app --plugin python3
 stdout_logfile=/dev/stdout
 stdout_logfile_maxbytes=0
 stderr_logfile=/dev/stderr
@@ -14,3 +14,5 @@ stdout_logfile=/dev/stdout
 stdout_logfile_maxbytes=0
 stderr_logfile=/dev/stderr
 stderr_logfile_maxbytes=0
+# Graceful stop, see http://nginx.org/en/docs/control.html
+stopsignal=QUIT

python3.6-alpine3.7/uwsgi.ini

@@ -2,3 +2,5 @@
 socket = /tmp/uwsgi.sock
 chown-socket = nginx:nginx
 chmod-socket = 664
+# Graceful shutdown on SIGTERM, see https://github.com/unbit/uwsgi/issues/849#issuecomment-118869386
+hook-master-start = unix_signal:15 gracefully_kill_them_all

python3.6/Dockerfile

@@ -1,12 +1,12 @@
 FROM python:3.6-stretch
 
 # Standard set up Nginx
-ENV NGINX_VERSION 1.13.7-1~stretch
-ENV NJS_VERSION   1.13.7.0.1.15-1~stretch
+ENV NGINX_VERSION 1.13.12-1~stretch
+ENV NJS_VERSION   1.13.12.0.2.0-1~stretch
 
 RUN set -x \
 	&& apt-get update \
-	&& apt-get install --no-install-recommends --no-install-suggests -y gnupg1 \
+	&& apt-get install --no-install-recommends --no-install-suggests -y gnupg1 apt-transport-https ca-certificates \
 	&& \
 	NGINX_GPGKEY=573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62; \
 	found=''; \
@@ -32,13 +32,13 @@ RUN set -x \
 	&& case "$dpkgArch" in \
 		amd64|i386) \
 # arches officialy built by upstream
-			echo "deb http://nginx.org/packages/mainline/debian/ stretch nginx" >> /etc/apt/sources.list \
+			echo "deb https://nginx.org/packages/mainline/debian/ stretch nginx" >> /etc/apt/sources.list.d/nginx.list \
 			&& apt-get update \
 			;; \
 		*) \
 # we're on an architecture upstream doesn't officially build for
 # let's build binaries from the published source packages
-			echo "deb-src http://nginx.org/packages/mainline/debian/ stretch nginx" >> /etc/apt/sources.list \
+			echo "deb-src https://nginx.org/packages/mainline/debian/ stretch nginx" >> /etc/apt/sources.list.d/nginx.list \
 			\
 # new directory for storing sources and .deb files
 			&& tempDir="$(mktemp -d)" \
@@ -79,7 +79,7 @@ RUN set -x \
 	&& apt-get install --no-install-recommends --no-install-suggests -y \
 						$nginxPackages \
 						gettext-base \
-	&& rm -rf /var/lib/apt/lists/* \
+	&& rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/nginx.list \
 	\
 # if we have leftovers from building, let's purge them (including extra, unnecessary build deps)
 	&& if [ -n "$tempDir" ]; then \
@@ -90,8 +90,9 @@ RUN set -x \
 # forward request and error logs to docker log collector
 RUN ln -sf /dev/stdout /var/log/nginx/access.log \
 	&& ln -sf /dev/stderr /var/log/nginx/error.log
-
 EXPOSE 80
+# Removed the section that breaks pip installations
+# && apt-get remove --purge --auto-remove -y apt-transport-https ca-certificates 
 # Standard set up Nginx finished
 
 # Expose 443, in case of LTS / HTTPS

python3.6/entrypoint.sh

@@ -11,6 +11,17 @@ USE_NGINX_WORKER_PROCESSES=${NGINX_WORKER_PROCESSES:-1}
 # Modify the number of worker processes in Nginx config
 sed -i "/worker_processes\s/c\worker_processes ${USE_NGINX_WORKER_PROCESSES};" /etc/nginx/nginx.conf
 
+# Set the max number of connections per worker for Nginx, if requested 
+# Cannot exceed worker_rlimit_nofile, see NGINX_WORKER_OPEN_FILES below
+if [[ -v NGINX_WORKER_CONNECTIONS ]] ; then
+    sed -i "/worker_connections\s/c\    worker_connections ${NGINX_WORKER_CONNECTIONS};" /etc/nginx/nginx.conf
+fi
+
+# Set the max number of open file descriptors for Nginx workers, if requested
+if [[ -v NGINX_WORKER_OPEN_FILES ]] ; then
+    echo "worker_rlimit_nofile ${NGINX_WORKER_OPEN_FILES};" >> /etc/nginx/nginx.conf
+fi
+
 # Get the listen port for Nginx, default to 80
 USE_LISTEN_PORT=${LISTEN_PORT:-80}
 # Modify Nignx config for listen port

python3.6/supervisord.conf

@@ -2,7 +2,7 @@
 nodaemon=true
 
 [program:uwsgi]
-command=/usr/local/bin/uwsgi --ini /etc/uwsgi/uwsgi.ini --die-on-term
+command=/usr/local/bin/uwsgi --ini /etc/uwsgi/uwsgi.ini --die-on-term --need-app
 stdout_logfile=/dev/stdout
 stdout_logfile_maxbytes=0
 stderr_logfile=/dev/stderr
@@ -14,3 +14,5 @@ stdout_logfile=/dev/stdout
 stdout_logfile_maxbytes=0
 stderr_logfile=/dev/stderr
 stderr_logfile_maxbytes=0
+# Graceful stop, see http://nginx.org/en/docs/control.html
+stopsignal=QUIT