-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
no servers inside upstream in /etc/nginx/conf.d/default.conf #1548
Comments
push for nginx-proxy/docker-gen#335 |
I ran into the same problem with Docker 20.10.3 on Debian Bullseye. @jwilder would you mind looking into (and merging) nginx-proxy/docker-gen#335 |
Hi, The same issue is here. However some of my containers have host names set in their docker-compose file, e.g.:
Some do not have the hostname set but all of them have a pingable/reachable hostname in /etc/hostname or by running hostname command, e.g. hostname=$(hostname):
this one does not have hostname set in the config:
both can be reached by either "gyventojams" or "637dbe8aec88" hostname from within the containers or the docker network. I am on Debian bullseye (11) and Docker 20.10. The development environment here has stopped working! Regards |
@christianuhlmann @mejo- @andriusdj You can find a fix here. |
I have the same issue and tried the fix by building myself the following image for the proxy (I changed the DOCKER_GEN_VERSION=0.7.5 to match the last tagged version which looks like embed the patch supposed to fix ): # setup build arguments for version of dependencies to use
ARG DOCKER_GEN_VERSION=0.7.5
ARG FOREGO_VERSION=0.16.1
# Use a specific version of golang to build both binaries
FROM golang:1.15.10 as gobuilder
# Build docker-gen from scratch
FROM gobuilder as dockergen
# Download the sources for the given version
ARG DOCKER_GEN_VERSION
ADD https://github.com/jwilder/docker-gen/archive/${DOCKER_GEN_VERSION}.tar.gz sources.tar.gz
# Move the sources into the right directory
RUN tar -xzf sources.tar.gz && \
mkdir -p /go/src/github.com/jwilder/ && \
mv docker-gen-* /go/src/github.com/jwilder/docker-gen
# Install the dependencies and make the docker-gen executable
WORKDIR /go/src/github.com/jwilder/docker-gen
RUN go get -v ./... && \
CGO_ENABLED=0 GOOS=linux go build -ldflags "-X main.buildVersion=${DOCKER_GEN_VERSION}" ./cmd/docker-gen
# Build forego from scratch
# Because this relies on golang workspaces, we need to use go < 1.8.
FROM gobuilder as forego
# Download the sources for the given version
ARG FOREGO_VERSION
ADD https://github.com/jwilder/forego/archive/v${FOREGO_VERSION}.tar.gz sources.tar.gz
# Move the sources into the right directory
RUN tar -xzf sources.tar.gz && \
mkdir -p /go/src/github.com/ddollar/ && \
mv forego-* /go/src/github.com/ddollar/forego
# Install the dependencies and make the forego executable
WORKDIR /go/src/github.com/ddollar/forego/
RUN go get -v ./... && \
CGO_ENABLED=0 GOOS=linux go build -o forego .
# Build the final image
FROM nginx:1.19.3
LABEL maintainer="Jason Wilder mail@jasonwilder.com"
# Install wget and install/updates certificates
RUN apt-get update \
&& apt-get install -y -q --no-install-recommends \
ca-certificates \
wget \
&& apt-get clean \
&& rm -r /var/lib/apt/lists/*
# Configure Nginx and apply fix for very long server names
RUN echo "daemon off;" >> /etc/nginx/nginx.conf \
&& sed -i 's/worker_processes 1/worker_processes auto/' /etc/nginx/nginx.conf
# Install Forego + docker-gen
COPY --from=forego /go/src/github.com/ddollar/forego/forego /usr/local/bin/forego
COPY --from=dockergen /go/src/github.com/jwilder/docker-gen/docker-gen /usr/local/bin/docker-gen
# Add DOCKER_GEN_VERSION environment variable
# Because some external projects rely on it
ARG DOCKER_GEN_VERSION
ENV DOCKER_GEN_VERSION=${DOCKER_GEN_VERSION}
COPY network_internal.conf /etc/nginx/
COPY . /app/
WORKDIR /app/
ENV DOCKER_HOST unix:///tmp/docker.sock
VOLUME ["/etc/nginx/certs", "/etc/nginx/dhparam"]
ENTRYPOINT ["/app/docker-entrypoint.sh"]
CMD ["forego", "start", "-r"] But got the same upstream error
EDIT: It looks like the upstream in default.conf is indeed empty where I run it locally it's filled, so maybe linked to Arch/Docker swarm |
The cat /proc/self/moutinfo inside the nginxproxy doesn't show the others containers
|
@daisukixci could you |
The |
Here it is @buchdag docker inspect web_reverseproxy.1.5axd16mkemdtpgm8xag8tk3og
[
{
"Id": "77f4d9c968d036e7c04600b71ce7e511d3c4f213bab2813523df2e18bc336a8e",
"Created": "2021-04-08T12:32:02.716284607Z",
"Path": "/app/docker-entrypoint.sh",
"Args": [
"forego",
"start",
"-r"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 3826314,
"ExitCode": 0,
"Error": "",
"StartedAt": "2021-04-08T12:32:08.871440706Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:a615f5f74df3ded2c800483c09d062d01c37d02a660510393678c573808269ca",
"ResolvConfPath": "/var/lib/docker/containers/77f4d9c968d036e7c04600b71ce7e511d3c4f213bab2813523df2e18bc336a8e/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/77f4d9c968d036e7c04600b71ce7e511d3c4f213bab2813523df2e18bc336a8e/hostname",
"HostsPath": "/var/lib/docker/containers/77f4d9c968d036e7c04600b71ce7e511d3c4f213bab2813523df2e18bc336a8e/hosts",
"LogPath": "/var/lib/docker/containers/77f4d9c968d036e7c04600b71ce7e511d3c4f213bab2813523df2e18bc336a8e/77f4d9c968d036e7c04600b71ce7e511d3c4f213bab2813523df2e18bc336a8e-json.log",
"Name": "/web_reverseproxy.1.5axd16mkemdtpgm8xag8tk3og",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": null,
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "default",
"PortBindings": {},
"RestartPolicy": {
"Name": "",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "private",
"Dns": null,
"DnsOptions": null,
"DnsSearch": null,
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": false,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": null,
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
0,
0
],
"Isolation": "default",
"CpuShares": 0,
"Memory": 0,
"NanoCpus": 0,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": null,
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": null,
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 0,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"Mounts": [
{
"Type": "bind",
"Source": "REDACTED/reverseproxy/conf.d",
"Target": "/etc/nginx/conf.d"
},
{
"Type": "bind",
"Source": "REDACTED/reverseproxy/vhost.d",
"Target": "/etc/nginx/vhost.d"
},
{
"Type": "bind",
"Source": "REDACTED/reverseproxy/html",
"Target": "/usr/share/nginx/html"
},
{
"Type": "bind",
"Source": "REDACTED/reverseproxy/certs",
"Target": "/etc/nginx/certs",
"ReadOnly": true
},
{
"Type": "bind",
"Source": "/etc/localtime",
"Target": "/etc/localtime",
"ReadOnly": true
},
{
"Type": "bind",
"Source": "/var/run/docker.sock",
"Target": "/tmp/docker.sock",
"ReadOnly": true
}
],
"MaskedPaths": [
"/proc/asound",
"/proc/acpi",
"/proc/kcore",
"/proc/keys",
"/proc/latency_stats",
"/proc/timer_list",
"/proc/timer_stats",
"/proc/sched_debug",
"/proc/scsi",
"/sys/firmware"
],
"ReadonlyPaths": [
"/proc/bus",
"/proc/fs",
"/proc/irq",
"/proc/sys",
"/proc/sysrq-trigger"
]
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/b2ade9e006d1fd9c6dac771cacb29639c99be9358710ff8a7098ccaa4826e929-init/diff:/var/lib/docker/overlay2/9535c238035a883a593cf82e3ead0a2b52169655046810269c18801346b15f30/diff:/var/lib/docker/overlay2/f24d9aba6af9e72c498c5b7948052319ecd16b060d72f3ed3002c0b64f3a82c2/diff:/var/lib/docker/overlay2/dd37898cacb36b4ff1c2586811af88fec013ef037d58703e68d478eb7eba81b1/diff:/var/lib/docker/overlay2/76c9404f1d346e007c54abf55bec84514cf0933f32591bd77c07e5805998b75a/diff:/var/lib/docker/overlay2/5a11291e8d43db11354cdbdfad8cc5de599ad8dbdf505556e7542ca98301fe78/diff:/var/lib/docker/overlay2/9ea87db8702a2ba81595448c37bfbdef2946dbf8f4b226d414a6d350cc161b3b/diff:/var/lib/docker/overlay2/a245da6b126496fd512367e50524683db743b34d14d8635a73098e844f3bcc9c/diff:/var/lib/docker/overlay2/601ade3c93d74617b1d25542bcd9d2bb0eb063e75e08863834a610bffcf94538/diff:/var/lib/docker/overlay2/601beabfc8f3ba4691f746dceec2b2805d04df67859942c138a2983f4087ae6a/diff:/var/lib/docker/overlay2/093d2e5d6e8fc016512b189781afa331467e2c3430cef24e363373acc6e3db5f/diff:/var/lib/docker/overlay2/7b6e18291277fb7fe2e53c788c2f8942318232c5eaef7fddd3f6fe4babb188f5/diff",
"MergedDir": "/var/lib/docker/overlay2/b2ade9e006d1fd9c6dac771cacb29639c99be9358710ff8a7098ccaa4826e929/merged",
"UpperDir": "/var/lib/docker/overlay2/b2ade9e006d1fd9c6dac771cacb29639c99be9358710ff8a7098ccaa4826e929/diff",
"WorkDir": "/var/lib/docker/overlay2/b2ade9e006d1fd9c6dac771cacb29639c99be9358710ff8a7098ccaa4826e929/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/var/run/docker.sock",
"Destination": "/tmp/docker.sock",
"Mode": "",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "b3727d891e3628a85ff31507d42e2a90f0f89f887ef209e0f23b190a562deba3",
"Source": "/var/lib/docker/volumes/b3727d891e3628a85ff31507d42e2a90f0f89f887ef209e0f23b190a562deba3/_data",
"Destination": "/etc/nginx/dhparam",
"Driver": "local",
"Mode": "",
"RW": true,
"Propagation": ""
},
{
"Type": "bind",
"Source": "REDACTED/reverseproxy/conf.d",
"Destination": "/etc/nginx/conf.d",
"Mode": "",
"RW": true,
"Propagation": "rprivate"
},
{
"Type": "bind",
"Source": "REDACTED/reverseproxy/vhost.d",
"Destination": "/etc/nginx/vhost.d",
"Mode": "",
"RW": true,
"Propagation": "rprivate"
},
{
"Type": "bind",
"Source": "REDACTED/reverseproxy/html",
"Destination": "/usr/share/nginx/html",
"Mode": "",
"RW": true,
"Propagation": "rprivate"
},
{
"Type": "bind",
"Source": "REDACTED/reverseproxy/certs",
"Destination": "/etc/nginx/certs",
"Mode": "",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "bind",
"Source": "/etc/localtime",
"Destination": "/etc/localtime",
"Mode": "",
"RW": false,
"Propagation": "rprivate"
}
],
"Config": {
"Hostname": "reverseproxy",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"80/tcp": {}
},
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"NGINX_VERSION=1.19.3",
"NJS_VERSION=0.4.4",
"PKG_RELEASE=1",
"DOCKER_GEN_VERSION=0.7.4",
"DOCKER_HOST=unix:///tmp/docker.sock"
],
"Cmd": [
"forego",
"start",
"-r"
],
"Image": "jwilder/nginx-proxy:alpine@sha256:ec8af88085f179858d57676ea9c7cce742fbaf1a154c7458d2f684b869db7604",
"Volumes": {
"/etc/nginx/certs": {},
"/etc/nginx/dhparam": {}
},
"WorkingDir": "/app",
"Entrypoint": [
"/app/docker-entrypoint.sh"
],
"OnBuild": null,
"Labels": {
"com.docker.stack.namespace": "web",
"com.docker.swarm.node.id": "2qk202sqpighsfg9gi9n8m1es",
"com.docker.swarm.service.id": "m2fp2dynu1agvia7g127lx9ew",
"com.docker.swarm.service.name": "web_reverseproxy",
"com.docker.swarm.task": "",
"com.docker.swarm.task.id": "5axd16mkemdtpgm8xag8tk3og",
"com.docker.swarm.task.name": "web_reverseproxy.1.5axd16mkemdtpgm8xag8tk3og",
"com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy": "true",
"maintainer": "Jason Wilder mail@jasonwilder.com",
"org.opencontainers.image.created": "2021-04-07T15:10:57.154Z",
"org.opencontainers.image.description": "Automated nginx proxy for Docker containers using docker-gen",
"org.opencontainers.image.licenses": "MIT",
"org.opencontainers.image.revision": "6613e272eb41c926605bc0411b7af9f70c2ebeee",
"org.opencontainers.image.source": "https://github.com/nginx-proxy/nginx-proxy",
"org.opencontainers.image.title": "nginx-proxy",
"org.opencontainers.image.url": "https://github.com/nginx-proxy/nginx-proxy",
"org.opencontainers.image.version": "alpine"
},
"StopSignal": "SIGTERM"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "bcc7e51a38387b44e7a3263dd7ddbc9542ae246240825cc5cb97d99bc9706fcd",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"80/tcp": null
},
"SandboxKey": "/var/run/docker/netns/bcc7e51a3838",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"ingress": {
"IPAMConfig": {
"IPv4Address": "10.0.0.169"
},
"Links": null,
"Aliases": [
"77f4d9c968d0",
"reverseproxy"
],
"NetworkID": "p65v6kntzcaa8bxz48qb91fp5",
"EndpointID": "c1c3af3d5204ba840013cf8c0d9cf19af35c033f3be9f4f4e5757c21e221183d",
"Gateway": "",
"IPAddress": "10.0.0.169",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:0a:00:00:a9",
"DriverOpts": null
},
"web_deathstar_network": {
"IPAMConfig": {
"IPv4Address": "10.0.40.89"
},
"Links": null,
"Aliases": [
"77f4d9c968d0",
"reverseproxy"
],
"NetworkID": "vo0alax7stymwx82fn46hvee4",
"EndpointID": "48d6bbcababae6c7a6b752a45276caffce0404777e1f237d5e59098435521b64",
"Gateway": "",
"IPAddress": "10.0.40.89",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:0a:00:28:59",
"DriverOpts": null
}
}
}
}
] |
There you go: "Config": {
"Hostname": "reverseproxy",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"80/tcp": {}
} nginx-proxy/docker-gen#336 introduced a regression that prevent docker-gen from getting its own container ID when you provide a custom hostname. |
@buchdag Still have the issue with hostname removed from the compose |
Out of curiosity, could you post:
|
|
This might help too
|
Sorry I meant |
@buchdag yes
FYI I chmod 400 the default.conf and fix it manually to get my stack working |
That's ... interesting. It doesn't explain why the method of nginx-proxy/docker-gen#336 does not work once you've removed the custom hostname though. The regular expression should match the correct ID on one of those three lines of |
I have opened an issue on their hand nginx-proxy/docker-gen#344, hope we will find a fix, I have been using nginxproxy for some time and it stopped working after an update. |
@buchdag I rebooted my server for an update and it's back to working, looking at the hash used for the docker image I may have confused things when doing debug. So when running my own image of nginxproxy with the 0.7.5 version of dockergen things works as expected once I removed the hostname like you said. |
Would you like to add any advices for the following information from an openSUSE Tumbleweed system?
|
I'm also receiving |
@DanHartman: How do you think about to share a bit more information about your test system? How will the involved software components evolve further? |
How do you think about to add any updates according to related bug reports? |
You can build the image upgrading docker-gen version to 7.5 but I think if it's not done right now, it's because this version introduce regression that prevent users to use hostname in the docker-compose file |
Are the software version dependencies especially challenging at the moment? |
cf #1585 and #1586 (comment) |
|
Same issue on Arch Linux. It's a blocker unfortunately but I would like to thank everybody for putting their efforts into this issue. Is there an indication when a potential fix could be available? Does a timeline exist? Is there at least an applicable work-around for the time being? |
#1548 (comment) |
Are the chances growing that another improved docker image “jwilder/nginx-proxy:alpine” will be published according to evolution with the software release “docker-gen 0.7.6”? |
I'm still having the issue :/ Some relevant info: docker inspect nginx-proxy
Docker version:
Docker compose example:
|
Description
On my x86 fedora 33 server environment I face the following issue with latest nginx-proxy image. The automatic default.conf generation seems to fail and results in an empty upstream configuration.
Steps to Reproduce
I use this minimal example:
Startup log:
As a consequence curl request returns the following:
Curl request
Please expand to see the resulting configuration:
default.conf
Pre-Analysis
I found one common issue which often results in this behaviour. It needs to be ensured the started containers are really in the same network and can reach each other. I tried many variants with external network and manually adding the containers to that network etc. All without success.
The second hint I found is that in some cases cgroup might be a reason
.Docker.CurrentContainerID
, used innginx.tmpl
, can't be resolved. Since Fedora33 uses cgroup2 there might be a link. However, I lack the debug knowledge on how to check whether.Docker.CurrentContainerID
is empty or not. I tried this very same example on an Debian Stretch server with 4.9 Kernel. Here the default.conf gets configured correctly. Thus, I assume some relation to the host environment.Please let me know the supported Information is not sufficient. I'm grateful for any kind of support. TIA
Thanks,
Steffen
Update #1
After some investigation I found that CurrentContainerID is empty and docker-gen can not get the information to fill the template.
I prepared a RFC PR and a preliminary fix here: nginx-proxy/docker-gen#335
The text was updated successfully, but these errors were encountered: