After possible update Apache no more reachable #5291

alessandriniluca · 2024-09-19T07:06:46Z

alessandriniluca
Sep 19, 2024

Hi to everyone, I would like first to apologize for the disturbance, but from a day I noticed a problem I am not able to solve

Problem

My AIO setup worked without problems for almost a year. After what I think was an auto update a couple of days ago (3 or 4, while I was not at home during the weekend and did not access nc), I can no more reach the apache container. Neither caddy, nor NC (apache container / mastercontainer) shows suspicious logs, except for the logs in the mastercontainer which returns some "404". In the following everything (tests I did and log I got) is reported, if more things are needed, I'll be glad to provide more information.
Apologize If I did not open an issue immediately, but first I wanted to ensure it was not a problem on my side.

Setup

The setup comprises:
a VM running caddy
a VM running AIO

Both in the same local network (2 VMs of a proxmox machine)

Port forwarding from the router is enabled, and the public URL correctly points to the static public IP of the network. It has worked correctly for a year, I already rebooted everything included the router.

Caddyfile set up according to the guide for reverse proxies

https://websiteurl:443 {
    reverse_proxy <LOCAL_IP_AIO_VM>:11000
}

Stack on portainer for nextcloud aio:

services:
  nextcloud-aio-mastercontainer:
    image: nextcloud/all-in-one:latest
    init: true
    restart: always
    container_name: nextcloud-aio-mastercontainer
    volumes:
      - nextcloud_aio_mastercontainer:/mnt/docker-aio-config
      - /var/run/docker.sock:/var/run/docker.sock:ro 
    ports:
      #- 80:80 
      - 8080:8080
      - 8443:8443
    environment:
      - APACHE_PORT=11000 
      - APACHE_IP_BINDING=0.0.0.0 
      - NEXTCLOUD_MOUNT=/mnt/nc-backup-disk/ 
      - NEXTCLOUD_ENABLE_DRI_DEVICE=true

volumes:
  nextcloud_aio_mastercontainer:
    name: nextcloud_aio_mastercontainer

Router forwards the URL chosen for nextcloud to the local IP of the Caddy VM.

Logs:

Apache container

To be able to access the AIO container and see logs, I stopped from bash the apache container, opened aio interface, stopped all containers and restarted. Logs:

2024-09-18T14:28:24.006957165Z Waiting for Nextcloud to start...
2024-09-18T14:28:29.008222043Z Waiting for Nextcloud to start...
2024-09-18T14:28:34.009957417Z Waiting for Nextcloud to start...
2024-09-18T14:28:39.011422805Z Waiting for Nextcloud to start...
2024-09-18T14:28:44.012864092Z Connection to nextcloud-aio-nextcloud (172.19.0.12) 9000 port [tcp/*] succeeded!
2024-09-18T14:28:45.168759415Z [Wed Sep 18 16:28:45.168479 2024] [mpm_event:notice] [pid 45:tid 45] AH00489: Apache/2.4.62 (Unix) configured -- resuming normal operations
2024-09-18T14:28:45.168780918Z [Wed Sep 18 16:28:45.168507 2024] [core:notice] [pid 45:tid 45] AH00094: Command line: '/usr/local/apache2/bin/httpd -D FOREGROUND'
2024-09-18T14:28:45.180233160Z {"level":"info","ts":1726669725.1800892,"msg":"using config from file","file":"/tmp/Caddyfile"}
2024-09-18T14:28:45.181190557Z {"level":"info","ts":1726669725.181122,"msg":"adapted config to JSON","adapter":"caddyfile"}

Mastercontainer

To be able to access the AIO container and see logs, I stopped from bash the apache container, opened aio interface, stopped all containers and restarted. Logs:

2024-09-18T14:26:19.921443186Z Deleting duplicate sessions
2024-09-18T14:26:28.200440020Z NOTICE: PHP message: 404 Not Found
2024-09-18T14:26:28.200482755Z Type: Slim\Exception\HttpNotFoundException
2024-09-18T14:26:28.200485353Z Code: 404
2024-09-18T14:26:28.200487636Z Message: Not found.
2024-09-18T14:26:28.200496609Z File: /var/www/docker-aio/php/vendor/slim/slim/Slim/Middleware/RoutingMiddleware.php
2024-09-18T14:26:28.200498871Z Line: 76
2024-09-18T14:26:28.200500857Z Trace: #0 /var/www/docker-aio/php/vendor/slim/slim/Slim/Routing/RouteRunner.php(62): Slim\Middleware\RoutingMiddleware->performRouting(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:26:28.200503271Z #1 /var/www/docker-aio/php/vendor/slim/csrf/src/Guard.php(482): Slim\Routing\RouteRunner->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:26:28.200505547Z #2 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(177): Slim\Csrf\Guard->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Slim\Routing\RouteRunner))
2024-09-18T14:26:28.200507879Z #3 /var/www/docker-aio/php/vendor/slim/twig-view/src/TwigMiddleware.php(115): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:26:28.200510193Z #4 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(129): Slim\Views\TwigMiddleware->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
2024-09-18T14:26:28.200512510Z #5 /var/www/docker-aio/php/src/Middleware/AuthMiddleware.php(38): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:26:28.200514792Z #6 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(280): AIO\Middleware\AuthMiddleware->__invoke(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
2024-09-18T14:26:28.200517249Z #7 /var/www/docker-aio/php/vendor/slim/slim/Slim/Middleware/ErrorMiddleware.php(77): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:26:28.200519454Z #8 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(129): Slim\Middleware\ErrorMiddleware->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
2024-09-18T14:26:28.200521764Z #9 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(73): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:26:28.200524522Z #10 /var/www/docker-aio/php/vendor/slim/slim/Slim/App.php(209): Slim\MiddlewareDispatcher->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:26:28.200526747Z #11 /var/www/docker-aio/php/vendor/slim/slim/Slim/App.php(193): Slim\App->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:26:28.200528852Z #12 /var/www/docker-aio/php/public/index.php(185): Slim\App->run()
2024-09-18T14:26:28.200530930Z #13 {main}
2024-09-18T14:26:28.200532891Z Tips: To display error details in HTTP response set "displayErrorDetails" to true in the ErrorHandler constructor.
2024-09-18T14:31:22.126486456Z NOTICE: PHP message: 404 Not Found
2024-09-18T14:31:22.126508080Z Type: Slim\Exception\HttpNotFoundException
2024-09-18T14:31:22.126510691Z Code: 404
2024-09-18T14:31:22.126512776Z Message: Not found.
2024-09-18T14:31:22.126523249Z File: /var/www/docker-aio/php/vendor/slim/slim/Slim/Middleware/RoutingMiddleware.php
2024-09-18T14:31:22.126525501Z Line: 76
2024-09-18T14:31:22.126527479Z Trace: #0 /var/www/docker-aio/php/vendor/slim/slim/Slim/Routing/RouteRunner.php(62): Slim\Middleware\RoutingMiddleware->performRouting(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:31:22.126530125Z #1 /var/www/docker-aio/php/vendor/slim/csrf/src/Guard.php(482): Slim\Routing\RouteRunner->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:31:22.126532446Z #2 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(177): Slim\Csrf\Guard->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Slim\Routing\RouteRunner))
2024-09-18T14:31:22.126534794Z #3 /var/www/docker-aio/php/vendor/slim/twig-view/src/TwigMiddleware.php(115): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:31:22.126537078Z #4 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(129): Slim\Views\TwigMiddleware->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
2024-09-18T14:31:22.126539455Z #5 /var/www/docker-aio/php/src/Middleware/AuthMiddleware.php(38): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:31:22.126541628Z #6 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(280): AIO\Middleware\AuthMiddleware->__invoke(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
2024-09-18T14:31:22.126543995Z #7 /var/www/docker-aio/php/vendor/slim/slim/Slim/Middleware/ErrorMiddleware.php(77): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:31:22.126546227Z #8 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(129): Slim\Middleware\ErrorMiddleware->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
2024-09-18T14:31:22.126548585Z #9 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(73): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:31:22.126551401Z #10 /var/www/docker-aio/php/vendor/slim/slim/Slim/App.php(209): Slim\MiddlewareDispatcher->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:31:22.126553584Z #11 /var/www/docker-aio/php/vendor/slim/slim/Slim/App.php(193): Slim\App->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:31:22.126555712Z #12 /var/www/docker-aio/php/public/index.php(185): Slim\App->run()
2024-09-18T14:31:22.126557806Z #13 {main}
2024-09-18T14:31:22.126559728Z Tips: To display error details in HTTP response set "displayErrorDetails" to true in the ErrorHandler constructor.
2024-09-18T14:39:46.823804640Z NOTICE: PHP message: 404 Not Found
2024-09-18T14:39:46.823825138Z Type: Slim\Exception\HttpNotFoundException
2024-09-18T14:39:46.823827628Z Code: 404
2024-09-18T14:39:46.823829620Z Message: Not found.
2024-09-18T14:39:46.823839115Z File: /var/www/docker-aio/php/vendor/slim/slim/Slim/Middleware/RoutingMiddleware.php
2024-09-18T14:39:46.823841364Z Line: 76
2024-09-18T14:39:46.823843320Z Trace: #0 /var/www/docker-aio/php/vendor/slim/slim/Slim/Routing/RouteRunner.php(62): Slim\Middleware\RoutingMiddleware->performRouting(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:39:46.823845776Z #1 /var/www/docker-aio/php/vendor/slim/csrf/src/Guard.php(482): Slim\Routing\RouteRunner->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:39:46.823848025Z #2 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(177): Slim\Csrf\Guard->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Slim\Routing\RouteRunner))
2024-09-18T14:39:46.823850834Z #3 /var/www/docker-aio/php/vendor/slim/twig-view/src/TwigMiddleware.php(115): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:39:46.823853157Z #4 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(129): Slim\Views\TwigMiddleware->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
2024-09-18T14:39:46.823855542Z #5 /var/www/docker-aio/php/src/Middleware/AuthMiddleware.php(38): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:39:46.823857792Z #6 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(280): AIO\Middleware\AuthMiddleware->__invoke(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
2024-09-18T14:39:46.823860149Z #7 /var/www/docker-aio/php/vendor/slim/slim/Slim/Middleware/ErrorMiddleware.php(77): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:39:46.823862395Z #8 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(129): Slim\Middleware\ErrorMiddleware->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
2024-09-18T14:39:46.823864788Z #9 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(73): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:39:46.823867512Z #10 /var/www/docker-aio/php/vendor/slim/slim/Slim/App.php(209): Slim\MiddlewareDispatcher->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:39:46.823869783Z #11 /var/www/docker-aio/php/vendor/slim/slim/Slim/App.php(193): Slim\App->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:39:46.823871969Z #12 /var/www/docker-aio/php/public/index.php(185): Slim\App->run()
2024-09-18T14:39:46.823874048Z #13 {main}
2024-09-18T14:39:46.823876032Z Tips: To display error details in HTTP response set "displayErrorDetails" to true in the ErrorHandler constructor.
2024-09-18T14:45:03.592538057Z NOTICE: PHP message: 404 Not Found
2024-09-18T14:45:03.592557200Z Type: Slim\Exception\HttpNotFoundException
2024-09-18T14:45:03.592559746Z Code: 404
2024-09-18T14:45:03.592561966Z Message: Not found.
2024-09-18T14:45:03.592571124Z File: /var/www/docker-aio/php/vendor/slim/slim/Slim/Middleware/RoutingMiddleware.php
2024-09-18T14:45:03.592573855Z Line: 76
2024-09-18T14:45:03.592576498Z Trace: #0 /var/www/docker-aio/php/vendor/slim/slim/Slim/Routing/RouteRunner.php(62): Slim\Middleware\RoutingMiddleware->performRouting(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:45:03.592579816Z #1 /var/www/docker-aio/php/vendor/slim/csrf/src/Guard.php(482): Slim\Routing\RouteRunner->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:45:03.592582767Z #2 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(177): Slim\Csrf\Guard->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Slim\Routing\RouteRunner))
2024-09-18T14:45:03.592586105Z #3 /var/www/docker-aio/php/vendor/slim/twig-view/src/TwigMiddleware.php(115): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:45:03.592588596Z #4 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(129): Slim\Views\TwigMiddleware->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
2024-09-18T14:45:03.592591115Z #5 /var/www/docker-aio/php/src/Middleware/AuthMiddleware.php(38): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:45:03.592593527Z #6 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(280): AIO\Middleware\AuthMiddleware->__invoke(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
2024-09-18T14:45:03.592596050Z #7 /var/www/docker-aio/php/vendor/slim/slim/Slim/Middleware/ErrorMiddleware.php(77): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:45:03.592598399Z #8 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(129): Slim\Middleware\ErrorMiddleware->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
2024-09-18T14:45:03.592600893Z #9 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(73): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:45:03.592603891Z #10 /var/www/docker-aio/php/vendor/slim/slim/Slim/App.php(209): Slim\MiddlewareDispatcher->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:45:03.592606182Z #11 /var/www/docker-aio/php/vendor/slim/slim/Slim/App.php(193): Slim\App->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T14:45:03.592608523Z #12 /var/www/docker-aio/php/public/index.php(185): Slim\App->run()
2024-09-18T14:45:03.592610747Z #13 {main}
2024-09-18T14:45:03.592612791Z Tips: To display error details in HTTP response set "displayErrorDetails" to true in the ErrorHandler constructor.
2024-09-18T15:42:18.289693254Z NOTICE: PHP message: 404 Not Found
2024-09-18T15:42:18.289709107Z Type: Slim\Exception\HttpNotFoundException
2024-09-18T15:42:18.289711676Z Code: 404
2024-09-18T15:42:18.289713695Z Message: Not found.
2024-09-18T15:42:18.289715827Z File: /var/www/docker-aio/php/vendor/slim/slim/Slim/Middleware/RoutingMiddleware.php
2024-09-18T15:42:18.289724782Z Line: 76
2024-09-18T15:42:18.289726903Z Trace: #0 /var/www/docker-aio/php/vendor/slim/slim/Slim/Routing/RouteRunner.php(62): Slim\Middleware\RoutingMiddleware->performRouting(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T15:42:18.289729416Z #1 /var/www/docker-aio/php/vendor/slim/csrf/src/Guard.php(482): Slim\Routing\RouteRunner->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T15:42:18.289731685Z #2 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(177): Slim\Csrf\Guard->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Slim\Routing\RouteRunner))
2024-09-18T15:42:18.289734455Z #3 /var/www/docker-aio/php/vendor/slim/twig-view/src/TwigMiddleware.php(115): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T15:42:18.289736806Z #4 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(129): Slim\Views\TwigMiddleware->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
2024-09-18T15:42:18.289739184Z #5 /var/www/docker-aio/php/src/Middleware/AuthMiddleware.php(38): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T15:42:18.289741362Z #6 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(280): AIO\Middleware\AuthMiddleware->__invoke(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
2024-09-18T15:42:18.289744662Z #7 /var/www/docker-aio/php/vendor/slim/slim/Slim/Middleware/ErrorMiddleware.php(77): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T15:42:18.289747798Z #8 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(129): Slim\Middleware\ErrorMiddleware->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
2024-09-18T15:42:18.289750996Z #9 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(73): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T15:42:18.289754589Z #10 /var/www/docker-aio/php/vendor/slim/slim/Slim/App.php(209): Slim\MiddlewareDispatcher->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T15:42:18.289757604Z #11 /var/www/docker-aio/php/vendor/slim/slim/Slim/App.php(193): Slim\App->handle(Object(GuzzleHttp\Psr7\ServerRequest))
2024-09-18T15:42:18.289760116Z #12 /var/www/docker-aio/php/public/index.php(185): Slim\App->run()
2024-09-18T15:42:18.289762311Z #13 {main}
2024-09-18T15:42:18.289764295Z Tips: To display error details in HTTP response set "displayErrorDetails" to true in the ErrorHandler constructor.

Caddy

through `journalctl -u cady --since today', logs from last caddy restart:

Sep 18 15:02:31 caddy systemd[1]: Started Caddy.
Sep 18 15:02:31 caddy caddy[162]: {"level":"info","ts":1726671751.263089,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/var/lib/caddy/.local/share/caddy","instance":"b45ed7f9-16de-4682-bd9a-f458e02d8e3d","try_again":1726758151.2630882,"try_again_in":86399.99999976}
Sep 18 15:02:31 caddy caddy[162]: {"level":"info","ts":1726671751.263214,"logger":"tls","msg":"finished cleaning storage units"}
Sep 18 15:02:31 caddy caddy[162]: {"level":"info","ts":1726671751.2636514,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0000e7780"}

Other tests:

From the caddy VM, wget <LOCAL_IP_AIO_VM>:11000 returns a page,
Outside from the local network, on my laptop from the internet, wget <PUBLIC_URL_NEXTCLOUD> hangs on:

--2024-09-18 17:58:18--  <PUBLIC_URL_NEXTCLOUD>
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving <PUBLIC_URL_NEXTCLOUD> (<PUBLIC_URL_NEXTCLOUD>)... <STATIC_PUBLIC_ROUTER_IP>
Connecting to <PUBLIC_URL_NEXTCLOUD> (<PUBLIC_URL_NEXTCLOUD>)|<STATIC_PUBLIC_ROUTER_IP>|:443...

So I am assuming the URL correctly redirects to it. Could it be possibly related to #5269?

I also spawned a simple http server on python on the VM hosting AIO, and used caddy to redirect traffic to it, it is accessible from the internet (hence, my routing, involving the router config and caddy, should be ok)
For this reason, I started doing some tests to check the routing among the containers (something I did not touch, hence I am assuming an update broke it), and the status with netstat is TIME_WAIT for everything inside the apache container.

szaimen · 2024-09-19T08:16:45Z

szaimen
Sep 19, 2024
Maintainer

From the caddy VM, wget <LOCAL_IP_AIO_VM>:11000 returns a page,

okay this proff thst AIO works correctly.

Outside from the local network, on my laptop from the internet, wget <PUBLIC_URL_NEXTCLOUD> hangs on:

--2024-09-18 17:58:18--  <PUBLIC_URL_NEXTCLOUD>
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving <PUBLIC_URL_NEXTCLOUD> (<PUBLIC_URL_NEXTCLOUD>)... <STATIC_PUBLIC_ROUTER_IP>
Connecting to <PUBLIC_URL_NEXTCLOUD> (<PUBLIC_URL_NEXTCLOUD>)|<STATIC_PUBLIC_ROUTER_IP

This looks like a bug in your your network or reverse proxy then (outside aio)

9 replies

alessandriniluca Sep 20, 2024
Author

Thanks a lot for your patience.

Sure, the result of sudo netstat -tulpn is:

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      725/systemd-resolve 
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      2153/docker-proxy   
tcp        0      0 0.0.0.0:8000            0.0.0.0:*               LISTEN      1304/docker-proxy   
tcp        0      0 0.0.0.0:9443            0.0.0.0:*               LISTEN      1277/docker-proxy   
tcp        0      0 0.0.0.0:8443            0.0.0.0:*               LISTEN      2131/docker-proxy   
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      840/sshd: /usr/sbin 
tcp        0      0 0.0.0.0:3478            0.0.0.0:*               LISTEN      1477190/docker-prox 
tcp        0      0 0.0.0.0:11000           0.0.0.0:*               LISTEN      1478858/docker-prox 
tcp6       0      0 :::8080                 :::*                    LISTEN      2159/docker-proxy   
tcp6       0      0 :::8000                 :::*                    LISTEN      1310/docker-proxy   
tcp6       0      0 :::9443                 :::*                    LISTEN      1292/docker-proxy   
tcp6       0      0 :::8443                 :::*                    LISTEN      2138/docker-proxy   
tcp6       0      0 :::22                   :::*                    LISTEN      840/sshd: /usr/sbin 
tcp6       0      0 :::3478                 :::*                    LISTEN      1477197/docker-prox 
udp        0      0 0.0.0.0:3478            0.0.0.0:*                           1477211/docker-prox 
udp        0      0 127.0.0.53:53           0.0.0.0:*                           725/systemd-resolve 
udp6       0      0 :::3478                 :::*                                1477217/docker-prox

szaimen Sep 20, 2024
Maintainer

tcp 0 0 0.0.0.0:11000 0.0.0.0:* LISTEN 1478858/docker-proxy

So as you can see, the service seems to be correctly listening. Also curling this endpoint returned something, so everything points towards the connection between your reverse proxy and the server running docker...

alessandriniluca Sep 20, 2024
Author

Thank you for the precious help. I'll dig deeper next week regarding the network part. If you don't mind, I will get back here when I have a solution for this, so if in the future someone faces the same problem, maybe can reach this discussion

szaimen Oct 7, 2024
Maintainer

Hi, is there any update here?

alessandriniluca Oct 7, 2024
Author

Hi, apologize for the delay. Unfortunately, I am a bit busy for two weeks due to a university deadline. If you want to mark it as closed for now, in case I'll add a comment later in time.

Sorry for the inconvenience, but we are in a rush to conclude, so I have no time to perform further analysis at the moment unfortunately

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

After possible update Apache no more reachable #5291

{{title}}

Replies: 1 comment 9 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

After possible update Apache no more reachable #5291

alessandriniluca Sep 19, 2024

Problem

Setup

Logs:

Apache container

Mastercontainer

Caddy

Other tests:

Replies: 1 comment · 9 replies

szaimen Sep 19, 2024 Maintainer

alessandriniluca Sep 20, 2024 Author

szaimen Sep 20, 2024 Maintainer

alessandriniluca Sep 20, 2024 Author

szaimen Oct 7, 2024 Maintainer

alessandriniluca Oct 7, 2024 Author

alessandriniluca
Sep 19, 2024

Replies: 1 comment 9 replies

szaimen
Sep 19, 2024
Maintainer

alessandriniluca Sep 20, 2024
Author

szaimen Sep 20, 2024
Maintainer

alessandriniluca Sep 20, 2024
Author

szaimen Oct 7, 2024
Maintainer

alessandriniluca Oct 7, 2024
Author