Docker compose ors-config.json not taken into account #1300
Comments
|
Probably linked to not accurate setup documentation #1298 |
|
Hey, I got a notification by mail that had the original issue text in it, and when creating this issue, you had the line that mounts the Are you sure that this mount point is correctly applied and respected by your container? Best regards |
Yes I tried a bunch of things and the first version in the issue didn't reflect the documentation recommendations I can confirm that it's not working even when this line is active. I've spent an entire day trying different things to make it work The issue is that the container doesn't use Would be great to be able to edit the file and recalculate graph but I couldn't find how. The only workaround right now to edit config file is to run EDITAlso tried mounting as a volume like this in Container won't launch, stuck at these lines in logs (graph doesn't get calculated and api throws 404 error): |
|
Hey, are you running the latest image, i.e. did you execute Best regards |
Yes I pulled it yesterday, when has it changed?
Here is the file: #!/usr/bin/env bash
echo "Running container as user $(whoami) with id $(id -u)"
ors_base=${1}
catalina_base=${ors_base}/tomcat
echo "ORS Path: ${ors_base}"
echo "Catalina Path: ${catalina_base}"
graphs=${ors_base}/ors-core/data/graphs
tomcat_ors_config=${catalina_base}/webapps/ors/WEB-INF/classes/ors-config.json
source_ors_config=${ors_base}/ors-core/ors-config.json
public_ors_config=${ors_base}/ors-conf/ors-config.json
if [ -z "${CATALINA_OPTS}" ]; then
export CATALINA_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost"
fi
if [ -z "${JAVA_OPTS}" ]; then
export JAVA_OPTS="-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:ParallelGCThreads=4 -Xms1g -Xmx2g"
fi
{
echo "CATALINA_BASE=\"${catalina_base}\""
echo "CATALINA_HOME=\"${catalina_base}\""
echo "CATALINA_PID=\"${catalina_base}/temp/tomcat.pid\""
echo "CATALINA_OPTS=\"${CATALINA_OPTS}\""
echo "JAVA_OPTS=\"${JAVA_OPTS}\""
} >"${catalina_base}"/bin/setenv.sh
if [ "${BUILD_GRAPHS}" = "True" ]; then
rm -rf "${graphs:?}"/*
fi
echo "### openrouteservice configuration ###"
# if Tomcat built before, copy the mounted ors-config.json to the Tomcat webapp ors-config.json, else copy it from the source
if [ -d "${catalina_base}/webapps/ors" ]; then
echo "Tomcat already built: Copying ${ors_base}/ors-conf/ors-config.json to tomcat webapp folder"
cp -f "${public_ors_config}" "${tomcat_ors_config}"
else
if [ ! -f "$public_ors_config" ]; then
echo "No ors-config.json in ors-conf folder. Copying config from ${source_ors_config}"
mkdir -p "${ors_base}"/ors-conf
cp -f "${source_ors_config}" "${public_ors_config}"
else
echo "ors-config.json exists in ors-conf folder. Copying config to ${source_ors_config}"
cp -f "${public_ors_config}" "${source_ors_config}"
fi
echo "### Package openrouteservice and deploy to Tomcat ###"
cp -f "${ors_base}"/ors-core/ors.war "${catalina_base}"/webapps/ors.war
fi
# so docker can stop the process gracefully
exec "${catalina_base}"/bin/catalina.sh runThanks for your help! |
|
Could you add the first dozen lines or so of a |
|
Sure thing, see below ❯ docker logs ors-app | head -n 15
NOTE: Picked up JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
02-Mar-2023 14:18:46.897 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version name: Apache Tomcat/8.5.69
02-Mar-2023 14:18:46.904 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built: Jun 30 2021 18:00:00 UTC
Running container as user root with id 0
02-Mar-2023 14:18:46.904 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version number: 8.5.69.0
ORS Path: /home/ors
Catalina Path: /home/ors/tomcat
02-Mar-2023 14:18:46.904 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name: Linux
### openrouteservice configuration ###
ors-config.json exists in ors-conf folder. Copying config to /home/ors/ors-core/ors-config.json
### Package openrouteservice and deploy to Tomcat ###
02-Mar-2023 14:18:46.905 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version: 5.15.49-linuxkit
02-Mar-2023 14:18:46.906 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture: amd64
02-Mar-2023 14:18:46.906 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home: /opt/java/openjdk
02-Mar-2023 14:18:46.907 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version: 11.0.18+10
02-Mar-2023 14:18:46.907 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: Eclipse Adoptium
02-Mar-2023 14:18:46.907 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE: /home/ors/tomcat
02-Mar-2023 14:18:46.908 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME: /home/ors/tomcat
02-Mar-2023 14:18:46.923 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.lang=ALL-UNNAMED
02-Mar-2023 14:18:46.925 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.io=ALL-UNNAMED
02-Mar-2023 14:18:46.926 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.util=ALL-UNNAMED
02-Mar-2023 14:18:46.926 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.util.concurrent=ALL-UNNAMED
02-Mar-2023 14:18:46.926 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
02-Mar-2023 14:18:46.926 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/home/ors/tomcat/conf/logging.properties
02-Mar-2023 14:18:46.927 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
02-Mar-2023 14:18:46.927 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.awt.headless=true
02-Mar-2023 14:18:46.927 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:TargetSurvivorRatio=75
02-Mar-2023 14:18:46.928 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:SurvivorRatio=64
02-Mar-2023 14:18:46.928 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:MaxTenuringThreshold=3
02-Mar-2023 14:18:46.928 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+UseG1GC
02-Mar-2023 14:18:46.928 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+ScavengeBeforeFullGC
02-Mar-2023 14:18:46.928 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:ParallelGCThreads=4
02-Mar-2023 14:18:46.929 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xms1g
02-Mar-2023 14:18:46.929 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xmx2g
02-Mar-2023 14:18:46.929 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048
02-Mar-2023 14:18:46.929 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
02-Mar-2023 14:18:46.929 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
02-Mar-2023 14:18:46.930 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.management.jmxremote
02-Mar-2023 14:18:46.930 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.management.jmxremote.port=9001
02-Mar-2023 14:18:46.931 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.management.jmxremote.rmi.port=9001
02-Mar-2023 14:18:46.932 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.management.jmxremote.authenticate=false
02-Mar-2023 14:18:46.932 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.management.jmxremote.ssl=false
02-Mar-2023 14:18:46.932 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.rmi.server.hostname=localhost
02-Mar-2023 14:18:46.933 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs=
02-Mar-2023 14:18:46.933 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/home/ors/tomcat
02-Mar-2023 14:18:46.933 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/home/ors/tomcat
02-Mar-2023 14:18:46.934 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/home/ors/tomcat/temp
02-Mar-2023 14:18:46.934 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent The Apache Tomcat Native library which allows using OpenSSL was not found on the java.library.path: [/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib]
02-Mar-2023 14:18:47.073 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"]
02-Mar-2023 14:18:47.110 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
02-Mar-2023 14:18:47.166 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 1515 ms
02-Mar-2023 14:18:47.400 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
02-Mar-2023 14:18:47.401 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet engine: [Apache Tomcat/8.5.69]
02-Mar-2023 14:18:47.492 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/home/ors/tomcat/webapps/ors.war]
02-Mar-2023 14:19:01.114 INFO [localhost-startStop-1] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
14:19:03.730 [localhost-startStop-1] INFO org.heigit.ors.config.AppConfig - Default path of 'ors-config.json' used for configuration
14:19:03.748 [localhost-startStop-1] INFO org.heigit.ors.config.AppConfig - Loading configuration from /home/ors/tomcat/webapps/ors/WEB-INF/classes/ors-config.json
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details. |
|
+1 |
|
+1 |
|
+1 |
The user config file is now used when the container is restarted or recreated. Fixes #1300
|
@nicolnt I pushed a fix that should take the config files into consideration now. We'll slowly introduce that to the affected versions tomorrow and also make a new release for |
|
Hey @MichaelsJP , In the file // ...
"profile-car": {
"profiles": "driving-car",
"parameters": {
// ...
"maximum_snapping_radius": 5,
// ...I still get the following error which shows that my settings are not considered (should say 5 meters instead of 350 meters): Here's my docker-compose.yml file: version: '2.4'
services:
ors-app:
platform: linux/amd64
container_name: ors-app
ports:
- "8888:8080"
- "9001:9001"
image: openrouteservice/openrouteservice:nightly
user: "${UID:-0}:${GID:-0}"
build:
context: ../
args:
ORS_CONFIG: ./openrouteservice/src/main/resources/ors-config-sample.json
OSM_FILE: ./extractions_OSM/230315_landes.osm.pbf
volumes:
- ./graphs:/home/ors/ors-core/data/graphs
- ./elevation_cache:/home/ors/ors-core/data/elevation_cache
- ./logs/ors:/home/ors/ors-core/logs/ors
- ./logs/tomcat:/home/ors/tomcat/logs
- ./conf:/home/ors/ors-conf
environment:
- BUILD_GRAPHS=True # Forces the container to rebuild the graphs, e.g. when PBF is changed
- "JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:ParallelGCThreads=4 -Xms1g -Xmx2g"
- "CATALINA_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost"
EDITAPI status (the build used is nightly) {"engine":{"version":"7.0.1","build_date":"2023-03-15T15:58:51Z"}} |
|
@nicolnt Thanks for getting back on this. Please do not use nightly but latest at least. Try the following docker-compose.yml version: '2.4'
services:
ors-app:
container_name: ors-app
ports:
- "8080:8080"
- "9001:9001"
image: openrouteservice/openrouteservice:latest
user: "${UID:-0}:${GID:-0}"
volumes:
- ./graphs:/home/ors/ors-core/data/graphs
- ./elevation_cache:/home/ors/ors-core/data/elevation_cache
- ./logs/ors:/home/ors/ors-core/logs/ors
- ./logs/tomcat:/home/ors/tomcat/logs
- ./conf:/home/ors/ors-conf
- ./extractions_OSM/230315_landes.osm.pbf:/home/ors/ors-core/data/osm_file.pbf
environment:
- BUILD_GRAPHS=True # Forces the container to rebuild the graphs, e.g. when PBF is changed
- "JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:ParallelGCThreads=4 -Xms1g -Xmx2g"
- "CATALINA_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost"commands docker-compose pull
docker-compose up -d && docker-compose logs -ft |
|
Thanks I get some errors with the graphhopper: EDITI can edit config file and pbf via the docker volume method, build parameters Tried with another PBF file for a different region, same issue with graphhopper. This bug seems to be related to graphhopper or osm files, not openrouteservice. Couldn't find how to fix it... |
|
I'll keep this issue open because there still is confusion as build parameters are still present in the template Should I open a new issue for the graphhopper thing? |
|
@nicolnt Nope, it's fine. No need for another issue. If it works for you, we can try to find out what's happening in this issue.
Additionally:
|
|
@nicolnt I could reproduce the error with dordogne.osm.pbf. On it! |
|
Thanks for your help!
Running
macOS Monterey 12.6 on MacBook Pro M2 chip
Any, other departments suffer from the same issue at the moment |
|
Thanks for your patience!
Yup. I could reproduce it in version 7.x. Please downgrade to v6.8.3, this bug doesn't affect that version. This will solve it for you until it's fixed.
No prob. In docker, you generally deal with So, you have two steps: 1. Build arguments only apply to step 1, Environment variables only to 2. BUT -> build:
context: ../
args:
ORS_CONFIG: ./openrouteservice/src/main/resources/ors-config-sample.json
OSM_FILE: ./openrouteservice/src/main/files/heidelberg.osm.gzI don't recommend you to do this since it would build the image on the latest master code. Better stick with a release version with either |
|
Just tried with v6.8.3, It is still ignoring my version: '2.4'
services:
ors-app:
container_name: ors-app
ports:
- "8088:8080"
- "9001:9001"
image: openrouteservice/openrouteservice:v6.8.3
user: "${UID:-0}:${GID:-0}"
platform: linux/amd64
volumes:
- ./graphs:/home/ors/ors-core/data/graphs
- ./elevation_cache:/home/ors/ors-core/data/elevation_cache
- ./logs/ors:/home/ors/ors-core/logs/ors
- ./logs/tomcat:/home/ors/tomcat/logs
- ./conf:/home/ors/ors-conf
- ./aquitaine.osm.pbf:/home/ors/ors-core/data/osm_file.pbf
environment:
- BUILD_GRAPHS=True # Forces the container to rebuild the graphs, e.g. when PBF is changed
- "JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:ParallelGCThreads=4 -Xms1g -Xmx2g"
- "CATALINA_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhostEDIT : Another problem I'm encounteringIt's maybe linked, I have no idea. Very often, it often doesn't want to build graph and ors get stuck at this point: And http://localhost:8088/ors/v2/health is replying What I do is: docker-compose down
# Then
rm -r graphs elevation_cache && mkdir graphs elevation_cache && docker-compose up -d && docker-compose logs -ftIt could also be expected behavior and it just takes a long time before initializing graph |
|
It's actually working. I was confused and instead of waiting for 15 min I was thinking there was a bug. I don't understand why it was not using my config file at first and now it does. If I can reproduce I'll let you now, I don't get what happened Here are my thoughts to reduce confusion:
ors-app | 2023-04-04T09:03:02.662984845Z 04-Apr-2023 09:03:02.662 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 38889 msWhen graph is being built, user is noticed with messages like that in logs: ors-app | 2023-04-04T11:56:12.934990379Z 04 Apr 11:56:12 INFO [core.PrepareCore] - 0, updates:0, nodes: 263 714, shortcuts:0, dijkstras:975 349, t(dijk):1.74, t(period):0.0, t(lazy):0.0, t(neighbor):0.0, meanDegree:1, algo:4MB, totalMB:1024, usedMB:378
[...]
[...]
ors-app | 2023-04-04T08:36:41.713111918Z 08:36:41.703 [localhost-startStop-1] INFO org.heigit.ors.config.AppConfig - Default path of 'ors-config.json' used for configuration
ors-app | 2023-04-04T08:36:41.721769710Z 08:36:41.721 [localhost-startStop-1] INFO org.heigit.ors.config.AppConfig - Loading configuration from /home/ors/tomcat/webapps/ors/WEB-INF/classes/ors-config.json
[...] |
Here's what I did
Run docker-compose on the docker provided
docker-compose.ymlconf/ors-config.jsonandaveyron-latest.osm.pbfare located in the same directory as the docker compose file so no need to change context for meI edited
ors-config.jsonto extend"maximum_snapping_radius"property from 350 to 1000.Here's what I got
Got the following error in QGIS when calculating fastest path between input point to one end point:
Here's what I was expecting
No error, or at least a 1000 meters error, not 350
Here's what I think could be improved
The custom config file is not taken into account.
Here's some logs from the console when running
mkdir -p conf elevation_cache graphs logs/ors logs/tomcat && docker-compose upWhen inspecting the file
/home/ors/tomcat/webapps/ors/WEB-INF/classes/ors-config.jsonusingdocker execI can see that it's the default configuration with 350 metersThe text was updated successfully, but these errors were encountered: