- Unidata THREDDS Docker
Dockerized THREDDS.
This repository contains files necessary to build and run a THREDDS Docker container. The Unidata THREDDS Docker images associated with this repository are available on DockerHub.
docker run -d -p 80:8080 unidata/thredds-docker:<version>See tags listed on dockerhub. Our security protocols have become stricter, and older images must be removed due to potential links with high profile CVEs. We strive to maintain the security of this project's DockerHub images by updating them with the latest upstream improvements. If you have any concerns in this area, please email us at security@unidata.ucar.edu to bring them to our attention.
Before you begin using this Docker container project, make sure your system has Docker installed. Docker Compose is optional but recommended.
You can either pull the image from DockerHub with:
docker pull unidata/thredds-docker:<version>Or you can build it yourself with:
- Clone the repository:
git clone https://github.com/Unidata/thredds-docker.git - Navigate to the project directory:
cd thredds-docker - Build the Docker image:
docker build -t thredds-docker:<version> .
Tomcat web applications and the TDS can require large amounts of memory to run. This container is setup to run Tomcat with a default 4 gigabyte memory allocation. When running this container, ensure your VM or hardware can accommodate this memory requirement.
To run the THREDDS Docker container, beyond a basic Docker setup, we recommend installing docker-compose. docker-compose serves two purposes:
-
Reduce headaches involving unwieldy
dockercommand lines where you are runningdockerwith multiple volume mounts and port forwards. In situations like these,dockercommands become difficult to issue and read. Instead, the lengthydockercommand is captured in adocker-compose.ymlthat is easy to read, maintain, and can be committed to version control. -
Coordinate the running of two or more containers to, for example, orchestrate the TDS and TDM. This can be useful for taking into account the same volume mountings, for example.
However, docker-compose use is not mandatory. There is an example docker-compose.yml in this repository.
Once you have completed your setup you can run the container with:
docker-compose up -d thredds-productionThe output of such command should be something like:
Creating thredds
To stop this container:
docker-compose stop thredds-productionTo clean the slate and remove the container (not the image, the container):
docker-compose rm -f thredds-productionUpgrading to a newer version of the container is easy. Simply stop the container via docker or docker-compose, followed by
docker pull unidata/thredds-docker:<version>and restart the container. Refer to the new version from the command line or in the docker-compose.yml.
At this point you should be able to do:
curl localhost:80/thredds/catalog/catalog.html
# or whatever port you mapped to outside the container in the docker-compose.ymland get back a response that looks something like
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>TDS Catalog</title>
<!-- Common metadata and styles. -->
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<!-- if webcrawler finds this page (say, from sitemap.xml), tell it to not follow the links -->
<meta name="robots" content="nofollow" />
<link rel="stylesheet" href="/thredds/tds.css" type="text/css"><link rel="stylesheet" href="/thredds/tds.css" type="text/css"><link rel="stylesheet" href="/thredds/tdsCat.css" type="text/css">
<script type="text/javascript">
document.getElementById("header-buffer").style.height = document.getElementById("header").clientHeight + "px";
document.getElementById("footer-buffer").style.height = document.getElementById("footer").clientHeight + "px";
</script>
</head>
...
</html>
If you encounter a problem there, you can also:
docker pswhich should give you output that looks something like this:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6c256c50a6cf unidata/thredds-docker:<version> "/entrypoint.sh catal" 6 minutes ago Up 6 minutes 0.0.0.0:8443->8443/tcp, 0.0.0.0:80->8080/tcp, 0.0.0.0:443->8443/tcp threddsdocker_thredds-quickstart_1
to obtain the ID of the running TDS container. You can enter the container with:
docker exec -it <ID> bashUse curl inside the container to verify the TDS is running:
curl localhost:8080/thredds/catalog/catalog.htmlyou should get a response that looks something like:
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>TDS Catalog</title>
<!-- Common metadata and styles. -->
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<!-- if webcrawler finds this page (say, from sitemap.xml), tell it to not follow the links -->
<meta name="robots" content="nofollow" />
<link rel="stylesheet" href="/thredds/tds.css" type="text/css"><link rel="stylesheet" href="/thredds/tds.css" type="text/css"><link rel="stylesheet" href="/thredds/tdsCat.css" type="text/css">
<script type="text/javascript">
document.getElementById("header-buffer").style.height = document.getElementById("header").clientHeight + "px";
document.getElementById("footer-buffer").style.height = document.getElementById("footer").clientHeight + "px";
</script>
</head>
...
</html>
Define directory and file paths for log files, Tomcat, THREDDS, and data in docker-compose.yml for the thredds-production image.
This project contains a docker-compose environment file named compose.env. This file contains default values for docker-compose to launch the TDS and TDM. You can configure these parameters:
| Parameter | Environment Variable | Default Value |
|-----------------------------+-----------------------+------------------------------|
| TDS Content Root | TDS_CONTENT_ROOT_PATH | /usr/local/tomcat/content |
| TDS JVM Max Heap Size (xmx) | THREDDS_XMX_SIZE | 4G |
| TDS JVM Min Heap Size (xms) | THREDDS_XMS_SIZE | 4G |
| TDM Password | TDM_PW | CHANGEME! |
| TDS HOST | TDS_HOST | http://thredds.yourhost.net/ |
| TDM JVM Max Heap Size (xmx) | TDM_XMX_SIZE | 6G |
| TDM JVM Min Heap Size (xms) | TDM_XMS_SIZE | 1G |
| Tomcat User ID | TOMCAT_USER_ID | 1000 |
| Tomcat Group ID | TOMCAT_GROUP_ID | 1000 |
If you wish to update your configuration, you can either update the compose.env file or create your own environments file by copying compose.env. If using your own file, you can export the suffix of the file name into an environment variable named THREDDS_COMPOSE_ENV_LOCAL. Also see the env_file key in docker-compose.yml.
For example:
cp compose.env compose_local.env
export THREDDS_COMPOSE_ENV_LOCAL=_local
< edit compose_local.env >
docker-compose up thredds-productionTHREDDS container is based off of the canonical Tomcat container with some additional security hardening measures. Tomcat configuration can be done by mounting over the appropriate directories in CATALINA_HOME (/usr/local/tomcat).
The Java configuration options (JAVA_OPTS) are configured in ${CATALINA_HOME}/bin/javaopts.sh (see javaopts.sh) inside the container. Note this file is copied inside the container during the Docker build. See the docker-compose section above for configuring some of the environment variables of this file.
To mount your own content/thredds directory with docker-compose.yml:
volumes:
- /path/to/your/thredds/directory:/usr/local/tomcat/content/threddsIf you just want to change a few files, you can mount them individually. Please note that the THREDDS cache is stored in the content directory. If you choose to mount individual files, you should also mount a cache directory.
volumes:
- /path/to/your/tomcat/logs/:/usr/local/tomcat/logs/
- /path/to/your/thredds/logs/:/usr/local/tomcat/content/thredds/logs/
- /path/to/your/tomcat-users.xml:/usr/local/tomcat/conf/tomcat-users.xml
- /path/to/your/thredds/directory:/usr/local/tomcat/content/thredds
- /path/to/your/data/directory1:/path/to/your/data/directory1
- /path/to/your/data/directory2:/path/to/your/data/directory2
- /path/to/your/server.xml:/usr/local/tomcat/conf/server.xml
- /path/to/your/web.xml:/usr/local/tomcat/conf/web.xml
- /path/to/your/keystore.jks:/usr/local/tomcat/conf/keystore.jksPlease see Tomcat parent container repository for HTTP over SSL instructions.
By default, Tomcat will start with two user accounts.
tdm- used by the THREDDS Data Manager for connecting to THREDDSadmin- can be used by everything else (has full privileges)
See the parent Tomcat container for information about creating passwords for these users.
TDS Remote Management is enabled for the admin user by default, and can be accessed via http(s)://<your server>/thredds/admin/debug.
To enable to ncSOS, change
<NCSOS>
<allow>false</allow>
</NCSOS>to true in threddsConfig.xml.
The THREDDS Data Manager (TDM) creates indexes for GRIB featureCollections, in a process separate from the TDS. It is a specialized utility typically employed in scenarios where the TDS is serving real-time data from the Unidata IDD (e.g., GFS Quarter Degree Analysis) and is referenced in the docker-compose.yml in this repository. In most scenarios, you can comment out the TDM section. The TDM Docker container is in its own repository where you can find instructions on how to run it.
This Docker project includes the installation of the netCDF-c project to allow for the downloading of netCDF files using the NetCDF Subset Service.
What to do when a version of the THREDDS data server is released?
- Update the
Dockerfilewith thewarfile corresponding to the new version of the TDS. E.g.,
ENV THREDDS_WAR_URL https://downloads.unidata.ucar.edu/tds/5.5/thredds-5.5.war- Check with the netCDF group if versions of HDF5, zlib, and netCDF referenced in the
Dockerfileneed to be updated. - Update TDS versions in
docker-compose.ymlanddocker-compose-swarm.yml. - Update the
CHANGELOG.mddocumenting updates to this project (not the TDS) since the last release. - Create a new git branch corresponding to this version of the TDS (e.g.,
5.5). - Push the new branch out to the
Unidata/thredds-dockerGitHub repository. This branch will remain frozen in time going forward. Any subsequent updates to this project should happen on the thelatestbranch. The only exception to this convention is if there is a critical (e.g., security related) update that needs to be applied to theDockerfileand associated files and eventually to the image (see below) - Build a docker image corresponding to the new version of the TDS (e.g., on the Docker build machine on Jetstream). E.g.,
docker build -t unidata/thredds-docker:5.5. - Test to ensure the image works.
- Push it out DockerHub e.g.,
docker push unidata/thredds-docker:5.5. - Note that this image does not remain frozen in time for two reasons.
- It can get rebuilt time and again as upstream image updates need to be incorporated into this THREDDS image. It may be confusing for a versioned image to evolve, but it is the convention in Dockerland.
- It can get rebuilt in the rare case the Dockerfile or associated files are updated on the branch as mentioned earlier.
In order to cite this project, please simply make use of the Unidata THREDDS Data Server DOI: https://doi.org/10.5065/D6N014KG https://doi.org/10.5065/D6N014KG
If you have a question or would like support for this THREDDS Docker container, consider submitting a GitHub issue. Alternatively, you may wish to start a discussion on the THREDDS Community mailing list: thredds@unidata.ucar.edu.
For general TDS questions, please see the THREDDS support page.