NeedForHeat Server Configuration

This repository contains configuration scripts and instructions for configuring a NeedForHeat Server, comprising of multiple Docker containers hosted on a Linux server, based on the following technologies: Traefik proxy, Portainer, MariaDB, CloudBeaver, Duplicati, NeedForHeat Server API and JupyterLab.

NB: Where you read energietransitiewindesheim.nl below, you should subsitute your own domain; where you read etw below, you should subsitute your own server abbreviation.

Table of contents

• Prerequisites
  • SSH setup
  • Domain setup
• Deploying
  • Setting up Traefik and Portainer
  • Deploying stacks with Portainer
• Updating
• Features
• License
• Credits

Prerequisites

SSH setup

For easy ssh/scp access to the server, copy your ssh certificate to the server. This requires the root password once; after that, your ssh key will be used for authentication.

Open a Linux command window on your local system. On Windows, you can use e.g. Windows Subsystem for Linux for this.

Generate a public/private key pair (files .ssh/etw.pub and .ssh/etw)

ssh-keygen -f ~/.ssh/etw

When asked for passphrase, leave that empty.

Now, upload your public key to the server.

ssh-copy-id -i ~/.ssh/etw root@energietransitiewindesheim.nl

Login with ssh, without using a password

ssh -i ~/.ssh/etw root@energietransitiewindesheim.nl

Append the following to your .ssh/config

Host etw
User root
HostName energietransitiewindesheim.nl
IdentityFile ~/.ssh/etw

With this config in place, login to the server is simply

ssh etw

Domain setup

We have production (prd) and test (tst) environment, each on a separate server. The production and test environment can be destinguished by their URL.

• Production URL's: <service>.energietransitiewindesheim.nl
• Test URL's: <service>.tst.energietransitiewindesheim.nl

If you wish to copy our configuration, then you will need to find and replace energietransitiewindesheim.nl with your own domain in all configuration files.

The URL's we use for our public services:

Service	Environment	URL
API	production	api.energietransitiewindesheim.nl
API	test	api.tst.energietransitiewindesheim.nl
Manuals	production	manuals.energietransitiewindesheim.nl
Manuals	test	manuals.tst.energietransitiewindesheim.nl

The URL's we use for system services:

Service	Environment	URL
Portainer	production	docker.energietransitiewindesheim.nl
Portainer	test	docker.tst.energietransitiewindesheim.nl
CloudBeaver	production	db.energietransitiewindesheim.nl
CloudBeaver	test	db.tst.energietransitiewindesheim.nl
Duplicati	production	backup.energietransitiewindesheim.nl
Duplicati	test	backup.tst.energietransitiewindesheim.nl
JupyterLab	production	analysis-<name>.energietransitiewindesheim.nl
JupyterLab	test	analysis-<name>.tst.energietransitiewindesheim.nl

Deploying

Setting up Traefik and Portainer

This setup has to be done with SSH access to the server. During these steps, portainer is shorly exposed to the public internet without protection of credentials or IP whitelists.

1. Log into the server using SSH.
2. Create a volume for portainer:

   docker volume create portainer_data

3. Start the container:

   docker run -d --rm -p 9443:9443 --name portainer-bootstrap \
   -v /var/run/docker.sock:/var/run/docker.sock \
   -v portainer_data:/data \
   portainer/portainer-ce:latest

4. Open Portainer in your browser: https://server-external-ip-address:9443.
5. Add portainer and traefik as a stack by following the steps described here.
6. Stop the bootstrapping container:

   docker stop portainer-bootstrap

7. Restart portainer:

   docker restart portainer

8. Portainer is now available at docker.energietransitiewindesheim.nl.

Deploying stacks with Portainer

Use the following steps to deploy a service as a Portainer stack.

At Windesheim we use Portainer stacks, but the docker-compose files can also be deployed without Portainer, using Docker Compose.

Add a new stack according to the following steps:

1. Go to stacks in portainer.
2. Click on Add stack.
3. Give the stack a name.
4. Select git repository as the build method.
5. Set the repository URL to:

   https://github.com/energietransitie/needforheat-server-configuration
   

6. Set the repository reference to:

   refs/heads/main
   

7. Set the compose path to point to the docker-compose.yml you want to deploy.

   Refer to the stack-specific README's for each stack's compose path.

8. Optionally add environment variables if used in the docker-compose file. Click on Add an environment variable to add additional variables.

   Refer to the stack-specific README's for each stack's environment variables.

9. Click on Deploy the stack.

Updating

When a docker-compose.yml for a stack is changed, the new configuration can be retrieved by following the steps below:

1. Go to Stacks in portainer.
2. Click on the stack you want to update.
3. You can change environment variables if you want.
4. Click on Pull and redeploy to retrieve the configuration from the main branch and update the container(s).
5. A pop-up will open. Click on Update to confirm.
6. The stack will update. Wait for the notification to know that the stack was updated successfully.

Features

List of features ready and TODOs for future development. Ready:

• Implementing automated backup solution
• Limiting access by IPv4 address
• Ratelimiting on the API
• Further implementation of Dependabot

To-do:

• Create only docker-compose file for Traefik instead of .toml files
• Integrated deployment of the entire NeedForHeat Server stack

Optional:

• force use of SSH tunnel to portainer container
• force use of SSH tunnel to cloudbeaver container
• force use of SSH tunnel to traefik container

License

This software is available under the Apache 2.0 license, Copyright 2022 Research group Energy Transition, Windesheim University of Applied Sciences

Credits

This configuration repository was originally created by:

• Arjan Peddemors · @arpe

It was extended by:

• Harris Mesic · @Labhatorian
• Nick van Ravenzwaaij · @n-vr
• Erik Krooneman · @Erikker21
• Leon Kampstra · @LeonKampstra
• Jorrin Kievit · @JorrinKievit
• Henri ter Hofte · @henriterhofte · Twitter @HeNRGi

Product owner:

• Henri ter Hofte · @henriterhofte · Twitter @HeNRGi

We use and gratefully acknowlegde the efforts of the makers of the following technologies:

• Traefik proxy, by Traefik Labs, licensed under MIT
• Portainer, by Portainer.io, licensed under Zlib
• MariaDB, licensed under GNU GPL-2.0
• CloudBeaver, by DBeaver Corp, licensed under Apache-2.0)
• Duplicati, by duplicati.com, licensed under LGPL-2.1
• NeedForHeat Server API, by Research group Energy Transition, Windesheim University of Applied Sciences, licensed under Apache-2.0
• JupyterLab, by Project Jupyter Contributors, licensed under this license