ISLE: Islandora Enterprise 8 Prototype

• Introduction
• Requirements
• Getting Started
• Local Development
• Custom Environment
• Secrets
• Services
  • Code Server
  • Watchtower
  • Traefik
  • ETCD
• Troubleshooting/Issues
• FAQ
  • Question: When doing an drush config:import -y I get one of the following errors
  • Question: When doing an drush sql:dump I get one of the following errors
• Development
• Maintainers/Sponsors
  • Architecture Team
• Sponsors
• License

Introduction

Docker Compose project for creating and managing an Islandora 8 instance using Docker containers from isle-buildkit.

In a nutshell, isle-dc generates a docker-compose.yml file for you based on configuration that you supply in a .env file. And there are three use cases we're trying to accomplish:

• demo (Example site for kicking the tires and looking at Islandora)
• local (Local development using composer/drush in the codebase folder)
• custom (A custom Dockerfile to deploy created from local)

On top of that, there's a lot of useful commands for managing an Islandora instance, such as database import/export and reindexing.

Requirements

• Composer 1.10+
• Desktop / laptop / VM (Docker must have sufficient resources to run GNU Make)
• Docker-CE 19.x+ (If using Docker Desktop for Windows, any stable release after 2.2.0.4, or use a 2.2.0.4 with a patch due to a bug)
• Docker-compose version 1.25.x+
• Drush 9.0+
• Git 2.0+
• GNU Make 4.0+

Getting Started

To get started with a demo environment, run:

make demo

This will pull down images from Dockerhub and generate

File	Purpose
.env	A configuration file that is yours to customize. This file controls how the docker-compose.yml file gets generated to meet your use case. It also allows you to set variables that make their way into the final docker-compose.yml file, such as your site's domain.
docker-compose.yml	A ready to run docker-compose.yml file based on your .env file. This file is considered disposable. When you change your .env file, you will generate a new one.

Your new Islandora instance will be available at https://islandora.traefik.me. Don't let the funny url fool you, it's a dummy domain that resolves to 127.0.0.1.

You can log into Drupal as admin using the default password, password.

Enjoy your Islandora instance! Check out the Islandora documentation to see all the things you can do. If you want to poke around, here's all the services that are available to visit:

Service	Url
Drupal	https://islandora.traefik.me
Traefik	https://islandora.traefik.me:8080
Fedora	https://islandora.traefik.me:8081/fcrepo/rest
Blazegraph	https://islandora.traefik.me:8082/bigdata
Activemq	https://islandora.traefik.me:8161
Solr	https://islandora.traefik.me:8983
Cantaloupe	https://islandora.traefik.me/cantaloupe
Matomo	https://islandora.traefik.me/matomo/
Code Server	https://islandora.traefik.me:8443/

When you're done with your demo environment, shut it down by running

docker-compose down

This will keep your data around until the next time you start your instance. If you want to completely destroy the repository and all ingested data, use

docker-compose down -v

Local Development

When developing locally, your Drupal site resides in the codebase folder and is bind-mounted into your Drupal container. This lets you update code using the IDE of your choice on your host machine, and the changes are automatically reflected on the Drupal container. Simply place any exported Drupal site as the codebase folder in isle-dc and you're good to go.

If you don't provide a codebase, you'll be given a vanilla Drupal 9 instance with the Islandora module installed and the bare minimum configured to run. This is useful if you want to build your repository from scratch and avoid islandora_defaults.

If you've included configuration in your exported site using drush config:export, then you'll need to set two values in your .env file:

INSTALL_EXISTING_CONFIG=true
DRUPAL_INSTALL_PROFILE=minimal

In either case, run this command to make a local environment.

make local

If you already have a Drupal site but don't know how to export it, log into your server, navigate to the Drupal root, and run the following commands:

• drush config:export
• git init
• git add -A .
• git commit -m "First export of site"

Then you can git push your site to Github and git clone it down whenever you want.

Custom Environment

This environment is used to run your custom drupal image which can be produced outside of this repository. You can specify the image in your .env file using the settings PROJECT_DRUPAL_DOCKERFILE if you want to build it in the context of this repository.

For convenience a sample.Dockerfile is provided from which you can generate a custom image from the codebase folder. For example if you followed the guide above to create the codebase folder from the islandora/demo image.

And then run it by changing ENVIRONMENT to be custom and regenerating the docker-compose.yml file and building the image.

make docker-compose.yml
make build

At this point you could run it using docker-compose:

docker-compose up -d

To specify an image created outside of this repository, you can add the following to docker-compose.env.yml:

drupal:
  image: YOUR_CUSTOM_IMAGE

Secrets

When running Islandora in the wild, you'll want to use secrets to store sensitive information such as credentials. Secrets are communicated from the docker host to the individual containers over an encrypted channel, making it much safer to run in production.

Some confd backends, such as etcd, can be used to serve secrets directly. Simply expose etcd over https and nothing else needs to be done. But for other backends, particuarly environment variables, you must mount the secrets into containers as files using docker-compose. During startup, the files' contents are read into the container environment and made available to confd.

To enable using secrets, set USE_SECRETS=true in your .env file. When you run make docker-compose.yml, a large block of secrets will be added at the top of your docker-compose.yml file.

secrets:
  ACTIVEMQ_PASSWORD:
    file: "./secrets/ACTIVEMQ_PASSWORD"
  ACTIVEMQ_WEB_ADMIN_PASSWORD:
    file: "./secrets/ACTIVEMQ_WEB_ADMIN_PASSWORD"
  ...

Each secret references a file in the secrets directory. Each secrets file is named the exact same as the environment variable it intends to replace. The contents of each file will be used as the value for the secret.

Services

Islandora is composed of many different services, this project has split these services up such that they have their own Container.

For in-depth documentation of the various islandora images see the isle-buildkit repository.

Other services will be documented below:

Code Server

The code-server container allows a user to edit / debug their Drupal site from their browser.

The code-server service can be disabled/enabled via the INCLUDE_CODE_SERVER_SERVICE variable in your .env file.

# Includes `code-server` as a service.
INCLUDE_CODE_SERVER_SERVICE=true

By default this will accessible at https://islandora.traefik.me:8443/.

You can login with the default password: password.

N.B: Do not expose this service on the internet without setting a strong password via the ./secrets/CODE_SERVER_PASSWORD, or better yet do not expose it at all, and instead use port forward to access it if you have the need. Exposing this service in an insecure way will allow root access to your server to the public.

To enable xdebug for your request, you must also send an XDEBUG_SESSION cookie with your request, this can be toggled on and off via a browser plugin such as the following.

• Chrome
• Firefox

Watchtower

The watchtower container monitors the running Docker containers and watches for changes to the images that those containers were originally started from. If watchtower detects that an image has changed, it will automatically restart the container using the new image. This allows for automatic deployment, and overall faster development time.

Note however Watchtower will not restart stopped container or containers that exited due to error. To ensure a container is always running, unless explicitly stopped, add restart: unless-stopped property to the container in the [docker-compose.yml] file. For example:

mariadb:
    image: islandora/mariadb:latest
    restart: unless-stopped

The watchtower can be disabled/enabled via the INCLUDE_WATCHTOWER_SERVICE variable in your .env file.

# Includes `watchtower` as a service.
INCLUDE_WATCHTOWER_SERVICE=true

Traefik

The traefik container acts as a reverse proxy, and exposes some containers through port 80/443/3306.

Since Drupal passes links to itself in the messages it passes to the microservices, and occasionally other urls need to be resolved on containers that do not have external access, we define aliases for most services on the internal network.

Aliases like so are defined on most services to mimic their routing rules in Traefik:

drupal:
    image: islandora/demo:latest
    # ...
    networks:
      default:
        aliases:
          - islandora-isle-dc.traefik.me

The traefik service can be disabled/enabled via the INCLUDE_TRAEFIK_SERVICE variable in your .env file.

# Includes `traefik` as a service.
INCLUDE_TRAEFIK_SERVICE=true

ETCD

The etcd container is a distributed reliable key-value store, which this project can use for configuration settings and secrets.

It is not enabled by default.

# Includes `etcd` as a service.
INCLUDE_ETCD_SERVICE=false

Troubleshooting/Issues

Post your questions here and subscribe for updates, meeting announcements, and technical support

• Islandora ISLE Interest Group - Meetings open to everybody!
  • Schedule is alternating Wednesdays, 3:00pm EDT
• Islandora ISLE Google group
• Islandora ISLE Slack channel #isle
• Islandora Group
• Islandora Dev Group

FAQ

Question: When doing an drush config:import -y I get one of the following errors

Error:

Site UUID in source storage does not match the target Storage.

This occurs when your configuration has come from a site other than the one that is currently installed. You can run the following command to override the site uuid so that you can import your configuration:

make set-site-uuid

Error:

Entities exist of type <em class="placeholder">Shortcut link</em> and <em class="placeholder">Shortcut set</em>
<em class="placeholder">Default</em>. These entities need to be deleted before importing

These are entities created by the standard installation profile, you can delete them with the following command in the codebase folder:

make delete-shortcut-entities

Error:

Error: Call to a member function getConfigDependencyName() on null in ... Entity/EntityDisplayBase.php on line 325 #0 ... /codebase/web/core/lib/Drupal/Core/Config/Entity/ConfigEntityBase.php(318): Drupal\Core\Entity\EntityDisplayBase->calculateDependencies()

There is some bug in the dependencies of the various modules. Until those dependencies issues are resolved just rerun the command until they go away.

drush config:import -y

Question: When doing an drush sql:dump I get one of the following errors

Error:

This arises from our use of MariaDB for the database in Docker, not matching the same client on your host system. Which probably uses the MySQL clients mysqldump executable. You can specify the following command using drush to get around it:

drush sql:dump --extra-dump="--column-statistics=0" > /tmp/dump.sql

Or you can use the provided method in the Makefile which should be more portable.

make database-dump DEST=/tmp/dump.sql

Development

If you would like to contribute to this project, please check out CONTRIBUTING.md. In addition, we have helpful Documentation for Developers info, as well as our Developers section on the Islandora.ca site.

drush --debug sql:dump --extra-dump="--column-statistics=0" > /tmp/dump.sql^C

Maintainers/Sponsors

Architecture Team

• Nigel Banks
• Jeffery Antoniuk, Canadian Writing Research Collaboratory
• Nia Kathoni, Canadian Writing Research Collaboratory
• Aaron Birkland, Johns Hopkins University
• Jonathan Green, LYRASIS
• Danny Lamb, Islandora Foundation
• Gavin Morris (Project Tech Lead), Born-Digital
• Mark Sandford (Documentation Lead), Colgate University
• Daniel Bernstein, LYRASIS

Sponsors

This project has been sponsored by:

• Grinnell College
• Tri-College (Bryn Mawr College, Haverford College, Swarthmore College)
• Wesleyan University
• Williams College
• Colgate University
• Hamilton College
• Amherst College
• Mount Holyoke College
• Franklin and Marshall College
• Whitman College
• Smith College
• Arizona State University
• Canadian Writing Research Collaboratory (CWRC)
• Johns Hopkins University
• Tulane University
• LYRASIS
• Born-Digital

License

MIT