Infinispan OpenShift Templates

This repository contains OpenShift templates for Infinispan. The templates are in sync with OpenShift Library.

This templates are designed for running on top of OpenShift Container Platform. For quick local testing, Minishift offers an easy way for running things locally. This README file concentrates on showing how to run the templates on top of Minishift.

DEPRECATED: These templates are no longer the recommended way to utilise Infinispan on Kubernetes/Openshift. Instead we recommend that future deployments make use of the Infinispan Operator

Minishift

Firstly, install Minishift.

Once installed Minishift, you should create a profile before starting Minishift. Profiles allow you to create and manage these isolated instances of Minishift. Here’s an example configuration with enough resources to run multiple applications and multiple Infinispan pods:

minishift profile set infinispan-openshift-test
minishift config set cpus 4
minishift config set memory 8192
minishift config set disk-size 50g

On top of this Minishift configuration, depending on your environment you’ll need to set the correct vm-driver. One option is to use xhyve, which is set like this:

minishift config set vm-driver xhyve

For ease of use, it’s also recommended to enable the following two addons:

minishift addons enable admin-user
minishift addons enable anyuid

Finally, start Minishift and add OpenShift client libraries to path:

minishift start
eval $(minishift oc-env)

Templates

In this section you will learn about the OpenShift templates offered in this repository. You will also find out how to install and use them.

All templates expose Infinispan’s configuration via a ConfigMap element. This makes it easy for the user to modify the XML directly in the template and re-install the template. Note that the ConfigMap can also be modified at runtime using the OpenShift console.

Ephemeral Template

This template demonstrates an Infinispan deployment that exclusively uses in-memory storage.

You can install the template by calling:

oc create -f infinispan-ephemeral.yaml

Next, instantiate the template to start an Infinispan Server cluster:

oc new-app infinispan-ephemeral \
  -p APPLICATION_USER=test \
  -p APPLICATION_PASSWORD=changeme \
  -p NUMBER_OF_INSTANCES=3

Once all pods are ready, let’s run a test to store and retrieve some data via the HTTP REST endpoint. Since the test is executed from outside OpenShift, you should first expose the HTTP REST endpoint as a route:

oc expose svc/infinispan-app-http

Next, store some data:

export ROUTE=`oc get route/infinispan-app-http -o jsonpath="{.spec.host}"`

curl -v \
  -u test:changeme \
  -H 'Content-type: text/plain' \
  -d 'test' \
  $ROUTE/rest/default/stuff

Then, retrieve and you should get the value stored:

curl -v \
  -u test:changeme \
  $ROUTE/rest/default/stuff

Persistent Template

This template demonstrates an Infinispan deployment where in-memory data is backed up by filesystem-based persistence. The template is very similar to the ephemeral one with a couple of exceptions:

• The default cache is configured with local rocksdb file-based persistence.

• The file path where data is persisted is backed by a PersistenVolume. This means that data persisted in the PersistentVolume survives an Infinispan cluster restart.

You can install the template by calling:

oc create -f infinispan-persistent.yaml

Before instantiating the template, you should make sure the persistent volumes will be writable by Infinispan Server. In Minishift, the directories that back persistent volumes are privileged and protected and owned by the root user. For ease of development on Minishift, you can relax the directory and permissions ownership for persistent volumes:

minishift ssh
sudo chmod -R a+w /var/lib/minishift/base/openshift.local.pv/pv*
sudo chmod -R 777 /var/lib/minishift/base/openshift.local.pv/pv*

Next, instantiate the template to start an Infinispan Server cluster:

oc new-app infinispan-persistent \
  -p APPLICATION_USER=test \
  -p APPLICATION_PASSWORD=changeme \
  -p NUMBER_OF_INSTANCES=3

Once all pods are ready, let’s run a test to verify persisted data survives complete restarts.

Since the test is executed from outside OpenShift, you should first expose the HTTP REST endpoint as a route:

oc expose svc/infinispan-persistent-app-http

Next, store some data:

export ROUTE=`oc get route/infinispan-persistent-app-http -o jsonpath="{.spec.host}"`

curl -v \
  -u test:changeme \
  -H 'Content-type: text/plain' \
  -d 'test' \
  $ROUTE/rest/default/stuff

To verify that data is accessible after a complete restart, scale down the Infinispan Server cluster to 0 instances:

oc scale statefulset infinispan-persistent-app --replicas=0

Then, scale it back up to 3 instances:

oc scale statefulset infinispan-persistent-app --replicas=3

Once all pods are ready, you should be able to retrieve the via:

curl -v \
  -u test:changeme \
  $ROUTE/rest/default/stuff

Extras

This section contains extra information related to these templates.

Maintenance

You might decide to make some changes to the template. Once you’ve made the changes, you can replace the template by calling:

oc replace -f infinispan-ephemeral.yaml

Also, you can remove the template:

oc delete template infinispan-ephemeral

Infinipan Server instances created by a specific template can be removed by calling:

oc delete all,secrets,sa,templates,configmaps,daemonsets,clusterroles,rolebindings,serviceaccounts --selector=template=infinispan-ephemeral || true

Note that any persistent volumes attached are not removed by default. These can be removed by calling:

oc delete pvc -l application=infinispan-persistent-app

Testing

test-ephemeral.sh and test-persistent.sh smoke test scripts are provided to quickly verify each template works as expected.