Merge branch 'main' of github.com:NOAA-GSL/idss-engine-commons

README.md

@@ -9,7 +9,6 @@ The `idss-engine-commons` is responsible for defining all implicit common depend
 The complete twelve-factors methodologies that the IDSS Engine Project adheres to can be found in the umbrella [idss-engine](https://github.com/NOAA-GSL/idss-engine) repository. The subset of the twelve factors that follows are specifics to this app only.
 
 ## Logging
----
 To support some standardization and best practices for IDSS Engine, developers should utilize the [idss-engine-commons logging](https://github.com/NOAA-GSL/idss-engine-commons) Java and python packages.
 
 The general guidelines for logging are as follows:
@@ -50,7 +49,7 @@ Any python based microservice within IDSS Engine should use this image as it's b
 
 > **Image Name** `idss.engine.commons.python.service`
 
-> **Recommended Tags** development `:dev` stable release `:major.minor` ie. `:1.0` targeted environment `:aws`
+> **Recommended Tags** development `:dev` stable release `:release` ie. `:alder` targeted environment `:aws`
 
 #### Build
 From the project root directory `idss-engine-commons`:
@@ -108,3 +107,51 @@ URLs:
 
 Using the port mapping `-p 15672:15672` allows access to the RabbitMQ Web UI Management Console: http://localhost:15672/
 
+### Couchdb Server
+---
+
+` idss.engine.commons.couchdb.server:<tag>`
+
+> **Recommended Tags** development `:dev` stable release `:major.minor` ie. `:1.0` targeted environment `:aws` test `:test`
+From the `idss-engine-commons/rabbitmq` directory:
+
+#### Build
+
+`$ docker build -t idss.engine.commons.couchdb.server:<tag> .`
+
+#### Volumes
+A data directory on the host system (outside the container) and mount this to a directory visible from inside the container. This places the database files in a known location on the host system, and makes it easy for tools and applications on the host system to access the files.
+
+> ```
+> -v <local dir>:/opt/couchdb/data
+> ```
+
+The ```-v <local dir>:/opt/couchdb/data``` part of the command mounts the ```<local dir>``` directory from the underlying host system as ```/opt/couchdb/data``` inside the container, where CouchDB by default will write its data files.
+
+**NOTE:** specifying a volume is **required** for data persistance. Any data creation, modification, or removal will not be persisted without a volume mapping will be lost if the container is stopped or removed.
+
+#### Run
+
+**Required arguments**:
+> ```
+> -e COUCHDB_USER=<couchdb default user>
+> -e COUCHDB_PASSWORD=<couchdb default password>
+> ```
+
+**Optional arguments**
+
+Use `--name` if running on a container network
+
+> ```
+> -v <local_dir>:/opt/couchdb/data
+> --network <network>
+> --rm
+> --name <host name (used by other containers as a name for connection)>
+> -p 5984:5984
+> ```
+
+Examples:
+
+`$ docker run --rm -d --name couchtest --network couch-test-network -e COUCHDB_USER=idss -e COUCHDB_PASSWORD=idss -p 5984:5984 idss.engine.commons.couchdb.server:<tag>`
+
+`$ docker run --rm -d --name couchtest --network couch-test-network -e COUCHDB_USER=idss -e COUCHDB_PASSWORD=idss -p 5984:5984 -v ~/Desktop/couchdb/data:/opt/couchdb/data idss.engine.commons.couchdb.server:<tag>`

couchdb/Dockerfile

@@ -0,0 +1 @@
+FROM couchdb:3.2.2

couchdb/test/Dockerfile

@@ -0,0 +1,10 @@
+FROM python:3.6
+
+# install additional packages specific to this microservice
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+WORKDIR /src
+COPY ./src/test.py /src/
+
+ENTRYPOINT [ "python3", "test.py" ]

couchdb/test/requirements.txt

@@ -0,0 +1 @@
+pycouchdb

couchdb/test/run_test.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+
+# command line arguments:
+#     [1] = couchdb server host name
+#     [2] = couchdb client/server username
+#     [3] = couchdb client/server password
+python ./test_driver.py "couchtest" "idss" "idss"

couchdb/test/src/test.py

@@ -0,0 +1,64 @@
+'''/*******************************************************************************
+ * Copyright (c) 2022 Regents of the University of Colorado. All rights reserved.
+ *
+ * Contributors:
+ *     Michael Rabellino
+ *******************************************************************************/'''
+import pycouchdb
+import sys
+import logging
+import argparse as ap
+
+# configure logging
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s [%(levelname)s] %(message)s",
+    handlers=[
+        logging.StreamHandler(sys.stdout)
+    ]
+)
+
+def main():
+    usage = "usage: %prog --host host --username couchdb_username --password couchdb_password"
+    parser = ap.ArgumentParser()
+    parser.add_argument('--host', dest='host', help='The host name used for the couchdb server')
+    parser.add_argument('--username', dest='username', help='The username used to connect to the couchdb server')
+    parser.add_argument('--password', dest='password', help='The password used to connect to the couchdb server')
+    args = parser.parse_args()
+
+    if args.host == None:
+        raise RuntimeError('Unknown couchdb host, check configuration')
+    host = args.host
+
+    if args.username == None:
+        raise RuntimeError('Unknown couchdb username, check configuration')
+    username = args.username
+
+    if args.password == None:
+        raise RuntimeError('Unknown couchdb password, check configuration')
+    password = args.password
+
+    return host, username, password
+
+if __name__ == '__main__':
+    logging.info('Running python test to verify connection to couchdb container')
+    try:
+        host, username, password = main()
+
+        logging.info('    host: %s', host)
+        logging.info('    username: %s', username)
+        logging.info('    password: %s', password)
+
+        # build the url for the couchdb
+        couch_url = 'http://' + username + ':' + password + '@' + host + ':5984'
+
+        # try to connect
+        logging.info('Attempting to connect to couchdb at: %s', couch_url)
+        server = pycouchdb.Server(couch_url)
+
+        # print something out from the server to demonstrate connectivity
+        logging.info('    connected to the couch server which is running version: %s', server.info()['version'])
+    except Exception as e:
+        logging.error('Failed to test couchdb driver due to exception: %s', str(e))
+    finally:
+        logging.info('Test Complete!')

couchdb/test/test_driver.py

@@ -0,0 +1,127 @@
+'''/*******************************************************************************
+ * Copyright (c) 2022 Regents of the University of Colorado. All rights reserved.
+ *
+ * Contributors:
+ *     Michael Rabellino
+ *******************************************************************************/'''
+
+import os
+import sys
+import logging
+import time
+import subprocess
+import urllib, urllib.request
+
+# configure logging
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s [%(levelname)s] %(message)s",
+    handlers=[
+        logging.StreamHandler(sys.stdout)
+    ]
+)
+
+if '__main__' == __name__:
+    logging.info('-----------------------------------------------------------')
+    logging.info('Running couchdb test with arguments: ' + str(sys.argv))
+
+    host = sys.argv[1]
+    username = sys.argv[2]
+    password = sys.argv[3]
+
+    # static test configuration
+    network = 'couch-test-network'
+    server_image = 'idss.engine.commons.couchdb.server:test'
+    test_image = 'idss.engine.commons.couchdb.client:test'
+
+    # create a network for the couchdb server and test to run on
+    try:
+        subprocess.check_output('docker network create ' + network, shell=True).decode('utf-8')
+        logging.info('Created docker network: %s', network)
+    except Exception as e:
+        logging.warning('Container network already detected, continuing')
+
+    # build the couchdb server container
+    try:
+        logging.info('Building couchdb server image')
+        subprocess.check_output('cd ..; docker build -t ' + server_image + ' .', shell=True).decode('utf-8')
+    except Exception as e:
+        logging.warning('Unable to build server container due to exception: %s', str(e))
+
+    # build the test container
+    try:
+        logging.info('Building test image')
+        subprocess.check_output('docker build -t ' + test_image + ' .', shell=True).decode('utf-8')
+    except Exception as e:
+        logging.warning('Unable to build test container due to exception: %s', str(e))
+
+    # run the couchdb server container
+    try:
+        logging.info('Running the couchdb server container')
+        couchdb_docker_server_cmd = (
+            'docker run --rm -d --name ' + host + ' ' +
+            '--network ' + network + ' ' +
+            '-e COUCHDB_USER=' + username + ' ' +
+            '-e COUCHDB_PASSWORD=' + password + ' ' +
+            '-p 5984:5984 ' +
+            server_image
+        )
+        logging.info(couchdb_docker_server_cmd)
+        subprocess.check_output(couchdb_docker_server_cmd, shell=True).decode('utf-8')
+    except Exception as e:
+        logging.warning('Unable to start the couchdb server container due to exception: %s', str(e))
+
+    # allow for some time for the couchdb server to start
+    logging.info('Waiting for couchdb server to start...')
+    time.sleep(10)
+
+    # run the test container
+    try:
+        logging.info('Running the test container')
+        couchdb_docker_test_cmd = (
+            'docker run --network ' + network + ' ' +
+            test_image + ' ' +
+            '--host=' + host + ' ' +
+            '--username=' + username + ' ' +
+            '--password=' + password + ' '
+        )
+        logging.info(couchdb_docker_test_cmd)
+        os.system(couchdb_docker_test_cmd)
+    except Exception as e:
+        logging.warning('Unable to run the test container due to exception: %s', str(e))
+
+    # cleanup
+    logging.info('-----------')
+    logging.info('Cleaning up')
+
+    # stop test server container
+    try:
+        subprocess.check_output('docker stop ' + host, shell=True).decode('utf-8')
+        logging.info('Removed test couchdb container: %s', host)
+    except Exception as e:
+        logging.warning('Unable to stop server container, please stop manually: $ docker stop %s', host)
+    # remove test server image
+    try:
+        subprocess.check_output('docker image rm -f ' + server_image, shell=True).decode('utf-8')
+        logging.info('Removed docker image: %s', server_image)
+    except Exception as e:
+        logging.warning('Unable to remove server image, please delete manually: $ docker image rm %s', server_image)
+
+    # remove test container
+    try:
+        subprocess.check_output('docker image rm -f ' + test_image, shell=True).decode('utf-8')
+        logging.info('Removed docker image: %s', test_image)
+    except Exception as e:
+        logging.warning('Unable to remove test image, please delete manually: $ docker image rm %s', test_image)
+
+    # remove docker network
+    try:
+        subprocess.check_output('docker network rm ' + network, shell=True).decode('utf-8')
+        logging.info('Removed docker network: %s', network)
+    except Exception as e:
+        logging.warning('Unable to remove container network, please delete manually: $ docker network rm %s', network)
+
+
+    logging.info('Done')
+    logging.info('-----------------------------------------------------------')
+