Skip to content
/ comimo Public

Web application repository for illegal gold mining monitoring application.

License

Notifications You must be signed in to change notification settings

sig-gis/comimo

Repository files navigation

Colombian Mining Monitoring (CoMiMo)

Installation Requirements

Required

Application Configuration

On startup, the COMIMO application reads a file called config.edn in the top-level directory of this repository, which contains various user-configurable system parameters. A sample file called config.example.edn is provided with this repository to get you started. To begin, simply copy it to config.edn. Then edit it according to the following sections.

Please note that sensitive information, such as access tokens and passwords, has been stripped from config.example.edn. You can see which fields will need to be filled in by looking for ; REVIEW comments.

You can find these configuration values while using the SIG VPN & credentials by visiting gemini://dev-docs.sig-gis.com/operations/servers/prod/kite, or by navigating to gemini://dev-docs.sig-gis.com -> Operations -> Application Servers -> kite (COMIMO - SIG).

Once you have finished setting up dependencies and building config.edn, you can validate that it passes the config spec set in Triangulum by navigating to the top level directory of COMIMO and running the following command:

clojure -M:config validate -f ./config.edn

Postgresql Database Setup

Postgresql needs to be installed on the machine that will be hosting this website. This installation task is system-specific and is beyond the scope of this README, so please follow the instructions for your operating system and Postgresql version. However, please ensure that the database server’s superuser account is named “postgres” and that you know its database connection password before proceeding.

Once the Postgresql database server is running on your machine, you should navigate to the toplevel directory (i.e., the directory containing this README) and run the database build command as follows:

clojure -M:build-db build-all [--dev-data]

This will begin by creating a new database and role called comimo and then add the postgis and pgcrypto extensions to it. Next, the script will populate the database with the schemas, tables, and functions that are necessary for storing and processing CoMiMo’s data. Finally, it will load some default data into these tables that is necessary for the website to function properly.

You can optionally load dev data with --dev-data. This will initialize the DB with 3 users, an imagery source, and a project. If you wish to use a live copy of the CoMiMo database instead of the dev data, please see the documentation of the build-db command in the Triangulum README.

Performance Settings

If you want to improve the performance of your Postgresql server, one way is to visit the pgtune website and input your system settings. This will generate a set of configuration options that you can add to the postgresql.conf file in your system-specific Postgresql data directory.

If you would like to keep these settings separate for your own reference, you can add them to a file called performance.conf that you should place in your Postgresql data directory. You can then import these settings into postgresql.conf by adding this line to the end of that file:

include = 'performance.conf'

Note that the Postgresql server will need to be restarted for these changes to take effect.

Environment Variable Setup

In order for your command-line shell to find the programs installed above, you will need to update your environment variables. Under GNU/Linux, *BSD, or MacOS X, you can set these through your shell’s .*rc or .*profile configuration files in your $HOME directory. In a Windows environment, you will need to set either the USER (for the current user only) or SYSTEM (for any user) environment variables under your system settings menu.

Step 1: Add JAVA_HOME

On a Unix machine using bash (e.g., GNU/Linux or MacOS X):

export JAVA_HOME=<jdk-installation-directory>

On a Unix machine using csh or tcsh (e.g., *BSD):

setenv JAVA_HOME <jdk-installation-directory>

On a Windows machine:

JAVA_HOME = C:\Program Files\Java\jdk-X.X.X

Replace X.X.X with the JDK version installed on your computer.

Step 2: Update PATH

On a Unix machine using bash (e.g., GNU/Linux or MacOS X):

export PATH=$PATH:$JAVA_HOME/bin

On a Unix machine using csh or tcsh (e.g., *BSD):

setenv PATH $PATH\:$JAVA_HOME/bin

On a Windows machine, add these entries to the PATH environment variable:

%JAVA_HOME%\bin
C:\Program Files\PostgreSQL\X\lib
C:\Program Files\PostgreSQL\X\bin
C:\Program Files\7-Zip

Replace X with the PostgreSQL version installed on your computer.

Configuration

Copy config.default.edn to config.edn and update any settings that you wish. The base set of defaults should be sufficient to run CoMiMo in a development setting.

Python dependencies

Use pip to install dependencies. The Python / Clojure interop does not support virtual environments.

pip install earthengine-api --upgrade

Development Web Server

Google Earth Engine authentication

You will need a Google Earth Engine account to use the GeoDash. To apply for a GEE account, visit https://signup.earthengine.google.com/#!/. Once you receive an account and installed the gcloud CLI, you can store local authentication information with:

earthengine authenticate

This command will open a window in your browser containing a key that you should copy and paste back into your terminal. You do not need to update the :gee section of config.edn if you use this method.

Create accounts

To be able to create accounts without having to send email for confirmation, set :auto-validate? of :mail to true in config.edn.

Compile and run

To compile and run the web application, navigate to the top-level project directory and run:

npm install
npm run bundle-dev # outputs to dist/public and starts Vite dev server on http://localhost:5173

Now, in a separate terminal window (because Vite needs to be running in the background), run the following commands:

clojure -M:server start

The website will then be available at http://localhost:8080 unless a port is specified. These can also be configured using the :server section in your config.edn file. When using the clojure -M:server start command (which is what npm run server-dev is an alias for) an http port can be specified with -p and an https port can be specified with -P. In dev-mode, server-side exceptions will be displayed in the browser and JavaScript source files will be reloaded whenever you refresh the page.

Going forward you will usually only need to run npm run bundle-dev and clojure -M:server start (still in separate terminals) to get your dev environment set up. Watch for updates to SQL files or package.json and run npm run build-db-functions or npm install respectively.

Checking for Reflection Warnings

From the top-level project directory run:

clojure -M:check-reflection

This will emit warnings when reflection is needed to resolve Java method calls or field accesses. To resolve any that appear, add type hints to your code. Resolving all reflection warnings can improve system performance.

Production Web Server

Sessions

It is very important to change the default :session-key in config.edn. This key is used to encrypt user session data and should be unique to each deployment. The key must be exactly 16 characters long.

Email Server

To set up the email server for system emails, open the config.edn file in the root directory of the application. Edit the default EDN object containing server details to the file, replacing the values with your own.

Google Earth Engine service account

For production it is recommended that you use a service account with a key file. You can obtain your key file by logging into your service account, navigating to the account menu, and clicking “Create key > JSON”. Then, download that JSON key file and place it in the root directory of CoMiMo. Set the email for your service account and key path in the :gee section of config.edn.

:gee {:ee-account  "example@gmail.com"
      :ee-key-path "comimo-gee-key.json"}

Google Analytics

To hook up Google Analytics to CoMiMo, simply add your Analtics ID as a string to the :ga-id field in your config.edn file.

Launching the Web Server

To compile and run the web application, navigate to the top-level project directory and run:

npm install
npm run bundle-prod # outputs to dist/public
clojure -M:build-db functions -d comimo
clojure -M:server start -m [dev|prod] [-p 8080] [-P 8443] [-r]

The website will then be available at http://localhost:8080 unless a port is specified. These can also be configured using the :server section in your config.edn file. An http port can be specified with -p and an https port can be specified with -P. In dev-mode, server-side exceptions will be displayed in the browser and Clojure source files will be reloaded whenever you refresh the page. These features are disabled in prod mode. If -m is unspecified, it will default to prod mode.

Running the Web Server as a System Service

View the Triangulum Systemd page for further instructions on enabling the app as a system service.

Maintaining Daily Logs

By default the server will log to standard out. If you would like to have the system log to YYYY-DD-MM.log, use the “-o path” option to specify an output path. You can either specify a path relative to the top-level directory of this repository or an absolute path on your filesystem. The logger will keep the 10 most recent logs.

Using the Announcement Banner

On each page load, Clojure will read the value of announcement.txt. If text is found, the value will be inserted into a HTML element that displays as a red banner at the top of the page. To add a new announcement, edit announcement.txt and add a new message. To remove the announcement, edit announcement.txt and remove all text.

License and Distribution

Copyright © 2016-2022 Spatial Informatics Group, LLC.

CoMiMo is distributed by Spatial Informatics Group, LLC. under the terms of the Eclipse Public License version 2.0 (EPLv2). See LICENSE.txt in this directory for more information.