Verso - Node.js LoopbackAPI-based storage middleware

Introduction

Verso is a boilerplate Loopback API REST application for use with the Library of Congress' BIBFRAME editor and its related profile editor. It offers endpoints for a few different kinds of storage:

1. /verso/api/bfs: Memory-based storage of BIBFRAME graphs for use with the BIBFRAME editor.
2. /verso/api/configs: Memory-based storage of JSON configuration objects for use with the profile editor.
3. /verso/api/Users: Memory-based storage of LoopBack User records. See Authentication and authorization below.

You can explore the API using the Loopback API explorer at /verso/explorer. The models themselves are in the common/models directory.

Verso can be configured to use MongoDB or a simple file to persist the memory-based storage. See Configuring storage below.

Saving a record from the Profile Editor results in corruption of the record when using a MongoDB storage backend. This makes MongoDB storage unusable with the Profile Editor at present, except as read-only.

Getting started

Verso is a Node.js application designed to be built and run with npm. It has been tested with node v.10.11.0 and npm v.6.4.1

Verso is compatible with PM2, a package manager for nodejs.

Installation

Create a directory to hold your application, and navigate to that directory.

If using PM2:

sudo npm install pm2 -g
git clone https://github.com/lcnetdev/verso
cd verso
npm install
npm start

For development:

git clone https://github.com/lcnetdev/verso
cd verso
npm install
npm run dev

Configuring storage

Verso's db datasource can be configured using the following environment variables:

DB_STORAGE: If set to file, the data for the db datasource will be stored and read from a JSON file generated by the memory connector. If set to mongodb, the data will be stored and read from a MongoDB instance. If this is not set, data will be stored in memory only, and not persisted between runs of the application.

DB_FILE: The path to the file for storage if DB_STORAGE=file. A file with sample BIBFRAME data (for the /verso/api/bfs endpoint) is included in this repository as bfpilot.json.

DB_URL: The MongoDB connection string if DB_STORAGE=mongodb. Format is mongodb://[user:password@]host[:port]/db.

AUTH: Default false, if set to true authorization is enabled.

You can configure these variables using a .env file in the root directory of the application. For example:

# Environment variables
AUTH=true
DB_STORAGE=mongodb
DB_URL=mongodb://localhost:27017/bf

The variables have been included in lcnetdev/verso/blob/master/ecosystem.config.js, the default environment is file. To start mongo, use npm run mongo.

Running Verso

TL;DR: npm start.

The first time you start Verso with authorization enabled, you will be prompted to create an admin user. This user can be used to create and manage other users (see Authentication and authorization below for more details). See above for creating an .env file. For example:

$ cat .env
AUTH=true
DB_STORAGE=mongodb
DB_URL=mongodb://127.0.0.1:27017/bf

$ npm run dev

> verso@1.0.0 start /opt/bibliomata/verso
> node .

/opt/bibliomata/verso/data/profiles not found!
Trying data/profiles
/opt/bibliomata/verso/data/vocabularies not found!
Trying data/vocabularies
Created 33 profile(s)
Created 1 vocabularies
Web server listening at: http://localhost:3001
Browse your REST API at http://localhost:3001/verso/explorer

No admin role present, assuming first run.
? Email for admin user? admin@example.org
? Password for admin user? [hidden]
? Confirm password for admin user? [hidden]
Admin user created.

The application runs on port 3001 by default.

Production deployment

Verso can be run using the PM2 Process Manager for a more production-appropriate deployment.

Important Caveat: If you do this, you must first configure storage and run using npm as above to create and persist the admin user, as pm2 will not allow for interactively creating the admin account.

To use pm2 to deploy Verso:

sudo pm2 start ecosystem.config.js --update-env

Note: this is the same script which is run using npm start

To check on the status of pm2 applications:

sudo pm2 status

You should get a status screen (with a short uptime).

To save your pm2 runtime configuration:

sudo pm2 save 

This will save a copy of the PM2 config in /root/.pm2/dump.pm2

To restart the cluster with the dump file:

sudo pm2 resurrect

Sample data

Sample configuration data for the profile editor is in the data directory. If there is no data in the data store, the sample data will be loaded on server start.

Authentication and authorization

Verso uses the built-in LoopBack mechanisms (Users, Roles, RoleMappings, and ACLs) to protect its endpoints. The User model has been extended to allow an administrative user to maintain user accounts and assign users to roles using the LoopBack web services API. Users and Roles are stored using the db datasource, so if you are using a MongoDB backend, the users will be persisted between runs of the application (see Configuring storage above).

For more information on the LoopBack security model, see the LoopBack documentation.

The following roles and ACLs are set up by default:

• admin role: users with this role have full access to all remote endpoints.
• profile_editor role: users with this role have full access to the configs endpoint.
• All authenticated users have full access to the bfs endpoint and read access to the configs endpoint.
• All access is denied for unauthenticated users.

On application startup, a bootscript runs to set up the default roles and to extend the User model. If there is no admin user present, the bootscript will prompt to create one (see Running Verso above).

On login to Verso, a custom hook on the User login remote method creates two cookies:

• access_token: a signed, HTTP-only session cookie that contains the LoopBack authentication token. Custom middleware looks for this cookie with every request, so that the Authorization HTTP header is not required. This allows applications like the BIBFRAME Editor and the Profile Editor to use Verso as an authentication layer without significant code changes.

• current_user: a session cookie that contains user identity information, as a JSON object. For example:

{
  "id": 1234,
  "username": "superuser",
  "roles": [
    "admin"
  ]
}

This cookie could be used by applications to present identity information on the page.

User management

You can create and manage users using the Users web services API. For example, to create a user:

1. Log into Verso as an administrative user

$ curl -w '\n' -D - -X POST -H "Content-Type: application/json" \
-d '{"username":"admin","password":"topsecret"}' \
http://localhost:3001/verso/api/Users/login

HTTP/1.1 200 OK
X-XSS-Protection: 1; mode=block
X-Frame-Options: DENY
X-Download-Options: noopen
X-Content-Type-Options: nosniff
Content-Type: application/json; charset=utf-8
Content-Length: 135
ETag: W/"87-nLgI/naEchBSDSJrT1iiVlBu4/Q"
Vary: Accept-Encoding
Date: Thu, 13 Sep 2018 03:40:14 GMT
Connection: keep-alive

{"id":"nCZX5oXWzLWY6UzgV0vqWzlSgxBBho7AKFn9cl4bCXZURPpMtwgXEmuW30zcFDCe","ttl":1209600,"created":"2018-09-13T03:40:14.810Z","userId":1}

The authorization token for use in the Authorization header of subsequent requests is returned in the id property of the response.

2. Create a user. Note that email is a required property.

$ curl -w '\n' -D - -X POST -H "Content-Type: application/json" \
-H "Authorization: nCZX5oXWzLWY6UzgV0vqWzlSgxBBho7AKFn9cl4bCXZURPpMtwgXEmuW30zcFDCe" \
-d '{"username":"new-user","password":"topsecret","email":"new-user@example.org"}' \
http://localhost:3001/verso/api/Users

HTTP/1.1 200 OK
X-XSS-Protection: 1; mode=block
X-Frame-Options: DENY
X-Download-Options: noopen
X-Content-Type-Options: nosniff
Content-Type: application/json; charset=utf-8
Content-Length: 61
ETag: W/"3d-X2TsLQGE0+X8tFjXyWQth8ndnro"
Vary: Accept-Encoding
Date: Thu, 13 Sep 2018 03:44:59 GMT
Connection: keep-alive

{"username":"new-user","email":"new-user@example.org","id":2}

This will create a user with full access to the bfs endpoint, and read access to the configs endpoint.

3. Add the user by ID to the profile_editor role.

$ curl -w '\n' -D - -X PUT \
-H "Authorization: nCZX5oXWzLWY6UzgV0vqWzlSgxBBho7AKFn9cl4bCXZURPpMtwgXEmuW30zcFDCe" \
http://localhost:3001/verso/api/Users/2/roles/profile_editor

HTTP/1.1 204 No Content
X-XSS-Protection: 1; mode=block
X-Frame-Options: DENY
X-Download-Options: noopen
X-Content-Type-Options: nosniff
Content-Type: application/json; charset=utf-8
Vary: Accept-Encoding
Date: Thu, 13 Sep 2018 03:49:37 GMT
Connection: keep-alive

This will grant the user full access to the configs endpoint.

You can explore the User API further using the Loopback API explorer at /verso/explorer.

LDAP setup

You will enter your LDAP settings in the server/providers.json file. Please see some examples at https://github.com/strongloop/loopback-example-passport/blob/master/providers.json.template and https://developer.ibm.com/recipes/tutorials/configuring-tokenbased-ldap-authentication-with-loopback-io-2/

There is a providers.json file included with this distribution but the account at azure.com is temporary and is likely not to connect.

There is also an example of a secure LDAP (ldaps) providers file called providers.ldaps.example.json. Set the tlsOptions.certFile to the appropriate filename. The cert file should be located in the same directory as providers.json. NOTE: The developer has not been able to test this functionallity becouse of a ERR_TLS_CERT_ALTNAME_INVALID error.

Shortfalls of the loopback-component-passport module:

• It stores the access_token in a regular (not HTTP Only) cookie.
• It doesn't create the current_user cookie. This is created by a script in the login.html file.
• Initial login will not create a roll for the new user, the administrator must manually map a roll after initial login.

Docker

The file Dockerfile allows Verso to built as a Docker Image.

Build verso: docker build -t ld4p/verso .

Running in the foreground: docker run -p 3000:3000 ld4p/verso

Test users

If the environment variable DEV_USER_PW is set, the users admin, profile_editor, and user will be created with the password set to the value in the variable.