Skip to content

Running ukbREST without Docker

Jump to bottom
meliao edited this page May 22, 2020 · 7 revisions

Table of Contents

This page shows how to run ukbREST without Docker. It could be useful for advanced use cases.

Using Flask

If for some reason you don't want to use the ukbREST Docker image, then you can run the code directly in your Python environment. Here we show how to run ukbREST using Flask directly, which is intended for testing purposes. If you want to run this in a production environment, then it could be better to use Gunicorn (see below).

Assuming you already followed the installation instructions, you can simply run:

$ python -m ukbrest.app --db-uri postgresql://test:test@localhost:5432/ukb

You can run the app.py script only with the --help parameter to see the options you can specify. Other options here are --users-file (the path of a YAML file with a list of users, thus this enables HTTP Basic Auth), and --ssl-mode (which enables SSL with Flask in adhoc mode). We'll show examples of these below with Gunicorn, and later using the ukbREST Docker image when loading real UK Biobank data.

Using Gunicorn

Gunicorn is a python WSGI HTTP Server ready for production purposes. In this case, you specify HTTP related parameters using Gunicorn arguments, and ukbREST options using environmental variables.

Assuming you are located in the repository folder and you activated your ukbrest python environment (conda activate ukbrest), you can run these commands:

$ export UKBREST_DB_URI="postgresql://test:test@localhost:5432/ukb"
$ gunicorn \
  --log-file=- \
  -k eventlet \
  --workers=4 \
  --timeout 10000 \
  -b 0.0.0.0:5000 \
  ukbrest.wsgi:app
[2018-07-23 17:07:13 -0500] [51182] [INFO] Starting gunicorn 19.7.1
[2018-07-23 17:07:13 -0500] [51182] [INFO] Listening at: http://0.0.0.0:5000 (51182)
[2018-07-23 17:07:13 -0500] [51182] [INFO] Using worker: eventlet
[2018-07-23 17:07:13 -0500] [51185] [INFO] Booting worker with pid: 51185
[...]

If you compare the environment variable UKBREST_DB_URI specified here with the one used when running the ukbREST docker image, you'll notice just one difference: the host here is localhost, while with the ukbREST container must be pg (the name of the PostgreSQL container).

Two common features you would probably like in a real scenario is authentication (to specify a list of users authorized to access the data) and SSL encryption. At the moment, ukbREST has support for both features, but regarding the former only HTTP Basic Authentication is supported. This means that you should activate also SSL when using authentication since with HTTP Basic Auth send the user and password in cleartext.

HTTP Basic authentication

To activate HTTP Basic authentication with Gunicorn you have to use environmental variables. This is the same if using the ukbREST docker image, where you set the variables using the -e docker option.

You first need a users file. Let's assume you havea file stored in /tmp/users with this content:

user1: password1
user2: password2

Then you run ukbREST with this command:

$ export UKBREST_DB_URI="postgresql://test:test@localhost:5432/ukb"
$ export UKBREST_HTTP_USERS_FILE_PATH="/tmp/users"
$ gunicorn \
  --log-file=- \
  -k eventlet \
  --workers=4 \
  --timeout 10000 \
  -b 0.0.0.0:5000 \
  ukbrest.wsgi:app

SSL support

Generate certificate and private key (it will ask a few questions):

$ openssl req -x509 -newkey rsa:4096 -nodes -out cert.pem -keyout key.pem -days 365
Generating a 4096 bit RSA private key
.......................................................................++
.....................................................................................++
writing new private key to 'key.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:Illinois
Locality Name (eg, city) []:Chicago
Organization Name (eg, company) [Internet Widgits Pty Ltd]:The University of Chicago
Organizational Unit Name (eg, section) []:ImLab - Section of Genetic Medicine
Common Name (e.g. server FQDN or YOUR name) []:127.0.0.1
Email Address []:

This certificate will be valid for 365 days; check out documentation of openssl for more options.

Pay special attention to Common Name, where I used the IP address of the host with the ukbREST server; clients will use this address to connect.

Then you can distribute the certificate file cert.pem to your team members.

You run Gunicorn like this to activate SSL and authentication at the same time:

$ export UKBREST_DB_URI="postgresql://test:test@localhost:5432/ukb"
$ export UKBREST_HTTP_USERS_FILE_PATH="/tmp/users"
$ gunicorn \
  --log-file=- \
  -k eventlet \
  --workers=4 \
  --timeout 10000 \
  -b 0.0.0.0:5000 \
  --certfile cert.pem \
  --keyfile key.pem \
  ukbrest.wsgi:app

Queries using curl

Once you activated authentication and SSL, you can use curl from the command line to make a query like the one below.

$ curl -G \
  -u user1:password1 \
  --cacert cert.pem \
  -HAccept:text/csv \
  "https://127.0.0.1:5000/ukbrest/api/v1.0/phenotype" \
  --data-urlencode "columns=c101_0_0 as variable_name"
Clone this wiki locally