Skip to content

UC3 R1.3

Jump to bottom
Fuqi Xu edited this page Apr 9, 2020 · 33 revisions

Recipe 1.3: How to deploy the service internally, using OLS as an examplar.

TODO update TOC

TODO remove all EFO examples, EFO is too big

TODO discuss a general ontology service or OLS in particular.

AI: feedback to Squad1. more ontology service

AI: change it to markdown

Main Objectives:

How to deploy public ontology services in local machines, take the EBI Ontology Lookup Service (OLS) as an example.

Ingredients:

Introduction

💡 Hint: This recipe is intended for users with, at least, basic command line and docker knowledge that allows them to download and install packages, as well as build and run docker images.

(TODO:is docker knowledge a must? I hope to use this recipe as a blackbox, dont learn to know docker?

No, not a must, following paragraph is perfect

? This recipe is intended for bioinformaticians or developers who wants to explore public ontologies and ontology services. The users are expected to be familiar with Bash and command line tools. Knowledge about building and running docker images allows users further customize the local OLS machine)

Ontology lookup service (OLS) is a repository for biomedical ontologies. It serves as a single point of access to query, browser and navigates different ontologies. OLS supports the Open Biological and Biomedical Ontology (OBO) Foundry guidelines and connects with other ontology services. It provides both web interface and API to search and browser ontologies.

Graphic view

💡 Hint: It seems here that 'from the EBI OLS' is the only one exporting to 'Load ontologies'. And maybe we can add a and/or between the options (not so sure about this last suggestion)

Step by step guide:

1. Installing dependencies

Software Version Description Installation
Git >=2.17.1 Get the versioned source file Official guide
Docker >=18.09.9 Deliver software in containers Official guide

3. Loading ontologies into OLS

Ontologies in both OBO and OWL formats can be loaded to OLS by adding ontology metadata to the configuration files ols-config.yaml, if the metadata follows the ols specification, or obo-config.yaml, if metadata follows the OBO Foundry specification. In either case, there are three mandatory fields:

💡 Hint: I think the mandatory fields, common to obo-config and ols-config, belong here, since they are not specific to ontologies you manually prepare, but for all.

Attribute Description
id Short unique id for the ontology
uri The ontology URL
ontology_purl URL to get the ontology from

To add an ontology from a PURL, specify the PURL must be specified in configuration file as follows: ontology_purl: <PURL>. For example:

## EFO

  - id: efo
    preferredPrefix: EFO
    title: Experimental Factor Ontology
    uri: http://www.ebi.ac.uk/efo/efo.owl
    description: "The Experimental Factor Ontology (EFO) provides a systematic description of many experimental variables available in EBI databases, and for projects such as the GWAS catalog"
    homepage: https://www.ebi.ac.uk/efo/
    ontology_purl: http://www.ebi.ac.uk/efo/efo.owl

Ontology metadata can be downloaded from either the OBO foundry, or the EBI OLS. Users can also provide their own ontology metadata.

3.1. Creating your own ontology metadata.

💡 Hint: not sure about the need of this section ("Creating your own ontology metadata") if we put the required fields and links to metadata specification in the previous paragraph, since it is obvious how to generate your metadata: just follow the specification.

I would organize it like:

From disk

From OLS EBI

From OBO Foundry [TODO]

OLS provides an example configuration file to load EFO ontology. The mandatory ontology metadata are listed in the table below:

Attribute Description
id Short unique id for the ontology
uri The ontology URL
ontology_purl URL to get the ontology from

To add an ontology from a PURL, specify the PURL must be specified in configuration file as follows: ontology_purl: <PURL>. For example:

## EFO

  - id: efo
    preferredPrefix: EFO
    title: Experimental Factor Ontology
    uri: http://www.ebi.ac.uk/efo/efo.owl
    description: "The Experimental Factor Ontology (EFO) provides a systematic description of many experimental variables available in EBI databases, and for projects such as the GWAS catalog"
    homepage: https://www.ebi.ac.uk/efo/
    ontology_purl: http://www.ebi.ac.uk/efo/efo.owl

2.2 Downloading configuration files from EBI OLS.

For ontologies included in the EBI OLS, the metadata can be downloaded directly using the OLS API using the ontology ID.

# EBI OLS endpoint for ontology metadata
# https://www.ebi.ac.uk/ols/api/ols-config\?ids\=<ontologies-ids-list>

# e.g. Download ontology metadata of EFO and AERO from the EBI OLS.
wget -O ols-config.yaml https://www.ebi.ac.uk/ols/api/ols-config\?ids\=aero,efo

2.3 Loading ontologies from disk

To add a local ontology to the service, first copy the ontology file inside the docker directory. In the configuration file ontology_purl: file:///opt/ols/<filename>.owl.

For example, to load the GO ontology from disk, the ontology file go.owl is placed inside the OLS directory and the following metadata is added to the configuration file ols-config.yaml:

ontologies:

## GO
  - id: go
    preferredPrefix: GO
    title: Gene Ontology
    uri: file:///opt/ols/go.owl
    description: "The logical structure describing the full complexity of the biology, comprised of the ‘classes’ for the many different kinds of biological functions, the pathways carrying out different biological programs, and the cellular locations where these occur; and equally important, the different types of specific relationships that indicate how each of these classes is related to other classes"
    homepage: http://geneontology.org/
    ontology_purl: file:///opt/ols/go.owl

2.4 Loading ontologies from OBO Foundry

3. Set up OLS in the local environment (TODO not sure if step2 or step 3 should come first)

I would say 2 and 3 come first, yes:

  1. Getting configuration files ready
  1. Setting up OLS
## Download OLS docker image
git clone https://github.com/EBISPOT/OLS-docker
cd OLS-docker

##  Start docker
sudo snap services
sudo snap start docker

## Build OLS docker image
sudo docker build -t ols .

## Run OLS docker image at port 8080
sudo docker run -d -p 80:80 -t ols

The local OLS service can be accessed at http://localhost:8080/index

Troubleshooting

  1. Building process runs out of memory with the following message: Exception in thread "cluster-2-127.0.0.1:27017" java.lang.OutOfMemoryError: Java heap space This issue can be solved by increasing the maximum heap size used by the Java Virtual Machine. Prior to building the image, inside the Dockerfile, change the parameter “JAVA_OPTS” from: ENV JAVA_OPTS "-Xmx1g" to: ENV JAVA_OPTS "-Xmx2g -Xms 2g"

  2. Image building exits with error: GC overhead limit exceeded

In order to fix, change ENV JAVA_OPTS "-Xmx2g -Xms1g for ENV JAVA_OPTS "-Xmx2g -Xms2g"

Summary

The local OLS provides :

  • API endpoints for retrieving, submitting, updating, and querying ontology data.
  • A user interface for searching and browsing ontology and ontology terms.
  • A automatic ontology update function for retrieving updates in ontologies it includes. (unfortunatelly NOT)

For a complete description of the service, please check the OLS documentations. (TODO)

Authors :

Name Institute ORCID Contributions
Fuqi Xu EMBL-EBI TODO TODO
Eva Martin BSC TODO TODO

License:

Clone this wiki locally