Skip to content

resource-watch/rw_metadata

This branch is 117 commits ahead of, 63 commits behind Skydipper/Metadata:master.

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ed46ff4 · Aug 30, 2023
Aug 30, 2023
Feb 24, 2020
Aug 30, 2023
Jul 18, 2018
Oct 17, 2019
Mar 28, 2017
Feb 18, 2020
Feb 18, 2020
Aug 30, 2023
Aug 30, 2023
Mar 10, 2022
Aug 30, 2023
Feb 24, 2021
Mar 7, 2018
Feb 22, 2022
Feb 22, 2022
Aug 30, 2023
Aug 30, 2023
Feb 18, 2020
Oct 18, 2019
Aug 30, 2023
Aug 30, 2023

Repository files navigation

Resource Watch Metadata Microservice

Build Status Test Coverage

Dependencies

You will need Control Tower up and running - either natively or with Docker. Refer to the project's README for information on how to set it up.

The Metadata microservice is built using Node.js, and can be executed either natively or using Docker, each of which has its own set of requirements.

Native execution requires:

Execution using Docker requires:

Dependencies on other Microservices:

Getting started

Start by cloning the repository from github to your execution environment

git clone https://github.com/resource-watch/rw_metadata.git && cd rw_metadata

After that, follow one of the instructions below:

Using native execution

1 - Set up your environment variables. See dev.env.sample for a list of variables you should set, which are described in detail in this section of the documentation. Native execution will NOT load the dev.env file content, so you need to use another way to define those values

2 - Install node dependencies using yarn:

yarn

3 - Start the application server:

yarn start

The endpoints provided by this microservice should now be available through Control Tower's URL.

Using Docker

1 - Create and complete your dev.env file with your configuration. The meaning of the variables is available in this section. You can find an example dev.env.sample file in the project root.

2 - Execute the following command to run Control tower:

./metadata.sh develop

The endpoints provided by this microservice should now be available through Control Tower's URL.

Testing

There are two ways to run the included tests:

Using native execution

Follow the instruction above for setting up the runtime environment for native execution, then run:

yarn test

Using Docker

Follow the instruction above for setting up the runtime environment for Docker execution, then run:

./metadata.sh test

Configuration

Environment variables

  • PORT => TCP port in which the service will run
  • NODE_PATH => relative path to the source code. Should be app/src
  • MICROSERVICE_TOKEN =>
  • S3_ACCESS_KEY_ID => AWS S3 key id
  • S3_SECRET_ACCESS_KEY => AWS S3 access key
  • STAMPERY_TOKEN => Stampery token
  • MONGO_PORT_27017_TCP_ADDR => IP/Address of the MongoDB server

This repository implements the metadata services that are available in the Resource Watch API.

If you are looking for the API Doc (Info and Usage) please go to the next link: View the documentation for this API (not yet)

Quick Overview

Metadata Entity

dataset: <String>, required
application: <String>, required
resource: {
    id: <String>, required
    type: <String>, required, [dataset, widget, layer]
},
language: <String>, required
name: <String>
description: <String>
source: <String>
citation: <String>
license: <String>
units: <Object>
info: <Object>
fields: <Object>
createdAt: <Date>
updatedAt: <Date>
status: <String>, [published, unpublished]

Dataset Metadata

GET: /dataset/:dataset/metadata
POST: /dataset/:dataset/metadata
PATCH: /dataset/:dataset/metadata
DELETE: /dataset/:dataset/metadata

Widget Metadata

GET: /dataset/:dataset/widget/:widget/metadata
POST: /dataset/:dataset/widget/:widget/metadata
PATCH: /dataset/:dataset/widget/:widget/metadata
DELETE: /dataset/:dataset/widget/:widget/metadata

Layer Metadata

GET: /dataset/:dataset/layer/:layer/metadata
POST: /dataset/:dataset/layer/:layer/metadata
PATCH: /dataset/:dataset/layer/:layer/metadata
DELETE: /dataset/:dataset/layer/:layer/metadata

GetAll Metadata

GET: /metadata

FindByIds Metadata

POST: /dataset/metadata/find-by-ids
POST: /dataset/:dataset/widget/metadata/find-by-ids
POST: /dataset/:dataset/layer/metadata/find-by-ids

POST, PATCH, DELETE

"application" and "language" attributes are required and it is mandatory to include them in the payload (when doing CRUD requests). This doesn't apply to the FindByIds Endpoints

GET Queryparam Filters:

application: gfw, gfw-climate, prep, rw, forest-atlas (select one or some of them)
language: select between available languages (select one or some of them)
limit: desired number

Custom param for /metadata endpoint

type: [dataset, widget, layer]

CRUD Examples

Getting

GET: /dataset/111123/metadata
GET: /dataset/111123/widget/134599/metadata
GET: /dataset/111123/layer/134599/metadata
GET: /dataset/111123/metadata?application=rw&language=es,en&limit=20
GET: /dataset/111123/widget/134599/metadata?application=rw,gfw&language=en
GET: /dataset/111123/layer/134599/metadata?language=en

Creating

POST: /dataset/111123/metadata -> payload: {"application": "rw", "language": "es"}
POST: /dataset/111123/widget/134599/metadata -> payload: {"application": "rw", "language": "es"}
POST: /dataset/111123/layer/134599/metadata -> payload: {"application": "rw", "language": "es"}
POST: /dataset/111123/layer/134599/metadata -> payload: {"application": "rw", "language": "es", "name": "M1", "info": {"a": "a", "b": "b"}}

Updating (partial)

PATCH: /dataset/111123/metadata -> payload: {"application": "rw", "language": "es", "name": "M0", "info": {"a": "A", "b": "B"}}
PATCH: /dataset/111123/widget/134599/metadata -> payload: {"application": "rw", "language": "es", "name": "M1", "info": {"c": "C", "d": "D"}}
PATCH: /dataset/111123/layer/134599/metadata -> payload: {"application": "rw", "language": "es", "name": "M2", "info": {"e": "E", "f": "F"}}
PATCH: /dataset/111123/layer/134599/metadata -> payload: {"application": "rw", "language": "es", "name": "M3", "info": {"g": "G", "h": "H"}}

Deleting

DELETE: /dataset/111123/metadata?application=rw&language=en
DELETE: /dataset/111123/widget/134599/metadata?application=rw&language=es
DELETE: /dataset/111123/layer/134599/metadata?application=gfw&language=en

Getting All

GET: /metadata
GET: /metadata?type=dataset
GET: /metadata?type=widget
GET: /metadata?application=rw&language=es,en&limit=20
GET: /metadata?application=rw,gfw&language=en&type=dataset
GET: /metadata?language=en

Finding By Ids

"ids" property is required in the payload, in other case the endpoint responds a 400 HTTP ERROR (Bad Request) This property can be an Array or a String (comma-separated) payload -> {"ids": ["112313", "111123"]} payload -> {"ids": "112313, 111123"}

POST: /dataset/metadata/find-by-ids -> payload: {"ids": ["112313", "111123"]}
POST: /dataset/metadata/find-by-ids?application=rw&language=es,en&limit=20 -> payload: {"ids": "112313, 111123"}
POST: /dataset/111123/widget/metadata/find-by-ids, -> payload: {"ids": "112313, 111123"}
POST: /dataset/111123/widget/metadata/find-by-ids?application=rw,gfw&language=es&limit=200 -> payload: {"ids": ["112313", "111123"]}

Ir order to contribute to this repo:

  1. Getting Started
  2. Deployment

Getting Started

OS X

First, make sure that you have the API gateway running locally.

We're using Docker which, luckily for you, means that getting the application running locally should be fairly painless. First, make sure that you have Docker Compose installed on your machine.

If you've not used Docker before, you may need to set up some defaults:

docker-machine create --driver virtualbox default
docker-machine start default
eval $(docker-machine env default)

You also need to configure an alias for your local machine:

Get your local IP:

ifconfig

Modify the /etc/hosts config file adding a new rule: mymachine

vim /etc/hosts

Now we're ready to actually get the application running:

git clone https://github.com/resource-watch/rw_metadata
cd rw_metadata
./metadata.sh develop

You can now access the microservice through the API gateway.

Deployment

In progress...

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • JavaScript 98.8%
  • Other 1.2%