Data Sync Agent Service

Microservice responsible for data synchronization of FitBit platform with OCARIoT platform.

Main features:

• Fitbit access token management;
• Automatically sync Fitbit data;
• Publish sync data to a message channel;
• Fitbit access token revocation.

Prerequisites

• Node 8.0.0+
• MongoDB Server 3.0.0+
• Redis Server 5.0.0+
• RabbitMQ 3.7.0+

---

Set the environment variables

Application settings are defined by environment variables. To define the settings, make a copy of the .env.example file, naming for .env. After that, open and edit the settings as needed. The following environments variables are available:

VARIABLE	DESCRIPTION	DEFAULT
NODE_ENV	Defines the environment in which the application runs. You can set: test (in this environment, the database defined in MONGODB_URI_TEST is used and the logs are disabled for better visualization of the test output), development (in this environment, all log levels are enabled) and production (in this environment, only the warning and error logs are enabled).	development
PORT_HTTP	Port used to listen for HTTP requests. Any request received on this port is redirected to the HTTPS port.	5000
PORT_HTTPS	Port used to listen for HTTPS requests. Do not forget to provide the private key and the SSL/TLS certificate. See the topic generate certificates.	5001
SSL_KEY_PATH	SSL/TLS certificate private key.	.certs/server.key
SSL_CERT_PATH	SSL/TLS certificate.	.certs/server.crt
MONGODB_URI	Database connection URI used if the application is running in development or production environment. The URI specifications defined by MongoDB are accepted. For example: mongodb://user:pass@host:port/database?options	mongodb://127.0.0.1:27017 /ocariot-ds-agent
MONGODB_URI_TEST	Database connection URI used if the application is running in test environment. The URI specifications defined by MongoDB are accepted. For example: mongodb://user:pass@host:port/database?options	mongodb://127.0.0.1:27017 /ocariot-ds-agent-test
REDIS_URI	Redis database connection URI. Using for sync jobs.	redis://127.0.0.1:6379
RABBITMQ_URI	URI containing the parameters for connection to the message channel RabbitMQ. The URI specifications defined by RabbitMQ are accepted. For example: amqp://user:pass@host:port	amqp://guest:guest @127.0.0.1:5672
RABBITMQ_CA_PATH	RabbitMQ CA file location. Must always be provided when using amqps protocol.	.certs/rabbitmqca.crt
FITBIT_CLIENT_ID	Client Id for Fitbit Application resposible to manage user data.	CIENT_ID_HERE
FITBIT_CLIENT_SECRET	Client Secret for Fitbit Application resposible to manage user data.	CIENT_SECRET_HERE
EXPRESSION_AUTO_SYNC	Defines how often the application will automatically sync user data in the background according to the crontab expression.	0 0 * * 0

Generate Certificates

For development and testing environments the easiest and fastest way is to generate your own self-signed certificates. These certificates can be used to encrypt data as well as certificates signed by a CA, but users will receive a warning that the certificate is not trusted for their computer or browser. Therefore, self-signed certificates should only be used in non-production environments, that is, development and testing environments. To do this, run the create-self-signed-certs.sh script in the root of the repository.

chmod +x ./create-self-signed-certs.sh
./create-self-signed-certs.sh

The following files will be created: ca.crt, server.crt and server.key.

In production environments its highly recommended to always use valid certificates and provided by a certificate authority (CA). A good option is Let's Encrypt which is a CA that provides free certificates. The service is provided by the Internet Security Research Group (ISRG). The process to obtain the certificate is extremely simple, as it is only required to provide a valid domain and prove control over it. With Let's Encrypt, you do this by using software that uses the ACME protocol, which typically runs on your host. If you prefer, you can use the service provided by the SSL For Free website and follow the walkthrough. The service is free because the certificates are provided by Let's Encrypt, and it makes the process of obtaining the certificates less painful.

Installation and Execution

1. Install dependencies

npm install    

2. Build

Build the project. The build artifacts will be stored in the dist/ directory.

npm run build    

3. Run Server

npm start

Build the project and initialize the microservice. Useful for production/deployment.

npm run build && npm start

Running the tests

All tests

Run unit testing, integration and coverage by Mocha and Instanbul.

npm test

Unit test

npm run test:unit

Integration test

npm run test:integration

Coverage test

npm run test:cov

Navigate to the coverage directory and open the index.html file in the browser to see the result. Some statistics are also displayed in the terminal.

Generating code documentation

npm run build:doc

The html documentation will be generated in the /docs directory by typedoc.

Using Docker

In the Docker Hub, you can find the image of the most recent version of this repository. With this image it is easy to create your own containers.

docker run ocariot/ds-agent

This command will download the latest image and create a container with the default settings.

You can also create the container by passing the settings that are desired by the environment variables. The supported settings are the same as those defined in "Set the environment variables". See the following example:

docker run -d --rm \
  --net=host \
  -e NODE_ENV=development \
  -e PORT_HTTP=5000 \
  -e PORT_HTTPS=5001 \
  -v $(pwd)/.certs:/etc \
  -e SSL_KEY_PATH=/etc/server.key \
  -e SSL_CERT_PATH=/etc/server.crt \
  -e RABBITMQ_URI="amqp://guest:guest@127.0.0.1:5672" \
  -e MONGODB_URI="mongodb://127.0.0.1:27017/ocariot-ds-agent" \
  -e REDIS_URI="redis://127.0.0.1:6379" \
  -e FITBIT_CLIENT_ID="YOUR_FITBIT_CLIENT_ID" \
  -e FITBIT_CLIENT_SECRET="YOUR_FITBIT_CLIENT_SECRET" \
  -e EXPRESSION_AUTO_SYNC="0 0 * * 0" \
  --name ocariot-ds-agent \
  ocariot/ocariot-ds-agent

If the MongoDB or RabbitMQ instance is in the host local, add the --net=host statement when creating the container, this will cause the docker container to communicate with its local host.

docker run -d --rm \
  --net=host \
  ...

To generate your own docker image, run the following command:

docker build -t image_name:tag .

---

Fitbit Client Errors Published in Message Bus

This microservice has a particular way of managing errors from the Fitbit Client, which is responsible for communicating with the Fitbit Server to interact with platform user data. When an error is generated by the client, it is published to the message channel with the following structure:

{
    "child_id": "5a62be07de34500146d9c544",
    "error": {
        "code": 1581, 
        "message": "The message is here", 
        "description": "The description is here"   
    }
}

The code parameter is an internal implementation of the API, which serves to map the generated errors. The following table illustrates the mapping of these errors as implemented in the API:

code	reference	message	description
1011	Expired Access Token	Access token expired.	The access token has been expired and needs to be refreshed.
1012	Invalid Access Token	Access token invalid.	The access token is invalid. Please make a new Fitbit Auth Data request and try again.
1021	Invalid Refresh Token	Refresh token invalid.	The refresh token is invalid. Please make a new Fitbit Auth Data request and try again.
1401	Invalid Client Credentials	Invalid Fitbit Client data.	The Fitbit Client credentials are invalid. The operation cannot be performed.
1429	Too Many Requests	Data request limit for user has expired.	Please wait a minimum of one hour and try make the operation again.
1500	Generic Error	The message from error.	The description from error.