Initial commit

Author: callmekatootie

.gitignore

@@ -0,0 +1,6 @@
+node_modules
+.nyc_output
+coverage
+
+.DS_Store
+.env

README.md

@@ -0,0 +1,96 @@
+# Identity - Data Processor
+
+## Dependencies
+
+- Nodejs(v12+)
+- Kafka
+
+## Configuration
+
+Configuration for the identity processor is at `config/default.js`.
+The following parameters can be set in config files or in env variables:
+
+- LOG_LEVEL: the log level; default value: 'debug'
+- KAFKA_URL: comma separated Kafka hosts; default value: 'localhost:9092'
+- KAFKA_CLIENT_CERT: Kafka connection certificate, optional; default value is undefined;
+    if not provided, then SSL connection is not used, direct insecure connection is used;
+    if provided, it can be either path to certificate file or certificate content
+- KAFKA_CLIENT_CERT_KEY: Kafka connection private key, optional; default value is undefined;
+    if not provided, then SSL connection is not used, direct insecure connection is used;
+    if provided, it can be either path to private key file or private key content
+- KAFKA_GROUP_ID: the Kafka group id, default value is 'identity-processor'
+- IDENTITY_NOTIFICATION_CREATE: the identity create entity Kafka message topic, default value is 'identity.notification.create'
+- AUTH0_URL: The auth0 url
+- AUTH0_UBAHN_AUDIENCE: The auth0 audience for accessing ubahn api(s)
+- AUTH0_TOPCODER_AUDIENCE: The auth0 audience for accessing topcoder api(s)
+- AUTH0_CLIENT_ID: The auth0 client id
+- AUTH0_CLIENT_SECRET: The auth0 client secret
+- AUTH0_PROXY_SERVER_URL: The auth0 proxy server url
+- TOKEN_CACHE_TIME: The token cache time
+- SLEEP_TIME: The pause time between two create operations, default value: 1000 ms
+- UBAHN_API_URL: The ubahn api url, default value: 'https://api.topcoder-dev.com/v5'
+- MEMBERS_API_URL: The topcoder member api url, default value: 'https://api.topcoder-dev.com/v5/members'
+- ATTRIBUTE_GROUP_NAME: The attribute group name
+- SKILL_PROVIDER_NAME: The skill provider name
+- ORGANIZATION_NAME: The organization name
+
+There is a `/health` endpoint that checks for the health of the app. This sets up an expressjs server and listens on the environment variable `PORT`. It's not part of the configuration file and needs to be passed as an environment variable
+
+## Local Kafka setup
+
+1. Navigate to the directory `docker-kafka`
+
+2. Run the following command
+
+    ```bash
+    docker-compose up -d
+    ```
+
+## Local deployment
+
+1. Make sure that Kafka is running as per instructions above.
+
+2. From the project root directory, run the following command to install the dependencies
+
+    ```bash
+    npm install
+    ```
+
+3. To run linters if required
+
+    ```bash
+    npm run lint
+    ```
+
+    To fix possible lint errors:
+
+    ```bash
+    npm run lint:fix
+    ```
+
+5. Start the processor and health check dropin
+
+    ```bash
+    npm start
+    ```
+
+## Local Deployment with Docker
+
+To run the Processor using docker, follow the below steps
+
+1. Navigate to the directory `docker`
+
+2. Rename the file `sample.api.env` to `api.env`
+
+3. Once that is done, run the following command
+
+    ```bash
+    docker-compose up
+    ```
+
+4. When you are running the application for the first time, It will take some time initially to download the image and install the dependencies
+
+
+## Verification
+
+see [VERIFICATION.md](VERIFICATION.md)

VERIFICATION.md

@@ -0,0 +1,48 @@
+# Verification
+
+```
+{
+  "topic":"identity.notification.create",
+  "originator":"u-bahn-api",
+  "timestamp":"2019-07-08T00:00:00.000Z",
+  "mime-type":"application/json",
+  "payload":{
+    "id":"90064000",
+    "modifiedBy":null,
+    "modifiedAt":"2021-01-05T14:01:40.336Z",
+    "createdBy":null,
+    "createdAt":"2021-01-05T14:01:40.336Z",
+    "handle":"theuserhandle",
+    "email":"foo@bar.com",
+    "firstName":"theuserfirstname",
+    "lastName":"theuserlastname",
+    "credential":{"activationCode":"FOOBAR2","resetToken":null,"hasPassword":false},
+    "profiles":null,
+    "status":"A",
+    "country":null,
+    "regSource":"null",
+    "utmSource":"null",
+    "utmMedium":"null",
+    "utmCampaign":"null",
+    "roles":null,
+    "ssoLogin":false,
+    "active":true,
+    "profile":null,
+    "emailActive":true
+  }
+}
+```
+
+Additionally, you will be entering the messages into only one topic:
+
+```
+docker exec -it identity-data-processor_kafka /opt/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic identity.notification.create
+```
+
+1. start kafka server, start processor app
+2. write message:
+  `{"recipients":[],"notificationType":"useractivation"}`
+3. Watch the app console, It will show error message.
+4. write message:
+  `{"topic":"identity.notification.create","originator":"u-bahn-api","timestamp":"2019-07-08T00:00:00.000Z","mime-type":"application/json","payload":{"id":"90064000","modifiedBy":null,"modifiedAt":"2021-01-05T14:01:40.336Z","createdBy":null,"createdAt":"2021-01-05T14:01:40.336Z","handle":"theuserhandle","email":"foo@bar.com","firstName":"theuserfirstname","lastName":"theuserlastname","credential":{"activationCode":"FOOBAR2","resetToken":null,"hasPassword":false},"profiles":null,"status":"A","country":null,"regSource":"null","utmSource":"null","utmMedium":"null","utmCampaign":"null","roles":null,"ssoLogin":false,"active":true,"profile":null,"emailActive":true}}`
+5. Watch the app console, It will show message successfully handled.

config/default.js

@@ -0,0 +1,34 @@
+/**
+ * The default configuration file.
+ */
+
+module.exports = {
+  LOG_LEVEL: process.env.LOG_LEVEL || 'debug',
+
+  KAFKA_URL: process.env.KAFKA_URL || 'localhost:9092',
+  // below are used for secure Kafka connection, they are optional
+  // for the local Kafka, they are not needed
+  KAFKA_CLIENT_CERT: process.env.KAFKA_CLIENT_CERT,
+  KAFKA_CLIENT_CERT_KEY: process.env.KAFKA_CLIENT_CERT_KEY,
+
+  // Kafka group id
+  KAFKA_GROUP_ID: process.env.KAFKA_GROUP_ID || 'identity-processor',
+
+  IDENTITY_NOTIFICATION_CREATE: process.env.IDENTITY_NOTIFICATION_CREATE || 'identity.notification.create',
+
+  AUTH0_URL: process.env.AUTH0_URL,
+  AUTH0_UBAHN_AUDIENCE: process.env.AUTH0_UBAHN_AUDIENCE,
+  AUTH0_TOPCODER_AUDIENCE: process.env.AUTH0_TOPCODER_AUDIENCE,
+  TOKEN_CACHE_TIME: process.env.TOKEN_CACHE_TIME,
+  AUTH0_CLIENT_ID: process.env.AUTH0_CLIENT_ID,
+  AUTH0_CLIENT_SECRET: process.env.AUTH0_CLIENT_SECRET,
+  AUTH0_PROXY_SERVER_URL: process.env.AUTH0_PROXY_SERVER_URL,
+
+  SLEEP_TIME: process.env.SLEEP_TIME ? parseInt(process.env.SLEEP_TIME, 10) : 1000,
+
+  UBAHN_API_URL: process.env.UBAHN_API_URL || 'https://api.topcoder-dev.com/v5',
+  ORGANIZATION_NAME: process.env.ORGANIZATION_NAME,
+  ATTRIBUTE_GROUP_NAME: process.env.ATTRIBUTE_GROUP_NAME,
+  SKILL_PROVIDER_NAME: process.env.SKILL_PROVIDER_NAME,
+  MEMBERS_API_URL: process.env.MEMBERS_API_URL || 'https://api.topcoder-dev.com/v5/members'
+}

docker-kafka/docker-compose.yml

@@ -0,0 +1,16 @@
+version: '3'
+services:
+  zookeeper:
+    image: wurstmeister/zookeeper
+    container_name: identity-data-processor_zookeeper
+    ports:
+      - "2181:2181"
+  kafka:
+    image: wurstmeister/kafka
+    container_name: identity-data-processor_kafka
+    ports:
+      - "9092:9092"
+    environment:
+      KAFKA_ADVERTISED_HOST_NAME: localhost
+      KAFKA_CREATE_TOPICS: "identity.notification.create:1:1"
+      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181

docker/Dockerfile

@@ -0,0 +1,12 @@
+# Use the base image with Node.js 8.11.3
+FROM node:12.16.3
+
+# Copy the current directory into the Docker image
+COPY . /identity-processor
+
+# Set working directory for future use
+WORKDIR /identity-processor
+
+# Install the dependencies from package.json
+RUN npm install
+CMD npm start

docker/docker-compose.yml

@@ -0,0 +1,10 @@
+version: '3'
+services:
+  identity-data-processor:
+    image: identity-data-processor:latest
+    build:
+      context: ../
+      dockerfile: docker/Dockerfile
+    env_file:
+      - api.env
+    network_mode: "host"

docker/sample.api.env

@@ -0,0 +1 @@
+KAFKA_URL=<KAFKA Host Endpoint>