Provides web interface for Mongoose - storage performance testing tool maintained by Dell EMC.
-
Overview
1.1 Launch Mongoose run
1.1.1 Nodes selection
1.1.2 Configuration
1.1.3 Scenario
1.2 Monitoring
1.2.1 Mongoose runs table
1.2.2 Mongoose run statistics
1.2.2.1 Mongoose logs
1.2.2.2 Mongoose charts
1.3 Terminate Mongoose run -
Configuration
2.1 Deploying ports
2.2 Image versions
2.3 Container configuration
2.3.1 Prometheus
2.4 Other parameters -
Deployment
4.1 Docker
4.2 Kubernetes
4.2.1 Test Darzee in Kubernetes using predefined Mongoose environment
4.2.1.1 Mongoose base
4.2.1.2 Mongoose Pravega driver -
Troubleshooting
5.1 Prometheus inactivity
5.2 Missing load step node address
5.3 Chart.js error on start up
Darzee lets you configure, run and monitor Mongoose runs.
Mongoose launching process is divided in 3 steps:
Multiple Mongoose run nodes could be selected. First selected node will be the entry one.
Configuration is presented as an editable JSON tree.
Initially, the configuration is fetched from the entry node.
Mongoose's scenarios are written in JavaScript. It's possible to write JavaScript code in the UI using code editor.
Discovered Mongoose's runs are displayed within the runs table.
There's 3 possible status of Mongoose runs:
- Finished - means Mongoose run has finished and its logs are available;
- Running - means Mongoose is still performing the benchmark. Charts will be updated every time Darzee received a new data;
- Unavailable - means info about Mongoose run has been found on the server, but the related data is lost or couldn't be loaded.
You could see Mongoose run status or run details by pressing the status icon.
By pressing run status icon, you'll be redirected to run statistics screen. See Mongoose logs, Mongoose charts.
If the run is still being processed, the data will be updating dynamically based on the Mongoose's exported metrics and generated logs.
Charts are drawn based on Mongoose's exported metrics. There're 4 types of Mongoose charts available:
- duration - mean duration of Mongoose's performed operations;
- bandwidth - amount of processed bytes per second;
- throughput - shows rate of successful and failed operations performed by Mongoose. It takes mean values for each metric;
- latency - shows minimal and maximal latency during Mongoose's run;
Processing Mongoose run could be terminated from the UI.
UI's configuration depends on the parameters listen within .env file.
The following parameters are being used to specify deploying ports of the services:
- DARZEE_PORT - for Darzee itself;
- PROMETHEUS_PORT - for Prometheus;
- MONGOOSE_PORT - for Mongoose;
The following parameters are being used to specify docker image version of a specific service:
- DARZEE_VER - Darzee's image version;
- PROMETHEUS_VER - Prometheus image version;
Prometheus configuration is being rewritten in a runtime in order to add targets dynamically. To implement this, we're using internal container volume.
- PROMETHEUS_CONFIGURATION_PATH - full path for Prometheus configuration;
- PROMETHEUS_CONFIGURATION_FOLDER_PATH - path to Prometheus folder;
- MONGOOSE_HOST - specifies host of an initially loaded Mongoose (optional);
Docker image is being builted via Gradle. To build Docker image, use:
$ ./gradlew buildImage
Darzee has been created with Angular 7.0. You could build it just like any Angular app.
$ npm install
$ ng build
Darzee is deploying via docker image. The image contains front-end files, nodeJS server to serving the app and Prometheus. In order to have the access from outside, NodeJS server should be mapped to port 8080 and Prometheus should be mapped to port 9090:
$ docker run --env-file .env -p 8080:8080 -p 9090:9090 emcmongoose/darzee
Note that you should use project's .env file.
Darzee has been created with Angular CLI. It could be ran in development mode using the appropriate command:
$ npm install
$ ng serve
Darzee image is being loaded into the docker hub.
bash
docker run \
--env-file .env \
-p {DARZEE_PORT}:{DARZEE_PORT} \
-p {PROMETHEUS_PORT}:{PROMETHEUS_PORT} \
emcmongoose/darzee[-<VERSION>]
Note that you should use project's .env file.
Darzee can be deployed in a kubernetes cluster. Examples of kubernetes object discription files are in the directory /console/kubernetes
.
In order to connect with the outside network, Darzee uses Kubernetes service /console/kubernetes/darzee-service.yml
Run Darzee as a deployment:
$ bash kubectl apply -f /console/kubernetes/darzee-deployment.yaml
$ bash kubectl apply -f /console/kubernetes/darzee-service.yml
The project contains scripts for deploying predefined Mongoose environment in order to run the UI.
Mongoose's environments are described via Kubernetes deplyoments. Note that there's 2 type of ports for Mongoose:
- Remote API port. You should specify remote API port when adding it into nodes list.
- RMI port (Remote Method Invocation port). You should specify RMI port when adding additional Mongoose nodes in order to run it in distributed mode. RMI ports should be specified in configuration (load-step-node-addrs).
$ bash cd /console/src/assets/configuration-examples/kubernetes/mongoose-base/tools
$ bash chmod +x create-mongoose-environment.sh
$ bash ./create-mongoose-environment.sh
This will deploy 2 pods, each of them will contain 5 Mongoose instances. Remote API ports are 9991...9994, 9999, RMI ports are 1091...1094, 1099. Every one of them are connected to the web outside of k8s cluster via the service. You can get service's external IP via
$ bash kubectl get svc mongoose-base-svc
$ bash kubectl get svc mongoose-base-svc-2
Combining external IP with Mongoose ports, you'll get an access to it from the network outside of cluster.
In order to delete the environment, use:
$ bash cd /console/src/assets/configuration-examples/kubernetes/mongoose-base/tools
$ bash chmod +x delete-mongoose-environment.sh
$ bash ./delete-mongoose-environment.sh
$ bash cd /console/src/assets/configuration-examples/kubernetes/mongoose-pravega/tools
$ bash chmod +x create-mongoose-pravega-environment.sh
$ bash ./create-mongoose-pravega-environment.sh
This will create a deployment and a service.
The deployment will contain instance of Mongoose Pravega driver, the service will connect it to the network outside of cluster via remote API port 9999.
Note that in kubernetes object description for Mongoose pravega driver, you should
specify a running Pravega controller IP (<PRAVEGA_CONTROLLER_IP>), otherwise it won't work.
args: [--run-node, --storage-namespace=mongoose-pravega-scope, --run-port=9999, --load-step-node-port=1099,
--storage-net-node-addrs=<PRAVEGA_CONTROLLER_IP>, --load-batch-size=100, --storage-driver-limit-queue-input=10000]
You can get service's external IP via
$ bash kubectl get svc mongoose-pravega-svc
In order to delete the environment, use:
$ bash cd /console/src/assets/configuration-examples/kubernetes/mongoose-pravega/tools
$ bash chmod +x delete-mongoose-pravega-environment.sh
$ bash ./delete-mongoose-pravega-environment.sh
If Darzee's container Prometheus is not active, it's possible to use a custom one with some limitations.
It's recommended to restart the image in case of Prometheus inactivity thought.
In case data for the load step is fetched from the Prometheus, but its IP address hasn't been found, it's possible to write it from the UI.
NOTE: Actual for 06.08.2019
With Angular builder version change, some errors are popping up. We've experienced issues with the charts library we're using: after building Darzee docker image from Gitlab CI, the UI hasn't been starting up because of invalid linking of the charts library with the UI via Webpack. Althought, it has been working correctly with manual build.
Different Angular CLI and Angular versions is the reason why the issue took a place. It was resolved here.
Related issues:
- ERROR Error: Uncaught (in promise): TypeError: Object(...) is not a function #1117 - same issue;
- "export 'ɵɵinject' was not found in '@angular/core' since v8.0.0-beta.12 - explains why some libraries are breaking because of different Angular versions usage;
We're constantly working on making the UI and Mongoose project overall better. In this section, you'd find a list of a high-priority issues that would be fixed as soon as possible.
Mongoose run's table is constructed based on the retrieved data from Prometheus: run ID, additional nodes, load step ID, etc. The list doesn't contain entry node address.
Now: Mongoose run's entry nodes addresses are stored within browser's local storage. It helps to retain the information about entry nodes as long as the local storage not getting cleaned up, yet it'd be saved even if the browser and/or tab has been closed.
We need entry node's address in order to get logs via POST request with Mongoose Logs API to its entry node.
Problem: while working outside of the browser from which Mongoose run has been launched (other browser, other computer, etc.), the data about run's entry node is missing, thus logs couldn't be gathered via the Logs API.
Temporary solution: see Missing load step node address.
Track progress on the issue here.