diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index 26da11db1..55291c4b6 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -2,14 +2,29 @@ This document will walk you through on what's needed to start contributing code to OpenSearch Benchmark. +### Table of Contents +- [Installation](#installation) + - [Prerequisites](#prerequisites) + - [Setup](#setup) +- [Importing the project into an IDE](#importing-the-project-into-an-ide) +- [Executing tests](#executing-tests) + - [Unit tests](#unit-tests) + - [Integration tests](#integration-tests) +- [Submitting your changes for a pull request](#submitting-your-changes-for-a-pull-request) +- [Developing breaking changes](#developing-breaking-changes) +- [Miscellaneous](#miscellaneous) + - [git-secrets](#git-secrets) + - [Adding new major and minor Python versions to OpenSearch Benchmark](#adding-new-major-and-minor-python-versions-to-opensearch-benchmark) + - [Debugging unittests in Visual Studio Code](#debugging-unittests-in-visual-studio-code) + ## Installation ### Prerequisites - - Pyenv : Install `pyenv` and follow the instructions in the output of `pyenv init` to set up your shell and restart it before proceeding. + - **Pyenv** : Install `pyenv` and follow the instructions in the output of `pyenv init` to set up your shell and restart it before proceeding. For more details please refer to the [PyEnv installation instructions](https://github.com/pyenv/pyenv#installation). - - JDK: Although OSB is a Python application, it optionally builds and provisions OpenSearch clusters. JDK version 17 is used to build the current version of OpenSearch. Please refer to the [build setup requirements](https://github.com/opensearch-project/OpenSearch/blob/ca564fd04f5059cf9e3ce8aba442575afb3d99f1/DEVELOPER_GUIDE.md#install-prerequisites). + - **JDK**: Although OSB is a Python application, it optionally builds and provisions OpenSearch clusters. JDK version 17 is used to build the current version of OpenSearch. Please refer to the [build setup requirements](https://github.com/opensearch-project/OpenSearch/blob/ca564fd04f5059cf9e3ce8aba442575afb3d99f1/DEVELOPER_GUIDE.md#install-prerequisites). Note that the `javadoc` executable should be available in the JDK installation. An earlier version of the JDK can be used, but not all the integration tests will pass. ``` @@ -17,16 +32,17 @@ This document will walk you through on what's needed to start contributing code ``` - - Install Docker and `docker-compose`. Start the Docker server. The user running the integration tests should have the permissions required to run docker commands. Test by running `docker ps`. + - **Docker**: Install Docker and `docker-compose`. Start the Docker server. The user running the integration tests should have the permissions required to run docker commands. Test by running `docker ps`. - - Git : git 1.9 or later. + - **Git** : supports versions 1.9+ ### Setup -Use the following command-line instructions to set up OpenSearch Benchmark for development: +To develop OSB properly, it is recommended that you fork the official OpenSearch Benchmark repository. + +After you git cloned the forked copy of OpenSearch Benchmark, use the following command-line instructions to set up OpenSearch Benchmark for development: ``` -git clone https://github.com/opensearch-project/OpenSearch-Benchmark -cd OpenSearch-Benchmark +cd opensearch-benchmark make prereq make install ``` @@ -50,8 +66,6 @@ Depending on the platform and shell you have, use the following command to activ For more information regarding activating virtual environments, please see https://docs.python.org/3/library/venv.html. -As an extra step, please consider configuring your JAVA_HOMES as mentioned in the `Important information related to integration tests`. - ## Importing the project into an IDE OpenSearch Benchmark builds using virtualenv. When importing into an IDE, such as PyCharm IDE, you will need to define an appropriate Python SDK, which is provided by virtualenv. @@ -65,10 +79,10 @@ In order to run tests within the PyCharm IDE, ensure the `Python Integrated Tool Once setup is complete, you may run the unit and integration tests. ### Unit Tests +Invoke unit tests by running the following command within the root directory of the repository: ``` make test - ``` ### Integration Tests @@ -80,12 +94,14 @@ Integration tests can be run on the following operating systems: * Amazon Linux 2 * MacOS +Invoke integration tests by running the following command within the root directory of the repository: ``` make it - ``` +Integration tests are expected to run for approximately 20-30 mins. + ## Submitting your changes for a pull request Once your changes and tests are ready to submit for review: @@ -109,7 +125,7 @@ Once your changes and tests are ready to submit for review: ## Developing Breaking Changes Breaking changes should not be directly added to the `main` branch. These should be developed in their own feature branch. Prior to a new release this feature branch should be rebased onto the latest changes from `main`. `main` can then `pull` or `cherry-pick` the breaking changes from the feature branch. -## Misc +## Miscellaneous ### git-secrets Security is our top priority. Avoid checking in credentials, install awslabs/git-secrets. @@ -120,7 +136,7 @@ cd git-secrets make install ``` ### Adding New Major and Minor Python Versions to OpenSearch-Benchmark -To streamline the process, please refer to [this guide](./PYTHON_SUPPORT_GUIDE.md) +To streamline the process, please refer to [this guide](https://github.com/opensearch-project/opensearch-benchmark/blob/main/PYTHON_SUPPORT_GUIDE.md) ### Debugging Unittests in Visual Studio Code To run and debug unittests in Visual Studio Code, add the following configuration to the Python Debugger `launch.json` file. See [the official Visual Studio Code documentation](https://code.visualstudio.com/docs/editor/debugging) for more information on setting up and accessing `launch.json` file. diff --git a/README.md b/README.md index 291935d03..707179607 100644 --- a/README.md +++ b/README.md @@ -20,103 +20,18 @@ We have also put considerable effort into OpenSearch Benchmark to ensure that be Documentation ------------- -Documentation for OpenSearch Benchmark is [available online](https://opensearch.org/docs/latest/benchmark/index/). +Official documentation for OpenSearch Benchmark is [available online](https://opensearch.org/docs/latest/benchmark/index/). Quick Start ----------- -OpenSearch Benchmark is developed for Unix and is actively tested on Linux, Ubuntu, and MacOS. OpenSearch Benchmark supports [benchmarking OpenSearch clusters running on Windows]() but OpenSearch Benchmark itself needs to be installed on machines running Unix. - -Installing OpenSearch Benchmark -------------------------------- - -**Note**: If you actively develop on OpenSearch, we recommend that you [install OpenSearch Benchmark in development mode]() instead as OpenSearch is fast moving and OpenSearch Benchmark always adapts accordingly to the latest main version. - -Install Python 3.8+ including ``pip3``, git 1.9+ and an [appropriate JDK to run OpenSearch]() Be sure that ``JAVA_HOME`` points to that JDK and that the corresponding Java executable in ``$JAVA_HOME/bin`` is picked up via your PATH environment variable. Then run the following command, optionally prefixed by ``sudo`` if necessary: - - python3 -m pip install opensearch-benchmark - - -If you have any trouble or need more detailed instructions, please look in the [detailed installation guide](). - -Run your first test execution ------------------------------ - -Now we're ready to run our first test execution: - - opensearch-benchmark execute-test --distribution-version=1.0.0 --workload=geonames --test-mode - -This will download OpenSearch 1.0.0 and run one of OpenSearch Benchmark's official workloads - the [geonames workload]() - against it. -Note that this uses the `--test-mode` argument to only run a single instance of each operation in order to reduce the time needed for a test execution. This argument is used as a sanity check and should be removed in an actual benchmarking scenario. -After the test execution, a summary report is written to the command line: - - ------------------------------------------------------ - _______ __ _____ - / ____(_)___ ____ _/ / / ___/_________ ________ - / /_ / / __ \/ __ `/ / \__ \/ ___/ __ \/ ___/ _ \ - / __/ / / / / / /_/ / / ___/ / /__/ /_/ / / / __/ - /_/ /_/_/ /_/\__,_/_/ /____/\___/\____/_/ \___/ - ------------------------------------------------------ - - | Metric | Task | Value | Unit | - |-------------------------------:|---------------------:|----------:|-------:| - | Total indexing time | | 28.0997 | min | - | Total merge time | | 6.84378 | min | - | Total refresh time | | 3.06045 | min | - | Total flush time | | 0.106517 | min | - | Total merge throttle time | | 1.28193 | min | - | Median CPU usage | | 471.6 | % | - | Total Young Gen GC | | 16.237 | s | - | Total Old Gen GC | | 1.796 | s | - | Index size | | 2.60124 | GB | - | Total written | | 11.8144 | GB | - | Heap used for segments | | 14.7326 | MB | - | Heap used for doc values | | 0.115917 | MB | - | Heap used for terms | | 13.3203 | MB | - | Heap used for norms | | 0.0734253 | MB | - | Heap used for points | | 0.5793 | MB | - | Heap used for stored fields | | 0.643608 | MB | - | Segment count | | 97 | | - | Min Throughput | index-append | 31925.2 | docs/s | - | Median Throughput | index-append | 39137.5 | docs/s | - | Max Throughput | index-append | 39633.6 | docs/s | - | 50.0th percentile latency | index-append | 872.513 | ms | - | 90.0th percentile latency | index-append | 1457.13 | ms | - | 99.0th percentile latency | index-append | 1874.89 | ms | - | 100th percentile latency | index-append | 2711.71 | ms | - | 50.0th percentile service time | index-append | 872.513 | ms | - | 90.0th percentile service time | index-append | 1457.13 | ms | - | 99.0th percentile service time | index-append | 1874.89 | ms | - | 100th percentile service time | index-append | 2711.71 | ms | - | ... | ... | ... | ... | - | ... | ... | ... | ... | - | Min Throughput | painless_dynamic | 2.53292 | ops/s | - | Median Throughput | painless_dynamic | 2.53813 | ops/s | - | Max Throughput | painless_dynamic | 2.54401 | ops/s | - | 50.0th percentile latency | painless_dynamic | 172208 | ms | - | 90.0th percentile latency | painless_dynamic | 310401 | ms | - | 99.0th percentile latency | painless_dynamic | 341341 | ms | - | 99.9th percentile latency | painless_dynamic | 344404 | ms | - | 100th percentile latency | painless_dynamic | 344754 | ms | - | 50.0th percentile service time | painless_dynamic | 393.02 | ms | - | 90.0th percentile service time | painless_dynamic | 407.579 | ms | - | 99.0th percentile service time | painless_dynamic | 430.806 | ms | - | 99.9th percentile service time | painless_dynamic | 457.352 | ms | - | 100th percentile service time | painless_dynamic | 459.474 | ms | - - ---------------------------------- - [INFO] SUCCESS (took 2634 seconds) - ---------------------------------- - -Creating Your Own Workloads ---------------------------- -For more information on how users can create their own workloads, see [the Create Workload Guide](./CREATE_WORKLOAD_GUIDE.md) +Want to get started with OpenSearch Benchmark quickly? See [OpenSearch Benchmark's Quick Start guide in the documentation](https://opensearch.org/docs/latest/benchmark/index/). Getting help ------------ * Quick help: ``opensearch-benchmark --help`` -* Look in [OpenSearch Benchmark's user guide]() for more information +* Want to contribute? Look at [OpenSearch Benchmark's Developer Guide]() for more information * For any questions or answers, visit our [community forum](). * File improvements or bug reports in our [Github repo]().