Skip to content

Latest commit

 

History

History
147 lines (118 loc) · 7.89 KB

README.md

File metadata and controls

147 lines (118 loc) · 7.89 KB
Raw

What is CyBench Launcher

NOTE: CyBench services moved to dedicated host https://app.cybench.io from previous https://www.gocypher.com/. This change is reflected in CyBench client version 1.3.0. For previous versions, set Java System Property sendReportToURL when running CyBench Launcher:

-DsendReportToURL=https://app.cybench.io/gocypher-benchmarks-reports/services/v1/reports/report

CyBench Launcher is a standalone Java application designed to execute JMH benchmarks. Speed-test Java classes, collections, modules, libraries and other application building blocks under varying runtime conditions. Share your results with the community, compare and choose the right libraries for your application. CyBench helps developers build better, faster Java code by benchmarking code performance.

CyBench Launcher contains a set of default performance tests developed by the CyBench team to benchmark Java Core services. These tests are executed by default every time CyBench benchmarks are executed via cybench shell script and results are automatically submitted to a public CyBench Repository.

Default benchmark execution and automated reporting can be configured via <cybench-home>/config/cybench-launcher.properties. Download and run CyBench here. Visit our page to analyze your results.

Running User-defined Benchmarks using CyBench Launcher

Add the runner and JMH dependencies to your project

<dependencies>
    <dependency>
        <groupId>org.openjdk.jmh</groupId>
        <artifactId>jmh-generator-annprocess</artifactId>
        <version>1.35</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.openjdk.jmh</groupId>
        <artifactId>jmh-core</artifactId>
        <version>1.35</version>
    </dependency>
    <dependency>
        <groupId>com.gocypher.cybench.client</groupId>
        <artifactId>gocypher-cybench-runner</artifactId>
        <version>1.3.5</version>
    </dependency>
</dependencies>

Run your benchmarks

  • Implement any benchmarks classes
  • Add call to the runner main class
    public static void main(String[]args)throws Exception{
        BenchmarkRunner.main(args);
        }
  • Analyze results in your console, reports folder or app.cybench.io
  • Optional: Add CyBench configuration file into <project-path>/config/cybench-launcher.properties directory to manage your benchmarking run configuration. Configuration example

CyBench launcher configuration

Property name Description Default value
javaOptions All the property fields that starts with name javaOptions will be used while benchmarking as JVM properties. -
javaToUsePath Provide full path to java.exe to be used e.g. D:/jdk180_162/bin/java.exe -
benchmarks Provide jar's with JMH benchmarks which shall be executed with CyBench. more here -
sendReport Choose if the report generated will be automatically uploaded. (true/false) true
reportUploadStatus Define public or private property for the uploaded report visibility. public
benchAccessToken By providing the "bench" token that you get after creating a workspace in CyBench UI, you can send reports to your private directory, which will be visible only to the users that you authorize. -
emailAddress Email property is used to identify report sender while sending reports to both private and public repositories -
reportName Choose the uploaded report name. E.g. -
benchmarkClasses Specify benchmarks by including fully qualified benchmark class names which are comma separated. For more information more here -
numberOfBenchmarkForks Number of separate full executions of a benchmark (warm up+measurement), this is returned still as one primary score item. 1
measurementIterations Number of measurements per benchmark operation, this is returned still as one primary score item. 5
warmUpIterations Number of iterations executed for warm up. 1
warmUpSeconds Number of seconds dedicated for each warm up iteration. 5
runThreadCount Number of threads for benchmark test execution. 1
benchmarkModes Benchmark modes to run. All, or ones defined by annotation
jmhArguments JMH CLI supported arguments line. See JMH Command line options for details -
benchmarkMetadata A property which adds extra properties to the benchmarks report such as category or version or context. Configuration pattern is <fully qualified benchmark class name>=<key1>:<value1>;<key2>:<value2>. Example which adds category for class CollectionsBenchmarks: com.gocypher.benchmarks.client.CollectionsBenchmarks=category:Collections; -
userProperties User defined properties which will be added to benchmarks report section environmentSettings->userDefinedProperties as key/value strings. Configuration pattern:<key1>:<value1>;<key2>:<value2>. Example which adds a project name:user.propname1=My Test Project; -

CyBench's configuration file cybench-launcher.properties can be created manually, or by using the CyBench Configuration Generator script, found here

Adding Custom Benchmarks for Execution

Update CyBench Launcher configuration located in <cybench-home>/config/cybench-launcher.properties:

  • required: add or update property benchmarks, set path to jar file which contains your JMH benchmark, this path will be added to the CLASSPATH of the JVM. Values must be semicolon separated!

    Rule:

    benchmarks=<path to custom jar file1>;<path to custom jar file2>;

    Example:

    benchmarks=gocypher-cybench-custom-1.0.0.jar;

  • optional: register categories for your tests in order to have correct tests classification and better readability and comparison in CyBench portal. If not set then default value (Custom) will be written and all custom tests will reside under this category. Values of different classes must be semicolon separated!

    Rule:

    benchmarkMetadata=<fully classified benchmark class name>=category:<category name>;\
  <fully classified benchmark class name>=category:<category name>;

    Example:

    benchmarkMetadata=com.gocypher.benchmarks.client.CollectionsBenchmarks=category:Collections;

Execute Only Custom Benchmarks

Update CyBench configuration in order to run only user-defined tests:

  • add or update property benchmarkClasses, specify class names of tests which shall be executed (values must be comma separated).

    Rule:

    benchmarkClasses=<fully qualified class name, or class name>,<fully qualified class name, or class name>

    Example:

    benchmarkClasses=com.gocypher.benchmarks.client.CollectionsBenchmarks,NumberBenchmarks

Benchmark reports comparator

Java app comparing most recently ran CyBench benchmarks to previously ran benchmarks hosted on the CyBench site.

See gocypher-cybench-comparator readme file for usage details.