Home

Welcome to the CyBench wiki!

Required Skills

• Java Developers, DevOps, CI/CD
• Familiarity with Java Micro-benchmarking Harness JMH.

Video Tutorials video library:

• Getting Started - Overview
• Getting Started - Creating your Account
• Getting Started - Maven Plugin
• Getting Started - Gradle Plugin
• Getting Started - Eclipse Plugin
• Getting Started - IntelliJ Plugin
• Getting Started - Comparing Benchmarks using the CyBench UI
• Getting Started - Performance Regression Testing for your CI/CD
• Getting Started - Automated Performance Regression Testing

Introduction

CyBench is a code benchmark platform to help developers, performance engineers with continuous performance regression testing. Quickly create, run and analyze code & API performance benchmarks based on JMH.

CyBench platform consists of the following components.

• CyBench Explorer -- Dashboard for displaying, sharing and analyzing CyBench benchmarks.
• CyBench Launcher -- Code micro-benchmarking harness designed to run JMH benchmarks.
• CyBench Java Benchmarks -- JMH default performance benchmarks for Java environment.
• CyBench Eclipse Plugin -- integration into Eclipse environment to help build, run benchmarks.
• CyBench IntelliJ Plugin -- integration into IntelliJ environment to help build, run benchmarks.
• CyBench Maven Plugin -- integration into Maven. Run benchmarks from Maven build scripts.
• CyBench Gradle Plugin -- integration into Gradle. Run benchmarks from Gradle build scripts.
• CyBench JUnit Plugin -- integration with JUnit, auto-generate and run benchmarks for unit tests.
• CyBench TestNG Plugin -- integration with TestNG, auto-generate and run benchmarks for unit tests.

All default benchmarks can be downloaded here, under Assets. Each benchmarks is packaged into a separate zip file and includes CyBench Launcher.

Upon execution, CyBench Launcher runs all configured JMH benchmarks and sends benchmark reports to CyBench Explorer, where benchmarks are stored and analyzed. Use it to view, analyze, compare and share your benchmarks. All benchmark reports (.cyb files) and communications are encrypted (reports (.cyb files) are sent via HTTPS).

By default all benchmark submissions are stored in a public workspace, visible to everyone. Private benchmarks (only visible to owner and team members) require setup of your own workspace (see tutorial here on how to create your own workspace and publish private benchmarks)

CyBench Launcher can also execute your user defined JMH benchmarks. See docs for details.

CyBench Tutorials

Performance Benchmark development using CyBench tools

• Benchmark development for Eclipse Java project
• Benchmark development for Eclipse Maven project
• Benchmark development for IntelliJ Java project
• Benchmark development for IntelliJ Maven project
• Benchmark development for IntelliJ Gradle project
• Annotating code benchmarks with metadata
• Getting started with CyBench Eclipse UI tools
• Performance Regression Testing for JUnit/TestNG
• Comparing benchmark scores with CyBench Comparator

Performance Benchmarks & Regression Testing Tutorials

• Code benchmarks for JUnit
• Code benchmarks in Eclipse and Maven

Performance Benchmark Case Studies

• Case 1: Detection of faulty code behavior using benchmarks
• Case 2: Verifying that code improvements works as expected using benchmarks
• Case 3: Systematic approach to benchmarking your code

Troubleshooting

A quick guide to help resolve known issues until fixes are released. Click here.