diff --git a/README.md b/README.md index 12938de..1b42d2e 100644 --- a/README.md +++ b/README.md @@ -3,57 +3,4 @@ **Hood** is a `Gradle` plugin to compare benchmarks and set the result as a `Github` status for a `Pull Request`. **Hood** is built on [Arrow](https://arrow-kt.io/), a Functional companion to Kotlin's Standard Library. -## How to use it - -**Hood** has two tasks to compare benchmarks: - - `compareBenchmarks`: Compare two benchmarks and print the result. - - `compareBenchmarksCI`: Compare two benchmarks and upload a `Github` status for a `Pull Request`. - -Both tasks have common parameters: - - **previousBenchmarkPath**: File with previous or master benchmark location. By default: `master.csv`. - - **currentBenchmarkPath**: List of files with current or pull request benchmark location. By default is an `empty list`. - - **keyColumnName**: Column name to distinct each benchmark on the comparison. By default: `Benchmark`. - - **compareColumnName**: Column name of the column to compare (the values must to be a `Double`). By default: `Score`. - - **thresholdColumnName**: Column name to get the threshold per benchmark. By default: `Score Error (99.9%)`. - - **generalThreshold**: Common threshold to all benchmarks overriding the value coming from `thresholdColumnName`. Optional. - - **benchmarkThreshold**: `Map` with a custom threshold per benchmark key overriding the value coming from `thresholdColumnName` or `generalThreshold`. Optional. - - **include**: Regular expression to include only the benchmarks with a matching key. Optional. - - **exclude**: Regular expression to exclude the benchmarks using its key. Optional. - -The `include/exclude` feature and `benchmarkThreshold` param use the cleaned key from benchmarks. -This means the key for `hood.comparing` will be `Comparing` with the capitalization. - -The `compareBenchmarksCI` task also needs some extra parameters: - - **token**: The `Github` access token. - - **repositoryOwner**: The repository owner. - - **repositoryName**: The repository name. - - **pullRequestSha**: The sha for the Pull Request. The environment variable `TRAVIS_PULL_REQUEST_SHA` on Travis CI. - - **pullRequestNumber**: The number of the Pull Request. The environment variable `TRAVIS_PULL_REQUEST` on Travis CI. - - **statusTargetUrl**: The URL to the CI job. The environment variable `TRAVIS_JOB_WEB_URL` on Travis CI. Optional. - -***Note***: Currently **Hood** only supports `CSV` and `JSON` based benchmarks with cross comparison available. -***Note 2***: If the `CI` integration is not available because one the requested fields above is not defined, - the task `compareBenchmarksCI` will be executed on the same way as `compareBenchmarks`. - -### Send output to a file - -Both task can send the result to a file, just need to fulfill the following parameters: - - **outputToFile**: If send the output to a file. By default: `false`. - - **outputPath**: The path to the output file. By default: `./hood/comparison`. - - **outputFormat**: The output file format, we support two formats `MD` and `JSON`. By default: `MD`. - -***Note***: To print a `JSON` output file, all the benchmarks must to be in `JSON` format. `CSV` benchmarks will be ignored. - -## Upload benchmarks - -**Hood** allow you to upload automatically the benchmarks -you want to maintain updated in your code thought commits during the CI. - -The task `uploadBenchmark` has the following parameters: - - **benchmarkFiles**: The list of benchmark files you want to upload. By default is an `empty list`. - - **uploadDirectory**: The path for the folder where you want to keep them, from project root directory. By default: `benchmarks`. - - **commitMessage**: The message for the task commit uploading the benchmark. By default: `Upload benchmark`. - - **token**: the `Github` access token. - - **repositoryOwner**: The repository owner. - - **repositoryName**: The repository name. - - **branch**: The branch where you want to upload those benchmarks. By default: `master`. \ No newline at end of file +Please see the [Hood Microsite](https://47deg.github.io/hood/) for more information. \ No newline at end of file diff --git a/deploy-scripts/deploy.sh b/deploy-scripts/deploy.sh index d357c1a..dc03d6b 100755 --- a/deploy-scripts/deploy.sh +++ b/deploy-scripts/deploy.sh @@ -7,9 +7,9 @@ if [ "$TRAVIS_BRANCH" == "master" ]; then if [[ "$VERSION_NAME" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then echo "Starting script for Release $VERSION_NAME" . $(dirname $0)/deploy_release.sh - elif [[ "$VERSION_NAME" == *-SNAPSHOT ]]; then - echo "Starting script for Snapshot Release $VERSION_NAME" - . $(dirname $0)/deploy_snapshot.sh +# elif [[ "$VERSION_NAME" == *-SNAPSHOT ]]; then +# echo "Starting script for Snapshot Release $VERSION_NAME" +# . $(dirname $0)/deploy_snapshot.sh else echo "No deploy script matched version '$VERSION_NAME' on master" fi diff --git a/docs/docs/CI/README.md b/docs/docs/CI/README.md index 0e4d240..d9512e7 100644 --- a/docs/docs/CI/README.md +++ b/docs/docs/CI/README.md @@ -5,25 +5,77 @@ permalink: docs/ci/ --- -# Continuous Integration h1 -## Generation h2 -### Generation h3 -#### Generation h4 -##### Generation h5 -###### Generation h6 - -Lorem ipsum dolor sit amet, [consectetur adipiscing](#) elit. Nam lacinia auctor nunc, in commodo ante faucibus ut. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Nunc vel neque ut nisi sodales aliquet. Ut nunc velit, feugiat et elit eget, lobortis sodales est. Integer turpis tellus, semper nec neque ut, elementum mollis metus. Ut `faucibus lorem` felis porta lorem vehicula accumsan. Duis sed tempor enim, ac porttitor mi. Maecenas non sollicitudin quam. Nam mollis scelerisque sapien eu viverra. Etiam tempor bibendum eros, a faucibus tellus ullamcorper a. Aliquam id lorem est. Vivamus in ipsum condimentum, vulputate nisl et, rutrum tellus. - -- item 1 -- item 2 -- item 3 -- item 4 - -```swift -func impureAdd(x: Int, y: Int) -> Int { - print("Received (\(x), \(y))") - return x + y +# Compare Benchmarks with CI + +**Hood** gives you the ability to integrate the benchmark comparison into your `CI` + and integrate the result into a `Github` pull request. + +The `compareBenchmarksCI` task has parameters in common with the `compareBenchmarks` task: + - **previousBenchmarkPath**: File with previous or master benchmark location. By default: `master.csv`. + - **currentBenchmarkPath**: List of files with current or pull request benchmark location. By default: an `empty list`. + - **keyColumnName**: Column name to distinguish each benchmark on the comparison. By default: `Benchmark`. + - **compareColumnName**: Column name of the column to compare (the values must be a `Double`). By default: `Score`. + - **thresholdColumnName**: Column name to get the threshold per benchmark. By default: `Score Error (99.9%)`. + - **generalThreshold**: Common threshold for all benchmarks overriding the value coming from `thresholdColumnName`. Optional. + - **benchmarkThreshold**: `Map` with a custom threshold per benchmark key overriding the value coming from `thresholdColumnName` or `generalThreshold`. Optional. + - **include**: Regular expression to include only the benchmarks with a matching key. Optional. + - **exclude**: Regular expression to exclude the benchmarks using its key. Optional. + +The `include/exclude` feature and `benchmarkThreshold` param use the cleaned key from benchmarks. +This means the key for `hood.comparing` will be `Comparing` with the capitalization. + +These extra parameters are necessary for the `CI` integration: + - **token**: The `Github` access token. + - **repositoryOwner**: The repository owner. + - **repositoryName**: The repository name. + - **pullRequestSha**: The sha for the Pull Request. The environment variable `TRAVIS_PULL_REQUEST_SHA` on Travis CI. + - **pullRequestNumber**: The number of the Pull Request. The environment variable `TRAVIS_PULL_REQUEST` on Travis CI. + - **statusTargetUrl**: The URL to the CI job. The environment variable `TRAVIS_JOB_WEB_URL` on Travis CI. Optional. + +***Note***: Currently, **Hood** only supports `CSV` and `JSON` based benchmarks with cross comparison available. +***Note 2***: If the `CI` integration is not available because one of the requested fields above is not defined, + the task `compareBenchmarksCI` will be executed in the same way as `compareBenchmarks`. + +### Send output to a file + +The task can send the result to a file with the following parameters: + - **outputToFile**: Sends the output to a file. By default: `false`. + - **outputPath**: The path to the output file. By default: `./hood/comparison`. + - **outputFormat**: The output file format; we support two formats `MD` and `JSON`. By default: `MD`. + +***Note***: To print a `JSON` output file, all the benchmarks must be in `JSON` format. `CSV` benchmarks will be ignored. + +### Configuration example + +```groovy +compareBenchmarksCI { + previousBenchmarkPath = file("$rootDir/hood_master/build/reports/master_benchmark.json") + currentBenchmarkPath = [file("$rootDir/build/reports/hood_benchmark.json")] + outputToFile = true + outputFormat = "json" + benchmarkThreshold = ["Parsing": 250.00, "Decodingfromraw": 250.00] + token = System.getenv("GITHUB_ACCESS_TOKEN") + repositoryOwner = "47deg" + repositoryName = "hood" + pullRequestSha = System.getenv("TRAVIS_PULL_REQUEST_SHA") + pullRequestNumber = (System.getenv("TRAVIS_PULL_REQUEST") != "false") ? System.getenv("TRAVIS_PULL_REQUEST")?.toInteger() : -1 + statusTargetUrl = System.getenv("TRAVIS_JOB_WEB_URL") ? URI.create(System.getenv("TRAVIS_JOB_WEB_URL")) : null } +``` -let impureSquare = impureAdd(x: 2, y: 5) * impureAdd(x: 2, y: 5) +```kotlin +tasks.compareBenchmarksCI { + previousBenchmarkPath = file("$rootDir/hood_master/build/reports/master_benchmark.json") + currentBenchmarkPath = listOf(file("$rootDir/build/reports/hood_benchmark.json")) + outputToFile = true + outputFormat = "json" + benchmarkThreshold = mapOf("Parsing" to 250.00, "Decodingfromraw" to 250.00) + token = System.getenv("GITHUB_ACCESS_TOKEN") + repositoryOwner = "47deg" + repositoryName = "hood" + pullRequestSha = System.getenv("TRAVIS_PULL_REQUEST_SHA") + pullRequestNumber = + if (System.getenv("TRAVIS_PULL_REQUEST") != "false") System.getenv("TRAVIS_PULL_REQUEST").toInt() else -1 + statusTargetUrl = (System.getenv("TRAVIS_JOB_WEB_URL") ?: URI.create(System.getenv("TRAVIS_JOB_WEB_URL"))) as URI? +} ``` diff --git a/docs/docs/README.md b/docs/docs/README.md index 07b90f1..80afb46 100644 --- a/docs/docs/README.md +++ b/docs/docs/README.md @@ -3,70 +3,99 @@ layout: docs title: Quick Start permalink: docs/ --- +# Hood -# Quick Start h1 -Lorem ipsum dolor sit amet, consectetur adipisicing elit. Quae sapiente quidem culpa sit ab ipsam alias repellat sequi. Quae tempora nulla consequuntur quibusdam aut cupiditate eum ratione, voluptatum error repellat. +**Hood** is a `Gradle` plugin to compare benchmarks and set the result as a `Github` status for a `Pull Request`. +**Hood** is built on [Arrow](https://arrow-kt.io/), a Functional companion to Kotlin's Standard Library. -## Titlte h2 -Lorem ipsum dolor sit amet, consectetur adipisicing elit. Quae sapiente quidem culpa sit ab ipsam alias repellat sequi. Quae tempora nulla consequuntur quibusdam aut cupiditate eum ratione, voluptatum error repellat. +## Tasks -### Titlte h3 -Lorem ipsum dolor sit amet, consectetur adipisicing elit. Quae sapiente quidem culpa sit ab ipsam alias repellat sequi. Quae tempora nulla consequuntur quibusdam aut cupiditate eum ratione, voluptatum error repellat. +**Hood** has two tasks to compare benchmarks: + - `compareBenchmarks`: Compare two or more benchmarks and print the result. + - `compareBenchmarksCI`: Compare two or more benchmarks and upload a `Github` status for a `Pull Request`. + +and an extra task `uploadBenchmarks` to upload Benchmarks files and keep them up to date. -#### Titlte h4 -Lorem ipsum dolor sit amet, consectetur adipisicing elit. Quae sapiente quidem culpa sit ab ipsam alias repellat sequi. Quae tempora nulla consequuntur quibusdam aut cupiditate eum ratione, voluptatum error repellat. +## Adding the Hood dependency -##### Titlte h5 -Lorem ipsum dolor sit amet, consectetur adipisicing elit. Quae sapiente quidem culpa sit ab ipsam alias repellat sequi. Quae tempora nulla consequuntur quibusdam aut cupiditate eum ratione, voluptatum error repellat. +To add the Hood plugin dependency on Gradle, you can use: -###### Titlte h6 -Lorem ipsum dolor sit amet, consectetur adipisicing elit. Quae sapiente quidem culpa sit ab ipsam alias repellat sequi. Quae tempora nulla consequuntur quibusdam aut cupiditate eum ratione, voluptatum error repellat. +### Declarative syntax(especially recommended for the Kotlin DSL) -* Lorem ipsum dolor sit amet, consectetur adipisicing elit -* Lorem ipsum dolor sit amet, consectetur -* Lorem ipsum dolor sit amet, consectetur adipisicing elit quae sapiente +```groovy +plugins { + id "com.47deg.hood" version "0.8.0" +} +``` -# Inline text elements -* Lorem ipsum dolor sit amet, *consectetur* adipisicing _elitquae_ sapiente -* Lorem ipsum dolor sit amet, **consectetur** adipisicing __elitquae__ sapiente -* Lorem ipsum dolor sit amet, **consectetur adipisicing _elitquae_** sapiente -* Lorem ipsum dolor sit amet, consectetur ~~adipisicing elitquae~~ sapiente +```kotlin +plugins { + id("com.47deg.hood") version "0.8.0" +} +``` -# Code -Lorem `ipsum dolor` sit amet, consectetur `adipisicing elitquae` sapiente +Don't forget to add the `pluginManagement` block at the top of your `settings.gradle/.kts` if you are not able to find it: -``` -repositories { - maven { url = uri("https://dl.bintray.com/47deg/helios") } +```groovy +pluginManagement { + repositories { + maven { url "https://dl.bintray.com/47deg/hood" } + gradlePluginPortal() + } } +``` -dependencies { - compile "com.47deg:helios-core:0.2.0" - compile "com.47deg:helios-parser:0.2.0" - compile "com.47deg:helios-optics:0.2.0" - kapt "com.47deg:helios-meta:0.2.0" - kapt "com.47deg:helios-dsl-meta:0.2.0" +```kotlin +pluginManagement { + repositories { + maven("https://dl.bintray.com/47deg/hood") + gradlePluginPortal() + } } ``` +### Imperative syntax + +To use plugin through imperative syntax, you need to first add the dependency on the `buildscript`: + +```groovy +buildscript { + repositories { + maven { url "https://dl.bintray.com/47deg/hood" } + } + + dependencies { + classpath "com.47deg:hood:0.8.0" + } +} ``` -val jsonStr = -"""{ - "name": "Simon", - "age": 30 - }""" - -val jsonFromString : Json = - Json.parseFromString(jsonStr).getOrHandle { - println("Failed creating the Json ${it.localizedMessage}, creating an empty one") - JsString("") + +```kotlin +buildscript { + repositories { + maven("https://dl.bintray.com/47deg/hood") } -val personOrError: Either = Person.decoder().decode(jsonFromString) + dependencies { + classpath("com.47deg:hood:0.8.0") + } +} +``` + +and then you will be able to add it with `apply`: + +```groovy +apply plugin: "com.47deg.hood" +``` -personOrError.fold({ - "Something went wrong during decoding: $it" -}, { - "Successfully decode the json: $it" -}) +```kotlin +apply(plugin = "com.47deg.hood") ``` + +## The visualizer companion + +**Hood** has a companion called [Robeen](https://github.com/47deg/robeen) to show the benchmark comparison in a visual way. + +## Projects using Hood + + - [Helios](https://47deg.github.io/helios/) diff --git a/docs/docs/benchmarks/README.md b/docs/docs/benchmarks/README.md index e9f0ff9..b1259e8 100644 --- a/docs/docs/benchmarks/README.md +++ b/docs/docs/benchmarks/README.md @@ -5,27 +5,52 @@ permalink: docs/benchmarks/ --- -# Benchmarks h1 -## Compile h2 -### Compile h3 -#### Compile h4 -##### Compile h5 -###### Compile h6 - -Lorem ipsum dolor sit amet, [consectetur adipiscing](#) elit. Nam lacinia auctor nunc, in commodo ante faucibus ut. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Nunc vel neque ut nisi sodales aliquet. Ut nunc velit, feugiat et elit eget, lobortis sodales est. Integer turpis tellus, semper nec neque ut, elementum mollis metus. Ut `faucibus lorem` felis porta lorem vehicula accumsan. Duis sed tempor enim, ac porttitor mi. Maecenas non sollicitudin quam. Nam mollis scelerisque sapien eu viverra. Etiam tempor bibendum eros, a faucibus tellus ullamcorper a. Aliquam id lorem est. Vivamus in ipsum condimentum, vulputate nisl et, rutrum tellus. - -- item 1 -- item 2 -- item 3 -- item 4 - - - -```swift -func impureAdd(x: Int, y: Int) -> Int { - print("Received (\(x), \(y))") - return x + y +# Compare Benchmarks + +The **Hood** basic task allows you to compare a set of benchmarks with the reference one, + printing the result to the console and failing if one of the benchmarks has downgraded values. + +The `compareBenchmarks` tasks have the following parameters: + - **previousBenchmarkPath**: File with previous or master benchmark location. By default: `master.csv`. + - **currentBenchmarkPath**: List of files with current or pull request benchmark location. By default: an `empty list`. + - **keyColumnName**: Column name to distinguish each benchmark on the comparison. By default: `Benchmark`. + - **compareColumnName**: Column name of the column to compare (the values must to be a `Double`). By default: `Score`. + - **thresholdColumnName**: Column name to get the threshold per benchmark. By default: `Score Error (99.9%)`. + - **generalThreshold**: Common threshold to all benchmarks overriding the value coming from `thresholdColumnName`. Optional. + - **benchmarkThreshold**: `Map` with a custom threshold per benchmark key overriding the value coming from `thresholdColumnName` or `generalThreshold`. Optional. + - **include**: Regular expression to include only the benchmarks with a matching key. Optional. + - **exclude**: Regular expression to exclude the benchmarks using its key. Optional. + +The `include/exclude` feature and `benchmarkThreshold` param use the cleaned key from benchmarks. +This means the key for `hood.comparing` will be `Comparing` with the capitalization. + +***Note***: Currently **Hood** only supports `CSV` and `JSON` based benchmarks with cross comparison available. + +## Send output to a file + +The task can send the result to a file with the following parameters: + - **outputToFile**: Sends the output to a file. By default: `false`. + - **outputPath**: The path to the output file. By default: `./hood/comparison`. + - **outputFormat**: The output file format; we support two formats: `MD` and `JSON`. By default: `MD`. + +***Note***: To print a `JSON` output file, all the benchmarks must be in `JSON` format. `CSV` benchmarks will be ignored. + +## Configuration example + +```groovy +compareBenchmarks { + previousBenchmarkPath = file("$rootDir/benchmarks/master_benchmark.json") + currentBenchmarkPath = [file("$rootDir/build/reports/hood_benchmark.json")] + outputFormat = "json" + benchmarkThreshold = ["Parsing": 500.00] } +``` -let impureSquare = impureAdd(x: 2, y: 5) * impureAdd(x: 2, y: 5) +```kotlin +tasks.compareBenchmarks { + previousBenchmarkPath = file("$rootDir/benchmarks/master_benchmark.json") + currentBenchmarkPath = listOf(file("$rootDir/build/reports/hood_benchmark.json")) + outputFormat = "json" + benchmarkThreshold = mapOf("Parsing" to 500.00) +} ``` diff --git a/docs/docs/benchmarks/more-benchmarks.md b/docs/docs/benchmarks/more-benchmarks.md deleted file mode 100644 index 3d68c5a..0000000 --- a/docs/docs/benchmarks/more-benchmarks.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -layout: docs -title: More benchmarks -permalink: docs/more-benchmarks/ - ---- - -# Benchmarks 2 h1 -## Compile h2 -### Compile h3 -#### Compile h4 -##### Compile h5 -###### Compile h6 - -Lorem ipsum dolor sit amet, [consectetur adipiscing](#) elit. Nam lacinia auctor nunc, in commodo ante faucibus ut. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Nunc vel neque ut nisi sodales aliquet. Ut nunc velit, feugiat et elit eget, lobortis sodales est. Integer turpis tellus, semper nec neque ut, elementum mollis metus. Ut `faucibus lorem` felis porta lorem vehicula accumsan. Duis sed tempor enim, ac porttitor mi. Maecenas non sollicitudin quam. Nam mollis scelerisque sapien eu viverra. Etiam tempor bibendum eros, a faucibus tellus ullamcorper a. Aliquam id lorem est. Vivamus in ipsum condimentum, vulputate nisl et, rutrum tellus. - -- item 1 -- item 2 -- item 3 -- item 4 - - - -```swift -func impureAdd(x: Int, y: Int) -> Int { - print("Received (\(x), \(y))") - return x + y -} - -let impureSquare = impureAdd(x: 2, y: 5) * impureAdd(x: 2, y: 5) -``` diff --git a/docs/docs/upload/README.md b/docs/docs/upload/README.md index 6dffa16..dd71429 100644 --- a/docs/docs/upload/README.md +++ b/docs/docs/upload/README.md @@ -5,25 +5,38 @@ permalink: docs/upload/ --- -# Upload h1 -## Integration h2 -### Integration h3 -#### Integration h4 -##### Integration h5 -###### Integration h6 +## Upload benchmarks -Lorem ipsum dolor sit amet, [consectetur adipiscing](#) elit. Nam lacinia auctor nunc, in commodo ante faucibus ut. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Nunc vel neque ut nisi sodales aliquet. Ut nunc velit, feugiat et elit eget, lobortis sodales est. Integer turpis tellus, semper nec neque ut, elementum mollis metus. Ut `faucibus lorem` felis porta lorem vehicula accumsan. Duis sed tempor enim, ac porttitor mi. Maecenas non sollicitudin quam. Nam mollis scelerisque sapien eu viverra. Etiam tempor bibendum eros, a faucibus tellus ullamcorper a. Aliquam id lorem est. Vivamus in ipsum condimentum, vulputate nisl et, rutrum tellus. +**Hood** allows you to automatically upload the benchmarks +you want to keep updated in your code thought commits during the CI. -- item 1 -- item 2 -- item 3 -- item 4 - -```swift -func impureAdd(x: Int, y: Int) -> Int { - print("Received (\(x), \(y))") - return x + y +The task `uploadBenchmarks` has the following parameters: + - **benchmarkFiles**: The list of benchmark files you want to upload. By default: an `empty list`. + - **uploadDirectory**: The path for the folder where you want to keep them, from project root directory. By default: `benchmarks`. + - **commitMessage**: The message for the task commit uploading the benchmark. By default: `Upload benchmark`. + - **token**: the `Github` access token. + - **repositoryOwner**: The repository owner. + - **repositoryName**: The repository name. + - **branch**: The branch where you want to upload those benchmarks. By default: `master`. + + ### Gradle configuration example + + ```groovy +uploadBenchmarks { + benchmarkFiles = [file("$rootDir/build/reports/master_benchmark.json"), file("$rootDir/build/reports/libraries_benchmark.json")] + token = System.getenv("GITHUB_ACCESS_TOKEN") + repositoryOwner = "47deg" + repositoryName = "hood" + commitMessage = "[ci skip] - Upload benchmark" } +``` -let impureSquare = impureAdd(x: 2, y: 5) * impureAdd(x: 2, y: 5) +```kotlin +tasks.uploadBenchmarks { + benchmarkFiles = listOf(file("$rootDir/build/reports/master_benchmark.json"), file("$rootDir/build/reports/libraries_benchmark.json")) + token = System.getenv("GITHUB_ACCESS_TOKEN") + repositoryOwner = "47deg" + repositoryName = "hood" + commitMessage = "[ci skip] - Upload benchmark" +} ``` diff --git a/gradle.properties b/gradle.properties index dbc3a87..b6b455d 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,6 +1,6 @@ # Package definitions group=com.47deg -version=0.7.1 +version=0.8.0 # Gradle options org.gradle.warning.mode=all diff --git a/src/main/kotlin/com/fortysevendeg/hood/Comparator.kt b/src/main/kotlin/com/fortysevendeg/hood/Comparator.kt index 1de7b06..d262296 100644 --- a/src/main/kotlin/com/fortysevendeg/hood/Comparator.kt +++ b/src/main/kotlin/com/fortysevendeg/hood/Comparator.kt @@ -148,25 +148,22 @@ object Comparator { val currentBenchmarksAfterFilter: Map> = currentBenchmarks.mapValues { it.value.filterBenchmarkIE(includeRegex, excludeRegex) } - //All the keys on master must to be on the branch benchmarks - val isConsistent = - previousBenchmarksAfterFilter.second.map { it.getKey() }.forAll { prevKey -> - currentBenchmarks.values.toList().forAll { list -> - list.map { it.getKey() }.exists { it == prevKey } - } + val currentKeys = currentBenchmarksAfterFilter.values.toList().flatMap { list -> list.map { it.getKey() } } + + val inconsistencies = previousBenchmarksAfterFilter.second.map { it.getKey() }.minus(currentKeys) + + println("[WARN] Found inconsistencies between benchmarks, only common keys will be compared.\n Inconsistencies: $inconsistencies") + + previousBenchmarksAfterFilter.second.flatMap { prev -> + val threshold = selectThreshold(prev, maybeGeneralThreshold, maybeBenchmarkThreshold) + val previousModified = + prev.withName(previousBenchmarksAfterFilter.first).withThreshold(threshold) + + getCompareResults(currentBenchmarksAfterFilter, prev, threshold).map { + buildBenchmarkComparison(prev.getKey(), previousModified, it) } + }.right() - if (isConsistent) - previousBenchmarksAfterFilter.second.flatMap { prev -> - val threshold = selectThreshold(prev, maybeGeneralThreshold, maybeBenchmarkThreshold) - val previousModified = - prev.withName(previousBenchmarksAfterFilter.first).withThreshold(threshold) - - getCompareResults(currentBenchmarksAfterFilter, prev, threshold).map { - buildBenchmarkComparison(prev.getKey(), previousModified, it) - } - }.right() - else BenchmarkComparisonError(BenchmarkInconsistencyError).left() }.fix().handleError { BenchmarkComparisonError(it).left() } diff --git a/src/main/kotlin/com/fortysevendeg/hood/HoodPlugin.kt b/src/main/kotlin/com/fortysevendeg/hood/HoodPlugin.kt index 35ef7ce..862a28a 100644 --- a/src/main/kotlin/com/fortysevendeg/hood/HoodPlugin.kt +++ b/src/main/kotlin/com/fortysevendeg/hood/HoodPlugin.kt @@ -1,8 +1,8 @@ package com.fortysevendeg.hood -import com.fortysevendeg.hood.tasks.CompareBenchmark -import com.fortysevendeg.hood.tasks.CompareBenchmarkCI -import com.fortysevendeg.hood.tasks.UploadBenchmark +import com.fortysevendeg.hood.tasks.CompareBenchmarks +import com.fortysevendeg.hood.tasks.CompareBenchmarksCI +import com.fortysevendeg.hood.tasks.UploadBenchmarks import org.gradle.api.Plugin import org.gradle.api.Project @@ -11,9 +11,9 @@ open class HoodPlugin : Plugin { override fun apply(project: Project): Unit = project.run { group = Settings.group - tasks.register("compareBenchmark", CompareBenchmark::class.java) - tasks.register("compareBenchmarkCI", CompareBenchmarkCI::class.java) - tasks.register("uploadBenchmark", UploadBenchmark::class.java) + tasks.register("compareBenchmarks", CompareBenchmarks::class.java) + tasks.register("compareBenchmarksCI", CompareBenchmarksCI::class.java) + tasks.register("uploadBenchmarks", UploadBenchmarks::class.java) } object Settings { diff --git a/src/main/kotlin/com/fortysevendeg/hood/tasks/CompareBenchmark.kt b/src/main/kotlin/com/fortysevendeg/hood/tasks/CompareBenchmarks.kt similarity index 96% rename from src/main/kotlin/com/fortysevendeg/hood/tasks/CompareBenchmark.kt rename to src/main/kotlin/com/fortysevendeg/hood/tasks/CompareBenchmarks.kt index 50c5943..566d59a 100644 --- a/src/main/kotlin/com/fortysevendeg/hood/tasks/CompareBenchmark.kt +++ b/src/main/kotlin/com/fortysevendeg/hood/tasks/CompareBenchmarks.kt @@ -5,7 +5,7 @@ import org.gradle.api.DefaultTask import org.gradle.api.tasks.* import java.io.File -open class CompareBenchmark : DefaultTask() { +open class CompareBenchmarks : DefaultTask() { //Benchmarks paths @get:InputFile @@ -50,7 +50,7 @@ open class CompareBenchmark : DefaultTask() { var exclude: String? = project.objects.property(String::class.java).orNull @TaskAction - fun compareBenchmark(): Unit = + fun compareBenchmarks(): Unit = HoodComparison.compare( previousBenchmarkPath, currentBenchmarkPath, diff --git a/src/main/kotlin/com/fortysevendeg/hood/tasks/CompareBenchmarkCI.kt b/src/main/kotlin/com/fortysevendeg/hood/tasks/CompareBenchmarksCI.kt similarity index 97% rename from src/main/kotlin/com/fortysevendeg/hood/tasks/CompareBenchmarkCI.kt rename to src/main/kotlin/com/fortysevendeg/hood/tasks/CompareBenchmarksCI.kt index 7865e9d..d33ab48 100644 --- a/src/main/kotlin/com/fortysevendeg/hood/tasks/CompareBenchmarkCI.kt +++ b/src/main/kotlin/com/fortysevendeg/hood/tasks/CompareBenchmarksCI.kt @@ -12,7 +12,7 @@ import org.gradle.api.tasks.* import java.io.File import java.net.URI -open class CompareBenchmarkCI : DefaultTask() { +open class CompareBenchmarksCI : DefaultTask() { //Benchmarks paths @get:InputFile @@ -77,7 +77,7 @@ open class CompareBenchmarkCI : DefaultTask() { var statusTargetUrl: URI? = project.objects.property(URI::class.java).orNull @TaskAction - fun compareBenchmarkCI() = + fun compareBenchmarksCI() = Option.applicative().map( token.toOption(), repositoryOwner.toOption(), diff --git a/src/main/kotlin/com/fortysevendeg/hood/tasks/UploadBenchmark.kt b/src/main/kotlin/com/fortysevendeg/hood/tasks/UploadBenchmarks.kt similarity index 97% rename from src/main/kotlin/com/fortysevendeg/hood/tasks/UploadBenchmark.kt rename to src/main/kotlin/com/fortysevendeg/hood/tasks/UploadBenchmarks.kt index 227c36a..9c4280e 100644 --- a/src/main/kotlin/com/fortysevendeg/hood/tasks/UploadBenchmark.kt +++ b/src/main/kotlin/com/fortysevendeg/hood/tasks/UploadBenchmarks.kt @@ -20,7 +20,7 @@ import org.gradle.api.tasks.InputFiles import org.gradle.api.tasks.TaskAction import java.io.File -open class UploadBenchmark : DefaultTask() { +open class UploadBenchmarks : DefaultTask() { @get:InputFiles var benchmarkFiles: List = @@ -69,7 +69,7 @@ open class UploadBenchmark : DefaultTask() { } @TaskAction - fun uploadBenchmark(): Unit = + fun uploadBenchmarks(): Unit = repositoryOwner.toOption().map2(repositoryName.toOption()) { (owner, repo) -> fx { val (ghToken: String) = token.toOption().getOrRaiseError { GradleException("Error getting Github token") }