Skip to content

Commit

Permalink
Run performance test for metrics
Browse files Browse the repository at this point in the history
Before this patch, performance tests ran together with unit and
integration with `--coverage` flag. Coverage analysis cropped the
result of performance tests to 10-15 times. For metrics integration
it resulted in timeout errors and drop of performance which is not
reproduces with coverage disabled. Moreover, before this patch log
capture was disabled and performance tests did not displayed any
results after run.

After this patch, `make -C build coverage` will run lightweight
version of performance test. `make -C build performance` will run real
performance tests.

This path also reworks current performance test. It adds new cases to
compare module performance with or without statistics, statistic
wrappers and compare different metrics drivers and reports new info:
average call time and max call time.

Closes #233, follows up #224
  • Loading branch information
DifferentialOrange committed Dec 17, 2021
1 parent 24096b3 commit 3332a01
Show file tree
Hide file tree
Showing 5 changed files with 417 additions and 167 deletions.
9 changes: 9 additions & 0 deletions .github/workflows/test_on_push.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,11 @@ jobs:
tarantool-version: ["1.10.6", "1.10", "2.2", "2.3", "2.4", "2.5", "2.6", "2.7", "2.8"]
metrics-version: [""]
remove-merger: [false]
perf-test: [false]
include:
- tarantool-version: "1.10"
metrics-version: "0.12.0"
perf-test: true
- tarantool-version: "2.7"
remove-merger: true
- tarantool-version: "2.8"
Expand All @@ -26,6 +30,7 @@ jobs:
- tarantool-version: "2.8"
coveralls: true
metrics-version: "0.12.0"
perf-test: true
fail-fast: false
runs-on: [ubuntu-latest]
steps:
Expand Down Expand Up @@ -72,6 +77,10 @@ jobs:
- name: Run tests and code coverage analysis
run: make -C build coverage

- name: Run performance tests
run: make -C build performance
if: ${{ matrix.perf-test }}

- name: Send code coverage to coveralls.io
run: make -C build coveralls
if: ${{ matrix.coveralls }}
Expand Down
8 changes: 8 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,14 @@ add_custom_target(luatest
COMMENT "Run regression tests"
)

set(PERFORMANCE_TESTS_SUBDIR "test/performance")

add_custom_target(performance
COMMAND PERF_MODE_ON=true ${LUATEST} -v -c ${PERFORMANCE_TESTS_SUBDIR}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
COMMENT "Run performance tests"
)

add_custom_target(coverage
COMMAND ${LUACOV} ${PROJECT_SOURCE_DIR} && grep -A999 '^Summary' ${CODE_COVERAGE_REPORT}
DEPENDS ${CODE_COVERAGE_STATS}
Expand Down
1 change: 1 addition & 0 deletions crud/stats/metrics_registry.lua
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ local metric_name = {

local LATENCY_QUANTILE = 0.99

-- Increasing tolerance threshold affects performance.
local DEFAULT_QUANTILES = {
[LATENCY_QUANTILE] = 1e-3,
}
Expand Down
Loading

0 comments on commit 3332a01

Please sign in to comment.