Performance tests interpretation

[masaeedu]

I'm trying to run the perf tests on the following setup:

• Windows 10
• Protractor 2.3.0 installed globally (Protractor 2.4.0+ causes this error)
• chromedriver_2.19
• selenium-server-standalone-2.47.1

Here's the output for a sample test:

BENCHMARK bufferCount Rx2
Description:
- forceGc: false
- iterations: 1000
- regressionSlopeMetric: scriptTime
- sampleSize: 20
- userAgent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36
Metrics:
- gcAmount: gc amount in kbytes
- gcTime: gc time in ms
- majorGcTime: time of major gcs in ms
- pureScriptTime: script execution time in ms, without gc nor render
- renderTime: render time in ms
- scriptTime: script execution time in ms, including gc and render

          gcAmount |             gcTime |        majorGcTime |     pureScriptTime |         renderTime |         scriptTime
------------------ | ------------------ | ------------------ | ------------------ | ------------------ | ------------------
              0.00 |               0.00 |               0.00 |               0.00 |               0.22 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.09 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.06 |               0.00
            201.75 |               0.75 |               0.00 |               0.00 |               0.16 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.10 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.14 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.06 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.18 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.14 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.20 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.05 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.05 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.17 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.06 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.24 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.05 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.12 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.14 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.09 |               0.00
            643.57 |               1.77 |               0.00 |               0.00 |               0.08 |               0.00
================== | ================== | ================== | ================== | ================== | ==================
       42.27+-342% |         0.13+-325% |         0.00+-299% |               0.00 |          0.12+-48% |               0.00

I'm trying to understand what this means. For example, the scriptTime, which is supposed to be the total of the pure, gc and render times (if I'm understanding correctly), is always 0, even in cases where the sum of those three values isn't zero.

Also, the pureScriptTime is zero in all tests, which I'm assuming means the test is completing in negligible time, or that something is wrong. Is there a problem with my setup?

[masaeedu]

According to the linked issue, there was a change in the selenium webdriver API that required a bunch of code in Angular 2 to be changed. This probably affected benchpress as well, which means it needs to be updated in order to support protractor 2.5.1, which this package depends on.

[kwonoj]

I remember similar number was seen and benchpress update resolves those. Not sure if it's exactly same issue by #1230 updates it but number doesn't changes.

[benlesh]

cc @jeffbcross @tbosch looks like we're facing benchpress issues again.

[masaeedu]

Is anyone else seeing the same thing when running perf tests?

[benlesh]

@jeffbcross I think benchpress is broken again.

[benlesh]

Closing this as stale.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.