Bench5 #33

atharva-2001 · 2023-04-26T12:33:59Z

No description provided.

…ests workflow

atharva-2001 · 2023-04-26T12:44:08Z

*beep* *bop*
Hi human,
I ran benchmarks as you asked comparing master (effacfb) and the latest commit (ff3c7c2).
Here are the logs produced by ASV.
Results can also be downloaded as artifacts here.

· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.10
·· Building ff3c7c25 for conda-py3.10
·· Installing ff3c7c25 into conda-py3.10
·· Error running /home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/bin/python /home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py discover /home/runner/work/tardis/tardis/benchmarks /tmp/tmpozn3i80d/result.json (exit status 1)
 STDOUT -------->
 
 STDERR -------->
 
Iterations:                                                                                0/? [00:00<?, ?it/s]
 
Packets:                                                                                   0/? [00:00<?, ?it/s]�[ATraceback (most recent call last):
   File "/home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py", line 1435, in <module>
     main()
   File "/home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py", line 1428, in main
     commands[mode](args)
   File "/home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py", line 1103, in main_discover
     list_benchmarks(benchmark_dir, fp)
   File "/home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py", line 1088, in list_benchmarks
     for benchmark in disc_benchmarks(root):
   File "/home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py", line 985, in disc_benchmarks
     for module in disc_modules(root_name, ignore_import_errors=ignore_import_errors):
   File "/home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py", line 967, in disc_modules
     for item in disc_modules(name, ignore_import_errors=ignore_import_errors):
   File "/home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py", line 950, in disc_modules
     module = import_module(module_name)
   File "/home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/lib/python3.8/importlib/__init__.py", line 127, in import_module
     return _bootstrap._gcd_import(name[level:], package, level)
   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
   File "<frozen importlib._bootstrap_external>", line 843, in exec_module
   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
   File "/home/runner/work/tardis/tardis/benchmarks/benchmark_run_tardis.py", line 3, in <module>
     from tardis.io.config_reader import Configuration
   File "/home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/lib/python3.8/site-packages/tardis/__init__.py", line 26, in <module>
     from tardis.io.util import yaml_load_file as yaml_load
   File "/home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/lib/python3.8/site-packages/tardis/io/__init__.py", line 6, in <module>
     from tardis.io.model_reader import (
   File "/home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/lib/python3.8/site-packages/tardis/io/model_reader.py", line 14, in <module>
     import h5py
   File "/home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/lib/python3.8/site-packages/h5py/__init__.py", line 33, in <module>
     from . import version
   File "/home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/lib/python3.8/site-packages/h5py/version.py", line 15, in <module>
     from . import h5 as _h5
   File "h5py/h5.pyx", line 1, in init h5py.h5
 ImportError: /home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/lib/python3.8/site-packages/h5py/defs.cpython-38-x86_64-linux-gnu.so: undefined symbol: H5Pset_fapl_ros3
 
Iterations:                                                                                0/? [00:00<?, ?it/s]
 
Packets:                                                                                   0/? [00:00<?, ?it/s

·· Failed: trying different commit/environment
·· Uninstalling from conda-py3.10
·· Building effacfb7 for conda-py3.10
·· Installing effacfb7 into conda-py3.10
·· Error running /home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/bin/python /home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py discover /home/runner/work/tardis/tardis/benchmarks /tmp/tmpgwq5vlnt/result.json (exit status 1)
 STDOUT -------->
 
 STDERR -------->
 
Iterations:                                                                                0/? [00:00<?, ?it/s]
 
Packets:                                                                                   0/? [00:00<?, ?it/s]�[ATraceback (most recent call last):
   File "/home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py", line 1435, in <module>
     main()
   File "/home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py", line 1428, in main
     commands[mode](args)
   File "/home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py", line 1103, in main_discover
     list_benchmarks(benchmark_dir, fp)
   File "/home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py", line 1088, in list_benchmarks
     for benchmark in disc_benchmarks(root):
   File "/home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py", line 985, in disc_benchmarks
     for module in disc_modules(root_name, ignore_import_errors=ignore_import_errors):
   File "/home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py", line 967, in disc_modules
     for item in disc_modules(name, ignore_import_errors=ignore_import_errors):
   File "/home/runner/.local/lib/python3.10/site-packages/asv/benchmark.py", line 950, in disc_modules
     module = import_module(module_name)
   File "/home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/lib/python3.8/importlib/__init__.py", line 127, in import_module
     return _bootstrap._gcd_import(name[level:], package, level)
   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
   File "<frozen importlib._bootstrap_external>", line 843, in exec_module
   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
   File "/home/runner/work/tardis/tardis/benchmarks/benchmark_run_tardis.py", line 3, in <module>
     from tardis.io.config_reader import Configuration
   File "/home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/lib/python3.8/site-packages/tardis/__init__.py", line 26, in <module>
     from tardis.io.util import yaml_load_file as yaml_load
   File "/home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/lib/python3.8/site-packages/tardis/io/__init__.py", line 6, in <module>
     from tardis.io.model_reader import (
   File "/home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/lib/python3.8/site-packages/tardis/io/model_reader.py", line 14, in <module>
     import h5py
   File "/home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/lib/python3.8/site-packages/h5py/__init__.py", line 33, in <module>
     from . import version
   File "/home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/lib/python3.8/site-packages/h5py/version.py", line 15, in <module>
     from . import h5 as _h5
   File "h5py/h5.pyx", line 1, in init h5py.h5
 ImportError: /home/runner/work/tardis/tardis/.asv/env/06715dc6ea55f2b02b6ebc21bca7e34a/lib/python3.8/site-packages/h5py/defs.cpython-38-x86_64-linux-gnu.so: undefined symbol: H5Pset_fapl_ros3
 
Iterations:                                                                                0/? [00:00<?, ?it/s]
 
Packets:                                                                                   0/? [00:00<?, ?it/s

·· Failed to build the project and import the benchmark suite.

atharva-2001 · 2023-04-26T13:00:31Z

*beep* *bop*
Hi human,
I ran benchmarks as you asked comparing master (effacfb) and the latest commit (50725c9).
Here are the logs produced by ASV.
Results can also be downloaded as artifacts here.

· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.10
·· Building 50725c9c for conda-py3.10
·· Installing 50725c9c into conda-py3.10
· Running 2 total benchmarks (2 commits * 1 environments * 1 benchmarks)
[  0.00%] · For tardis commit effacfb7 (round 1/2):
[  0.00%] ·· Building for conda-py3.10
[  0.00%] ·· Benchmarking conda-py3.10
[ 25.00%] ··· Running (benchmark_run_tardis.Benchmarkruntardis.time_run_tardis--).
[ 25.00%] · For tardis commit 50725c9c (round 1/2):
[ 25.00%] ·· Building for conda-py3.10
[ 25.00%] ·· Benchmarking conda-py3.10
[ 50.00%] ··· Running (benchmark_run_tardis.Benchmarkruntardis.time_run_tardis--).
[ 50.00%] · For tardis commit 50725c9c (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.10
[ 75.00%] ··· ...un_tardis.Benchmarkruntardis.time_run_tardis            1.01±0s
[ 75.00%] · For tardis commit effacfb7 (round 2/2):
[ 75.00%] ·· Building for conda-py3.10
[ 75.00%] ·· Benchmarking conda-py3.10
[100.00%] ··· ...un_tardis.Benchmarkruntardis.time_run_tardis         1.08±0.03s

BENCHMARKS NOT SIGNIFICANTLY CHANGED.

atharva-2001 · 2023-04-26T13:23:57Z

*beep* *bop*
Hi human,
I ran benchmarks as you asked comparing master (effacfb) and the latest commit (b8c73e3).
Here are the logs produced by ASV.
Results can also be downloaded as artifacts here.

· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.10
·· Building b8c73e37 for conda-py3.10
·· Installing b8c73e37 into conda-py3.10
· Running 2 total benchmarks (2 commits * 1 environments * 1 benchmarks)
[  0.00%] · For tardis commit effacfb7 (round 1/2):
[  0.00%] ·· Building for conda-py3.10
[  0.00%] ·· Benchmarking conda-py3.10
[ 25.00%] ··· Running (benchmark_run_tardis.Benchmarkruntardis.time_run_tardis--).
[ 25.00%] · For tardis commit b8c73e37 (round 1/2):
[ 25.00%] ·· Building for conda-py3.10
[ 25.00%] ·· Benchmarking conda-py3.10
[ 50.00%] ··· Running (benchmark_run_tardis.Benchmarkruntardis.time_run_tardis--).
[ 50.00%] · For tardis commit b8c73e37 (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.10
[ 75.00%] ··· ...un_tardis.Benchmarkruntardis.time_run_tardis         694±0.03ms
[ 75.00%] · For tardis commit effacfb7 (round 2/2):
[ 75.00%] ·· Building for conda-py3.10
[ 75.00%] ·· Benchmarking conda-py3.10
[100.00%] ··· ...un_tardis.Benchmarkruntardis.time_run_tardis            701±4ms

BENCHMARKS NOT SIGNIFICANTLY CHANGED.

atharva-2001 · 2023-04-27T05:01:04Z

*beep* *bop*
Hi human,
I ran benchmarks as you asked comparing master (effacfb) and the latest commit (4dba800).
Here are the logs produced by ASV.
Results can also be downloaded as artifacts here.

· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.10
·· Building 4dba8000 for conda-py3.10
·· Installing 4dba8000 into conda-py3.10
· Running 2 total benchmarks (2 commits * 1 environments * 1 benchmarks)
[  0.00%] · For tardis commit effacfb7 (round 1/2):
[  0.00%] ·· Building for conda-py3.10
[  0.00%] ·· Benchmarking conda-py3.10
[ 25.00%] ··· Running (benchmark_run_tardis.Benchmarkruntardis.time_run_tardis--).
[ 25.00%] · For tardis commit 4dba8000 (round 1/2):
[ 25.00%] ·· Building for conda-py3.10
[ 25.00%] ·· Benchmarking conda-py3.10
[ 50.00%] ··· Running (benchmark_run_tardis.Benchmarkruntardis.time_run_tardis--).
[ 50.00%] · For tardis commit 4dba8000 (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.10
[ 75.00%] ··· ...un_tardis.Benchmarkruntardis.time_run_tardis          974±0.1ms
[ 75.00%] · For tardis commit effacfb7 (round 2/2):
[ 75.00%] ·· Building for conda-py3.10
[ 75.00%] ·· Benchmarking conda-py3.10
[100.00%] ··· ...un_tardis.Benchmarkruntardis.time_run_tardis            969±3ms

BENCHMARKS NOT SIGNIFICANTLY CHANGED.

atharva-2001 · 2023-04-27T06:13:36Z

*beep* *bop*
Hi human,
I ran benchmarks as you asked comparing master (effacfb) and the latest commit (caf6595).
Here are the logs produced by ASV.
Results can also be downloaded as artifacts here.

· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.10
·· Building caf65951 for conda-py3.10
·· Installing caf65951 into conda-py3.10
· Running 2 total benchmarks (2 commits * 1 environments * 1 benchmarks)
[  0.00%] · For tardis commit effacfb7 (round 1/2):
[  0.00%] ·· Building for conda-py3.10
[  0.00%] ·· Benchmarking conda-py3.10
[ 25.00%] ··· Running (benchmark_run_tardis.Benchmarkruntardis.time_run_tardis--).
[ 25.00%] · For tardis commit caf65951 (round 1/2):
[ 25.00%] ·· Building for conda-py3.10
[ 25.00%] ·· Benchmarking conda-py3.10
[ 50.00%] ··· Running (benchmark_run_tardis.Benchmarkruntardis.time_run_tardis--).
[ 50.00%] · For tardis commit caf65951 (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.10
[ 75.00%] ··· ...un_tardis.Benchmarkruntardis.time_run_tardis            953±1ms
[ 75.00%] · For tardis commit effacfb7 (round 2/2):
[ 75.00%] ·· Building for conda-py3.10
[ 75.00%] ·· Benchmarking conda-py3.10
[100.00%] ··· ...un_tardis.Benchmarkruntardis.time_run_tardis           919±10ms

BENCHMARKS NOT SIGNIFICANTLY CHANGED.

atharva-2001 · 2023-04-28T10:43:18Z

*beep* *bop*
Hi human,
I ran benchmarks as you asked comparing master (effacfb) and the latest commit (2af1659).
Here are the logs produced by ASV.
Results can also be downloaded as artifacts here.

· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.10
·· Building 2af16594 for conda-py3.10
·· Installing 2af16594 into conda-py3.10
· Running 2 total benchmarks (2 commits * 1 environments * 1 benchmarks)
[  0.00%] · For tardis commit effacfb7 (round 1/2):
[  0.00%] ·· Building for conda-py3.10
[  0.00%] ·· Benchmarking conda-py3.10
[ 25.00%] ··· Running (benchmark_run_tardis.Benchmarkruntardis.time_run_tardis--).
[ 25.00%] · For tardis commit 2af16594 (round 1/2):
[ 25.00%] ·· Building for conda-py3.10
[ 25.00%] ·· Benchmarking conda-py3.10
[ 50.00%] ··· Running (benchmark_run_tardis.Benchmarkruntardis.time_run_tardis--).
[ 50.00%] · For tardis commit 2af16594 (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.10
[ 75.00%] ··· ...un_tardis.Benchmarkruntardis.time_run_tardis         1.06±0.02s
[ 75.00%] · For tardis commit effacfb7 (round 2/2):
[ 75.00%] ·· Building for conda-py3.10
[ 75.00%] ·· Benchmarking conda-py3.10
[100.00%] ··· ...un_tardis.Benchmarkruntardis.time_run_tardis         1.13±0.02s

BENCHMARKS NOT SIGNIFICANTLY CHANGED.

atharva-2001 · 2023-04-28T11:37:05Z

*beep* *bop*
Hi human,
I ran benchmarks as you asked comparing master (effacfb) and the latest commit (420692a).
Here are the logs produced by ASV.
Results can also be downloaded as artifacts here.

All benchmarks:

     before           after         ratio
   [effacfb7]       [420692aa]
        867±1ms        849±0.7ms     0.98  benchmark_run_tardis.Benchmarkruntardis.time_run_tardis

atharva-2001 · 2023-05-01T12:29:11Z

*beep* *bop*
Hi human,
I ran benchmarks as you asked comparing master (effacfb) and the latest commit (0a4d97a).
Here are the logs produced by ASV.
Results can also be downloaded as artifacts here.
Significantly changed benchmarks:

```diff

</details>

All benchmarks:
<details>
```diff

All benchmarks:

     before           after         ratio
   [effacfb7]       [0a4d97a6]
     1.04±0.01s          1.02±0s     0.99  benchmark_run_tardis.Benchmarkruntardis.time_run_tardis

atharva-2001 · 2023-05-01T12:57:39Z

*beep* *bop*
Hi human,
I ran benchmarks as you asked comparing master (effacfb) and the latest commit (9a11894).
Here are the logs produced by ASV.
Results can also be downloaded as artifacts here.
Significantly changed benchmarks:

All benchmarks:

All benchmarks:

     before           after         ratio
   [effacfb7]       [9a11894d]
        997±1ms        999±0.5ms     1.00  benchmark_run_tardis.Benchmarkruntardis.time_run_tardis

atharva-2001 added 30 commits March 8, 2023 18:55

Initial commit

93fb089

Comment old benchmarks temporarily

8904b1c

Add timeout

f8fb484

Change the tests workflow to run benchmarks

ac99c71

Replace ASV config file by template one, make changes there, modify t…

d19b17b

…ests workflow

Change the tests workflow back

f25d369

Change token

a2d20e2

reduce the number of packets

8942ba3

change config again

7d64f2e

Change config again

28eb478

Remove environment

8277f7b

Fix paths

02395be

Display comment

8e89893

Try PAT

ddc2de6

Bot token

aa31c29

asv continuous fails sporadically, continue-on-error temporarily

1b8530c

Rename asv directory to benchmarks

e333e42

Try running for 5 commits

fa3a44d

Change branch on my workflow

e3aa565

asv -> benchmarks

a4d1526

Clean up benchmarks workflow

140128d

Move publish to separate step

fdb38fb

do not show logs and progress bars

693fa34

Change branch in asv config file

66e0b7a

Fix commits in asv cont.

828e5b8

Set target directory in push-to-another-repo

c0bdfb9

Change trigger to pull request and push on master

04bc7dd

Delete commented permissions

3a41f33

Modify the comment, trigger via [bench] word

ec27009

Send comments via github actions

255ab0a

atharva-2001 added 9 commits April 19, 2023 19:19

change to fetch-depth:0

1242151

asv machine accept all questions

62db512

Only one os is sufficient

0ac99ae

Check if benchmarks failed without showing stderr

d6e9756

change destination repo for results

2e08f7c

pull request on any branch

d53d5aa

Prevent tests from running on forks

09a7a5f

ccache

434473f

token

ff3c7c2

atharva-2001 added the run-benchmarks label Apr 26, 2023

env problem

50725c9

atharva-2001 added 2 commits April 26, 2023 18:43

run agaain

2c9c79a

ccache not saved

b8c73e3

Run workflows

4dba800

save all asv data as artifact for testing

caf6595

atharva-2001 added 2 commits April 28, 2023 15:53

compare

a864018

checkkout v3

2af1659

bench more

420692a

Significantly changed benchmarks

0a4d97a

needs spaces

9a11894

atharva-2001 force-pushed the master branch from abde1a6 to f5cfd50 Compare September 7, 2023 11:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bench5 #33

Bench5 #33

atharva-2001 commented Apr 26, 2023

atharva-2001 commented Apr 26, 2023

atharva-2001 commented Apr 26, 2023

atharva-2001 commented Apr 26, 2023

atharva-2001 commented Apr 27, 2023

atharva-2001 commented Apr 27, 2023

atharva-2001 commented Apr 28, 2023

atharva-2001 commented Apr 28, 2023

atharva-2001 commented May 1, 2023

atharva-2001 commented May 1, 2023

Bench5 #33

Are you sure you want to change the base?

Bench5 #33

Conversation

atharva-2001 commented Apr 26, 2023

atharva-2001 commented Apr 26, 2023

atharva-2001 commented Apr 26, 2023

atharva-2001 commented Apr 26, 2023

atharva-2001 commented Apr 27, 2023

atharva-2001 commented Apr 27, 2023

atharva-2001 commented Apr 28, 2023

atharva-2001 commented Apr 28, 2023

atharva-2001 commented May 1, 2023

atharva-2001 commented May 1, 2023