Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark Workflow #162

Open
3 tasks
alecandido opened this issue Dec 2, 2022 · 2 comments
Open
3 tasks

Benchmark Workflow #162

alecandido opened this issue Dec 2, 2022 · 2 comments
Assignees
Labels
benchmarks Benchmark (or infrastructure) related bug Something isn't working good first issue Good for newcomers refactor Refactor code

Comments

@alecandido
Copy link
Member

alecandido commented Dec 2, 2022

Ok, blaming in Git it seems like that the problem described in #161 (comment) it is my fault, but something I did one year ago.

This means that yadism is approximately unmaintained, and almost no development happened in the meanwhile...

However, we need to drop asserts, since we never developed a suitable benchmark metric to automatically applied, see #83.
For this, I propose:

  • add in banana a no-assert mode, to be triggered by the workflow
  • add in banana report extraction, such that we can upload them as artifacts in the CI
    • the other, sensible, option is to upload the whole database, but I'd rather do both, since it is less clear to people how to do both...
  • re-enable .github/workflows/benchmark.yml, deactivated with the on value content in ef75c7a

Even though once you have the db and the extraction method this should be trivial (if well-documented), the workflow outcome should be intuitive to consume on its own.
It is as much trivial to run the conversion in the CI and upload both.

@alecandido alecandido self-assigned this Dec 2, 2022
@alecandido alecandido added the bug Something isn't working label Dec 2, 2022
@felixhekhorn felixhekhorn added the refactor Refactor code label Dec 5, 2022
@felixhekhorn
Copy link
Contributor

actually, should we transfer this to banana? (which has even less traffic by the way, but this means it is working just fine, I'd say ...)

@alecandido
Copy link
Member Author

The issue at the moment is here, so I'd keep where it arose.

You can open a further issue in banana, exporting the todo items from here.

@felixhekhorn felixhekhorn added good first issue Good for newcomers benchmarks Benchmark (or infrastructure) related labels Sep 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
benchmarks Benchmark (or infrastructure) related bug Something isn't working good first issue Good for newcomers refactor Refactor code
Projects
None yet
Development

No branches or pull requests

2 participants