Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add hello-world benchmark #231

Merged
merged 44 commits into from
Dec 12, 2024
Merged

Add hello-world benchmark #231

merged 44 commits into from
Dec 12, 2024

Conversation

PEZ
Copy link
Collaborator

@PEZ PEZ commented Dec 11, 2024

Hello,

I started to add a hello-world benchmark. Two drivers for me:

  1. To have a benchmark that shows how quick a program written in a language is to start. Related to:
  2. To learn how Hello World looks in all these cool languages. And what the toolchains look like.
    • Since I picked up Clojure, I have stopped testing other languages almost completely, so it is probably healthy as an exercise.

When it comes to toolchains I at some point decided I wanted CI to run the benchmark. Making all these toolchains co-exist in a Github Actions workflow is VERY time consuming, though, so it put a bit of a stop in my implementing of Hello Worlds. But it is probably good to have one fast-running benchmark that can test the clean-compile-run of all languages reliably.

Anyway, @bddicken, before I prioritize this PR too much, I'd like your feedback on the idea and the way I am going at it. Is it something you would even consider merging? I'll have use for it even if you don't so don't hesitate to tell me the truth. 😄

As for the state of this PR, I have added some 15 languages so far, maybe less, maybe more. And CI runs some of them. I think my next steps will be to add more languages and then see how many of them I can get to run in CI.

@bddicken
Copy link
Owner

This is actually a very cool idea. One of the big complains outlined in this issue was that this test suite includes startup times, so it would be super cool just to see what the cost actually is for all of them!

@bddicken
Copy link
Owner

Is this ready to merge, or do you want to work on it more first?

@PEZ
Copy link
Collaborator Author

PEZ commented Dec 12, 2024

Good question! I was thinking to just show you what I was up to, but I do also think it can be merged and we can take it from there. Maybe other people will want to add languages and help with the CI setup.

The thing to keep in mind is that it's not all put together in CI. So there are languages that have hello-world, that won't have a tool chain set up in CI. But I guess that's fine as long as you are aware.

@bddicken
Copy link
Owner

Ok great. I'll just go ahead and merge this as a start, and others will be welcome to add!

@bddicken bddicken merged commit 53e44bd into bddicken:main Dec 12, 2024
@PEZ
Copy link
Collaborator Author

PEZ commented Dec 12, 2024

One of the big complains outlined in this issue was that this test suite includes startup times, so it would be super cool just to see what the cost actually is for all of them!

Indeed. Seeing the startup times was my main driver for this. Let's see if it gives us any ideas how to mitigate. Hello World is not dealing with the JIT aspects, though. But maybe we can figure that out too, eventually.

@igouy
Copy link

igouy commented Dec 16, 2024

I started to add a hello-world benchmark.

I had kind-of hoped people wouldn't just repeat the same mistakes:

https://salsa.debian.org/benchmarksgame-team/archive-alioth-benchmarksgame/-/tree/master/contributed-source-code/shootout/hello

:but would do better.

@igouy
Copy link

igouy commented Dec 16, 2024

One of the big complains outlined in this issue was that this test suite includes startup times, so it would be super cool just to see what the cost actually is for all of them!

So measure it ?

https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/startup.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants