Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow configuring runtime cap - 60 seconds might be too long in some cases #5

Open
haxscramper opened this issue Apr 3, 2021 · 3 comments

Comments

@haxscramper
Copy link

I decided to add this package on playground (PMunch/nim-playground#7), but because maximum execution time is capped at 20s it might be possible for the benchmark to be terminated externally without even finishing.

I know that 60s cap is not always reached, but considering playground has relatively low execution power it might happen more often compared to regular use.

at least 10 times but possibly more to figure out the standard deviation. It will keep running it until things look like they stabilized. It will stop after 60s though.

I thought about adding cappedTimeIt(maxSeconds: int, name: string, body: untyped) that would allow configuring maximum running time. Make timeIt an alias for cappedTimeIt(60, ...)

Does the implementation idea sound good, or you would prefer to keep this package as simple as possible, without any configuration complexity?

@treeform
Copy link
Owner

treeform commented Apr 5, 2021

Sorry I don't understand you want it to be infinite? or 20seconds?

The time cap is there really as a fail safe. Most benches should finish in 1-5s seconds. No one has time to wait a full minute!

You can already pass it max number of iterations:
https://github.com/treeform/benchy/blob/master/tests/test.nim#L52

I can see another param that passes max time too.

@haxscramper
Copy link
Author

Sorry I don't understand you want it to be infinite? or 20seconds?

Just as the title says - I want to be configurable. I didn't know you can pass the maximum number of iterations (it is not mentioned in README, and I didn't look into tests (although I should've done this before opening an issue)). I think that solves an issue, though an option to have "another param that passes max time too." would be exactly what I need in this case.

The time cap is there really as a fail safe. Most benches should finish in 1-5s seconds. No one has time to wait a full minute!

Yes, of course, but in PMunch/nim-playground#7 PMunch was concerned with the possibility of the benchmark executing for longer than twenty seconds.

@PMunch
Copy link

PMunch commented Nov 19, 2021

Oh, to be honest I thought that all the benchmarks ran for a minute. If it only runs for a couple seconds it should be fine to run on the playground.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants