Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Mooncake to Lux benchmarking suite #405

Open
yebai opened this issue Dec 2, 2024 · 2 comments
Open

Add Mooncake to Lux benchmarking suite #405

yebai opened this issue Dec 2, 2024 · 2 comments
Labels
enhancement (testing) Would improve the tests

Comments

@yebai
Copy link
Contributor

yebai commented Dec 2, 2024

For more comprehensive benchmarking, let's add Mooncake to the Lux's benchmarking set:

Cc @avik-pal, who might be able to provide pointers on what is needed.

(this issue is copied from #361 (comment))

@willtebbutt willtebbutt added the enhancement (testing) Would improve the tests label Dec 2, 2024
@avik-pal
Copy link

avik-pal commented Dec 3, 2024

Yes, let's do this. I need to cleanup the benchmarks on Lux end (the benchmarks are somewhat broken since the monorepo setup). I will update here once that is done

@yebai
Copy link
Contributor Author

yebai commented Dec 18, 2024

It'd be incredibly helpful if we could have a set of (Lux + basic NN library functions) benchmark cases and a benchmarking script that leads to an (autograd) community leaderboard like this one for probabilistic programming:

https://nsiccha.github.io/StanBlocks.jl/performance.html#visualization

@avik-pal, what are your thoughts? The Turing.jl team would be happy to contribute to a leaderboard if you could lead it.

Related Flux benchmark cases: #361 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement (testing) Would improve the tests
Projects
None yet
Development

No branches or pull requests

3 participants