Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a readme under each benchmark. #88

Closed
Anindyadeep opened this issue Dec 6, 2023 · 1 comment
Closed

Add a readme under each benchmark. #88

Anindyadeep opened this issue Dec 6, 2023 · 1 comment
Assignees
Labels
documentation Improvements or additions to documentation enhancement New feature or request high priority

Comments

@Anindyadeep
Copy link
Member

Under the docs section we have a information about ML-Engines and benchmarking information for Llama2 models in Llama2-template. However, we do not provide information about each of the benchmark. So, this issue proposes to provide a readme under benchmark folder. Under the readme the following information should be provided:

  • A little desc about of the benchmark.
  • A original links of the implementation.
  • device supported.
  • quantization.
  • The limitations of the benchmark.
  • How to run that particular benchmark on custom data example.

This will help newcomers to easily get started with a new framework or a backend and use it properly.

@nsosio nsosio added enhancement New feature or request documentation Improvements or additions to documentation labels Dec 11, 2023
@Anindyadeep
Copy link
Member Author

Anindyadeep commented Jan 10, 2024

This is the updated structure:

  • About (This includes)

    • general desc
    • paper (if any)
    • implementation source
  • Benchmarks info

    • The numerical info
    • Any specific information like supported devices and limitations (can also reference the issues related to it)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request high priority
Projects
None yet
Development

No branches or pull requests

2 participants