Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Re-Iterate, Fix CTransformers Benchmark and add Readme #130

Merged
merged 13 commits into from
Jan 25, 2024
Merged

Re-Iterate, Fix CTransformers Benchmark and add Readme #130

merged 13 commits into from
Jan 25, 2024

Conversation

Anindyadeep
Copy link
Member

@Anindyadeep Anindyadeep commented Jan 23, 2024 •
edited
Loading

Checks:

  • Checksetup.sh and bench.sh have Python files configured for both Mac and Linux and installation working.
  • Check if int8 < int4 patterns follow
  • Check the log file name and where it is kept
  • Check if all precisions can be covered or not and if not put an issue
  • Check if all devices can be covered or not otherwise put an issue or find an issue
  • Check the Benchmark README

@Anindyadeep Anindyadeep requested a review from nsosio January 23, 2024 07:53
nsosio
nsosio reviewed Jan 23, 2024
View reviewed changes
README.md Outdated Show resolved Hide resolved
docs/llama2.md Outdated Show resolved Hide resolved
bench_ctransformers/README.md Outdated Show resolved Hide resolved
bench_ctransformers/README.md Show resolved Hide resolved
@Anindyadeep @nsosio
Update bench_ctransformers/README.md
fe9315f 
Co-authored-by: Nicola Sosio <sosio.nicola94@tiscali.it>
@Anindyadeep Anindyadeep self-assigned this Jan 23, 2024
@Anindyadeep Anindyadeep marked this pull request as draft January 24, 2024 10:43
@Anindyadeep
add model specific information and info about gpu layers
bdd63c3
@Anindyadeep
Refactor: Default config changed for setup and bench.sh
1a1d39a 
- created a Logs folder to store the logs
- benchmaek log folder starts with benchmark name
- python/python3 support
- changed the default prompt to something where model needs to generate more tokens
- changed the max_tokens value to 512, to get significant differences
@Anindyadeep
removed exceptions information
1a338a8
@Anindyadeep
added latest benchmark numbers for cuda for ctransformers and removed…
5da712f 
… exceptions
@Anindyadeep
Metal installation fix
f44de56
@Anindyadeep
Add latest default commands, benchmark scores for metal/cpu.
ecffcc1
@Anindyadeep Anindyadeep marked this pull request as ready for review January 24, 2024 17:37
@Anindyadeep Anindyadeep requested a review from nsosio January 24, 2024 17:37
@Anindyadeep Anindyadeep changed the title Readme for CTransformers CTransformer fixes Jan 24, 2024
@Anindyadeep Anindyadeep changed the title CTransformer fixes Re-Iterate, Fix CTransformers Benchmark and add Readme Jan 25, 2024
nsosio
nsosio reviewed Jan 25, 2024
View reviewed changes
Copy link
Collaborator

@nsosio nsosio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just some minor things and we can merge it :)

bench_ctransformers/bench.sh Outdated Show resolved Hide resolved
bench_ctransformers/bench.sh Show resolved Hide resolved
bench_ctransformers/setup.sh Show resolved Hide resolved
nsosio
nsosio reviewed Jan 25, 2024
View reviewed changes
bench_ctransformers/bench.sh Outdated Show resolved Hide resolved
bench_ctransformers/bench.sh Outdated Show resolved Hide resolved
Anindyadeep and others added 2 commits January 25, 2024 16:08
@Anindyadeep @nsosio
Update bench_ctransformers/bench.sh
ec73161 
Co-authored-by: Nicola Sosio <sosio.nicola94@tiscali.it>
@Anindyadeep @nsosio
Update bench_ctransformers/bench.sh
905bb98 
Co-authored-by: Nicola Sosio <sosio.nicola94@tiscali.it>
@nsosio nsosio merged commit 43e0954 into premAI-io:main Jan 25, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nsosio nsosio nsosio left review comments

Assignees

@Anindyadeep Anindyadeep

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Anindyadeep @nsosio @actions-user