This folder contains instructions to evaluate LLM360 models in alignment with the HuggingFace Open LLM Leaderboard.
Please run the following commands or visit the official documentation:
git clone https://github.com/huggingface/lm-evaluation-harness.git
cd lm-evaluation-harness
git checkout adding_all_changess
pip install -e .[math,ifeval,sentencepiece]An example usage is provided in the demo.ipynb, which can be executed with a single A100 80G GPU.