Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add batch files for automatic installation on Windows #189

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions Windows-installer.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
@echo off
::Install Llama.cpp dependencies, such as Python and CMake (for future building of llama.exe and quantize.exe)

if not defined PYTHON (set PYTHON=python)
if not defined VENV_DIR (set "VENV_DIR=%~dp0%venv")

%PYTHON% -V
if %ERRORLEVEL% == 0 goto :create_venv
echo Python is not installed, installing
goto :install_python

:install_python
call bitsadmin /transfer Python-3.10.6 /download /priority FOREGROUND "https://www.python.org/ftp/python/3.10.6/python-3.10.6-amd64.exe" "%CD%/python-3.10.6-amd64.exe"
python-3.10.6-amd64.exe /quiet InstallAllUsers=1 PrependPath=1 Include_test=0
call refreshenv
goto :create_venv

:: Should probably impliment check for pip before installing dependicies
:create_venv
:: Check if venv already exists
dir "%VENV_DIR%\Scripts\Python.exe" -V
if %ERRORLEVEL% == 0 goto :initiate_venv

:: Otherwise create new venv
echo Creating venv in %VENV_DIR%
%PYTHON% -m venv "%VENV_DIR%"
if %ERRORLEVEL% == 0 goto :initiate_venv
echo Unable to create venv in "%VENV_DIR%"
pause
exit

:: Activate venv
:initiate_venv
set PYTHON="%VENV_DIR%\Scripts\Python.exe"
echo venv %PYTHON%
goto :install_dependencies

:install_dependencies
%PYTHON% -m pip install cmake torch numpy sentencepiece %*
echo Llama.cpp dependencies are now installed!
echo Put your LLaMA models into the models folder, and run model_conversion to convert and quantize them.
pause
exit
31 changes: 31 additions & 0 deletions Windows-model_conversion.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
@echo off
:: Convert models to 4 bit

set "VENV_DIR=%~dp0%venv"
set PYTHON="%VENV_DIR%\Scripts\Python.exe"
echo venv is located at: %PYTHON%
goto :convert_ggml

:: Convert the 7B model to ggml FP16 format
:convert_ggml
%PYTHON% "convert-pth-to-ggml.py" models/7B/ 1
if %ERRORLEVEL% == 1 goto :fast_quit
goto :build_cmake

:: Build llama.exe and quantize.exe
:build_cmake
cmake -S . -B build/ -D CMAKE_BUILD_TYPE=Release
cmake --build build/ --config Release
if %ERRORLEVEL% == 1 goto :fast_quit
goto :quantize

:quantize
.\build\Release\quantize.exe .\models\7B\ggml-model-f16.bin .\models\7B\ggml-model-q4_0.bin 2
echo All tasks completed successfully.
pause
exit

:fast_quit
echo Something went wrong!
pause
exit