Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Investigate model aliasing #39

Closed
abetlen opened this issue Apr 6, 2023 · 3 comments
Closed

Investigate model aliasing #39

abetlen opened this issue Apr 6, 2023 · 3 comments
Labels
enhancement New feature or request server

Comments

@abetlen
Copy link
Owner

abetlen commented Apr 6, 2023 •
edited
Loading

Allow the user to alias their local models to OpenAI model names as many tools have those hard-coded.

This may cause unexpected issues with tokenization mismatches.
@abetlen abetlen added the server label Apr 6, 2023
@MillionthOdin16
Copy link
Contributor

Is this still an issue, or you meant to delete?

@abetlen
Copy link
Owner Author

abetlen commented Apr 7, 2023

Just tracking an idea I had related to #17 basically allow the user to alias their local models to OpenAI model names as many tools have those hard-coded.

@MillionthOdin16
Copy link
Contributor

Ahhhhh, okay, I thought you were somehow having models aliasing as an issue 😆

@abetlen abetlen added the enhancement New feature or request label Apr 7, 2023
@abetlen abetlen mentioned this issue Apr 16, 2023
Closed
carmonajca added a commit to carmonajca/llama-cpp-python that referenced this issue May 17, 2023
@carmonajca @abetlen
@dependabot @zxybazh @aneeshjoy
Update to current status (#2)
0df288e 
* Bugfix: Ensure logs are printed when streaming

* Update llama.cpp

* Update llama.cpp

* Add missing tfs_z paramter

* Bump version

* Fix docker command

* Revert "llama_cpp server: prompt is a string". Closes abetlen#187

This reverts commit b9098b0.

* Only support generating one prompt at a time.

* Allow model to tokenize strings longer than context length and set add_bos. Closes abetlen#92

* Update llama.cpp

* Bump version

* Update llama.cpp

* Fix obscure Wndows DLL issue. Closes abetlen#208

* chore: add note for Mac m1 installation

* Add winmode arg only on windows if python version supports it

* Bump mkdocs-material from 9.1.11 to 9.1.12

Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.1.11 to 9.1.12.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](squidfunk/mkdocs-material@9.1.11...9.1.12)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update README.md

Fix typo.

* Fix CMakeLists.txt

* Add sampling defaults for generate

* Update llama.cpp

* Add model_alias option to override model_path in completions. Closes abetlen#39

* Update variable name

* Update llama.cpp

* Fix top_k value. Closes abetlen#220

* Fix last_n_tokens_size

* Implement penalize_nl

* Format

* Update token checks

* Move docs link up

* Fixd CUBLAS dll load issue in Windows

* Check for CUDA_PATH before adding

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Andrei Betlen <abetlen@gmail.com>
Co-authored-by: Anchen <anchen.li+alias@pepperstone.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Xiyou Zhou <xiyou.zhou@gmail.com>
Co-authored-by: Aneesh Joy <aneeshjoy@gmail.com>
@shamitv shamitv mentioned this issue Jul 28, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request server
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@abetlen @MillionthOdin16