Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEAT: Support Phi-2 #828

Merged
merged 3 commits into from
Dec 29, 2023
Merged

FEAT: Support Phi-2 #828

merged 3 commits into from
Dec 29, 2023

Conversation

Bojun-Feng
Copy link
Contributor

@Bojun-Feng Bojun-Feng commented Dec 27, 2023

Fix #806

Added support for Phi-2 in GGUF and PyTorch. GGUF works as expected when tested locally. Would appreciate it if someone can test PyTorch locally as well.

Phi-2 supports a variety of input formats, so I thought the generate UI would be more appropriate.

Input formats from Hugging Face page](https://huggingface.co/microsoft/phi-2)

QA Format:
You can provide the prompt as a standalone question as follows:

Write a detailed analogy between mathematics and a lighthouse.

where the model generates the text after "." . To encourage the model to write more concise answers, you can also try the following QA format using "Instruct: \nOutput:"

Instruct: Write a detailed analogy between mathematics and a lighthouse.
Output: Mathematics is like a lighthouse. Just as a lighthouse guides ships safely to shore, mathematics provides a guiding light in the world of numbers and logic. It helps us navigate through complex problems and find solutions. Just as a lighthouse emits a steady beam of light, mathematics provides a consistent framework for reasoning and problem-solving. It illuminates the path to understanding and helps us make sense of the world around us.

where the model generates the text after "Output:".

Chat Format:

Alice: I don't know why, I'm struggling to maintain focus while studying. Any suggestions?
Bob: Well, have you tried creating a study schedule and sticking to it?
Alice: Yes, I have, but it doesn't seem to help much.
Bob: Hmm, maybe you should try studying in a quiet environment, like the library.
Alice: ...

where the model generates the text after the first "Bob:".

@XprobeBot XprobeBot added this to the v0.8.0 milestone Dec 27, 2023
@aresnow1
Copy link
Contributor

Is it ready for review?

@aresnow1 aresnow1 marked this pull request as ready for review December 29, 2023 02:37
@aresnow1
Copy link
Contributor

I've rebased and tested on local, ready to review.

@ChengjieLi28 ChengjieLi28 merged commit 3907fc6 into xorbitsai:main Dec 29, 2023
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

FEAT: 增加对phi-2模型的支持
4 participants