Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add llama3.np / llama3.cuda port to README.md #525

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

likejazz
Copy link

@likejazz likejazz commented Jun 6, 2024

No description provided.

@jameswdelancey
Copy link
Contributor

These appear to be llama2 ports vs llama3. They need to at least use tiktoken start tokens to work with the base, not 0 like sentencepiece, and if they work with tinystories, i suspect they are using sentencepiece tokenizer. Chat/instruct require a stack of tokens to start dialogue else you will be unimpressed with the output. There is a predictable output at zero temperature for both base/pretrained and chat/inspect that you can use to validate. You can copy this diff for the meat of what you need: jameswdelancey@7815cd3#diff-8935a7a088435e2ddf7315451f07fae16810932fb3a0a5d706a2eead1618af26R850-R854

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants