Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GPTNeoX Model #1052

Open
4 of 5 tasks
shivance opened this issue May 23, 2023 · 1 comment
Open
4 of 5 tasks

Add GPTNeoX Model #1052

shivance opened this issue May 23, 2023 · 1 comment
Assignees
Labels
type:feature New feature or request

Comments

@shivance
Copy link
Collaborator

shivance commented May 23, 2023

Pythia is a suite of 16 LLMs all trained on public data seen in the exact same order and ranging in size from 70M to 12B parameters. The model was developed with intention to facilitate research in many areas. That's why I think this would be a good addition to KerasNLP. I'll work on adding following as a part of Google Summer of Code

@mattdangerw mattdangerw added the type:feature New feature or request label May 23, 2023
@shivance shivance changed the title Add PythiaBackbone Add GPTNeoXBackbone May 29, 2023
@innat
Copy link

innat commented Jun 2, 2023

#929

This was referenced Jun 4, 2023
@shivance shivance changed the title Add GPTNeoXBackbone Add GPTNeoX Model Jun 17, 2023
@shivance shivance self-assigned this Jul 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:feature New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants