Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Op] Add attention and bias_gelu ops #41

Merged
merged 7 commits into from
Feb 7, 2023
Merged

[Op] Add attention and bias_gelu ops #41

merged 7 commits into from
Feb 7, 2023

Conversation

comaniac
Copy link
Contributor

@comaniac comaniac commented Feb 6, 2023

Description

This is a prerequisite PR for adding HF GPT-2 schedule.

  • Implement attention ops that use flash-attention and xformrs.
  • Implement bias_gelu ops that use torchscript or torch compiler.
  • Change the GPT-Neo schedule to use these ops, so that now GPT-Neo schedule doesn't depend on epoi anymore. Later we will update the schedules of other example models accordingly.
  • [Test] Add -rxXs to let pytest print reasons of skipped tests.
  • [Docker] Update flash-attention commit hash which improves the kernel performance by ~12%. CI image is not updated because this change doesn't impact the functionality.

Checklist

  • PR's title starts with a category (e.g. [Bugfix], [Model], [Tutorial], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

@szhengac @chhzh123

examples/gpt/schedule.py Show resolved Hide resolved
@comaniac comaniac merged commit d2dbaeb into awslabs:main Feb 7, 2023
@comaniac
Copy link
Contributor Author

comaniac commented Feb 7, 2023

Thanks @chhzh123

@comaniac comaniac deleted the op branch February 7, 2023 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants