Skip to content

Navigation Menu

Explore
By company size
By use case
By industry
View all solutions
Topics
- AI
- DevOps
- Security
- Software Development
- View all
Explore
- GitHub Sponsors
  Fund open source developers
- The ReadME Project
  GitHub community articles
Repositories
- Enterprise platform
  AI-powered developer platform
Available add-ons
Pricing

Search code, repositories, users, issues, pull requests...

Search

Clear

Search syntax tips

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our documentation.

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

bghira / SimpleTuner Public

Notifications You must be signed in to change notification settings
Fork 184
Star 1.9k

Code
Issues 7
Pull requests 1
Discussions
Actions
Projects
Security
Insights

Additional navigation options

Code
Issues
Pull requests
Discussions
Actions
Projects
Security
Insights

Add SageAttention for substantial training speed-up#1182

Merged

bghira merged 18 commits intomainbghira/SimpleTuner:mainfrom feature/sage-attentionbghira/SimpleTuner:feature/sage-attentionCopy head branch name to clipboard

Dec 2, 2024

+327-38

Conversation Commits18 (18)Checks Files changed

Commits

Commits on Nov 30, 2024

flux: use sage attention if available
bghira
committed
update deepspeed call to ensure compliance with new transformers version
bghira
committed

Commits on Dec 2, 2024

configurator should offer option to enable SageAttention for user
bghira
committed
add --attention_mechanism option for sageattention
bghira
committed
flux: remove model-specific sageattention code
bghira
committed
sageattention cannot be enabled concurrently to xformers
bghira
committed
sageattention should overwrite sdpa at startup if enabled
bghira
committed
flux: remove more model-specific sageattn code
bghira
committed
deprecate --enable_xformers_memory_efficient_attention in favour of --attention_mechanism=xformers
bghira
committed
hackish awful workaround for VAE decode in SageAttention
bghira
committed
add more sageattention API choices
bghira
committed
SD 1.5/2.x fix for SageAttention decode
bghira
committed
set attention mechanism to the default for tests
bghira
committed
add sageattention to OPTIONS doc, update recommendations in --help output
bghira
committed
kolors: enable vae decode hack for sageattention
bghira
committed
add sageattention to the quickstart docs and a specific section under "Precision" in the OPTIONS doc
bghira
committed
sd3 skip_layer_guidance fix from upstream for num images per prompt > 1
bghira
committed
disable nf4 + sageattention
bghira
committed

Footer

© 2024 GitHub, Inc.

Footer navigation

Terms
Privacy
Security
Status
Docs
Contact

You can’t perform that action at this time.