Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use fused_multi_head_attention_inference_v2 to implement CrossAttention #459

Merged
merged 7 commits into from
Dec 26, 2023
Merged

use fused_multi_head_attention_inference_v2 to implement CrossAttention #459

merged 7 commits into from
Dec 26, 2023

Conversation

fpzh2011
Copy link
Contributor

主要修改

  • 用 fused_multi_head_attention_inference_v2 完成 CrossAttention 的核心计算。UNetModel 从 4.49it/s 提升到 6.71it/s。
  • 更新 readme 中的性能数据,base 由 torch 改为 TensorRT。
  • time_embed 的替换会触发 mock,最新的 onediff 需要将 sgm 包路径添加到 sys.path,否则会报 import 错误。(已经跟冯文确认过)
  • 将 sd-webui 和 comfyUI readme 中 onediff 的 git 地址由 oneflow 改为 siliconflow。
  • 修复 sd-webui 安装说明中两个命令的顺序。

@fpzh2011 fpzh2011 requested a review from strint December 26, 2023 07:01
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
@@ -171,6 +133,8 @@ def compile(sd_model):
return
global _compiled
_compiled = oneflow_compile(sd_model.model.diffusion_model, use_graph=True)
import sgm, sys
sys.path.append(sgm.__path__[0][:-4])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里注明下原因吧,看起来比较奇怪

Copy link
Contributor

@ccssu ccssu Dec 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oneflow.mock.enable():下找不到 sgm 这个 模块 , 以前拷贝文件夹那种方式 没有这个问题 是因为用有提前加入sys.path
image

具体是因为 mock不是根据 disable_mod_cache 中 对应 sgm 的 spec 找 , 导致 相对的路径 oneflow_mock下找不到模块。

@strint strint merged commit b032272 into siliconflow:main Dec 26, 2023
0 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants