Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[PPDiffusers] add photomaker & InstantID model #401

Merged
merged 17 commits into from
Feb 7, 2024
Merged

[PPDiffusers] add photomaker & InstantID model #401

merged 17 commits into from
Feb 7, 2024

Conversation

JiehangXie
Copy link

No description provided.

@JunnYu
Copy link
Member

JunnYu commented Jan 30, 2024

这么多图片，其实不太想在repo里面存放，你可以存到别的地方啥的

@JunnYu JunnYu self-requested a review January 30, 2024 13:55
JunnYu
JunnYu reviewed Jan 30, 2024
View reviewed changes
ppdiffusers/examples/PhotoMaker/photomaker/pipeline.py Outdated
self.trigger_word = trigger_word


id_encoder = PhotoMakerIDEncoder()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个encoder看看ip adapter里面的设置就差不多了

JunnYu
JunnYu reviewed Jan 31, 2024
View reviewed changes
ppdiffusers/examples/PhotoMaker/gradio_demo/app.py Outdated Show resolved Hide resolved
ppdiffusers/examples/PhotoMaker/predict.py Outdated Show resolved Hide resolved
ppdiffusers/examples/PhotoMaker/predict.py Outdated Show resolved Hide resolved
@JunnYu JunnYu changed the title add photomaker model [PPDiffusers] add photomaker model Jan 31, 2024
@JunnYu
Copy link
Member

JunnYu commented Jan 31, 2024

image
测试成功。

JunnYu
JunnYu approved these changes Jan 31, 2024
View reviewed changes
Copy link
Member

@JunnYu JunnYu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JiehangXie JiehangXie changed the title [PPDiffusers] add photomaker model [PPDiffusers] add photomaker & InstantID model Feb 1, 2024
JunnYu
JunnYu reviewed Feb 2, 2024
View reviewed changes
ppdiffusers/examples/InstantID/attention_processor.py Outdated
from ppdiffusers.models.attention_processor import Attention
from ppdiffusers.utils import USE_PEFT_BACKEND

class AttnProcessor(paddle.nn.Layer):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这应该没必要重写吧，直接参考ip adapter训练的那个，因为可能未来版本会变，里面的__call__会变化

JunnYu
JunnYu reviewed Feb 2, 2024
View reviewed changes
ppdiffusers/examples/InstantID/resampler.py Outdated
Args:
x (torch.Tensor): image features
shape (b, n1, D)
latent (torch.Tensor): latent features
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

所有的torch可以改成paddle

JunnYu
JunnYu reviewed Feb 2, 2024
View reviewed changes
ppdiffusers/examples/InstantID/pipeline_stable_diffusion_xl_instantid.py Outdated

if needs_upcasting:
self.upcast_vae()
latents = latents.to(next(iter(self.vae.post_quant_conv.parameters())).dtype)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个应该不对吧， latents.to(next(iter(self.vae.post_quant_conv.named_parameters())[1]).dtype),
因为 named_parameters 才是迭代器返回的东西，迭代器是lazy的
而 parameters 直接返回一个list，然后消耗的资源不一样，list要所有都返回

@JunnYu
Copy link
Member

JunnYu commented Feb 2, 2024

可以加个gradio，加个readme啥的

@JunnYu
Copy link
Member

JunnYu commented Feb 2, 2024

提交的时候使用 pre-commit 格式化一下代码，详细参考 这种使用方式。https://github.com/PaddlePaddle/PaddleNLP/blob/develop/CONTRIBUTING.md#6-commit

@JunnYu JunnYu self-requested a review February 2, 2024 05:53
This was referenced Feb 2, 2024
Open
Open
JunnYu
JunnYu reviewed Feb 4, 2024
View reviewed changes
ppdiffusers/examples/InstantID/gradio_demo/app.py Outdated


def main(pretrained_model_name_or_path="wangqixun/YamerMIX_v8", enable_lcm_arg=False):
vae = AutoencoderKL.from_pretrained(pretrained_model_name_or_path, subfolder="vae", from_hf_hub=True,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这就不用手动加载了吧，直接模型加载吧

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个模型是基于sdxl 1.0的，在pipe直接加载vae也有问题，只能手动加载

JunnYu
JunnYu reviewed Feb 4, 2024
View reviewed changes
ppdiffusers/examples/InstantID/predict.py

base_model_path = "stabilityai/stable-diffusion-xl-base-1.0"

vae = AutoencoderKL.from_pretrained(base_model_path, subfolder="vae")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这

@JunnYu
Copy link
Member

JunnYu commented Feb 4, 2024

文档跟代码的写法都同步统一改一下，发现可能文档的代码跟实际的代码稍微有点出入

@JunnYu JunnYu merged commit d00d571 into PaddlePaddle:upgrade_ppdiffusers0240 Feb 7, 2024
1 check passed
@luotao1
Copy link
Collaborator

luotao1 commented Feb 21, 2024

hi, @JiehangXie

  • 非常感谢你对飞桨的贡献，我们正在运营一个PFCC组织，会通过定期分享技术知识与发布开发者主导任务的形式持续为飞桨做贡献，详情可见 https://github.com/luotao1 主页说明。
  • 如果你对PFCC有兴趣，请发送邮件至 ext_paddle_oss@baidu.com，我们会邀请你加入~

@LokeZhou LokeZhou mentioned this pull request Mar 26, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JunnYu JunnYu JunnYu approved these changes

Assignees
No one assigned
Labels
contributor
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@JiehangXie @JunnYu @luotao1