Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Text2Image into Taskflow #2988

Merged
merged 8 commits into from
Aug 10, 2022

Conversation

JunnYu
Copy link
Member

@JunnYu JunnYu commented Aug 8, 2022

PR types

New features

PR changes

Others

Description

Add Text2Image into Taskflow

import paddle
from paddlenlp import Taskflow
from IPython.display import display

model_text = {
    "pai-painter-painting-base-zh": ["风阁水帘今在眼,且来先看早梅红", "见说春风偏有贺,露花千朵照庭闹"],
    "pai-painter-scenery-base-zh": ["海边的日出", "黄昏日落"],
    "pai-painter-commercial-base-zh":
    ["女童套头毛衣打底衫秋冬针织衫童装儿童内搭上衣", "春夏真皮工作鞋女深色软皮久站舒适上班面试职业皮鞋"],
}
for model, prompts in model_text.items():
    text_to_image = Taskflow("text_to_image", model=model)
    images = text_to_image(prompts)
    for image in images:
        display(image)
    paddle.device.cuda.empty_cache()

0 (1)
1 (1)
2
3
4
5

import paddle
from paddlenlp import Taskflow
from IPython.display import display

prompts = [
    "New York Skyline with 'Google Research Pizza Cafe' written with fireworks on the sky.",
    "Dali painting of WALL·E"
]
model_text = {
    "dalle-mini": prompts,
    "dalle-mega-v16": prompts,
    "dalle-mega": prompts
}

for model, prompts in model_text.items():
    text_to_image = Taskflow("text_to_image", model=model)
    images = text_to_image(prompts)
    for image in images:
        display(image)
    paddle.device.cuda.empty_cache()

dalle_0
dalle_1
dalle_2
dalle_3
dalle_4
dalle_5

@guoshengCS guoshengCS requested a review from wawltor August 9, 2022 02:24
* `top_p`:解码参数top_p,默认为1.0。
* `conditional_scale`:dalle-mini模型使用的参数,可参考[推特](https://twitter.com/RiversHaveWings/status/1478093658716966912),默认为10.0。
* `num_return_images`:返回图片的数量,默认为4,即4张图片水平拼接形成一张长图。
* `use_faster`:是否使用faster_generation,默认为False,目前artist模型支持,而dalle-mini模型不支持。
Copy link
Contributor

@guoshengCS guoshengCS Aug 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里也尽量隐去 artist 吧,这个模型名确实不见得有认可度,先用 pai这些模型权重来体现吧

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use_faster:是否使用faster_generation,默认为False,目前支持faster_generation的模型有["pai-painter-painting-base-zh", "pai-painter-scenery-base-zh", "pai-painter-commercial-base-zh"]

guoshengCS
guoshengCS previously approved these changes Aug 9, 2022
if self._model.base_model_prefix == "dallebart":
images = (images.clip(0, 1) * 255).astype("uint8")
elif self._model.base_model_prefix == "gpt":
images = ((images + 1.0) * 127.5).clip(0, 255).astype("uint8")
Copy link
Contributor

@guoshengCS guoshengCS Aug 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这些后处理如果是和模型绑定的话后面也可以看看将这些方法放在transformers模型代码中提供出来,这里进行统一调用,方便更多模型扩展

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, 现已添加到model的generate内部。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants