Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

新增图文对话模型X-GPT #3541

Closed
chenjjcccc opened this issue Oct 24, 2023 · 0 comments
Closed

新增图文对话模型X-GPT #3541

chenjjcccc opened this issue Oct 24, 2023 · 0 comments
Assignees

Comments

@chenjjcccc
Copy link
Collaborator

问题描述 Please describe your issue

新增图文对话模型X-GPT

任务描述

任务背景

  • X-Decoder 集成了图像理解的多类任务,结合GPT和SD相关生成模型就可以实现All-in-One的图文对话式agnet。参考代码

完成步骤

  1. 基于Paddle复现X-Decoder,其可以进行检测、分割、VQA、取标题等多种应用。(如果训练对齐遇到不可抗问题,可以仅进行前向对齐。)
  2. 结合PaddleMix中基础模型ppdiffuser,进行生成模型迁移。
  3. 结合开源对话模型例如chatglm v2或者llama v2,来实现XGPT,给出使用示例和文档以及UI。提交至PaddleSeg/contrib/XGPT。
  4. 参考PR提交规范提交代码PR到ppseg中。

提交内容:

  1. 代码提交到PaddleSeg。
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants