Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About alignment metric #2

Open
XiaoqiWang opened this issue Sep 11, 2023 · 2 comments
Open

About alignment metric #2

XiaoqiWang opened this issue Sep 11, 2023 · 2 comments

Comments

@XiaoqiWang
Copy link

I would like to confirm whether the paper evaluates the alignment metric by calculating the score for all samples, or 80% for training and 20% for testing.

@lcysyzxdxc
Copy link
Owner

Hi. Alignment is the score for all samples

@XiaoqiWang
Copy link
Author

谢谢你的回复,我想再确认一下:论文表3(a)中,作者使用一些预训练模型例如CLIP,直接测试数据集中所有样本关于图文对齐的性能,没有finetuning?如果是这样,我测试出来的结果和表3(a)的差距很大,例如ImageReward 的SRCC只有0.4765,而作者的是0.7298. 在图文对齐测试中,论文并未对数据划分方式进行说明。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants