Skip to content

Issues: open-compass/opencompass

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

252 Open 361 Closed
252 Open 361 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Feature] expose max_task_size in run.py, for quick debug
#1869 opened Feb 13, 2025 by yxdyc
1 task
@acylam
[Feature] 是否可以有输出结果后，单独去计算一下评分的功能
#1866 opened Feb 12, 2025 by Dagoli
1 task
@MaiziXiao
[Feature] 请问如何加载本地模型进行测评
#1865 opened Feb 11, 2025 by Castrol68
1 task
@liushz
在没有 ppl 的 dataset 中如何测试 base model？
#1861 opened Feb 10, 2025 by WangRongsheng
2 tasks done
@bittersweet1999
[Bug] 使用提示词攻击，出现导入模块失败
#1860 opened Feb 8, 2025 by pomliuxj
2 tasks done
@tonysy
[Bug] Only Debug Mode can perform eval tasks correctly.
#1859 opened Feb 8, 2025 by GenerallyCovetous
2 tasks done
3
[Bug] chinese simpleQA dataset is not working
#1858 opened Feb 7, 2025 by hailsham
2 tasks done
@tonysy
[Bug] 在对DeepSeek-R1-Distill-Qwen-1.5B模型评测livecodebench数据集时，lcb_test_output为什么为0呢？
#1856 opened Feb 7, 2025 by guoguo1314
2 tasks done
@tonysy
1
[Bug] MBPP score significantly lower than official results
#1855 opened Feb 7, 2025 by GenerallyCovetous
2 tasks done
[Feature] humaneval_postprocess历史无效的单测清理或case更新
#1847 opened Jan 24, 2025 by anfreshman
1 task done
@tonysy
[Bug] 部分数据集中对question进行增广时使用浅拷贝会导致选项与答案错位
#1845 opened Jan 23, 2025 by QuanWanLear
2 tasks done
@acylam
1
Missing one required positional argument in tools/collect_code_preds.py#L190
#1844 opened Jan 23, 2025 by grassFlamingo
@acylam
Miss using GPT3 in GPT4.py?
#1836 opened Jan 20, 2025 by xcfcode
@bittersweet1999
[Bug] 无法使用多卡评测
#1828 opened Jan 16, 2025 by GenerallyCovetous
2 tasks done
@acylam
6
[Bug] 使用leval测评时报错，求助
#1825 opened Jan 15, 2025 by fxnie
2 tasks done
@MaiziXiao
8
[Bug] longbench报错
#1824 opened Jan 15, 2025 by fxnie
2 tasks done
@MaiziXiao
3
[Feature] 请问是否能有一个汇总，就是opencompass包含的所有评测数据集、这些数据集的metrics和链接(paper/github)？
#1820 opened Jan 14, 2025 by 141forever
1 task
@liushz
2
[Bug] 部分样本上opencompass的评测方法是否可能有点片面？
#1818 opened Jan 14, 2025 by laogonggong847
2 tasks done
@liushz
[Bug] SuperGLUE_RTE_gen KeyError
#1816 opened Jan 10, 2025 by shedding-ash
2 tasks done
@tonysy
[Bug] 使用vllm加速时无法启动多卡
#1810 opened Jan 8, 2025 by Lichunyan3
2 tasks done
1
[Bug] 缺乏game24以及其他数据集
#1807 opened Jan 7, 2025 by nuochenpku
2 tasks done
@liushz
2
[Feature] 支持基于ray的多卡多机评测
#1804 opened Jan 4, 2025 by wenxueru
1 task
@tonysy
1
[Feature] 如何添加评价指标
#1803 opened Jan 3, 2025 by Lichunyan3
1 task
@tonysy
2
[Bug] SuperGLUE_BoolQ_ppl Unable to load file
#1799 opened Jan 2, 2025 by shutttttdown
2 tasks done
@acylam
4
[Bug] gsm8k_gen_17d0dc-BOT思考提示语错误
#1788 opened Dec 27, 2024 by TBoyBack
2 tasks done
@liushz
2
ProTip! Mix and match filters to narrow down what you’re looking for.