Skip to content

Issues: open-compass/opencompass

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

204 Open 341 Closed
204 Open 341 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Bug] v0.3.5版本评测Qwen/Qwen2.5-72B得分显著下降
#1675 opened Nov 11, 2024 by guoshengCS
2 tasks done
@bittersweet1999
1
[Feature] dataset for humaneval-multipl
#1673 opened Nov 9, 2024 by jyshee
1 task
@tonysy
[Feature] Support Math23k
#1667 opened Nov 7, 2024 by 00INDEX
1 task
@acylam
[Feature] MMLU CEVAL MATH test results for some models
#1664 opened Nov 4, 2024 by jinmingyi1998
1 task
@bittersweet1999
1
[Bug] mmlupro 正则提取错误
#1661 opened Nov 4, 2024 by bittersweet1999
2 tasks done
@tonysy
[Bug] Error in downloading GSM8k dataset
#1654 opened Oct 31, 2024 by acse-ym722
2 tasks done
@acylam
1
[Bug] 多种测试方法测试结果差异大
#1649 opened Oct 29, 2024 by DietDietDiet
2 tasks done
@liushz
[Bug] TruthfulQA同时使用多个评估metric时报错
#1646 opened Oct 28, 2024 by XEric7
2 tasks done
@bittersweet1999
[Bug] 通过调用vllm启动服务的api接口，多次运行测试，得分结果不一致
#1634 opened Oct 23, 2024 by luhairong11
2 tasks done
@MaiziXiao
2
[Bug] Qwen2-7B测试过程中分数远远低于旧版本的分数
#1629 opened Oct 22, 2024 by Alexyuanfun
2 tasks done
@liushz
2
SafetyBench数据集评测bug
#1622 opened Oct 18, 2024 by shutttttdown
2 tasks done
@tonysy
[Bug] 使用Circular方式评测，few-shot数据拼接错误
#1615 opened Oct 17, 2024 by Alexyuanfun
2 tasks done
@acylam
[Feature] 请求添加BigCodeBench评测集 backlog
#1613 opened Oct 16, 2024 by BinBrent
1 task
@MaiziXiao
2
[Bug] 评估wikitext ppl数据集时候由于没有reference无法计算结果
#1608 opened Oct 15, 2024 by LanDisen
2 tasks done
@liushz
[Bug] hf推理与vllm推理评测结果不一致
#1594 opened Oct 9, 2024 by luhairong11
2 tasks done
@MaiziXiao
[Bug] AttributeError: 'ConfigDict' object has no attribute 'reader_cfg'
#1591 opened Oct 9, 2024 by xtuhcy
2 tasks done
@MaiziXiao
1
[Bug] Silent GPU failures, works with --debug
#1585 opened Oct 4, 2024 by anuragprat1k
2 tasks done
@tonysy
[Bug] pid_params.py dumps not successfully.
#1580 opened Sep 29, 2024 by tonysy
2 tasks done
@tonysy
[Bug] It seems run wrong number of prompts when max-num-workers > 1.
#1579 opened Sep 29, 2024 by zhulinJulia24
2 tasks done
@MaiziXiao
[Bug] debug不显示日志
#1578 opened Sep 29, 2024 by HaltonJiang
2 tasks done
@tonysy
1
[Bug] mtbench101：batch_size设置
#1571 opened Sep 27, 2024 by zhang-junjian
2 tasks done
@tonysy @bittersweet1999
1
[Feature] Support MMMLU backlog
#1560 opened Sep 24, 2024 by tonysy
1 task
@liushz
1
[Bug] HuggingFacewithChatTemplate
#1557 opened Sep 24, 2024 by ZCzzzzzz
2 tasks done
2
[Bug] 试图添加mlc后端遇到问题，找不到cuda
#1551 opened Sep 23, 2024 by XJY990705
2 tasks done
@bittersweet1999
[Bug] SciCode miss with_background Bug Something isn't working
#1540 opened Sep 18, 2024 by tonysy
2 tasks done
@MaiziXiao
ProTip! Mix and match filters to narrow down what you’re looking for.