UltraEval is an open-source framework for evaluating the capabilities of foundational models, offering a lightweight and easy-to-use evaluation system. The overall framework structure is illustrated in the following diagram:
Following the operational sequence, it is divided into three main modules: 【Data Preparation】, 【Model Deployment】, and 【Task Evaluation】, corresponding to
Additionally, UltraEval is highly extensible. To facilitate users in extending other tasks or models, we provide a customization evaluation process.