Composite Scoring for Frameworks
The following is a work-in-progress feature for the TFB project. This page will be moved within this Wiki if and when the feature described here is completed.
Coincident with efforts to measure and provide a score for hardware performance, we will be providing a new composite score for frameworks. The Wiki entry for TPR contains several details about how scoring works, so it is recommended that you read that first.
Below is a work-in-progress rendering of composite scores from Round 18 on Citrine.
The objective of a composite score is to give a single, unified view of the relative performance of frameworks across all of our test types. While the value of this score remains to be determined, we feel it is relatively low effort to provide. We will see how the community consumes this and adjust as needed.
Only frameworks that implement all of our test types will be included in the composite scoring chart.
See the section on scoring within the description of TPR for details on scoring. In brief, each Round will compute new weights for each test type. These weights will be applied to each frameworks' results per test type and then summed. The resulting sum is then divided by 1,000 to yield a shorter weighted score.