Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider changing from MSE to MAE #6

Open
evantey14 opened this issue Aug 12, 2022 · 0 comments
Open

Consider changing from MSE to MAE #6

evantey14 opened this issue Aug 12, 2022 · 0 comments

Comments

@evantey14
Copy link
Owner

Parity scorers currently use mean squared error (MSE) on a target value to evaluate scores since that seemed the more natural error to use. However, we want an empty league to have a meaningful score, so I've normalized the MSE (otherwise the scores would just be zeros until there are at least a large # of kids on each team, meaning assignment wouldn't actually use the score in any decisionmaking). This normalization means that at the end of the process, the error is very very small so all the scores look like 1s.

I think this means we're making more trades in the early / mid-game but less in the late game because the scores are used linearly to calculate the final score.

Other options in general seem to be:

  • picking a different normalization so we are more effective late game
  • using a different error (like MAE)
  • have a running target value based on currently assigned players (I believe this would be pretty slow to compute)
  • have the normalization be calculated on construction (normalization depends on already assigned players from the input file)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant