Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

linker.parameter_estimate_comparisons_chart() improvements #1014

Closed
samnlindsay opened this issue Feb 2, 2023 · 0 comments · Fixed by #1301
Closed

linker.parameter_estimate_comparisons_chart() improvements #1014

samnlindsay opened this issue Feb 2, 2023 · 0 comments · Fixed by #1301
Labels
enhancement New feature or request good first issue Good for newcomers graphs

Comments

@samnlindsay
Copy link
Contributor

linker.parameter_estimate_comparisons_chart() currently displays the m and u probabilities as log odds. This has its advantages, but the original chart was designed for probabilities (fixed x-axis between 0 and 1).

Need to make some visual tweaks to make the log odds version easier to interpret, and enable switching between probabilities and log odds (a log_odds=False argument or a button in the chart to toggle different formats).

Issues

  • Parameters are estimated as probabilities with an effective tolerance of em_convergence. If this is set to 0.01 then we must expect the same m value to potentially be estimated as 0.0004 (log odds = -7.824) or 0.0104 (log odds = -4.5564). These values are consistent with one another given our em_convergence but on a log odds scale these small differences are amplified, implying that different training runs have estimated significantly different values.
  • The probability scale is simple to interpret (left extreme = 0, right extreme=1) but the log odds scale is (potentially) $-\infty$ to $+\infty$. This means it's not immediately clear what range is being shown, and where the zero line is. (See example below)
  • Related to the previous point, because of this moveable range, it's important to have the x-axis labels visible for each chart to be able to judge differences between points. (See the below)

Fixes

  • MUST - Add zero line
  • SHOULD - Allow probability version of chart (either with an argument or a toggle switch in the chart itself)
  • COULD - Add axis labels to all facets for log odds version
  • COULD - Add subtle lines extending from zero to the points (resembling match_weights_chart()) to highlight positive or negative log odds
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers graphs
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants