Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

leaderboard: objective functions are hard to interpret #85

Closed
KrisThielemans opened this issue Aug 15, 2024 · 1 comment
Closed

leaderboard: objective functions are hard to interpret #85

KrisThielemans opened this issue Aug 15, 2024 · 1 comment
Assignees
Labels
question Further information is requested wontfix This will not be worked on

Comments

@KrisThielemans
Copy link
Member

KrisThielemans commented Aug 15, 2024

There are a few issues with how we plot the objective functions via TensorBoard:

  • BSREM maximises the SIRF objective function
  • ISTA minimises minus the SIRF objective function
  • OSEM maximises the SIRF log-likelihood, but update_objective() just does return 0 (as it's not used by the OSEM update())

We will not fix this, as:

  • the objective function is not part of the metrics (evaluation phase: skip objective #82)
  • participants could use other conventions for the objective function (e.g. the CIL KL implementation uses a different offset)

We will keep the objective function on TensorBoard for now to help participants debug, but they will likely need to select/filter one algorithm at a time to benefit from y-axis autoscaling.

@KrisThielemans KrisThielemans added the wontfix This will not be worked on label Aug 15, 2024
@casperdcl casperdcl changed the title Objective functions plts in Leaderboard are hard to interprete (won't fix) leaderboard: objective functions are hard to interpret Aug 15, 2024
@casperdcl casperdcl added the question Further information is requested label Aug 15, 2024
@casperdcl
Copy link
Member

this issue is cross-referenced in https://github.com/SyneRBI/PETRIC/wiki/FAQ

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested wontfix This will not be worked on
Projects
None yet
Development

No branches or pull requests

2 participants