[Explainability Evaluation] - Characterization score, and other ways to combine Fid+/- scores #6188
Conversation
| def get_characterization_score(weights: Tensor, | ||
| fidelities: Tensor) -> 'Tensor': | ||
| r"""Returns the componentwise characterization score of | ||
| fidelities[0] and fidelities[1] |
There was a problem hiding this comment.
Would be nice to add here a bit more detail: do the two rows corresponding to fid+ and fid-? Which one is which?
And does the column corresponds to num nodes (explanation)?
There was a problem hiding this comment.
I have added explanation to which row corresponds to what.
As for the columns I'm unsure but shouldn't it be the fidelities of different explanations? As far as I understand it the two fidelities are computed as the sum average over all nodes and serve as a way to gauge the "explanatory power" of different explanations, or am I mistaken here?
Codecov Report
@@ Coverage Diff @@
## master #6188 +/- ##
==========================================
+ Coverage 84.50% 84.51% +0.01%
==========================================
Files 374 376 +2
Lines 20889 20905 +16
==========================================
+ Hits 17652 17668 +16
Misses 3237 3237
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
There was a problem hiding this comment.
This is great @seemanne, thank you! I'm not sure about the docstrings, but once we get confirmation that they compile, we can merge this 👍🏻
Co-authored-by: Blaž Stojanovič <Blaz@stojanovic.si>
|
@BlazStojanovic The docs compile now, I've added a section about these evaluation metrics in |
seemanne
changed the title
|
Super, will take a final look tomorrow. |
This PR implements the features from #5960
The tests and the math part is done, I'm somewhat lost with the sphinx documentation and very much open to suggestions for the docstrings.
As #5962 also requires calculating an AUC and probably a weighted harmonic mean for F_1 type scores these parts have been implemented as separate helper functions.