Add a generic reference histogram comparator #2317
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aferrero2707
requested review from
Barthelemy,
knopers8 and
justonedev1
as code owners
aferrero2707
force-pushed
the
common-add-reference-plots-comparator
branch
3 times, most recently
from
c18cd63
to
f1e582f
Compare
Barthelemy
reviewed
Jun 14, 2024
| // | ||
| // Helper function for retrieving the last MonitorObject for a give run number | ||
|
|
||
| static std::shared_ptr<MonitorObject> getMOFromRun(repository::DatabaseInterface* qcdb, std::string fullPath, uint32_t run, Activity activity) |
There was a problem hiding this comment.
we should consider moving this to the Framework probably.
There was a problem hiding this comment.
fullPath could also be a reference here
There was a problem hiding this comment.
Reference added.
Barthelemy
reviewed
Jun 14, 2024
| // | ||
| // Helper function for retrieving the last MonitorObject for a give run number | ||
|
|
||
| static std::shared_ptr<MonitorObject> getMOFromRun(repository::DatabaseInterface* qcdb, std::string fullPath, uint32_t run, Activity activity) |
There was a problem hiding this comment.
fullPath could also be a reference here
The code introduces some tools that allow to compare the plots produced by given task(s) with that of a reference run:
- `ReferenceComparatorTask` draws histograms with their reference superimposed, as well as the ratio between histograms and reference
- `ReferenceComparatorCheck` compares the histograms with their corresponding reference, and assesses their compatibility. The comparison is performed by a dynamically loadable module that provides the actual comparison algorithm
- `ReferenceComparatorPlot` is an utility class to draw the current and reference histograms, and their ratio
- `ObjectComparator*` are dynamically loadable modules than implement different histogram comparison methods
Three examples are provided:
- `ObjectComparatorDeviation` evaluates the average relative difference between the histogram bins
- `ObjectComparatorChi2` compares the histograms using a chi2-based test
- `ObjectComparatorKolmogorov` compares the histograms using a Kolmogorov test
aferrero2707
force-pushed
the
common-add-reference-plots-comparator
branch
from
f1e582f
to
916d1ce
Compare
|
@Barthelemy I have pushed an update that should address all the comments, let me know if anything is still missing. Thanks a lot! |
Barthelemy
approved these changes
Jun 18, 2024
|
all good, thanks ! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The code introduces some tools that allow to compare the plots produced by given task(s) with that of a reference run:
ReferenceComparatorTaskdraws histograms with their reference superimposed, as well as the ratio between histograms and referenceReferenceComparatorCheckcompares the histograms with their corresponding reference, and assesses their compatibility. The comparison is performed by a dynamically loadable module that provides the actual comparison algorithmReferenceComparatorPlotis an utility class to draw the current and reference histograms, and their ratioObjectComparator*are dynamically loadable modules than implement different histogram comparison methodsThree examples are provided:
ObjectComparatorDeviationevaluates the average relative difference between the histogram binsObjectComparatorChi2compares the histograms using a chi2-based testObjectComparatorKolmogorovcompares the histograms using a Kolmogorov testThis is an example of output of the task and checker, also showing the flags associated to the quality objects:
