- Post-processing
← Go back to Modules Development | ↑ Go to the Table of Content ↑ | Continue to Advanced Topics →
This framework is intended for planned post-processing of objects generated by QC Tasks, Checks and correlating them with other data. The most common use-cases include correlation and trending of different properties of the detectors.
The users can write their own Post-processing Tasks or use the ones provided by the framework (see Convenience classes) which are supposed to cover the usual needs. Post-processing Tasks run asynchronously to data-taking, but can be triggered by a set of selected events.
Any Post-processing Task should inherit PostProcessingInterface, which includes four methods:
configure(optional) - provides the task with configuration.initialize- initializes the task and its data, given the event which it was triggered by.update- updates the task and its data, given the event which it was triggered by.finalize- finalizes the processing, given the event which it was triggered by.
Interfaces to databases and other services are accesible via ServiceRegistry, which is an argument to the last three methods. They are invoked when any of the specified triggers is up, which can be:
- Start Of Run - triggers when receives SOSOR message from
aliecs.runkafka topic which has DIFFERENT run number AND environment id thanActivityclass in config - End Of Run - triggers when receives SOEOR message from
aliecs.runkafka topic which has DIFFERENT run number AND environment id thanActivityclass in config - Start Of Fill (SOF, not implemented yet)
- End Of Fill (EOF, not implemented yet)
- Periodic - triggers when a specified period of time passes
- New Object - triggers when an object in QCDB is updated
- For Each Object - triggers for each object in QCDB which matches an Activity
- For Each Latest - triggers for the latest object in QCDB for each matching Activity, sorted by Period, Pass and Run
- Once - triggers only first time it is checked
- Always - triggers each time it is checked
Triggers are complemented with:
- timestamps which correspond the time when trigger started to be valid, in form of ms since epoch, just like in CCDB and QCDB,
lastflag, beingtrueif it is the last time trigger will fire,Activityobject, which contains metadata such as run type and number, pass name, period name, data provenance.
For example, the periodic trigger will provide evenly spaced timestamps, even if the trigger is checked more rarely. The New Object trigger provide the timestamp of the updated object. The timestamps and Activites should be used to access databases, so any Post-processing Task can be rerun at any time for any run and reconstruction pass.
The Activity specified at the top of the configuration file is used to for triggers to match objects which belong to certain run, pass, period. A lack of a parameter or a default value are treated as a wildcard. Since AliECS overwrites the run number during initialization, one may force the run number wildcard by adding the following key-value pair:
{
"qc": {
"config": {
"postprocessing": {
"matchAnyRunNumber": "true"
}
},MonitorObjects may be saved by registering them in ObjectManager, similarly to normal QC Tasks (recommended, see examples linked below), or by using DatabaseInterface directly. Please note, that created objects have to registered in ObjectManager to make them accessible by Checks.
Please refer to SkeletonPostProcessing for a minimal illustration of inheriting the interface, or to TrendingTask for a fully functional example. One can generate their own post-processing task by using the o2-qc-module-configurator helper, as described in the Module Creation chapter.
Running the post-processing is configured in a similar manner as it is for QC Tasks and Checks - the configuration parameters are stored in a JSON file or in the Configuration database (at later development stage). The configuration's path should be passed to the application running a task.
This is a snippet of a JSON structure which configures a post-processing task:
{
"qc": {
"config": {
...
"postprocessing": {
"periodSeconds": 10.0
}
},
"postprocessing": {
"MyPostProcessingTaskID": {
"active": "true",
"taskName": "MyPostProcessingTask",
"className": "o2::quality_control_modules::my_module::MyPPTask",
"moduleName": "QcMyModule",
"detectorName": "TST",
"initTrigger": [
"SOR"
],
"updateTrigger": [
"10mins"
],
"stopTrigger": [
"EOR",
"10hours"
]
},
...
}
}
}Each task is uniquely identified by its id (MyPostProcessingTaskID). One can activate it by setting the "active" field to "true". The task is loaded given its full "className" and a "moduleName" where it is located. The "detectorName" might be used by tasks to store generated data in correct paths in QCDB. The "initTrigger", "updateTrigger" and "stopTrigger" lists contain triggers which should invoke corresponding interface methods. The "periodSeconds" parameter in the common section defines how often should the triggers be checked. Values larger than 10 seconds should be applied when running synchronously to data taking, while very small periods can be used when processing batches of already existing objects.
Checks can be applied to the results of Post-processing Tasks just as for normal QC Tasks. However, one should use
data source type of "PostProcessing" instead of "Task":
...
"checks": {
"ExamplePPCheck": {
"active": "true",
"className": "o2::quality_control_modules::skeleton::SkeletonCheck",
"moduleName": "QcSkeleton",
"policy": "OnAny",
"detectorName": "TST",
"dataSource": [{
"type": "PostProcessing",
"name": "ExampleTrend",
"MOs": ["mean_of_histogram"]
}]
}
},
...
A postprocessing task can access custom parameters declared in the configuration file at qc.postprocessing.<task_id>.extendedTaskParameters. They are stored inside an object of type CustomParameters named mCustomParameters, which is a protected member of TaskInterface.
More details can be found about this feature in the Tasks (same behaviour).
Each of the three methods can be invoked by one or more triggers. Below are listed the possible options (case insensitive).
"sor"or"startofrun"- start of a different run (useful for long-running post-processing which observes many data taking runs). Please ensure that the configuration file includes a kafka broker."eor"or"endofrun"- end of a different run (useful for long-running post-processing which observes many data taking runs). Please ensure that the configuration file includes a kafka broker."sof"or"startoffill"- Start Of Fill (not implemented yet)"eof"or"endoffill"- End Of Fill (not implemented yet)"<x><sec/min/hour>"- Periodic - triggers when a specified period of time passes. For example: "5min", "0.001 seconds", "10sec", "2hours"."newobject:[qcdb/ccdb]:<path>"- New Object - triggers when an object in QCDB or CCDB is updated (applicable for synchronous processing). For example:"newobject:qcdb:qc/TST/MO/QcTask/Example""foreachobject:[qcdb/ccdb]:<path>"- For Each Object - triggers for each object in QCDB or CCDB which matches the activity indicated in the QC config file (applicable for asynchronous processing)."foreachlatest:[qcdb/ccdb]:<path>"- For Each Latest - triggers for the latest object version in QCDB or CCDB for each matching activity (applicable for asynchronous processing). It sorts objects in ascending order by period, pass and run."once"- Once - triggers only first time it is checked"always"- Always - triggers each time it is checked
It might happen that one wants to get data and store data in different databases. Typically if you want to test with production data but store the object in test.
This can be achieved by setting the extra parameter sourceRepo in the task. You have to add it to all your tasks as this is not a global
parameter. It is optional.
The destination repository is always the global one defined in the global configuration under qc.config.database.
"postprocessing": {
"MyPostProcessingTaskID": {
...
"sourceRepo": {
"implementation": "CCDB",
"host": "another-test.cern.ch:8080"
},
...
}
...
}
The post-processing tasks can be run in three ways. First uses the usual o2-qc executable which relies on DPL and
it is the only one which allows to run checks over objects generated in post-processing tasks. This is will be one
of two ways to run PP tasks in production.
To try it out, use it like for any other QC configuration:
o2-qc -b --config json://${QUALITYCONTROL_ROOT}/etc/postprocessing.json
All declared and active tasks in the configuration file will be run in parallel.
To change how often triggers are evaluated, modify the value for qc.config.postprocessing.periodSeconds in the config file.
To run a different configuration which trends all the qc/TST/MO/QcTask/example objects existing in QCDB, try the following:
o2-qc -b --config json://${QUALITYCONTROL_ROOT}/etc/postprocessing-async.json
Debugging post-processing tasks might be easier when using the o2-qc-run-postprocessing application (only for
development) or with o2-qc-run-postprocessing-occ (both development and production), as they are one-process
executables, running only one, chosen task.
To run the basic example, use the command below. The --config parameter should point to the configuration file.
o2-qc-run-postprocessing --config json://${QUALITYCONTROL_ROOT}/etc/postprocessing.json --id ExamplePostprocessing
As it is configured to invoke each method only "once", you will see it initializing, entering the update method, then finalizing the task and exiting.
This executable also allows to run a Post-processing task in batch mode, i.e. with selected timestamps (see the
--timestamps argument). This way, one can rerun a task over old data, if such a task actually respects given
timestamps.
To have more control over the state transitions or to run a standalone post-processing task in production, one should
use o2-qc-run-postprocessing-occ. It is run almost exactly as the previously mentioned application, however one has
to use peanut to drive
its state transitions and push the configuration.
To try it out locally, run the following in the first terminal window (we will try out a different task this time):
o2-qc-run-postprocessing-occ --name ExampleTrend --period 10
In the logs you will see a port number which listens for RPC commands. Remember it.
no control port configured, defaulting to 47100
no role configured, defaulting to default-role
gRPC server listening on port 47100
In the second window, run the following. Use the port number from the output of the QC executable.
# If you haven't built it:
# aliBuild build Coconut --defaults o2-dataflow
alienv enter coconut/latest
OCC_CONTROL_PORT=47100 peanut
A simple terminal user interface will open, which will allow you to trigger state transitions. Use it to load the configuration by entering the path to the configuration file. The usual transition sequence, which you might want to try out, is CONFIGURE, START, STOP, RESET, EXIT.
We aim to provide some convenience classes which should cover the most common post-processing use-cases. Everyone is free to propose extensions to them or write their own tasks for more specific usages taking these as a starting point.
TrendingTask is a post-processing task which uses a TTree to trend objects in the QC database and produce basic plots. The Post-processing example in the QuickStart showcases the possibilities of this class.
The following scheme shows how the class is designed. It can access data sources which are Monitor Objects and Quality Objects from the QCDB, as well as custom objects from the CCDB.
The objects' characteristics which should be tracked are extracted by Reductors - simple plugins. The framework provides a set of Reductors for commonly used data structures, but any custom Reductor might be used as well. In such case, to reduce a Monitor Object or Quality Object, one has to inherit ReductorTObject, while reducing objects from CCDB can be achieved by implementing ReductorConditionAny.
All the values are stored in a TTree.
Each data source forms a separate branch, with its leaves being the individual values.
Additionally added columns include a time branch and a metadata branch (now consisting only of runNumber).
The TTree is stored back to the QC database each time it is updated.
In addition, the class exposes the TTree::Draw interface, which allows to instantaneously generate plots with trends, correlations or histograms that are also sent to the QC database.
Multiple graphs can be drawn on one plot, if needed.
As this class is a post-processing task, it inherits also its configuration JSON template. It extends it, though, some additional parameters.
{
"qc": {
...
"postprocessing": {
"ExampleTrend": {
"active": "true",
"className": "o2::quality_control::postprocessing::TrendingTask",
"moduleName": "QualityControl",
"detectorName": "TST",
"resumeTrend": "false",
"producePlotsOnUpdate": "true",
"trendingTimestamp": "validUntil",
"dataSources": [],
"plots": [],
"initTrigger": [ "once" ],
"updateTrigger": [ "5 seconds" ],
"stopTrigger": []
}
}
}
}Data sources are defined by filling the corresponding structure, as in the example below. For the key "type" use:
"repository"if you access a Monitor Object."repository-quality"if that should be a Quality Object."condition"for trending an object of any type in CCDB.
The "names" array should point to one or more objects under a common "path" in the repository. The values of "reductorName" and "moduleName" should point to a full name of a data Reductor and a library where it is located. One can use the Reductors available in the Common module or write their own by inheriting the interface class.
{
...
"dataSources": [
{
"type": "repository",
"path": "TST/MO/QcTask",
"names": [ "example" ],
"reductorName": "o2::quality_control_modules::common::TH1Reductor",
"moduleName": "QcCommon"
},
{
"type": "repository-quality",
"path": "TST/QO",
"names": [ "QcCheck" ],
"reductorName": "o2::quality_control_modules::common::QualityReductor",
"moduleName": "QcCommon"
},
{
"type": "condition",
"path": "GRP/Calib",
"names": [ "LHCClockPhase" ],
"reductorName": "o2::quality_control_modules::common::LHCClockPhaseReductor",
"moduleName": "QcCommon"
}
],
...
}Similarly, plots are defined by adding proper structures to the "plots" list, as shown below. The plot will be
stored under the "name" value and it will have the "title" value shown on the top.
The "graphs" list can include one more objects, where the "varexp", "selection" and "option" fields correspond to the arguments of the TTree::Draw method.
If there is more than one graph drawn on a plot, a legend will be automatically created.
Optionally, one can use "graphError" to add x and y error bars to a graph, as in the first plot example.
The "name" and "varexp" are the only compulsory arguments, others can be omitted to reduce configuration files size.
"graphAxisLabel" allows the user to set axis labels in the form of "Label Y axis: Label X axis". With "graphYRange" numerical values for fixed ranges of the y axis can be provided in the form of "Min:Max".
{
...
"plots": [
{
"name": "mean_of_histogram",
"title": "Mean trend of the example histogram",
"graphAxisLabel": "Mean X:time",
"graphYRange": "-2.3:6.7",
"graphs" : [{
"title": "mean trend",
"varexp": "example.mean:time",
"selection": "",
"option": "*L",
"graphErrors": "5:example.stddev"
}]
},
{
"name": "histogram_of_means",
"title": "Distribution of mean values in the example histogram",
"graphs" : [{
"varexp": "example.mean",
"selection": "",
"option": "*L"
}]
},
{
"name": "example_quality",
"title": "Trend of the example histogram's quality",
"graphs" : [{
"varexp": "QcCheck.name:time",
"selection": "",
"option": "*"
}]
}
],
...
}To decide whether plots should be generated during each update or just during finalization,
use the boolean flag "producePlotsOnUpdate".
To pick up the last existing trend which matches the specified Activity, set "resumeTrend" to "true".
To generate plots only when all input objects are available, set "trendIfAllInputs".
"trendingTimestamp" allows to select which timestamp should be used as the trending point.
The available options are "trigger" (timestamp provided by the trigger), "validFrom" (validity start in activity provided by the trigger), "validUntil" (validity end in activity provided by the trigger, default).
The SliceTrendingTask is a complementary task to the standard TrendingTask. This task allows the trending of canvas objects that hold multiple histograms (which have to be of the same dimension, e.g. TH1) and the slicing of histograms. The latter option allows the user to divide a histogram into multiple subsections along one or two dimensions which are trended in parallel to each other. The task has specific reductors for TH1 and TH2 objects which are o2::quality_control_modules::common::TH1SliceReductor and o2::quality_control_modules::common::TH2SliceReductor.
Similar to the TrendingTask, the configuration of the SliceTrendingTask is divided into "dataSources" and "plots", where both parts have been extended in respect to the standard trending. Here, only changes in respect to the standard trending task are highlighted.
The data sources are extended by "axisDivision" which configures the slicing of the histograms. The inner most brackets relate the the actual axis. Its configuration can be understood as "axisDivision": [ [x-Axis], [y-Axis] ] where [y-Axis] does not need to be provided in case of one-dimensional objects. The values provided in [x(y)-Axis] are the numerical boundaries of the x(y)-axis. For n slices, one thus needs to provide n+1 values in ascending order. Protections are added such that each bin is part of only one slice. If the outer brackets are left empty (i.e. "axisDivision": [ ]), no slicing is applied and the whole histogram is trended as in the standard trending task.
{
...
"dataSources": [
{
"type": "repository",
"path": "TST/MO/QcTask",
"names": [ "example" ],
"reductorName": "o2::quality_control_modules::common::TH1SliceReductor",
"axisDivision": [ [ "0", "4500", "10500" ] ],
"moduleName": "QcCommon"
}
],
...
}
The "plot" configuration has changed in respect to the standard trending task as follows:
The "varexp" selection is still set up as "Histogram.Var:TrendingType" where "Histogram.Var" is trended vs "TrendingType". The options for "Var"are:
"entries": Number of entries of the slice"meanX": Mean along the x-axis of the slice"stddevX": Stddev along the x-axis of the slice"errMeanX": Error of the mean along the x-axis of the slice"meanY": Mean along the y-axis of the slice."stddevY": Stddev along the y-axis of the slice"errMeanY": Error of the mean along the y-axis of the slice
In case of 1 dimensional objects, "meanY" is calculated as the arithmetic mean of all the bin values in the slice. The respective "stddevY" and "errMeanY" are provided as well.
The options for "TrendingType" are limited to:
"time": The quantity"Histogram.Var"of all slices is trended as a function of time. Each slice-trending has its own graph which are all published on one canvas."multigraphtime": The quantity"Histogram.Var"of all slices is trended as a function of time. All slice-trendings are published on one"TMultiGraph". A legend is provided which contains the numerical boundaries of the slices."slices": The quantity"Histogram.Var"of all slices is trended as a function of the geometrical center of the slices. Always the latest timestamp is plotted."slices2D": The quantity"Histogram.Var"of all slices is trended as a function of the geometrical center of the slices in two dimensions. Always the latest timestamp is plotted. Errors (if used) are stored per bin but are not visualized.
The field "graphErrors" is set up as "graphErrors":"Var1:Var2" where Var1 is the error along y and Var2 the error along x. For Var1(2) numerical values or the options listed for Var above can be set. The original histogram does not need to be provided as the task will take the histogram specified in "varexp": "Histogram.Var:TrendingType". In "graphYRange" and "graphXRange" numerical values for fixed ranges of the x and y axis can be provided in the form of "Min:Max". If provided, the task will set all x (or y) axis on the canvas to this range. "graphAxisLabel" allows the user to set axis labels in the form of "Label Y axis: Label X axis".
{
...
"plots": [
{
"name": "ExtendedTrending_meanX_of_histogram",
"title": "Mean X trend of the example histogram",
"varexp": "example.meanX:time",
"selection": "",
"option": "*L",
"graphErrors": "errMeanX:0.5",
"graphYRange": "",
"graphXRange": "",
"graphAxisLabel": "Mean X:time"
},
{
"name": "ExtendedTrending_meanY_of_histogram_slices",
"title": "Mean Y trend of the example histogram",
"varexp": "example.meanY:slices",
"selection": "",
"option": "*L",
"graphErrors": "errMeanY:errMeanX",
"graphYRange": "",
"graphXRange": "-500.0:10000",
"graphAxisLabel": "Mean Y:Center of slices along x"
},
{
"name": "ExtendedTrending_meanY_of_histogram_timeMultigraph",
"title": "Mean Y trend of the example histogram",
"varexp": "example.meanY:multigraphtime",
"selection": "",
"option": "*L",
"graphErrors": "errMeanY:0.5",
"graphYRange": "",
"graphXRange": "",
"graphAxisLabel": "Mean Y:time"
}
],
...
}
This post-processing task draws a given set of plots in comparison with their corresponding references, both as superimposed histograms and as current/reference ratio histograms.
Currently the source of reference data is specified as a run-type and beam-type specific referenceRun number. This will be modified once a centralized way of accessing reference plots will become available in the framework.
The notOlderThan option allows to ignore monitor objects that are older than a given number of seconds. A value of -1 means "no limit".
The ignorePeriodForReference and ignorePassForReference boolean parameters control whether the period and/or pass names should be matched or not when querying the reference plots from the database.
A value of "1" (default) means that the reference plots are not required to match the period and/or pass names of the current run, while a value of "0" means that the reference plot is retrieved only if the corresponding period and/or pass names match those of the current run.
The input MonitorObjects to be processed are logically divided in dataGroups. Each group is configured via the following parameters:
inputPath: path in the QCDB where the input objects are locatedreferencePath(optional): specifies the path for the reference objects, if not set theinputPathis usedoutputPath: path in the QCDB where the output objects are storeddrawRatioOnly: boolean parameter specifying wether to only draw the ratio plots, or the current/reference comparisons as welldrawOption1D: the ROOT draw option to be used for the 1-D histogramsdrawOption2D: the ROOT draw option to be used for the 2-D histograms
The input objects are searched within the inputPath, and the output plots are stored inside the outputPath.
It is also possible to optionally specify a different path for the reference objects, via the referencePath parameter. If not given, the referencePath will coincide with the inputPath.
The normalizeReference boolean parameter controls wether the reference histogram is scaled such that its integral matches that of the current plot.
The checker extracts the current and reference plots from the stored MO, and compares them using external modules, specified via the moduleName and comparatorName parameters. The threshold parameter specifies the value used to discriminate between good and bad matches between the histograms.
Four comparison modules are provided in the framework:
o2::quality_control_modules::common::ObjectComparatorDeviation: comparison based on the average relative deviation between the bins of the current and reference histograms; the module accepts the following configuration parameters:threshold: the maximum allowed average relative deviation between current and reference histogramsrangeX,rangeY: if set, the comparison is restricted to the bins in the specified X and Y ranges; bins outside the ranges are ignored
o2::quality_control_modules::common::ObjectComparatorBinByBinDeviation: comparison based on the relative deviation between each bin of the current and reference histograms; the module accepts the following configuration parameters:threshold: the maximum allowed relative deviation for each bin between current and reference histogramsrangeX,rangeY: if set, the comparison is restricted to the bins in the specified X and Y ranges; bins outside the ranges are ignoredmaxAllowedBadBins: the maximum number of bins above threshold for which the quality is still considered Good
o2::quality_control_modules::common::ObjectComparatorChi2: comparison based on a standard chi2 test between the current and reference histograms; the module accepts the following configuration parameters:threshold: the minimum allowed chi2 probabilityrangeX,rangeY: if set, the Chi2 test is restricted to the bins in the specified X and Y ranges; bins outside the ranges are ignored
o2::quality_control_modules::common::ObjectComparatorKolmogorov: comparison based on a standard Kolmogorov test between the current and reference histograms; the module accepts the following configuration parameters:threshold: the minimum allowed Kolmogorov probability
All the configuration parameters of the comparison modules optionally allow to restrict their validity to specific plots.
The following example specifies a threshold value common to all the plots, and then overrides the threshold and X range for all plots named TrackEta:
"extendedCheckParameters": {
"default": {
"default": {
"moduleName" : "QualityControl",
"comparatorName" : "o2::quality_control_modules::common::ObjectComparatorChi2",
"threshold" : "0.5",
"threshold:TrackEta" : "0.2",
"rangeX:TrackEta" : "-3.5,-2.5"
}
}
}In the example configuration below, the relationship between the input and output histograms is the following:
MCH/MO/Tracks/WithCuts/TrackEta(1-D histogram)MCH/MO/RefComp/TracksMCH/WithCuts/TrackEta- 1-D version, current and reference plots drawn superimposed in the same canvas, with the ratio below
- comparison with a chi2 test method
MCH/MO/Tracks/WithCuts/TrackEtaPhi(2-D histogram)MCH/MO/RefComp/TracksMCH/WithCuts/TrackEtaPhi- 2-D version, ratio between plots drawn on top with the current and reference plots drawn smaller at the bottom
- comparison with a chi2 test method (
"comparatorName" : "o2::quality_control_modules::common::ObjectComparatorChi2)
{
"qc": {
"config": {
"": "The usual global configuration variables"
},
"postprocessing": {
"ExampleRefComp": {
"active": "true",
"className": "o2::quality_control_modules::common::ReferenceComparatorTask",
"moduleName": "QualityControl",
"detectorName": "MCH",
"extendedTaskParameters": {
"default": {
"default": {
"notOlderThan" : "300",
"referenceRun" : "551875",
"ignorePeriodForReference": "1",
"ignorePassForReference": "1"
}
},
"PHYSICS": {
"PROTON-PROTON": {
"referenceRun" : "551890"
}
}
},
"dataGroups": [
{
"name": "Tracks",
"inputPath": "MCH/MO/Tracks/WithCuts",
"referencePath": "MCH/MO/Tracks",
"outputPath": "Tracks/WithCuts",
"normalizeReference": "true",
"drawRatioOnly": "false",
"drawOption1D": "E",
"drawOption2D": "COL",
"inputObjects": [
"TrackEta",
"TrackEtaPhi"
]
}
],
"initTrigger": [
"userorcontrol"
],
"updateTrigger": [
"60 seconds"
],
"stopTrigger": [
"userorcontrol"
]
}
},
"checks": {
"ExampleRefCheck": {
"active": "true",
"className": "o2::quality_control_modules::common::ReferenceComparatorCheck",
"moduleName": "QualityControl",
"detectorName": "MCH",
"policy": "OnAny",
"extendedCheckParameters": {
"default": {
"default": {
"moduleName" : "QualityControl",
"comparatorName" : "o2::quality_control_modules::common::ObjectComparatorChi2",
"threshold" : "0.5",
"threshold:TrackEta" : "0.2",
"rangeX:TrackEta" : "-3.5,-2.5"
}
}
},
"dataSource": [
{
"type": "PostProcessing",
"name": "ExampleRefComp",
"MOs" : [
"Tracks/WithCuts/TrackEta",
"Tracks/WithCuts/TrackEtaPhi"
]
}
]
}
}
}
}A post-processing task that checks the existence, time stamp and validity of CCDB/QCDB objects. The task produces a 2-D plot with object indexes in the X-axis (the bin labels are set to the mnemonic name of the object defined in the configuration) and the check status on the Y-axis, where the first bin corresponds to OK and the following bins to different errors. The 2-D bins are populated according to the result of the object inspection.
A CcdbInspectorCheck task receives the 2-D histogram produced by the CCDB inspector and outputs an overall quality based on the status flag of each object in the histogram.
The input objects are specified in the DataSources section. Each object is identified by the following parameters:
name: the name of the object, which is used to label the X-axis bins of the output histogrampath: the path of the object in the databaseupdatePolicy: the policy with wich the object is updated. Possible values are:atSOR: the object is only created once after start-of-runatEOR: the object is only created once at end-of-runperiodic: the object is created periodically during the run
cycleDuration: for periodic objects, the time interval between updatesvalidatorName: (optional) name of the software module used to validate the contents of the objectmoduleName: library where the validator module is located
The task accepts the following configuration parameters:
timeStampTolerance: tolerance (in seconds) applied when comparing the actual and expected object time stampdatabaseType: type of input database. Possible values areccdborqcdb(default:ccdb)databaseUrl: address of the database (default:https://alice-ccdb.cern.ch)retryTimeout: timeout (in seconds) for accessing the objects at the task finalizationretryDelay: delay (in seconds) between the retries when accessing the objects at the task finalizationverbose: print additional debugging messages
{
"qc": {
"config": {
"": "The usual global configuration variables"
},
"postprocessing": {
"CcdbInspector": {
"active": "true",
"className": "o2::quality_control_modules::common::CcdbInspectorTask",
"moduleName": "QualityControl",
"detectorName": "GLO",
"extendedTaskParameters": {
"default": {
"default": {
"verbose" : "1",
"timeStampTolerance": "60",
"retryTimeout" : "60",
"retryDelay": "10",
"databaseType": "ccdb",
"databaseUrl": "https://alice-ccdb.cern.ch"
}
}
},
"dataSources": [
{
"name": "Mean Vertex",
"path": "GLO/Calib/MeanVertex",
"updatePolicy": "periodic",
"cycleDuration": "200",
"validatorName": "o2::quality_control_modules::glo::MeanVertexValidator",
"moduleName": "QcGLO"
},
{
"name": "CTP Config",
"path": "CTP/Config/Config",
"updatePolicy": "atSOR"
},
{
"name": "CTP Scalers",
"path": "CTP/Calib/Scalers",
"updatePolicy": "atEOR"
}
],
"initTrigger": [
"userorcontrol"
],
"updateTrigger": [
"30 seconds"
],
"stopTrigger": [
"userorcontrol"
]
}
},
"checks": {
"CcdbInspectorCheck": {
"active": "true",
"className": "o2::quality_control_modules::common::CcdbInspectorCheck",
"moduleName": "QualityControl",
"detectorName": "GLO",
"policy": "OnAll",
"dataSource": [
{
"type": "PostProcessing",
"name": "CcdbInspector",
"MOs" : [
"ObjectsStatus"
]
}
]
}
}
}
}This task allows to trend a set of QualityObjects (QO) stored in the QCDB, and to display their name and value in human-readable format on a canvas (see the figure below). For each QualityObject, it also creates a 1-D distribution of the corresponding quality values (Good/Medium/Bad/Null). Each Quality can be optionally accompanied by a message that can provide instructions or further details based on the quality value. Each time the post-processing task is triggered, the QualityObjects are fetched from the CCDB and the canvas is updated with the current values. Moreover, the trending plot and 1-D distribution associated to each QO is updated accordingly. Hence, the trending and 1-D distribution can be used to estimate the fraction of good data for a given run.
The QualityObjects to be monitored and displayed are passed as qualityGroups, each containing inputObjects for a specific, line-separated group. Each group requires a name and path to the contained objects. A title can be added optionally, which appears at the top of the group in the canvas. By listing certain qualities in ignoreQualitiesDetails one can ask to ignore Flags associated to QualityObjects.
The inputObjects list should contain Quality Object names in a given group. A title can be added, which is used in the summary canvas to denote given Quality Object. If it is absent, name is used instead. Optionally, one can add messageBad, messageMedium, messageGood, messageNull to add a message when a particular Quality is seen.
At each update, the task retrieves the latest version of each input QualityObject, even if their validity range ends in the past. A task configuration parameter, called maxObjectAgeSeconds, allows to define the maximum allowed age (in seconds) of the retrieved objects. The age is defined as the difference between the the time stamp of the task update and the creation time stamp of the retrieved object.
Here is a complete example of QualityTask configuration:
{
"qc": {
"config": {
"": "The usual global configuration variables"
},
"postprocessing": {
"ExampleQualityTask": {
"active": "true",
"className": "o2::quality_control_modules::common::QualityTask",
"moduleName": "QualityControl",
"detectorName": "TST",
"taskParameters": {
"maxObjectAgeSeconds": "300"
},
"qualityGroups": [
{
"name" : "global",
"title" : "GLOBAL TST QUALITY",
"path": "TST/QO",
"ignoreQualitiesDetails" : ["Null", "Good", "Medium", "Bad"],
"inputObjects": [
{
"name" : "QcCheck",
"title" : "Aggregated TST Quality",
"messageBad" : "Inform XYZ on-call immediately",
"messageMedium": "Add bookkeeping entry",
"messageGood": "All checks are OK",
"messageNull": "Some histograms are empty!!!"
}
]
},
{
"name" : "details",
"title" : "TST DETAILS",
"path": "TST/QO",
"ignoreQualitiesDetails" : [],
"inputObjects": [
{
"name" : "QcCheck",
"title" : ""
},
{
"name" : "someNumbersCheck",
"title" : ""
},
{
"name" : "XYZCheck",
"title" : ""
}
]
}
],
"initTrigger": [
"userorcontrol"
],
"updateTrigger": [
"60 seconds"
],
"stopTrigger": [
"userorcontrol", "10 minutes"
]
}
}
}
}This task allows to transform a set of QualityObjects stored QCDB across certain timespan (usually for the duration of a data acquisition run) into a QualityControlFlagCollection. It is meant to be run after for each detector/subsystem separately and when all QualityObjects for a run are generated. After generating timestamps, final data tags can be computed as the next step. The data formats for tagging data quality are described here.
The task should be run asynchronously to data-taking and should be given the start and end of a time range to process. For example:
o2-qc-run-postprocessing --config json://${QUALITYCONTROL_ROOT}/Modules/Common/etc/flagcollection-example.json \
--name FlagCollectionQcCheck --timestamps 1612707603626 1613999652000The task is configured as follows:
{
"qc": {
"config": {
"": "The usual global configuration variables"
},
"postprocessing": {
"FlagCollectionQcCheck": {
"active": "true",
"className": "o2::quality_control_modules::common::FlagCollectionTask",
"moduleName": "QcCommon",
"detectorName": "TST", "": "One task should concatenate Qualities from detector, defined here.",
"initTrigger": [], "": "The triggers can be left empty,",
"updateTrigger": [], "": "because we run the task with a defined set of timestamps.",
"stopTrigger": [],
"": "The list of Quality Object to process.",
"QOs": [
"QcCheck"
]
}
}
}
}QualityControlFlagCollections are meant to be used as a base to derive Data Tags for analysis (WIP).
This task allows to display the aggregated quality flags of each system (detectors + global QC) in a single canvas, arranged as a matrix of colored boxes. The system names are displayed above each box, while the quality flag is displayed inside the box. The box color follows the usual convention:
- Good: green
- Medium: orange
- Bad: red
- Null: magenta
In addition, the boxes are filled with a grey color is the corresponding QualityObjects cannot be retrieved or are too old.
The color of the canvas background and of the detector labels can be customized with the "foregroundColor" and "backgroundColor" parameters. They accept interger values corresponding to the indexes of the default ROOT colors or the indexes defined in the color wheel. The example below shows a color combination with white text over a dark gray background.
The task is configured as follows:
{
"qc": {
"config": {
"": "The usual global configuration variables"
},
"postprocessing": {
"BigScreen": {
"active": "true",
"className": "o2::quality_control_modules::common::BigScreen",
"moduleName": "QualityControl",
"detectorName": "GLO",
"extendedTaskParameters": {
"default": {
"default": {
"nRows": "4",
"nCols": "5",
"borderWidth": "1",
"": "white text over dark gray background",
"foregroundColor": "0",
"backgroundColor": "923",
"maxObjectTimeShift": "10000",
"ignoreActivity": "0",
"labels": "CPV,EMC,FDD,FT0,FV0,HMP,ITS,MCH,MFT,MID,PHS,TPC,TOF,TRD,,TRK,MTK,VTX,PID"
}
}
},
"dataSources": [
{
"names": [
"CPV:MCH/QO/CPVQuality/CPVQuality",
"EMC:MCH/QO/EMCQuality/EMCQuality",
"FDD:MCH/QO/FDDQuality/FDDQuality",
"FT0:MCH/QO/FT0Quality/FT0Quality",
"FV0:MCH/QO/FV0Quality/FV0Quality",
"HMP:MCH/QO/HMPQuality/HMPQuality",
"ITS:MCH/QO/ITSQuality/ITSQuality",
"MCH:MCH/QO/MCHQuality/MCHQuality",
"MFT:MCH/QO/MFTQuality/MFTQuality",
"MID:MCH/QO/MIDQuality/MIDQuality",
"PHS:MCH/QO/PHSQuality/PHSQuality",
"TOF:MCH/QO/TOFQuality/TOFQuality",
"TPC:MCH/QO/TPCQuality/TPCQuality",
"TRD:MCH/QO/TRDQuality/TRDQuality",
"ZDC:MCH/QO/ZDCQuality/ZDCQuality",
"TRK:MCH/QO/TRKQuality/TRKQuality",
"MTK:MCH/QO/MTKQuality/MTKQuality",
"VTX:MCH/QO/VTXQuality/VTXQuality",
"PID:MCH/QO/PIDQuality/PIDQuality"
]
}
],
"initTrigger": [
"userorcontrol"
],
"updateTrigger": [
"60 seconds"
],
"stopTrigger": [
"userorcontrol"
]
}
}
}
}The following options allow to configure the appearence and behavior of the task:
nRows/nCols: size of the X-Y gridborderWidth: size of the border around the boxesmaxObjectTimeShift: ignore quality objects that are older than a given number of seconds. A value of -1 means "no limit".ignoreActivity: if different from 0, the task will fetch objects regardless of their activity number and type.labels: comma-separated list of labels with boxes to be displayed in the canvas. Some places in the grid of boxes can be left empty by inserting two consecutive commas in the list, like betweenTRDandTRKin the example above
The names in the data sources are composed of two parts, separated by a colon:
LABEL:OBJECT_PATH
The LABEL should match one of the elements of the labels parameter. The quality object will be associated to the corresponding box.
This section contains examples of how to approach usual use-cases.
Use NewObject as the update trigger:
"updateTrigger": [ "newobject:qcdb:TST/MO/QcTask/example" ],If the post-processing runs in a different AliECS environment than the acquisition run, one should add the following flag. Since AliECS adds a concrete run number to the workflow, the triggers would match only objects from the same run.
"qc": {
"config": {
...
"postprocessing": {
"matchAnyRunNumber": "true"
}
}Use Periodic or NewObject (but with ccdb as the DB) as the update trigger:
"updateTrigger": [ "newobject:ccdb:TPC/Calib/IDC_0_A" ],or
"updateTrigger": [ "60s" ],In your trending task, make sure to retrieve the object from the CCDB, not QCDB.
If you want to keep the task running regardless of the data-taking activity, please contact the QC developers to set up a long-running workflow in Nomad.
In your QC task, enable the moving window feature on the selected plot. More details can be found in Advanced/Moving window.
Use the NewObject trigger on the moving window to update the task:
"updateTrigger": [ "newobject:qcdb:TST/MO/QcTask/mw/example" ],In your postprocessing task, retrieve the object you want to trend. For TrendingTask, it would be:
"dataSources": [
{
"type": "repository",
"path": "TST/MO/QcTask/mw",
"names": [ "example" ],
"reductorName": "o2::quality_control_modules::common::TH1Reductor",
"moduleName": "QcCommon"
}
]Use ForEachObject as the update trigger:
"updateTrigger": [ "foreachobject:qcdb:TST/MO/QcTask/example" ],Since objects are usually published in collections at the same time, you can use a path for one object to be triggered for a collection of them (all objects produced by a QC Task).
Use the Activity which matches the run, and (optionally) period and pass name:
"Activity": {
"number": "3212",
"type": "",
"passName": "apass2",
"periodName" : "OCT",
"provenance" : "qc"
},
"postprocessing": {
"periodSeconds": 0.01
}I want to run postprocessing for all objects in all the runs of a given reconstruction pass and period
Use ForEachObject as the update trigger:
"updateTrigger": [ "foreachobject:qcdb:TST/MO/QcTask/example" ],Use the Activity which leaves the run number empty, but indicate the pass and period names.
"Activity": {
"number": "",
"type": "",
"passName": "apass2",
"periodName" : "OCT",
"provenance" : "qc"
},
"postprocessing": {
"periodSeconds": 0.01
}I want to run postprocessing for all objects in all the runs of a given reconstruction pass and period which are valid in given time interval
Use ForEachObject as the update trigger:
"updateTrigger": [ "foreachobject:qcdb:TST/MO/QcTask/example" ],Use the Activity which leaves the run number empty, but indicate the pass and period names.
Add start and end values in ms since epoch to restrict the validity start of objects.
"Activity": {
"number": "",
"type": "",
"passName": "apass2",
"periodName" : "OCT",
"provenance" : "qc",
"start" : "1649417693630",
"end" : "1649417800000"
},
"postprocessing": {
"periodSeconds": 0.01
}I want to run postprocessing for the latest object for each available run in a given pass and period
Use ForEachObject as the update trigger:
"updateTrigger": [ "foreachlatest:qcdb:TST/MO/QcTask/example" ],This way you will avoid iterating on potential duplicates and intermediate objects, and get only the final versions instead.
Use the Activity which leaves the run number empty, but indicate the pass and period names.
"Activity": {
"number": "",
"type": "",
"passName": "apass2",
"periodName" : "OCT",
"provenance" : "qc"
},
"postprocessing": {
"periodSeconds": 0.01
}← Go back to Modules Development | ↑ Go to the Table of Content ↑ | Continue to Advanced Topics →