← Go back to Modules Development | ↑ Go to the Table of Content ↑ | Continue to Advanced Topics →
This framework is intended for planned post-processing of objects generated by QC Tasks, Checks and correlating them with other data. The most common use-cases include correlation and trending of different properties of the detectors.
The users can write their own Post-processing Tasks or use the ones provided by the framework (see Convenience classes) which are supposed to cover the usual needs. Post-processing Tasks run asynchronously to data-taking, but can be triggered by a set of selected events.
Any Post-processing Task should inherit PostProcessingInterface, which includes four methods:
configure(optional) - configures the task, given its name and a configuration interface.initialize- initializes the task and its data, given the event which it was triggered by.update- updates the task and its data, given the event which it was triggered by.finalize- finalizes the processing, given the event which it was triggered by.
Interfaces to databases and other services are accesible via ServiceRegistry, which is an argument to the last three methods. They are invoked when any of the specified triggers is up, which can be:
- Start Of Run (SOR)
- End Of Run (EOR)
- Start Of Fill (SOF)
- End Of Fill (EOF)
- Periodic - triggers when a specified period of time passes
- New Object - triggers when an object in QCDB is updated
- Once - triggers only first time it is checked
- Always - triggers each time it is checked
Please refer to SkeletonPostProcessing for a minimal illustration of inheriting the interface, or to TrendingTask for a fully functional example. One can generate their own post-processing task by using the o2-qc-module-configurator helper, as described in the Module Creation chapter.
Running the post-processing is configured in a similar manner as it is for QC Tasks and Checks - the configuration parameters are stored in a JSON file or in the Configuration database (at later development stage). The configuration's path should be passed to the application running a task.
This is a snippet of a JSON structure which configures a post-processing task:
{
"qc": {
"config": {
...
},
"postprocessing": {
"MyPostProcessingTask": {
"active": "true",
"className": "o2::quality_control_modules::my_module::MyPPTask",
"moduleName": "QcMyModule",
"detectorName": "TST",
"initTrigger": [
"SOR"
],
"updateTrigger": [
"10mins"
],
"stopTrigger": [
"EOR",
"10hours"
]
},
...
}
}
}Each task is identified by its name (MyPostProcessingTask). One can activate it by setting the "active" field to "true". The task is loaded given its full "className" and a "moduleName" where it is located. The "detectorName" might be used by tasks to store generated data in correct paths in QCDB. The "initTrigger", "updateTrigger" and "stopTrigger" lists contain triggers which should invoke corresponding interface methods.
Each of the three methods can be invoked by one or more triggers. Below are listed the possible options (case insensitive).
"sor"or"startofrun"- Start Of Run"eor"or"endofrun"- End Of Run"sof"or"startoffill"- Start Of Fill"eof"or"endoffill"- End Of Fill"<x><sec/min/hour>"- Periodic - triggers when a specified period of time passes. For example: "5min", "0.001 seconds", "10sec", "2hours"."newobject:<path>"- New Object - triggers when an object in QCDB is updated. For example:"newobject:/qc/TST/QcTask/Example""once"- Once - triggers only first time it is checked"always"- Always - triggers each time it is checked
The post-processing tasks can be run by using the o2-qc-run-postprocessing application. At the moment it does not use DPL nor OCC library, which means, that it cannot be integrated with AliECS yet, i.e. cannot be run from GUI within an FLP Suite.
To run the basic example, use the command below. The --config parameter should point to the configuration file. The --rate parameter specifies how often the specified triggers should be checked.
o2-qc-run-postprocessing --config json://${QUALITYCONTROL_ROOT}/etc/postprocessing.json --name ExamplePostprocessing --rate 10
As it is configured to invoke each method only "once", you will see it initializing, entering the update method, then finalizing the task and exiting.
We aim to provide some convenience classes which should cover the most common post-processing use-cases. Everyone is free to propose extensions to them or write their own tasks for more specific usages taking these as a starting point.
TrendingTask is a post-processing task which uses a TTree to trend objects in the QC database and produce basic plots. The Post-processing example in the QuickStart showcases the possibilities of this class.
The following scheme shows how the class is designed. It can access data sources which are Monitor Objects and Quality Objects from the Quality Control Database - anything that is generated by other Tasks and Checks. In the future we will also support access to the CCDB.
The objects' characteristics which should be tracked are extracted by Reductors - simple plugins. The framework provides a set of Reductors for commonly used data structures, but any custom Reductor might be used as well.
All the values are stored in a TTree.Each data source forms a separate branch, with its leaves being the individual values. Additionally added columns include a time branch and a metadata branch (now consisting only of runNumber).
The TTree is stored back to the QC database each time it is updated. In addition, the class exposes the TTree::Draw interface, which allows to instantaneously generate plots with trends, correlations or histograms that are also sent to the QC database.
As this class is a post-processing task, it inherits also its configuration JSON template. It extends it, though, with two additional lists - "dataSources" and "plots":
{
"qc": {
...
"postprocessing": {
"ExampleTrend": {
"active": "true",
"className": "o2::quality_control::postprocessing::TrendingTask",
"moduleName": "QualityControl",
"detectorName": "TST",
"dataSources": [],
"plots": [],
"initTrigger": [ "once" ],
"updateTrigger": [ "5 seconds" ],
"stopTrigger": []
}
}
}
}Data sources are defined by filling the corresponding structure, as in the example below. For the key "type" use the value "repository" if you access a Monitor Object and "repository-quality" if that should be a Quality (this will be unified in the future). The fields "path" and "name" should point to the object in the repository. The values of "reductorName" and "moduleName" should point to a full name of a data Reductor and a library where it is located. One can use the Reductors available in the Common module or write their own by inheriting the interface class.
{
...
"dataSources": [
{
"type": "repository",
"path": "qc/TST/QcTask",
"name": "example",
"reductorName": "o2::quality_control_modules::common::TH1Reductor",
"moduleName": "QcCommon"
},
{
"type": "repository-quality",
"path": "qc/checks",
"name": "QcCheck",
"reductorName": "o2::quality_control_modules::common::QualityReductor",
"moduleName": "QcCommon"
}
],
...
}Similarly, plots are defined by adding proper structures to the "plots" list, as shown below. The plot will be stored under the "name" value and it will have the "title" value shown on the top. The "varexp", "selection" and "option" fields correspond to the arguments of the TTree::Draw method.
{
...
"plots": [
{
"name": "mean_of_histogram",
"title": "Mean trend of the example histogram",
"varexp": "example.mean:time",
"selection": "",
"option": "*L"
},
{
"name": "histogram_of_means",
"title": "Distribution of mean values in the example histogram",
"varexp": "example.mean",
"selection": "",
"option": ""
},
{
"name": "example_quality",
"title": "Trend of the example histogram's quality",
"varexp": "QcCheck.name:time",
"selection": "",
"option": "*"
}
],
...
}← Go back to Modules Development | ↑ Go to the Table of Content ↑ | Continue to Advanced Topics →