Improve CGMES import and export of OperationalLimits

[rcourtier]

Please check if the PR fulfills these requirements

• The commit message follows our guidelines
• Tests for the changes have been added (for bug fixes / features)
• Docs have been added / updated (for bug fixes / features)

Does this PR already have an issue describing the problem?

No

What kind of change does this PR introduce?

Feature.

What is the current behavior?

There are multiple concerns with the current CGMES import and export.

At export:

• For a given FlowsLimitsHolder, only the selected OperationalLimitsGroup is exported.
• There are as many OperationalLimitSet as OperationalLimit.
• There are as many OperationalLimitType as OperationalLimit.

At import:

• When several OperationalLimitSet are attached to the same terminal end:
  • If the sets contain OperationalLimit of different subclass (current/active power/apparent power), the sets are always merged in the same OperationalLimitGroup (the different subclasses of OperationalLimit are imported into the different LoadingLimit of the same OperationalLimitGroup).
  • If the sets contain OperationalLimit of the same subclass, the OperationalLimit values are merged (the lowest value is kept)
• As a consequence, there is always at most 1 OperationalLimitGroup per terminal end. This OperationalLimitGroup is the selected one.

What is the new behavior (if this is a feature change)?

At export:

• A new parameter for CGMES export can be set to indicate whether for a given FlowsLimitsHolder, only the selected OperationalLimitsGroup is exported or if all OperationalLimitsGroup are exported. By default, all OperationalLimitsGroup are exported.
• There is 1 CGMES OperationalLimitSet per IIDM OperationalLimitsGroup.
• If the IIDM identifiable holding the OperationalLimitsGroup has a property of the form: key="CGMES.OperationalLimitSet_<rdfid>", value="<name>", then the CGMES OperationalLimitSet will get this rdfid/name.
• There is only one OperationalLimitType for a given kind (patl/tatl) + acceptableDuration.

At import:

• Multiple OperationalLimitSet attached to the same terminal end are all imported:
  • The CGMES OperationalLimitSet rdfid is mapped to IIDM OperationalLimitsGroup id
  • The CGMES OperationalLimitSet name is stored as a property of the Equipment in the following form: key="CGMES.OperationalLimitSet_<rdfid>", value="<name>"
• As a consequence, there can be more than 1 OperationalLimitGroup per terminal end.
  • In case there is only one, it gets to be the selected one.
  • In case there is several, none is selected.

Does this PR introduce a breaking change or deprecate an API?

• Yes
• No

Other information:

In case of multiple OperationalLimitsGroup on the same terminal end, since the selected group must be set manually by the user, the Pypowsybl API should allow it.

[zamarrenolm]

The operational limit sets for the same terminal are merged by name. Why not merge them by identifier of the operational limit set? (names in CGMES are not required to be unique, two different objects may have the same name), is there a strong reason for preferring the name?

[rcourtier]

The operational limit sets for the same terminal are merged by name. Why not merge them by identifier of the operational limit set? (names in CGMES are not required to be unique, two different objects may have the same name), is there a strong reason for preferring the name?

To my experience, TSOs always model CGMES OperationalLimitSet with the same subclass of OperationalLimit inside. Meaning if they have CurrentLimit and ActivePowerLimit for the same terminal, there would be two OperationalLimitSet. However, in IIDM, it is possible to model multiple limit kinds in the same OperationalLimitsGroup.
So how to do the mapping between CGMES and IIDM? Sure a strict 1:1 mapping between CGMES OperationalLimitSet and IIDM OperationalLimitsGroup would work, but that doesn't cover the case where limits of different kinds and so in different sets but for the same season should be merged in the same group (note: they should be merged because then we can set the whole group as selected (active)). So I decided to merge the sets in the same group on the basis of the set name. Here is what I have written in the documentation :

In case of multiple OperationalLimitSet attached to the same end:

• If the sets have different names, they are imported in different OperationalLimitsGroup. For instance, a WINTER set and a SUMMER set will be imported in two distinct groups.
• If the sets have the same name, the sets are merged in the same OperationalLimitsGroup. For instance, a WINTER set of CurrentLimit and a WINTER set of ActivePowerLimit will be imported in the same group.

To be noticed, there is a slight evolution regarding the merging of limits (these cases are quite odd and could be regarded as modeling errors):

• Before the changes in that PR, 2 OperationalLimit were merged in case they had the same : terminal, subclass, duration (the lowest value is kept).
• Now with the changes in that PR, 2 OperationalLimit are merged in case they have the same : terminal, set name, subclass, duration (the lowest value is kept).

[zamarrenolm]

The operational limit sets for the same terminal are merged by name. Why not merge them by identifier of the operational limit set? (names in CGMES are not required to be unique, two different objects may have the same name), is there a strong reason for preferring the name?

To my experience, TSOs always model CGMES OperationalLimitSet with the same subclass of OperationalLimit inside. Meaning if they have CurrentLimit and ActivePowerLimit for the same terminal, there would be two OperationalLimitSet. However, in IIDM, it is possible to model multiple limit kinds in the same OperationalLimitsGroup. So how to do the mapping between CGMES and IIDM? Sure a strict 1:1 mapping between CGMES OperationalLimitSet and IIDM OperationalLimitsGroup would work, but that doesn't cover the case where limits of different kinds and so in different sets but for the same season should be merged in the same group (note: they should be merged because then we can set the whole group as selected (active)). So I decided to merge the sets in the same group on the basis of the set name. Here is what I have written in the documentation :

In case of multiple OperationalLimitSet attached to the same end:

• If the sets have different names, they are imported in different OperationalLimitsGroup. For instance, a WINTER set and a SUMMER set will be imported in two distinct groups.
• If the sets have the same name, the sets are merged in the same OperationalLimitsGroup. For instance, a WINTER set of CurrentLimit and a WINTER set of ActivePowerLimit will be imported in the same group.

To be noticed, there is a slight evolution regarding the merging of limits (these cases are quite odd and could be regarded as modeling errors):

• Before the changes in that PR, 2 OperationalLimit were merged in case they had the same : terminal, subclass, duration (the lowest value is kept).
• Now with the changes in that PR, 2 OperationalLimit are merged in case they have the same : terminal, set name, subclass, duration (the lowest value is kept).

I understand your use case trying to create a single set of limits based on the name (i.e. WINTER, SUMMER, ...) BUT ...

Now that we can have multiple groups in IIDM, I still have a concern about mapping multiple operational sets to a single group in IIDM (even if the name is the same). We are losing a separation that the modeller that created the CGMES data enforced. Separate operational sets with the same name may have been prepared for different situations: see DLR, dynamic line ratings, where different ratings can be calculated for different environmental conditions (https://www.entsoe.eu/Technopedia/techsheets/dynamic-line-rating-dlr).

Also, if the modeller has decided to keep in different sets in CGMES limits of different class for the same terminal, I think we should respect it. In CGMES, there is no problem in defining multiple classes in the same set.

By merging sets explicitly defined as separate in CGMES, we are losing the ability to activate in IIDM exactly one of the sets (and only that one) that was defined in CGMES, where the modeller created the network.

[rcourtier]

I understand. But sometimes, the modeled data is not consistent and that wouldn't make sense to import it as is in IIDM.

Typically when different sets (attached to the same end) have the same name.

• If the two sets are identical (they contain the same limits of the same subclass with the same duration and values), it doesn't make sense to import duplicates.
• If the two sets somehow differ, still it doesn't make sense to import them separately. Mainly because of the naming:
  • You can't map CGMES OperationalLimitSet.name to IIDM OperationalLimitsGroup.id (if you don't merge the sets) because in a FlowsLimitsHolder, the multiple OperationalLimitsGroup must have distinct ids.
  • Sure you could map the CGMES OperationalLimitSet.rdfid to IIDM OperationalLimitsGroup.id, since its unique, but you lose a lot in readability, and it becomes harder to later select the active set when calling the FlowsLimitsHolder.setOperationalLimitGroup(String id) if the method parameter is a rdfid.

In the case of DLR, from what I have seen in the vendor's implementation: there is no multiple set created in modelization. There is only one set modeled and the DLR application updates the limits value based on the actual conditions.

[zamarrenolm]

The update of values for operational limits brings another argument to preserve the identifier. In CIM100 (CGMES3) we have the normalValue in the EQ, and the (actual) value in the SSH.

Eventually in the import we should be able to import a single EQ and then later update the IIDM Network with values read from SSH. If we have merged different sets based on normalValue, we won't be able to know how to perform the update once we receive the actual values from the SSH.

[rcourtier]

Ok then, from that point of view that makes sense to map the CGMES OperationalLimitSet.rdfid to IIDM OperationalLimitsGroup.id, thus ensuring the id unicity and a strict 1:1 mapping.
I'll make the changes.

[zamarrenolm]

Ok then, from that point of view that makes sense to map the CGMES OperationalLimitSet.rdfid to IIDM OperationalLimitsGroup.id, thus ensuring the id unicity and a strict 1:1 mapping. I'll make the changes.

You are right in the point of having the name, and letting the user know about it. I think it is important too.

[rcourtier]

The OperationalLimitSet rdfid and name are preserved in a property attached to the Branch/DanglingLine/ThreeWindingTransformer during CGMES import. The property has the following key/value:
CGMES.OperationalLimitSet_<rdfid>/<name>
The presence of this property is checked during CGMES export. If there, the rdfid/name will be used for the export.

[flo-dup]

nicely documented 🙏

[flo-dup]

I think that to avoid all these loops we could do the selection in LoadingLimitsMapping::addAll.

[rcourtier]

The difficulty is that we should loop over all FlowsLimitsHolder to set the selected set, but there is no easy link to go from OperationalLimitsGroup which we have in LoadingLimitsMapping to FlowsLimitsHolder.

[zamarrenolm]

Just a minor comment: in IIDM we decided to use always mRIDs instead of the rdf:ID of the objects (the only difference is the prefix "_" in the rdf:IDs).

If you decide to make this adjustment, you can get the limit set identifier from the PropertyBag using getId instead of getLocal in the operational limit conversion.

[zamarrenolm]

I've verified the changes using real-world CGMES data in a network that defined several operational limit sets for many terminals, some of them with the same name by different identifiers.

Checked it doing import-export and calculation of overloads.

The limit sets (and their names) are correctly preserved.

We do not preserve CGMES names for limit types, but it is ok for me. For example in the input data we found "Normal" and "Emergency" limits, that simply give a name for PATL and TATL 1800. We keep the right values, and export them as "PATL" and "TATL 1800".

When multiple limit sets with the same name are found, they kept separated in the IIDM and in the export.

I found some warnings for limits not mapped to IIDM, but all of them were ok:

• When multiple TATL with the same acceptable duration. We correctly choose the lowest vale and keep a single TATL.
• When the related ACLineSegment has not been mapped to IIDM because it had R=0, X=0.

Calculation of overloads for this real-world case gives the same results as the previous PowSyBl version (main branch) when the group with the most restrictive limit value is selected.

My conclusion is: functionally the PR is ok.

[rcourtier]

Just a minor comment: in IIDM we decided to use always mRIDs instead of the rdf:ID of the objects (the only difference is the prefix "_" in the rdf:IDs).

If you decide to make this adjustment, you can get the limit set identifier from the PropertyBag using getId instead of getLocal in the operational limit conversion.

done

[sonarqubecloud]

Quality Gate passed

Issues
9 New issues
0 Accepted issues

Measures
0 Security Hotspots
94.7% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud