CGMES export. Handle CGM export dependencies

[zamarrenolm]

Please check if the PR fulfills these requirements

• The commit message follows our guidelines
• Tests for the changes have been added (for bug fixes / features)
• Docs have been added / updated (for bug fixes / features)

Does this PR already have an issue describing the problem?

What kind of change does this PR introduce?

Feature.

What is the current behavior?

CGMES export only considers IGM (Individual Grid Model) dependencies when exporting.

What is the new behavior (if this is a feature change)?
CGMES export will consider the dependencies required for CGM (Common Grid Model) export.

A Merging Agent using PowSyBl will export first an updated SSH for each IGM,
that should contain a Supersedes reference to the IGM’s original SSH CIMXML file.

Then, the Merging Agent should export a single SV for the whole CGM. This SV file must contain a DependentOn reference to each IGM’s updated SSH. The power flow of a CGM is calculated without topology processing. No new TP CIMXML files are created. This means that the CGM SV file must contain a DependentOn reference to each IGM’s original TP.

Does this PR introduce a breaking change or deprecate an API?

• Yes
• No

If yes, please check if the following requirements are fulfilled

• The Breaking Change or Deprecated label has been added
• The migration steps are described in the following section

What changes might users need to make in their application due to this PR? (migration steps)

Other information:

This change is based on #2898

[zamarrenolm]

When exporting a CGM, we need a Network with Subnetworks, and we expect the "profiles" to be exported to be "SSH,SV".

We have to export an updated SSH for every Subnetwork (for every IGM) and a single SV for the main Network (that represents the CGM).

We could have the parameter explicit, defined as the rest of configuration values in the CgmesImport, or just assume that it will be true if the network that is being exported has more than one Subnetwork and the profiles requested are "SSH,SV". We could save in the context if we are exporting a CGM solution:

context.isExportCommongGridModelSolution = network.getSubnetworks().size() > 1
                && parts.contains("SSH")
                && parts.contains("SV");

It would be nice to refactor the current contents of the CgmesExport::export function, splitting the code in separate methods to:

• Determine basename for exported files.
• Determine reference data (boundaries that will be used for exported data).
• Create the Context (the object that will store relevant data for export).
• Determine the modeling authority.
• Prepare the metadata Models for the exported instance files.
• Export instance files.

Then, when exporting the SSH instance files, we should consider whether we are exporting a CGM solution:

if (context.isExportCommongGridModelSolution) {
    for (Network sn : network.getSubnetworks()) {
        exportUpdatedSSHForIGM(sn, ds, context, baseName, parts, masUri);
    }
} else {
    exportSSH(network, ds, context, baseName, parts);
}

The method that outputs the updated SSH for an IGM should look like the following:

void exportUpdatedSSHForIGM(Subnetwork subnetwork, ...) {
  CgmesMetadataModel updatedSSHModel = new CgmesMetadataModel(CgmesSubset.STEADY_STATE_HYPOTHESIS, modelingAuthoritySet);
  subnetwork
      .getExtension(CgmesMetadataModels.class)
      .getModelForPart(CgmesSubset.STEADY_STATE_HYPOTHESIS)
      .ifPresent(ssh -> context.prepareExportedModelFrom(updatedSSHModel, ssh));
  SteadyStateHypothesisExport.write(subnetwork, writer, context, updatedSSHModel);
  context.igmUpdatedSshModels.add(updatedSSHModel.getId());
}

The non-null check for result of subnetwork.getExtension is omitted in this pseudo-code.
Also, a call to context.addIidmMappingsControlArea(subnetwork) before SteadyStateHypothesis.write will be required to initialise data related to control areas required in the export.

[zamarrenolm]

To build the filename template for exported SSH we should look into QoCDC rule FilenameMD. We should use a sourcing actor reference that is composed of <sourcingRSC>-<cgmRegion>-<sourcingTSO> (the merging agent, the region for the CGM and each TSO sourcing actor identifier). Currently, we are using only the country id.

[zamarrenolm]

why not use primitive boolean's here? the conditions below will be more concise and readable

[zamarrenolm]

I think we must keep the dependencies of the original SSH. If we clear the dependencies, we lose the SSH dependent on the EQ. This dependency has been correctly prepared in the method initializeModelForExport, but is removed if we clear the dependencies here.

[zamarrenolm]

When exporting a CGM, we need a Network with Subnetworks, and we expect the "profiles" to be exported to be "SSH,SV".
We have to export an updated SSH for every Subnetwork (for every IGM) and a single SV for the main Network (that represents the CGM).
We could have the parameter explicit, defined as the rest of configuration values in the CgmesImport, or just assume that it will be true if the network that is being exported has more than one Subnetwork and the profiles requested are "SSH,SV". We could save in the context if we are exporting a CGM solution:

context.isExportCommongGridModelSolution = network.getSubnetworks().size() > 1
                && parts.contains("SSH")
                && parts.contains("SV");

It would be nice to refactor the current contents of the CgmesExport::export function, splitting the code in separate methods to:

• Determine basename for exported files.
• Determine reference data (boundaries that will be used for exported data).
• Create the Context (the object that will store relevant data for export).
• Determine the modeling authority.
• Prepare the metadata Models for the exported instance files.
• Export instance files.

Then, when exporting the SSH instance files, we should consider whether we are exporting a CGM solution:

if (context.isExportCommongGridModelSolution) {
    for (Network sn : network.getSubnetworks()) {
        exportUpdatedSSHForIGM(sn, ds, context, baseName, parts, masUri);
    }
} else {
    exportSSH(network, ds, context, baseName, parts);
}

The method that outputs the updated SSH for an IGM should look like the following:

void exportUpdatedSSHForIGM(Subnetwork subnetwork, ...) {
  CgmesMetadataModel updatedSSHModel = new CgmesMetadataModel(CgmesSubset.STEADY_STATE_HYPOTHESIS, modelingAuthoritySet);
  subnetwork
      .getExtension(CgmesMetadataModels.class)
      .getModelForPart(CgmesSubset.STEADY_STATE_HYPOTHESIS)
      .ifPresent(ssh -> context.prepareExportedModelFrom(updatedSSHModel, ssh));
  SteadyStateHypothesisExport.write(subnetwork, writer, context, updatedSSHModel);
  context.igmUpdatedSshModels.add(updatedSSHModel.getId());
}

The non-null check for result of subnetwork.getExtension is omitted in this pseudo-code. Also, a call to context.addIidmMappingsControlArea(subnetwork) before SteadyStateHypothesis.write will be required to initialise data related to control areas required in the export.

Can you transfer a part of this to the documentation (readthedocs now). We have now an "exported.md" in CGMES to explain how to export CGM. Thanks !

Done. I added documentation for the new parameters and a brief explanation of the two main use-cases for export: IGM and CGM.

[olperr1]

This parameter key is not used anymore (unless in some unit tests).

[zamarrenolm]

removed the obsolete parameter name and its uses in unit tests

[annetill]

I disagree with this part: if we want to export a CGM with EQ and TP files, we can do it too. We have 2 methods to export CGM, a quick one that you are describing and a more complex one manipulating extensions and exporting profile by profile. Both must be explained and illustrated here.

[zamarrenolm]

Added documentation for the two ways of exporting CGM: quick and manual. Included code to illustrate both ways of generating the required files. Also added a unit test for the "manual" option, showing how metadata can be defined by the user by setting information in the related extension.

[annetill]

Small changes in doc.

[annetill]

What happens here if I put "EQ", "TP", "SSH" and "SV" ?

[annetill]

Indeed, after a small test, changing this list does not change anything. We always export SV and SSH, even if I only put SV.

[zamarrenolm]

clarified the documentation to explicitly state that CGM quick export ignores the profiles parameter and always exports SSH IGMs and CGM SV. Removed the parameter from the code example in the docs and from the main unit test showing its use.

[annetill]

Have you pushed?

[annetill]

Are you sure update dependencies parameter is used for CGM export?

[zamarrenolm]

yes, the update dependencies parameter is considered for both for CGM quick export and in manual network export. For CGM quick export dependencies are updated in CgmesExport::updateDependenciesCGM. For manual network export they are handled in CgmesExport::updateDependenciesIGM.

[annetill]

Thanks a lot, looks good!

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
95.1% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud