Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix CGM export dependencies issues #3128

Merged
merged 6 commits into from
Sep 16, 2024
Merged

Fix CGM export dependencies issues #3128

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
b8fa338
Add dependentOn relation of SV model to all TP_BD models in case of a…
rcourtier Sep 3, 2024
fef831b
Update documentation
rcourtier Sep 3, 2024
0793f9a
Merge branch 'main' into cgmes_cgm_export_fix_dependencies_issues
zamarrenolm Sep 4, 2024
013750e
Add dependency (dependentOn) from updated SSHs to original EQs
rcourtier Sep 4, 2024
6e8586a
Check that the BOUNDARY_TP_ID parameter is considered in the export d…
rcourtier Sep 4, 2024
a01d65e
Merge branch 'main' into cgmes_cgm_export_fix_dependencies_issues
zamarrenolm Sep 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 19 additions & 11 deletions cgmes/cgmes-conversion/src/main/java/com/powsybl/cgmes/conversion/CgmesExport.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,17 +112,18 @@ private void exportCGM(Network network, DataSource dataSource, CgmesExportContex
Map<Network, IgmModelsForCgm> igmModels = new HashMap<>();
for (Network subnetwork : network.getSubnetworks()) {
IgmModelsForCgm igmModelsForCgm = new IgmModelsForCgm(
initializeModelForExport(subnetwork, CgmesSubset.STEADY_STATE_HYPOTHESIS, context, false, false),
initializeModelForExport(subnetwork, CgmesSubset.STEADY_STATE_HYPOTHESIS, context, false, true),
initializeModelForExport(subnetwork, CgmesSubset.TOPOLOGY, context, false, false)
initializeModelForExport(subnetwork, CgmesSubset.STEADY_STATE_HYPOTHESIS, context, false, false),
initializeModelForExport(subnetwork, CgmesSubset.TOPOLOGY, context, false, false),
initializeModelForExport(subnetwork, CgmesSubset.TOPOLOGY_BOUNDARY, context, false, false)
);
igmModels.put(subnetwork, igmModelsForCgm);
}
CgmesMetadataModel updatedCgmSvModel = initializeModelForExport(network, CgmesSubset.STATE_VARIABLES, context, true, true);

// Update dependencies
if (context.updateDependencies()) {
updateDependenciesCGM(igmModels.values(), updatedCgmSvModel);
updateDependenciesCGM(igmModels.values(), updatedCgmSvModel, context.getBoundaryTpId());
}

// Export the SSH for the IGMs and the SV for the CGM
Expand Down Expand Up @@ -227,22 +228,27 @@ public static CgmesMetadataModel initializeModelForExport(
}

/**
* Update cross dependencies between the subset models through the dependentOn relationship.
* Update cross dependencies between the subset models (including boundaries) through the dependentOn relationship.
* The IGMs updated SSH supersede the original ones.
* The CGM updated SV depends on the IGMs updated SSH and on the IGMs original TP.
* @param igmModels For each IGM: the original SSH model, the updated SSH model and the original TP model.
* The CGM updated SV depends on the IGMs updated SSH and on the IGMs original TP and TP_BD.
* @param igmModels For each IGM: the updated SSH model and the original SSH, TP and TP_BD models.
* @param updatedCgmSvModel The SV model for the CGM.
* @param boundaryTpId The model id for the TP_BD subset.
*/
private void updateDependenciesCGM(Collection<IgmModelsForCgm> igmModels, CgmesMetadataModel updatedCgmSvModel) {
private void updateDependenciesCGM(Collection<IgmModelsForCgm> igmModels, CgmesMetadataModel updatedCgmSvModel, String boundaryTpId) {
// Each updated SSH model supersedes the original one
// Clear previous dependencies
igmModels.forEach(m -> m.updatedSsh.clearDependencies());
igmModels.forEach(m -> m.updatedSsh.clearSupersedes());
igmModels.forEach(m -> m.updatedSsh.addSupersedes(m.originalSsh.getId()));

// Updated SV model depends on updated SSH models and original TP models
// Updated SV model depends on updated SSH models and original TP and TP_BD models
updatedCgmSvModel.addDependentOn(igmModels.stream().map(m -> m.updatedSsh.getId()).collect(Collectors.toSet()));
updatedCgmSvModel.addDependentOn(igmModels.stream().map(m -> m.originalTp.getId()).collect(Collectors.toSet()));
updatedCgmSvModel.addDependentOn(igmModels.stream().map(m -> m.originalTpBd.getId()).collect(Collectors.toSet()));
if (boundaryTpId != null) {
updatedCgmSvModel.addDependentOn(boundaryTpId);
}
}

/**
Expand Down Expand Up @@ -492,14 +498,16 @@ private String getBaseName(CgmesExportContext context, DataSource dataSource, Ne
* when setting the relationships (dependOn, supersedes) between them in a CGM export.
*/
private static class IgmModelsForCgm {
CgmesMetadataModel originalSsh;
CgmesMetadataModel updatedSsh;
CgmesMetadataModel originalSsh;
CgmesMetadataModel originalTp;
CgmesMetadataModel originalTpBd;

public IgmModelsForCgm(CgmesMetadataModel originalSsh, CgmesMetadataModel updatedSsh, CgmesMetadataModel originalTp) {
this.originalSsh = originalSsh;
public IgmModelsForCgm(CgmesMetadataModel updatedSsh, CgmesMetadataModel originalSsh, CgmesMetadataModel originalTp, CgmesMetadataModel originalTpBd) {
this.updatedSsh = updatedSsh;
this.originalSsh = originalSsh;
this.originalTp = originalTp;
this.originalTpBd = originalTpBd;
}
}

Expand Down
22 changes: 17 additions & 5 deletions ...ion/src/test/java/com/powsybl/cgmes/conversion/test/export/CommonGridModelExportTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,9 @@ void testCgmExportNoModelsNoProperties() throws IOException {
String updatedNlSshId = "urn:uuid:Network_NL_N_STEADY_STATE_HYPOTHESIS_2021-02-03T04:30:00Z_2_1D__FM";
String originalBeTpId = "urn:uuid:Network_BE_N_TOPOLOGY_2021-02-03T04:30:00Z_1_1D__FM";
String originalNlTpId = "urn:uuid:Network_NL_N_TOPOLOGY_2021-02-03T04:30:00Z_1_1D__FM";
Set<String> expectedDependencies = Set.of(updatedBeSshId, updatedNlSshId, originalBeTpId, originalNlTpId);
String originalBeTpBdId = "urn:uuid:Network_BE_N_TOPOLOGY_BOUNDARY_2021-02-03T04:30:00Z_1_1D__FM";
String originalNlTpBdId = "urn:uuid:Network_NL_N_TOPOLOGY_BOUNDARY_2021-02-03T04:30:00Z_1_1D__FM";
Set<String> expectedDependencies = Set.of(updatedBeSshId, updatedNlSshId, originalBeTpId, originalNlTpId, originalBeTpBdId, originalNlTpBdId);
assertEquals(expectedDependencies, getOccurrences(updatedCgmSvXml, REGEX_DEPENDENT_ON));

// Each updated IGM SSH should supersede the original one
Expand Down Expand Up @@ -195,7 +197,9 @@ void testCgmExportWithModelsForSubnetworks() throws IOException {
String updatedNlSshId = "urn:uuid:Network_NL_N_STEADY_STATE_HYPOTHESIS_2021-02-03T04:30:00Z_2_1D__FM";
String originalBeTpId = "urn:uuid:Network_BE_N_TOPOLOGY_2021-02-03T04:30:00Z_1_1D__FM";
String originalNlTpId = "urn:uuid:Network_NL_N_TOPOLOGY_2021-02-03T04:30:00Z_1_1D__FM";
Set<String> expectedDependencies = Set.of(updatedBeSshId, updatedNlSshId, originalBeTpId, originalNlTpId);
String originalBeTpBdId = "urn:uuid:Network_BE_N_TOPOLOGY_BOUNDARY_2021-02-03T04:30:00Z_1_1D__FM";
String originalNlTpBdId = "urn:uuid:Network_NL_N_TOPOLOGY_BOUNDARY_2021-02-03T04:30:00Z_1_1D__FM";
Set<String> expectedDependencies = Set.of(updatedBeSshId, updatedNlSshId, originalBeTpId, originalNlTpId, originalBeTpBdId, originalNlTpBdId);
assertEquals(expectedDependencies, getOccurrences(updatedCgmSvXml, REGEX_DEPENDENT_ON));

// Each updated IGM SSH should supersede the original one
Expand Down Expand Up @@ -254,8 +258,11 @@ void testCgmExportWithModelsForAllNetworks() throws IOException {
String updatedNlSshId = "urn:uuid:Network_NL_N_STEADY_STATE_HYPOTHESIS_2022-03-04T05:30:00Z_4_1D__FM";
String originalBeTpId = "urn:uuid:Network_BE_N_TOPOLOGY_2022-03-04T05:30:00Z_1_1D__FM";
String originalNlTpId = "urn:uuid:Network_NL_N_TOPOLOGY_2022-03-04T05:30:00Z_1_1D__FM";
String originalBeTpBdId = "urn:uuid:Network_BE_N_TOPOLOGY_BOUNDARY_2022-03-04T05:30:00Z_1_1D__FM";
String originalNlTpBdId = "urn:uuid:Network_NL_N_TOPOLOGY_BOUNDARY_2022-03-04T05:30:00Z_1_1D__FM";
String additionalDependency = "Additional dependency";
Set<String> expectedDependencies = Set.of(updatedBeSshId, updatedNlSshId, originalBeTpId, originalNlTpId, additionalDependency);
Set<String> expectedDependencies = Set.of(updatedBeSshId, updatedNlSshId, originalBeTpId,
originalNlTpId, originalBeTpBdId, originalNlTpBdId, additionalDependency);
assertEquals(expectedDependencies, getOccurrences(updatedCgmSvXml, REGEX_DEPENDENT_ON));

// Each updated IGM SSH should supersede the original one
Expand Down Expand Up @@ -314,7 +321,9 @@ void testCgmExportWithProperties() throws IOException {
String updatedNlSshId = "urn:uuid:Network_NL_N_STEADY_STATE_HYPOTHESIS_2021-02-03T04:30:00Z_4_1D__FM";
String originalBeTpId = "urn:uuid:Network_BE_N_TOPOLOGY_2021-02-03T04:30:00Z_1_1D__FM";
String originalNlTpId = "urn:uuid:Network_NL_N_TOPOLOGY_2021-02-03T04:30:00Z_1_1D__FM";
Set<String> expectedDependencies = Set.of(updatedBeSshId, updatedNlSshId, originalBeTpId, originalNlTpId);
String originalBeTpBdId = "urn:uuid:Network_BE_N_TOPOLOGY_BOUNDARY_2021-02-03T04:30:00Z_1_1D__FM";
String originalNlTpBdId = "urn:uuid:Network_NL_N_TOPOLOGY_BOUNDARY_2021-02-03T04:30:00Z_1_1D__FM";
Set<String> expectedDependencies = Set.of(updatedBeSshId, updatedNlSshId, originalBeTpId, originalNlTpId, originalBeTpBdId, originalNlTpBdId);
assertEquals(expectedDependencies, getOccurrences(updatedCgmSvXml, REGEX_DEPENDENT_ON));

// Each updated IGM SSH should supersede the original one
Expand Down Expand Up @@ -375,8 +384,11 @@ void testCgmExportWithModelsAndProperties() throws IOException {
String updatedNlSshId = "urn:uuid:Network_NL_N_STEADY_STATE_HYPOTHESIS_2022-03-04T05:30:00Z_4_1D__FM";
String originalBeTpId = "urn:uuid:Network_BE_N_TOPOLOGY_2022-03-04T05:30:00Z_1_1D__FM";
String originalNlTpId = "urn:uuid:Network_NL_N_TOPOLOGY_2022-03-04T05:30:00Z_1_1D__FM";
String originalBeTpBdId = "urn:uuid:Network_BE_N_TOPOLOGY_BOUNDARY_2022-03-04T05:30:00Z_1_1D__FM";
String originalNlTpBdId = "urn:uuid:Network_NL_N_TOPOLOGY_BOUNDARY_2022-03-04T05:30:00Z_1_1D__FM";
String additionalDependency = "Additional dependency";
Set<String> expectedDependencies = Set.of(updatedBeSshId, updatedNlSshId, originalBeTpId, originalNlTpId, additionalDependency);
Set<String> expectedDependencies = Set.of(updatedBeSshId, updatedNlSshId, originalBeTpId,
zamarrenolm marked this conversation as resolved.
Show resolved Hide resolved
originalNlTpId, originalBeTpBdId, originalNlTpBdId, additionalDependency);
assertEquals(expectedDependencies, getOccurrences(updatedCgmSvXml, REGEX_DEPENDENT_ON));

// Each updated IGM SSH should supersede the original one
Expand Down
4 changes: 2 additions & 2 deletions docs/grid_exchange_formats/cgmes/export.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ The quick CGM export will always write updated SSH files for IGMs and a single S

If the dependencies have to be updated automatically (see parameter **iidm.export.cgmes.update-dependencies** below), the exported instance files will contain metadata models where:
* Updated SSH for IGMs supersede the original ones.
* Updated SV for the CGM depends on the updated SSH from IGMs and on the original TP from IGMs.
* Updated SV for the CGM depends on the updated SSH from IGMs and on the original TP and TP_BD from IGMs.

The filenames of the exported instance files will follow the pattern:
* For the CGM SV: `<basename>_SV.xml`.
Expand Down Expand Up @@ -70,7 +70,7 @@ exampleBase_NL_SSH.xml
exampleBase_SV.xml
```

where the updated SSH files will supersede the original ones, and the SV will contain the correct dependencies of new SSH and original TPs.
where the updated SSH files will supersede the original ones, and the SV will contain the correct dependencies of new SSH and original TPs and TP_BD.

## CGM (Common Grid Model) manual export

Expand Down