Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify CGMES naming strategy so that the IDs are stable #2804

Merged
merged 66 commits into from
Jan 15, 2024
Merged

Modify CGMES naming strategy so that the IDs are stable #2804

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
c664e5d
Add namespace in UUID generation
colinepiloquet Nov 28, 2023
f4a7315
change way to generate uuid
colinepiloquet Dec 1, 2023
a6478a5
merge main, solve conflicts
zamarrenolm Dec 1, 2023
434d25e
update number of params in exporter
zamarrenolm Dec 1, 2023
7225758
by default do not use namespace for stable UUID unique ids based on n…
zamarrenolm Dec 1, 2023
baecdf6
Merge branch 'main' into cgmes_naming_strategy
zamarrenolm Dec 1, 2023
4dc594f
naming strategy keeps uuid namespace
zamarrenolm Dec 1, 2023
4294f2a
Merge branch 'main' into cgmes_naming_strategy
zamarrenolm Dec 4, 2023
7aa5088
type of uuid namespace is UUID
zamarrenolm Dec 4, 2023
9564dee
remove mapping file
zamarrenolm Dec 5, 2023
3fd5d22
temporary check unit tests using random UUIDs
zamarrenolm Dec 5, 2023
e1a9b1f
Merge branch 'main' into cgmes_naming_strategy
zamarrenolm Dec 5, 2023
2bcd91e
Start implementing specific seed creation for UUID generation
colinepiloquet Dec 8, 2023
3ab1d31
Fixes
colinepiloquet Dec 11, 2023
aa95b11
Improve limits ID generation.
colinepiloquet Dec 11, 2023
b0db66a
Improve tap changers ID generation
colinepiloquet Dec 11, 2023
7716b08
Improve nodes ID generation
colinepiloquet Dec 11, 2023
25a7ca9
Modify remaining strings in getUniqueId
colinepiloquet Dec 11, 2023
d6b4821
Fix bug
colinepiloquet Dec 11, 2023
b03e023
fix unit test
zamarrenolm Dec 12, 2023
15b8f40
comment
zamarrenolm Dec 12, 2023
778fcc1
separate suffixes for RC of ratio and phase tap changers
zamarrenolm Dec 12, 2023
09b24a1
activate name-based uuids
zamarrenolm Dec 12, 2023
2adbb2b
fix suffix for the terminal of the equivalent injection
zamarrenolm Dec 12, 2023
2027e3c
Merge branch 'main' into cgmes_naming_strategy
zamarrenolm Dec 12, 2023
236c038
use random uuids for Sv objects (they are not persistent)
zamarrenolm Dec 13, 2023
cc9cb2e
unique ids are provided by the naming strategy
zamarrenolm Dec 13, 2023
1b051e4
fix calls to unique ids in missing containers preprocessor
zamarrenolm Dec 13, 2023
2e9805f
naming strategy unique ids based on list of cgmes object references
zamarrenolm Dec 15, 2023
b81e0c4
Merge branch 'main' into cgmes_naming_strategy
zamarrenolm Dec 18, 2023
28ee7cc
Merge branch 'main' into cgmes_naming_strategy
zamarrenolm Dec 18, 2023
18059c4
simplify code: naming strategy is not based in an external provided m…
zamarrenolm Dec 18, 2023
9ee274d
debug name-based uuids
zamarrenolm Dec 19, 2023
deb5c7a
base voltage stable uuids
zamarrenolm Dec 19, 2023
770b34d
keep the load groups created
zamarrenolm Dec 19, 2023
8c73a17
fix loading limits export: proper CGMES class name and avoid CGMES ob…
zamarrenolm Dec 19, 2023
4f4cdd9
fix writing temporary limits
zamarrenolm Dec 19, 2023
eea93ef
Merge branch 'main' into cgmes_naming_strategy
zamarrenolm Dec 20, 2023
48d4c06
use suffixes for prime objects
zamarrenolm Dec 20, 2023
24f94ac
a _ prefix is used at naming strategy, not when combining object refe…
zamarrenolm Dec 20, 2023
6122e12
uuid namespace parameter name for name-based uuids is public
zamarrenolm Dec 20, 2023
e63e39b
fix suffix for reactive capability curve
zamarrenolm Dec 20, 2023
19eefde
Fix some suffixes for generatingUnits, transformerEnd and tap changer…
colinepiloquet Dec 21, 2023
969f5b8
suffix for types of generating units moved to cgmes object reference …
zamarrenolm Dec 21, 2023
be06706
sometimes we do not need to add the suffix for the identifiable type
zamarrenolm Dec 21, 2023
65f2b17
Fix refGeneratingUnit + remove useless reference to AC Line Segment
colinepiloquet Dec 22, 2023
6f7853b
cleanup: uniform method name, avoid multiple calls to unique id with …
zamarrenolm Dec 27, 2023
5c29116
cleanup: remove unused naming conversion of geo tag
zamarrenolm Dec 27, 2023
ffb7c0d
cleanup: debug
zamarrenolm Dec 27, 2023
4bba6dd
cleanup: remove obsolete methods
zamarrenolm Dec 27, 2023
b5a16c4
minor fixes after testing against real-world cases
zamarrenolm Dec 28, 2023
6d51fba
consistent dc topological nodes
zamarrenolm Dec 29, 2023
0096fea
consistent dc topological nodes
zamarrenolm Dec 29, 2023
c25a200
code cleanup; get rid of old mechanism for storing uuids read from ma…
zamarrenolm Dec 31, 2023
6a4cf78
always use name-based uuids
zamarrenolm Jan 1, 2024
7be424b
fix sonar issues
zamarrenolm Jan 1, 2024
47669fc
remove potential urn:uuid prefix from ids before using them
zamarrenolm Jan 1, 2024
65d6dd2
fix sonar issues
zamarrenolm Jan 1, 2024
edae45c
Merge branch 'main' into cgmes_naming_strategy
annetill Jan 8, 2024
4d48d14
Merge branch 'main' into cgmes_naming_strategy
zamarrenolm Jan 9, 2024
020556f
A few fixes to make it work on my usecase
colinepiloquet Jan 9, 2024
588c404
Add condition in debug that should not have been removed
colinepiloquet Jan 10, 2024
d06b2a9
use ref and refTyped for building references for identifiables
zamarrenolm Jan 11, 2024
c619797
use combo for joining two references without using a separator
zamarrenolm Jan 11, 2024
940ef88
Merge branch 'main' into cgmes_naming_strategy
annetill Jan 15, 2024
6e4fb1f
Merge branch 'main' into cgmes_naming_strategy
annetill Jan 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 15 additions & 11 deletions ...etion/src/main/java/com/powsybl/cgmes/completion/CreateMissingContainersPreProcessor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
import com.powsybl.cgmes.extensions.CimCharacteristicsAdder;
import com.powsybl.cgmes.model.CgmesModel;
import com.powsybl.cgmes.model.CgmesNames;
import com.powsybl.cgmes.model.CgmesSubset;
import com.powsybl.cgmes.model.triplestore.CgmesModelTripleStore;
import com.powsybl.commons.PowsyblException;
import com.powsybl.commons.config.PlatformConfig;
Expand Down Expand Up @@ -45,6 +46,9 @@
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;

import static com.powsybl.cgmes.conversion.naming.CgmesObjectReference.ref;
import static com.powsybl.cgmes.conversion.naming.CgmesObjectReference.refTyped;

/**
* <p>
* A CGMES pre-processor that defines missing containers in input data.
Expand All @@ -71,7 +75,7 @@ public class CreateMissingContainersPreProcessor implements CgmesImportPreProces

public static final String NAME = "createMissingContainers";
public static final String FIXES_FOLDER_NAME = "iidm.import.cgmes.fixes-for-missing-containers-folder";
public static final double DEFAULT_NOMINAL_VALUE_FOR_MISSING_VOLTAGE_LEVELS = 100.0;
public static final double DEFAULT_NOMINAL_VALUE_FOR_MISSING_VOLTAGE_LEVELS = 1.2345;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for this change?

Copy link
Member

@zamarrenolm zamarrenolm Jan 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do NOT want to use a valid, reasonable nominal voltage that could be defined in reference data. We want to express that we do not have information about this nominal voltage.


private static final Logger LOG = LoggerFactory.getLogger(CreateMissingContainersPreProcessor.class);
private static final Parameter FIXES_FOLDER_NAME_PARAMETER = new Parameter(FIXES_FOLDER_NAME,
Expand Down Expand Up @@ -154,8 +158,8 @@ private static void buildZipFileWithFixes(CgmesModel cgmes, Set<String> missingV
try (ZipOutputStream zout = new ZipOutputStream(Files.newOutputStream(fixesFile))) {
zout.putNextEntry(new ZipEntry(basename + "_EQ.xml"));
XMLStreamWriter writer = XmlUtil.initializeWriter(true, " ", zout);
writeHeader(writer, context);
RegionContainers regionContainers = writeRegionContainers(writer, context);
writeHeader(network, writer, context);
RegionContainers regionContainers = writeRegionContainers(network, writer, context);
for (String missingVoltageLevel : missingVoltageLevels) {
writeMissingVoltageLevel(missingVoltageLevel, writer, context, regionContainers);
}
Expand All @@ -179,16 +183,16 @@ private static Network prepareEmptyNetworkForExport(CgmesModel cgmes) {
return network;
}

private static RegionContainers writeRegionContainers(XMLStreamWriter writer, CgmesExportContext context) throws XMLStreamException {
private static RegionContainers writeRegionContainers(Network network, XMLStreamWriter writer, CgmesExportContext context) throws XMLStreamException {
String cimNamespace = context.getCim().getNamespace();

// An alternative to replicate this code would be to make public the method
// EquipmentExport::writeFictitiousSubstationFor and use it here.
// We could group all missing voltage levels in the same (fictitious) substation
RegionContainers regionContainers = new RegionContainers();
regionContainers.subGeographicalRegionId = CgmesExportUtil.getUniqueId();
regionContainers.subGeographicalRegionId = context.getNamingStrategy().getCgmesId(refTyped(network), ref("SubgeographicalRegionId"));
String subGeographicalRegionName = "SGR fix for missing data";
regionContainers.geographicalRegionId = CgmesExportUtil.getUniqueId();
regionContainers.geographicalRegionId = context.getNamingStrategy().getCgmesId(refTyped(network), ref("GeographicalRegionId"));
String geographicalRegionName = "GR fix for missing data";
SubGeographicalRegionEq.write(regionContainers.subGeographicalRegionId, subGeographicalRegionName, regionContainers.geographicalRegionId, cimNamespace, writer, context);
GeographicalRegionEq.write(regionContainers.geographicalRegionId, geographicalRegionName, cimNamespace, writer, context);
Expand All @@ -202,21 +206,21 @@ private static void writeMissingVoltageLevel(String voltageLevelId, XMLStreamWri
// we do not have additional information about the voltage level,
// we create a different substation and base voltage for every missing voltage level
String voltageLevelName = voltageLevelId + " VL";
String substationId = CgmesExportUtil.getUniqueId();
String substationId = context.getNamingStrategy().getCgmesId(ref(voltageLevelId), ref("Substation"));
String substationName = voltageLevelId + "SUB for missing VL " + voltageLevelId;
String baseVoltageId = CgmesExportUtil.getUniqueId();
String baseVoltageId = context.getNamingStrategy().getCgmesId(ref(voltageLevelId), ref("BaseVoltage"));

VoltageLevelEq.write(voltageLevelId, voltageLevelName, Double.NaN, Double.NaN, substationId, baseVoltageId, cimNamespace, writer, context);
SubstationEq.write(substationId, substationName, regionContainers.subGeographicalRegionId, cimNamespace, writer, context);
BaseVoltageEq.write(baseVoltageId, DEFAULT_NOMINAL_VALUE_FOR_MISSING_VOLTAGE_LEVELS, cimNamespace, writer, context);
}

private static void writeHeader(XMLStreamWriter writer, CgmesExportContext context) throws XMLStreamException {
private static void writeHeader(Network network, XMLStreamWriter writer, CgmesExportContext context) throws XMLStreamException {
String cimNamespace = context.getCim().getNamespace();
String euNamespace = context.getCim().getEuNamespace();
CgmesExportUtil.writeRdfRoot(cimNamespace, context.getCim().getEuPrefix(), euNamespace, writer);
if (context.getCimVersion() >= 16) {
ModelDescriptionEq.write(writer, context.getEqModelDescription(), context);
CgmesExportUtil.writeModelDescription(network, CgmesSubset.EQUIPMENT, writer, context.getEqModelDescription(), context);
}
}

Expand Down Expand Up @@ -256,7 +260,7 @@ public void process(CgmesModel cgmes) {
}
}

private static class RegionContainers {
private static final class RegionContainers {
String subGeographicalRegionId;
String geographicalRegionId;
}
Expand Down
4 changes: 2 additions & 2 deletions ...0.3/MicroGrid/BaseCase/BC_Assembled_v2_bad_ids/MicroGridTestConfiguration_BC_BE_EQ_V2.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1023,7 +1023,7 @@
<cim:CurveData.y2value>200.000000</cim:CurveData.y2value>
<cim:CurveData.Curve rdf:resource="#_59ff1e53-0e1a-44c0-ada5-7a0b3a660170"/>
</cim:CurveData>
<cim:EnergyConsumer rdf:ID="_cb459405-cc14-4215-a45c-416789205904">
<cim:EnergyConsumer rdf:ID="_cb459405-cc14-4215-a45c-416789205904badidload">
<cim:IdentifiedObject.name>BE-Load_1</cim:IdentifiedObject.name>
<entsoe:IdentifiedObject.shortName>BE-L_1</entsoe:IdentifiedObject.shortName>
<cim:IdentifiedObject.description>Electrabel</cim:IdentifiedObject.description>
Expand Down Expand Up @@ -2030,7 +2030,7 @@
<entsoe:IdentifiedObject.shortName>BE-L_1</entsoe:IdentifiedObject.shortName>
<cim:ACDCTerminal.sequenceNumber>1</cim:ACDCTerminal.sequenceNumber>
<cim:Terminal.phases rdf:resource="http://iec.ch/TC57/2013/CIM-schema-cim16#PhaseCode.ABC"/>
<cim:Terminal.ConductingEquipment rdf:resource="#_cb459405-cc14-4215-a45c-416789205904"/>
<cim:Terminal.ConductingEquipment rdf:resource="#_cb459405-cc14-4215-a45c-416789205904badidload"/>
</cim:Terminal>
<cim:Terminal rdf:ID="_a036b765-1669-4f64-acd3-1e8fbd513312">
<cim:IdentifiedObject.name>BE-Load_2 - T1</cim:IdentifiedObject.name>
Expand Down
2 changes: 1 addition & 1 deletion ....3/MicroGrid/BaseCase/BC_Assembled_v2_bad_ids/MicroGridTestConfiguration_BC_BE_SSH_V2.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@
<cim:EnergyConsumer.p>1.000000</cim:EnergyConsumer.p>
<cim:EnergyConsumer.q>0e+000</cim:EnergyConsumer.q>
</cim:EnergyConsumer>
<cim:EnergyConsumer rdf:about="#_cb459405-cc14-4215-a45c-416789205904">
<cim:EnergyConsumer rdf:about="#_cb459405-cc14-4215-a45c-416789205904badidload">
<cim:EnergyConsumer.p>200.000000</cim:EnergyConsumer.p>
<cim:EnergyConsumer.q>90.000000</cim:EnergyConsumer.q>
</cim:EnergyConsumer>
Expand Down
11 changes: 11 additions & 0 deletions cgmes/cgmes-conversion/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@
</plugins>
</build>

<properties>
<jug.version>4.3.0</jug.version>
</properties>

<dependencies>
<!-- Compilation dependencies -->
<dependency>
Expand Down Expand Up @@ -78,6 +82,13 @@
<artifactId>slf4j-api</artifactId>
</dependency>

<!-- UUID generator for the naming strategy -->
<dependency>
<groupId>com.fasterxml.uuid</groupId>
<artifactId>java-uuid-generator</artifactId>
<version>${jug.version}</version>
</dependency>

<!-- Test dependencies -->
<dependency>
<groupId>ch.qos.logback</groupId>
Expand Down
204 changes: 0 additions & 204 deletions ...nversion/src/main/java/com/powsybl/cgmes/conversion/AbstractCgmesAliasNamingStrategy.java
View file
Open in desktop

This file was deleted.

Loading