Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CGMES: use only header to identify CGMES profile instead of looking e.g. for _EQ_ in the file name #3205

Open
jeandemanged opened this issue Nov 8, 2024 · 1 comment
Assignees
Labels

Comments

@jeandemanged
Copy link
Member

jeandemanged commented Nov 8, 2024

Describe the current behavior

CGMES importer looks for _EQ_ or _EQ. string presence in file name to filter for EQ profiles.

Describe the expected behavior

A more robust solution would rather check only the model header md:Model.profile to determine what profile(s) a file contains.

Describe the motivation

Got an issue with non-obvious stack trace with one IGM where files were named (allegedly weirdly):

  • confidential_EQ_confidential_EQ.xml
  • confidential_EQ_confidential_SSH.xml
  • confidential_EQ_confidential_SV.xml
  • confidential_EQ_confidential_TP.xml

All of them are then considered as EQ. Because they all have same md:Model.modelingAuthoritySet the import then fails with default importer settings.

Workarounds are:

  • renaming files
  • changing iidm.import.cgmes.cgm-with-subnetworks-defined-by to FILENAME (consequences may be acceptable or not)
  • changing iidm.import.cgmes.cgm-with-subnetworks to false (consequences may be acceptable or not)

Extra Information

powsybl-core 6.5.0

DEBUG:powsybl:Duplicate key MAS (attempted merging values [confidential_EQ_confidential_SV.xml] and [confidential_EQ_confidential_TP.xml])
java.lang.IllegalStateException: Duplicate key MAS (attempted merging values [confidential_EQ_confidential_SV.xml] and [confidential_EQ_confidential_TP.xml])
        at java.base@17.0.11/java.util.stream.Collectors.duplicateKeyException(Collectors.java:135)
        at java.base@17.0.11/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:182)
        at java.base@17.0.11/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
        at java.base@17.0.11/java.util.stream.Streams$StreamBuilderImpl.forEachRemaining(Streams.java:411)
        at java.base@17.0.11/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
        at java.base@17.0.11/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
        at java.base@17.0.11/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
        at java.base@17.0.11/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
        at java.base@17.0.11/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1707)
        at java.base@17.0.11/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base@17.0.11/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base@17.0.11/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
        at java.base@17.0.11/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base@17.0.11/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
        at com.powsybl.cgmes.conversion.CgmesImport$MultipleGridModelChecker.separateByModelingAuthority(CgmesImport.java:260)
        at com.powsybl.cgmes.conversion.CgmesImport$MultipleGridModelChecker.separate(CgmesImport.java:247)
        at com.powsybl.cgmes.conversion.CgmesImport.importData(CgmesImport.java:163)
        at com.powsybl.iidm.network.Network.read(Network.java:128)
        at com.powsybl.iidm.network.Network.read(Network.java:135)
        at com.powsybl.python.network.NetworkCFunctions.lambda$loadNetwork$6(NetworkCFunctions.java:171)
        at com.powsybl.python.commons.Util.doCatch(Util.java:132)
        at com.powsybl.python.network.NetworkCFunctions.loadNetwork(NetworkCFunctions.java:163)
@zamarrenolm
Copy link
Member

Agree on the fact that the names may not follow the expected CIMXML pattern used by CGMES exchanges:

<effectiveDateTime>_<businessProcess>_<sourcingActor>_<modelPart>_<fileVersion>.xml

From the names of your files, EQ should be a businessProcess, which I think is not.

Here we were looking only at the filename to have a fast way of splitting a data source that contained multiple IGMs, trying to avoid reading "inside" each XML. But we have to obtain the modelling authority anyway (by reading the header), so it makes sense to check also the profile.

@zamarrenolm zamarrenolm self-assigned this Nov 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants