Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an IIDM import post-processor to fill geographical extensions from ODRE data #2998

Merged
merged 42 commits into from
May 31, 2024
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
5cd3fb1
Add module to load positions on a network using ODRE csv files
HugoKulesza Mar 27, 2024
fdca677
setting geodata using an import postprocessor
HugoKulesza Apr 2, 2024
5765576
Merge branch 'main' into geodata
HugoKulesza Apr 2, 2024
ca95e64
fix checkstyle
HugoKulesza Apr 2, 2024
9b601e0
fix powsybl version
HugoKulesza Apr 2, 2024
25fc769
adding package to distribution
HugoKulesza Apr 2, 2024
786cbdc
adding tests
HugoKulesza Apr 2, 2024
9cc4786
add tests
HugoKulesza Apr 2, 2024
a1e37e2
fix style + add multiple coordinate components line
HugoKulesza Apr 3, 2024
dee125d
add test with valid line name
HugoKulesza Apr 3, 2024
8b178af
wrong path
HugoKulesza Apr 4, 2024
19cac42
set right Test import
HugoKulesza Apr 4, 2024
8636204
adding all work coming from rte-core
HugoKulesza May 2, 2024
3c410b2
fix javadoc for authors
HugoKulesza May 2, 2024
e20f1da
fix smells
HugoKulesza May 2, 2024
0a39f91
add coverage + license
HugoKulesza May 2, 2024
e0d3b67
fix smell
HugoKulesza May 2, 2024
a23ba6b
Merge branch 'main' into geodata
HugoKulesza May 6, 2024
61a8f35
fix get elements
HugoKulesza May 13, 2024
954997d
Merge branch 'main' into geodata
HugoKulesza May 13, 2024
aaf7578
Merge branch 'main' into geodata
olperr1 May 14, 2024
4b39791
fix code smells
HugoKulesza May 14, 2024
902732b
Merge branch 'main' into geodata
olperr1 May 14, 2024
a50e5fa
Fix groupId in pom.xml
olperr1 May 14, 2024
0e33b62
rework copyright dates
HugoKulesza May 17, 2024
e4200e3
add spdx to pom
HugoKulesza May 21, 2024
7e97c32
separate specific odre and general utility methods and classes
HugoKulesza May 24, 2024
f40b159
Merge branch 'main' into geodata
HugoKulesza May 24, 2024
7721c85
remove smell
HugoKulesza May 24, 2024
795c2dd
Merge branch 'main' into geodata
HugoKulesza May 27, 2024
499952a
adding geographical data documentation
HugoKulesza May 27, 2024
70eea20
Merge branch 'main' into geodata
HugoKulesza May 30, 2024
bff2555
remove french variables names, reuse FileValidator methods
HugoKulesza May 30, 2024
dbebd7b
update doc + name of post-processor parameters in config
HugoKulesza May 30, 2024
626b734
Merge branch 'main' into geodata
HugoKulesza May 30, 2024
f8fff20
fix typo + links
HugoKulesza May 30, 2024
597c79f
Merge branch 'main' into geodata
HugoKulesza May 30, 2024
28d6049
move odreGeoDataImporter doc to import_post_processor
HugoKulesza May 30, 2024
9e5b6f7
Merge branch 'main' into geodata
HugoKulesza May 30, 2024
a99b81f
move position extensions to extensions.md
HugoKulesza May 30, 2024
20f3aa0
Fix post-processor module name in the documentation + ODRE license me…
olperr1 May 31, 2024
a42b849
renaming dto package to elements
HugoKulesza May 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions distribution-core/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,12 @@
<version>${project.version}</version>
</dependency>

<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>powsybl-iidm-geodata</artifactId>
<version>${project.version}</version>
</dependency>

<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>powsybl-loadflow-api</artifactId>
Expand Down
68 changes: 68 additions & 0 deletions docs/geographical_data.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
Network geographical data
olperr1 marked this conversation as resolved.
Show resolved Hide resolved
=========================

Geographical positions extensions
---------------------------------

It is possible to add geographical positions to a network using dedicated extensions.

For these extensions, all coordinates are described in (latitude, longitude).

There are two such extensions :
olperr1 marked this conversation as resolved.
Show resolved Hide resolved

* [SubstationPosition](https://javadoc.io/doc/com.powsybl/powsybl-iidm-extensions/latest/com.powsybl.iidm.extensions/com/powsybl/iidm/network/extensions/SubstationPosition.html) can be used to add the position of substation :

```java
Substation station = network.getSubstation("P1")
station.newExtension(SubstationPositionAdder.class)
.withCoordinate(new Coordinate(48, 2))
.add();
Coordinate stationCoordinate = station.getExtension(SubstationPosition.class)
.getCoordinate();
```

* [LinePosition](https://javadoc.io/doc/com.powsybl/powsybl-iidm-extensions/latest/com.powsybl.iidm.extensions/com/powsybl/iidm/network/extensions/LinePosition.html) can be used to add the list of coordinates describing the position of a line :

```java
Line line = network.getLine("L1");
line.newExtension(LinePositionAdder.class)
.withCoordinates(List.of(new Coordinate(48, 2), new Coordinate(48.1, 2.1)))
.add();
List<Coordinate> lineCoordinates = line.getExtension(LinePosition.class)
.getCoordinates();
```

Geographical data import post-processor
---------------------------------------

One way to add geographical positions on a network is to use the import post-processor named OdreGeoDataAdderPostProcessor, that will automatically add the geographical positions extensions to the network model.
olperr1 marked this conversation as resolved.
Show resolved Hide resolved

This processor uses geographical position data formatted in multiple csv files, as it can be obtained on the website [OpenData Réseaux-Energie](https://odre.opendatasoft.com) for the network of the French TSO RTE.
Here are the links to obtain the RTE data CSV files, to be used as reference for input data formatting :

| Network element type | RTE data CSV file link |
|----------------------|--------------------------------------------------------------------------------------------------------|
| Substations | [https://odre.opendatasoft.com/api/explore/v2.1/catalog/datasets/postes-electriques-rte/exports/csv]() |
| Aerial lines | [https://odre.opendatasoft.com/api/explore/v2.1/catalog/datasets/lignes-aeriennes-rte-nv/exports/csv]() |
| Underground lines | [https://odre.opendatasoft.com/api/explore/v2.1/catalog/datasets/lignes-souterraines-rte-nv/exports/csv]() |
olperr1 marked this conversation as resolved.
Show resolved Hide resolved

To use this import post-processor, two things must be added to the PowSyBl configuration file.

Firstly, to activate the post-processor :

```yaml
import:
postProcessors:
- odreGeoDataImporter
```

Then, to precise where to find the data files :

```yaml
odre-geo-data-importer-post-processor:
substations: /path/to/substations.csv
aerial-lines: /path/to/aerial-lines.csv
underground-lines: /path/to/underground-lines.csv
```

The paths to the different files can be absolute paths or paths relative to the directory where your command is launched.
121 changes: 121 additions & 0 deletions iidm/iidm-geodata/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2019, RTE (http://www.rte-france.com)
This Source Code Form is subject to the terms of the Mozilla Public
License, v. 2.0. If a copy of the MPL was not distributed with this
file, You can obtain one at http://mozilla.org/MPL/2.0/.
SPDX-License-Identifier: MPL-2.0
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<artifactId>powsybl-iidm</artifactId>
<groupId>com.powsybl</groupId>
<version>6.4.0-SNAPSHOT</version>
</parent>

<artifactId>powsybl-iidm-geodata</artifactId>
<name>IIDM geographical data loader</name>
<description>The IIDM module to load geographical data on a network</description>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifestEntries>
<Automatic-Module-Name>com.powsybl.iidm.geodata</Automatic-Module-Name>
</manifestEntries>
</archive>
</configuration>
</plugin>
</plugins>
</build>

<properties>
<supercsv.version>2.4.0</supercsv.version>
</properties>

<dependencies>
<!-- compile dependencies -->
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>powsybl-iidm-api</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>powsybl-iidm-extensions</artifactId>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
</dependency>
<dependency>
<groupId>net.sf.supercsv</groupId>
<artifactId>super-csv</artifactId>
<version>${supercsv.version}</version>
</dependency>

<!-- test dependencies -->
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava-testlib</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>powsybl-commons-test</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>powsybl-iidm-impl</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>powsybl-iidm-serde</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>powsybl-iidm-test</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>powsybl-config-test</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
</dependency>
</dependencies>

</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/**
* Copyright (c) 2024, RTE (http://www.rte-france.com)
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
* SPDX-License-Identifier: MPL-2.0
*/
package com.powsybl.iidm.geodata.dto;
olperr1 marked this conversation as resolved.
Show resolved Hide resolved

import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.powsybl.iidm.geodata.utils.GeoShapeDeserializer;
import com.powsybl.iidm.network.extensions.Coordinate;

import java.util.List;

/**
* @author Hugo Marcellin {@literal <hugo.marcelin at rte-france.com>}
*/
@JsonDeserialize(using = GeoShapeDeserializer.class)
public record GeoShape(List<Coordinate> coordinates) { }
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
/**
* Copyright (c) 2019, RTE (http://www.rte-france.com)
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
* SPDX-License-Identifier: MPL-2.0
*/
package com.powsybl.iidm.geodata.dto;

import com.powsybl.iidm.network.extensions.Coordinate;

import java.util.List;

/**
* @author Geoffroy Jamgotchian {@literal <geoffroy.jamgotchian at rte-france.com>}
* @author Chamseddine Benhamed {@literal <chamseddine.benhamed at rte-france.com>}
*/
public class LineGeoData {

private String id;
private String country1;
private String country2;
private String substationStart;
private String substationEnd;
private List<Coordinate> coordinates;

public LineGeoData(String id, String country1, String country2, String substationStart, String substationEnd, List<Coordinate> coordinates) {
this.id = id;
this.country1 = country1;
this.country2 = country2;
this.substationStart = substationStart;
this.substationEnd = substationEnd;
this.coordinates = coordinates;
}

public String getId() {
return id;
}

public String getCountry1() {
return country1;
}

public String getCountry2() {
return country2;
}

public String getSubstationStart() {
return substationStart;
}

public String getSubstationEnd() {
return substationEnd;
}

public List<Coordinate> getCoordinates() {
return coordinates;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/**
* Copyright (c) 2019, RTE (http://www.rte-france.com)
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
* SPDX-License-Identifier: MPL-2.0
*/
package com.powsybl.iidm.geodata.dto;

import com.powsybl.iidm.network.extensions.Coordinate;

/**
* @author Chamseddine Benhamed {@literal <chamseddine.benhamed at rte-france.com>}
*/
public class SubstationGeoData {

private String id;
private String country;
private Coordinate coordinate;

public SubstationGeoData(String id, String country, Coordinate coordinate) {
this.id = id;
this.country = country;
this.coordinate = coordinate;
}

public String getId() {
return id;
}

public String getCountry() {
return country;
}

public Coordinate getCoordinate() {
return coordinate;
}
}
Loading