Distributed slack management in sensitivity analysis #178

Djazouli · 2020-12-03T17:37:15Z

Signed-off-by: Gael Macherel gael.macherel@artelys.com

Please check if the PR fulfills these requirements (please use '[x]' to check the checkboxes, or submit the PR and then click the checkboxes)

The commit message follows our guidelines
Tests for the changes have been added (for bug fixes / features)
Docs have been added / updated (for bug fixes / features)

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Create a feature allowing the user to use distributed slack in sensitivity analysis

Does this PR introduce a breaking change or deprecate an API? If yes, check the following:

The migration guide has been updated in the github wiki (What changes might users need to make in their application due to this PR?)

Other information:

This is currently WIP, I opened the PR to request feedback

Djazouli · 2020-12-03T17:39:08Z

src/main/java/com/powsybl/openloadflow/ac/AbstractDistributedSlackOuterLoop.java

@@ -30,11 +30,11 @@
     */
    private static final double SLACK_P_RESIDUE_EPS = Math.pow(10, -5);

-    protected static class ParticipatingElement<T> {
+    public static class ParticipatingElement<T> {


I really want to get rid of these "public" changes, but I need to get those values somewhere else in the code
Should I just create a protected method that returns something understandable by the rest of the code ?
(returning Pair<String, Number> instead of participatingElement for example)

src/main/java/com/powsybl/openloadflow/dc/equations/AbstractClosedBranchDcFlowEquationTerm.java

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

Djazouli · 2020-12-03T17:42:09Z

src/test/java/com/powsybl/openloadflow/sensi/OpenSensitivityAnalysisProviderTest.java

@@ -131,6 +145,38 @@ void testDc4buses() {
        assertEquals(0.25d, getValue(result, "g4", "l13"), LoadFlowAssert.DELTA_POWER);
    }

+    @Test
+    void testDc4busesDistributed() {


Test does not pass at the moment, it's [WIP]

Djazouli · 2020-12-03T18:00:07Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

+        if (loadFlowParameters.isDistributedSlack()) {
+            switch (loadFlowParameters.getBalanceType()) {
+                case PROPORTIONAL_TO_GENERATION_P_MAX:
+                    DistributedSlackOnGenerationOuterLoop outerLoop = new DistributedSlackOnGenerationOuterLoop(openLoadFlowParameters.isThrowsExceptionInCaseOfSlackDistributionFailure());


This is were I need all this outer loop stuff to be Public, I would love to avoid this

Djazouli · 2020-12-03T18:01:19Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

+     * @param oldFactors
+     * @return
+     */
+    private List<SensitivityFactor> getDistributedFactors(Network network, LfNetwork lfNetwork,


This method uses network to get generators. But lfNetwork may not contain all these generators (because it is a connected component of network), so I would love to use only lfNetwork, but I do not know how to access generators from LfNetwork

Djazouli · 2020-12-03T18:03:29Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

+     * @param sensitivityValues
+     * @param remainder
+     * @return
+     */


hardly understandable, needs refactoring

Indeed, it is the part of the code that is the less understable for me.

Djazouli · 2020-12-03T18:05:05Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

+        List<SensitivityValue> sensitivityValues = calculateSensitivityValues(lfNetwork, equationSystem, factorsByVarConfig, states);
+
+        if (!lfParameters.isDistributedSlack()) {
+            return sensitivityValues;


Having a return in the middle of the function makes it harder to understand, I need to find a better way to distinguish the distributed/ not distributed case

annetill · 2020-12-08T16:03:08Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

+        }
+
+        switch (balanceType) {
+            case PROPORTIONAL_TO_GENERATION_P_MAX:


@Djazouli Geoffory prefers that we do not use factors that whould remain in the API. So we have to fix that.

annetill · 2020-12-08T16:03:49Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

+     * @param loadFlowParameters
+     * @return
+     */
+    private List<Pair<String, Number>> getParticipatingElements(LfNetwork lfNetwork, LoadFlowParameters loadFlowParameters) {


I think here we just have to have as argument the balance type.

annetill

Tests have to be added. I think that this PR should not have more features but should be tested and improved before review.

annetill · 2020-12-08T16:05:54Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

+        }
+
+        // compute sensitivities corrected from the slack distribution effects
+        for (SensitivityValue sensitivityValue : sensitivityValues) {


Here a fix is needed because I don't have sort the sensitivity values to give only those that have been ask in the API. @Djazouli your code was functionaly right but it was to remove the use of factors.

Djazouli · 2020-12-09T20:12:41Z

src/test/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysisTest.java

+        assertEquals(0.275d, getValue(result, "g1", "l12"), LoadFlowAssert.DELTA_POWER);
+        assertEquals(-0.125d, getValue(result, "g1", "l23"), LoadFlowAssert.DELTA_POWER);
+        assertEquals(0.025d, getValue(result, "g1", "l34"), LoadFlowAssert.DELTA_POWER);
+        assertEquals(0.15d, getValue(result, "g1", "l13"), LoadFlowAssert.DELTA_POWER);
    }


Todo:
add tests:

where we are injecting on a load node that has no generator

Where the slack bus is participating in the compensation

Where the balance type is PROPORTIONAL_TO_LOAD

Djazouli · 2020-12-09T21:34:00Z

We still need to test balance type : PROPORTIONAL_TO_LOAD to improve coverage

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

…rs required for distribution Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

Improve variable naming. Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>

Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>

annetill

Maybe some slight variable names have to be changed to improve readability. I have a question about injection increase for a generator or a load that is connected to the slack bus: for the moment, we throw an exception. Is it the final behaviour that we aim ?

annetill · 2020-12-14T16:32:30Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

        Map<SensitivityVariableConfiguration, SensitivityFactorGroup> factorsByVarConfig = new LinkedHashMap<>(factors.size());
+        Map<String, Number> participationMap = getParticipationMap(network, lfNetwork, loadFlowParameters); // empty if slack is not distributed


Maybe participationFactorMap ?

annetill · 2020-12-14T16:35:15Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

        Map<SensitivityVariableConfiguration, SensitivityFactorGroup> factorsByVarConfig = new LinkedHashMap<>(factors.size());
+        Map<String, Number> participationMap = getParticipationMap(network, lfNetwork, loadFlowParameters); // empty if slack is not distributed
+        participationMap.remove(lfNetwork.getSlackBus().getId()); // the injection on the slack bus will not appear in the rhs
+        participationMap.replaceAll((key, value) -> -value.doubleValue()); // the compensation on a bus will be the opposite of its participation


compensation -> slack distribution

annetill · 2020-12-14T16:39:15Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

-                    SensitivityVariableConfiguration varConfig = new SensitivityVariableConfiguration(Collections.singleton(bus.getId()), Collections.emptySet());
+                    Map<String, Number> busCompensation = new HashMap<>(participationMap);
+                    // add 1 where we are making the injection
+                    busCompensation.put(bus.getId(), busCompensation.getOrDefault(bus.getId(), 0).doubleValue() + 1);


busCompensation is not the good name choice indeed.

annetill · 2020-12-14T16:48:35Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

+                } else if (participatingElement.getElement() instanceof LfBus) {
+                    busId = ((LfBus) participatingElement.getElement()).getId();
+                } else {
+                    throw new UnsupportedOperationException("Only buses can have an impact in slack distribution");


Indeed I am not sure that you can cover easily this line.

Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

annetill · 2020-12-15T13:06:25Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

@@ -142,7 +142,7 @@ private DenseMatrix initRhs(LfNetwork lfNetwork, EquationSystem equationSystem,
            for (Map.Entry<String, Number> busAndInjection : configuration.busInjectionById().entrySet()) {
                LfBus lfBus = lfNetwork.getBusById(busAndInjection.getKey());
                if (lfBus.isSlack()) {
-                    throw new PowsyblException("Cannot analyse sensitivity of slack bus");


I am not sure to get why this line is not cover by our test.

It is because we never reach this line, the slack bus will never be inside the businjectionbyId. But the exception here stays for the safety of future developments

src/main/java/com/powsybl/openloadflow/network/impl/LfBranchImpl.java

geofjamg · 2020-12-15T22:03:05Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

+            participatingElements = step.getParticipatingElements(lfNetwork);
+            ParticipatingElement.normalizeParticipationFactors(participatingElements, "bus");
+
+            participatingElements.forEach(participatingElement -> {


You can directly create the map like:

Map<String, Number> participationFactorByBusMap = participatingElements.stream().collect(Collectors.toMap(..));

geofjamg · 2020-12-15T22:04:22Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

+        // run DC load
+        Map<String, Double> functionReferenceByBranch = new HashMap<>();
+        DcLoadFlowParameters dcLoadFlowParameters = new DcLoadFlowParameters(lfParametersExt.getSlackBusSelector(),
+                new SparseMatrixFactory(), true, lfParametersExt.isDcUseTransformerRatio(), lfParameters.isDistributedSlack(),  lfParameters.getBalanceType());


MatrixFactory should be taken from parameters.

Done, but tp be checked by @Djazouli !

geofjamg · 2020-12-15T22:06:16Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

+     * @param loadFlowParameters
+     * @return
+     */
+    private Map<String, Number> getParticipationFactorByBus(Network network, LfNetwork lfNetwork, LoadFlowParameters loadFlowParameters) {


Map<String, Double> ?

geofjamg · 2020-12-15T22:09:08Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

@@ -36,17 +39,17 @@

    static class SensitivityVariableConfiguration {


Maybe we should remove SensitivityVariableConfiguration? I added it to avoid computing same sensibility multiple times but there is no reason for that to happen frequently (user input mistake?) and it adds useless complexity.
So instead of Map<SensitivityVariableConfiguration, SensitivityFactorGroup> factorsByVarConfig we would only have List<SensitivityFactor<?, ?>>

I feel like SensitivityVariableConfigurations are useful when the user gives a provider that contains all the generators of the network. Not sure if this is considered as an "user input mistake". But in this case, the SensitivityVariableConfiguration is useful to remove duplicate (generators connected on the same bus).

Ok so let s keep it

geofjamg · 2020-12-15T22:10:55Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

+        Map<String, Double> functionReferenceByBranch = new HashMap<>();
+        DcLoadFlowParameters dcLoadFlowParameters = new DcLoadFlowParameters(lfParametersExt.getSlackBusSelector(),
+                new SparseMatrixFactory(), true, lfParametersExt.isDcUseTransformerRatio(), lfParameters.isDistributedSlack(),  lfParameters.getBalanceType());
+        DcLoadFlowResult dcLoadFlowResult = new DcLoadFlowEngine(lfNetworks, dcLoadFlowParameters).run();


As we calculate sensitivity only on first component do we really need to run the initial DC loadflow on all components?

Indeed, in the DCLoadFlowEngine, we explicitely compute the LF just on the first connected component. So I leave it like that?

geofjamg · 2020-12-15T22:22:07Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

+            participatingElements.forEach(participatingElement -> {
+                String busId;
+                if (participatingElement.getElement() instanceof LfGenerator) {
+                    busId = network.getGenerator(((LfGenerator) participatingElement.getElement()).getId()).getTerminal().getBusView().getBus().getId();


We should avoid searching generator into IIDM model. We could just get LfBus from LfGenerator.

Indeed there is no existing method, you mean that we have to implement the getter on all LfGenerator or there is something easier?

I have tried something in the latest commit, tell me if this works for you:

geofjamg · 2020-12-15T22:33:49Z

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java

+        Map<String, Double> functionReferenceByBranch = new HashMap<>();
+        DcLoadFlowParameters dcLoadFlowParameters = new DcLoadFlowParameters(lfParametersExt.getSlackBusSelector(),
+                new SparseMatrixFactory(), true, lfParametersExt.isDcUseTransformerRatio(), lfParameters.isDistributedSlack(),  lfParameters.getBalanceType());
+        DcLoadFlowResult dcLoadFlowResult = new DcLoadFlowEngine(lfNetworks, dcLoadFlowParameters).run();


Maybe for later but running a DC LF using DcLoadFlowEngine is not efficient as we later need to build the same equation system and jacobian for sensitivity analysis.

Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>

… factor map Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

Signed-off-by: Geoffroy Jamgotchian <geoffroy.jamgotchian@rte-france.com>

sonarqubecloud · 2020-12-16T20:42:53Z

SonarCloud Quality Gate failed.

0 Bugs
0 Vulnerabilities
0 Security Hotspots
6 Code Smells

91.9% Coverage
3.8% Duplication

annetill

Thanks a lot @Djazouli and @Hadrien-Godard for this great PR !

Djazouli requested review from geofjamg and annetill December 3, 2020 17:37

Djazouli self-assigned this Dec 3, 2020

Djazouli commented Dec 3, 2020

View reviewed changes

src/main/java/com/powsybl/openloadflow/dc/equations/AbstractClosedBranchDcFlowEquationTerm.java Outdated Show resolved Hide resolved

Djazouli commented Dec 3, 2020

View reviewed changes

src/main/java/com/powsybl/openloadflow/sensi/DcSensitivityAnalysis.java Outdated Show resolved Hide resolved

Djazouli commented Dec 3, 2020

View reviewed changes

Base automatically changed from sensi to master December 4, 2020 10:20

annetill changed the title ~~[WIP] first commit for distributed slack in sensitivity analysis~~ [WIP] Distributed slack management in sensitivity analysis Dec 4, 2020

Djazouli force-pushed the sensi_compense branch from 76dc05f to 71ee251 Compare December 4, 2020 10:55

annetill reviewed Dec 8, 2020

View reviewed changes

Djazouli force-pushed the sensi_compense branch from 642b814 to db45230 Compare December 9, 2020 17:08

Djazouli linked an issue Dec 9, 2020 that may be closed by this pull request

Manage distributedSlack in Sensitivity Analysis #185

Closed

Djazouli commented Dec 9, 2020

View reviewed changes

Djazouli and others added 9 commits December 10, 2020 16:54

first commit for distributed slack in sensitivity analysis

31cccbd

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

fix checkstyle

3d65393

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

add sensitivity tests

3c13cb3

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

fix tests for sensitivity analysis with distributed slack

3c5d2af

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

remove dependency to network in the function that generates all facto…

7c8e3d6

…rs required for distribution Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

Add DC LF computation before sensitivity values computation.

3f8cb09

Improve variable naming. Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>

First import of distribution on load correction.

89e7abd

Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>

refactor compensed sensitivity to avoid creating new factors

b949128

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

adding test

47665b0

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

Djazouli added 3 commits December 10, 2020 16:55

fix checkstyle

380e2da

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

refactor code to make it more understandable

47af784

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

rebase for distribution refactor

60cf4da

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

Djazouli force-pushed the sensi_compense branch from 6e3b924 to 60cf4da Compare December 10, 2020 16:40

annetill and others added 2 commits December 14, 2020 13:30

Merge branch 'master' into sensi_compense

71dbbce

Add unit tests.

e710cea

Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>

annetill changed the title ~~[WIP] Distributed slack management in sensitivity analysis~~ Distributed slack management in sensitivity analysis Dec 14, 2020

annetill marked this pull request as ready for review December 14, 2020 13:53

annetill added 2 commits December 14, 2020 17:10

Add unit tests.

51a3bf3

Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>

Fix unit test and add unit test.

a264a4a

Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>

annetill reviewed Dec 14, 2020

View reviewed changes

annetill and others added 6 commits December 15, 2020 08:42

Add a test with two generators connected to the same bus.

f6714f4

Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>

fix some code smells and tests

6001930

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

fix code smell

515be22

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

rename variables

39a8b23

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

Proposal.

f279733

Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>

allow computation of sensitivity on slack bus if slack is distributed

28dbf00

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

annetill reviewed Dec 15, 2020

View reviewed changes

geofjamg requested changes Dec 15, 2020

View reviewed changes

geofjamg reviewed Dec 15, 2020

View reviewed changes

annetill and others added 3 commits December 16, 2020 07:27

Some fixes after review.

c137716

Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>

change Number to Double, and use Collectors to create the sensitivity…

06204d0

… factor map Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

Add bus getter in LfGenerator

c434272

Signed-off-by: Gael Macherel <gael.macherel@artelys.com>

Djazouli requested a review from geofjamg December 16, 2020 12:10

Remove BusElement

e2c041a

Signed-off-by: Geoffroy Jamgotchian <geoffroy.jamgotchian@rte-france.com>

geofjamg approved these changes Dec 16, 2020

View reviewed changes

annetill approved these changes Dec 17, 2020

View reviewed changes

annetill merged commit 8b32eff into master Dec 17, 2020

annetill deleted the sensi_compense branch December 17, 2020 08:29

		Map<SensitivityVariableConfiguration, SensitivityFactorGroup> factorsByVarConfig = new LinkedHashMap<>(factors.size());
		Map<String, Number> participationMap = getParticipationMap(network, lfNetwork, loadFlowParameters); // empty if slack is not distributed

		@@ -36,17 +39,17 @@

		static class SensitivityVariableConfiguration {

Distributed slack management in sensitivity analysis #178

Distributed slack management in sensitivity analysis #178

Conversation

Djazouli commented Dec 3, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Djazouli Dec 3, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

annetill left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Djazouli commented Dec 9, 2020

annetill left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Dec 16, 2020

annetill left a comment

Choose a reason for hiding this comment

Djazouli Dec 3, 2020 •

edited

Loading