Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Security analysis functional logs #576

Merged
merged 5 commits into from
Jul 6, 2022
Merged

Security analysis functional logs #576

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
67d2492
Security analysis functional logs
geofjamg Jul 3, 2022
825ac97
Fix hierarchy
geofjamg Jul 3, 2022
a86224c
Add more logs
geofjamg Jul 3, 2022
3cb9d98
Fix Sonar issue
geofjamg Jul 3, 2022
f47e316
Merge branch 'main' into sa_report
geofjamg Jul 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/main/java/com/powsybl/openloadflow/ac/outerloop/AcloadFlowEngine.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,8 @@ public AcLoadFlowResult run() {

LOGGER.info("Ac loadflow complete on network {} (result={})", context.getNetwork(), result);

Reports.reportAcLfComplete(context.getNetwork().getReporter(), result.getNewtonRaphsonStatus().name());

return result;
}

Expand Down
12 changes: 8 additions & 4 deletions src/main/java/com/powsybl/openloadflow/network/LfNetwork.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ public class LfNetwork extends AbstractPropertyBag implements PropertyBag {

private GraphDecrementalConnectivity<LfBus, LfBranch> connectivity;

private final Reporter reporter;
private Reporter reporter;

public LfNetwork(int numCC, int numSC, SlackBusSelector slackBusSelector,
GraphDecrementalConnectivityFactory<LfBus, LfBranch> connectivityFactory, Reporter reporter) {
Expand Down Expand Up @@ -107,6 +107,10 @@ public Reporter getReporter() {
return reporter;
}

public void setReporter(Reporter reporter) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here you're breaking in a way the way the reporter was at first designed... It sure is handy to put it in LfNetwork but it was first thought to be passed along all the methods and not retained, in the javadoc:

A reporter is not meant to be shared with other threads nor to be saved as a class parameter

Copy link
Member Author

@geofjamg geofjamg Jul 6, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A LF network won't ever be shared between multiple threads as it is used as working data (including writing working data)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it was first thought to be passed along all the methods

Sure not, this is a usage detail which has nothing to do with reporter API. The only thing that is clear is that it is not thread safe. From that it is up to each user to decide the best way to use it. In case of structural single thread part of the code it is ok to put it in the applicative thread context (and this is what LfNetwork is).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. I missed the thing that LfNetwork is (and will be) always working data and thus by nature will never be shared.

this.reporter = Objects.requireNonNull(reporter);
}

private void invalidateSlack() {
slackBus = null;
}
Expand Down Expand Up @@ -427,7 +431,7 @@ public void writeJson(Writer writer) {
}

private void reportSize(Reporter reporter) {
Reports.reportNetworkSize(reporter, numCC, numSC, busesById.values().size(), branches.size());
Reports.reportNetworkSize(reporter, busesById.values().size(), branches.size());
LOGGER.info("Network {} has {} buses and {} branches",
this, busesById.values().size(), branches.size());
}
Expand All @@ -444,7 +448,7 @@ public void reportBalance(Reporter reporter) {
reactiveLoad += b.getLoadTargetQ() * PerUnit.SB;
}

Reports.reportNetworkBalance(reporter, numCC, numSC, activeGeneration, activeLoad, reactiveGeneration, reactiveLoad);
Reports.reportNetworkBalance(reporter, activeGeneration, activeLoad, reactiveGeneration, reactiveLoad);
LOGGER.info("Network {} balance: active generation={} MW, active load={} MW, reactive generation={} MVar, reactive load={} MVar",
this, activeGeneration, activeLoad, reactiveGeneration, reactiveLoad);
}
Expand All @@ -468,7 +472,7 @@ private void validateBuses(boolean dc, Reporter reporter) {
}
if (!hasAtLeastOneBusVoltageControlled) {
LOGGER.error("Network {} must have at least one bus voltage controlled", this);
Reports.reportNetworkMustHaveAtLeastOneBusVoltageControlled(reporter, numCC, numSC);
Reports.reportNetworkMustHaveAtLeastOneBusVoltageControlled(reporter);
valid = false;
}
}
Expand Down
6 changes: 5 additions & 1 deletion src/main/java/com/powsybl/openloadflow/sa/AbstractSecurityAnalysis.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
*/
package com.powsybl.openloadflow.sa;

import com.powsybl.commons.reporter.Reporter;
import com.powsybl.computation.CompletableFutureTask;
import com.powsybl.computation.ComputationManager;
import com.powsybl.contingency.ContingenciesProvider;
Expand Down Expand Up @@ -40,12 +41,15 @@ public abstract class AbstractSecurityAnalysis {

protected final StateMonitorIndex monitorIndex;

protected final Reporter reporter;

protected AbstractSecurityAnalysis(Network network, MatrixFactory matrixFactory, GraphDecrementalConnectivityFactory<LfBus, LfBranch> connectivityFactory,
List<StateMonitor> stateMonitors) {
List<StateMonitor> stateMonitors, Reporter reporter) {
this.network = Objects.requireNonNull(network);
this.matrixFactory = Objects.requireNonNull(matrixFactory);
this.connectivityFactory = Objects.requireNonNull(connectivityFactory);
this.monitorIndex = new StateMonitorIndex(stateMonitors);
this.reporter = Objects.requireNonNull(reporter);
}

public CompletableFuture<SecurityAnalysisReport> run(String workingVariantId, SecurityAnalysisParameters securityAnalysisParameters,
Expand Down
23 changes: 17 additions & 6 deletions src/main/java/com/powsybl/openloadflow/sa/AcSecurityAnalysis.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
import com.powsybl.openloadflow.network.impl.PropagatedContingency;
import com.powsybl.openloadflow.network.util.ActivePowerDistribution;
import com.powsybl.openloadflow.network.util.PreviousValueVoltageInitializer;
import com.powsybl.openloadflow.util.Reports;
import com.powsybl.security.LimitViolationsResult;
import com.powsybl.security.SecurityAnalysisParameters;
import com.powsybl.security.SecurityAnalysisReport;
Expand All @@ -42,8 +43,8 @@
public class AcSecurityAnalysis extends AbstractSecurityAnalysis {

protected AcSecurityAnalysis(Network network, MatrixFactory matrixFactory, GraphDecrementalConnectivityFactory<LfBus, LfBranch> connectivityFactory,
List<StateMonitor> stateMonitors) {
super(network, matrixFactory, connectivityFactory, stateMonitors);
List<StateMonitor> stateMonitors, Reporter reporter) {
super(network, matrixFactory, connectivityFactory, stateMonitors, reporter);
}

private static SecurityAnalysisResult createNoResult() {
Expand All @@ -57,6 +58,8 @@ private static SecurityAnalysisResult createNoResult() {
@Override
SecurityAnalysisReport runSync(String workingVariantId, SecurityAnalysisParameters securityAnalysisParameters, ContingenciesProvider contingenciesProvider,
ComputationManager computationManager) {
var saReporter = Reports.createAcSecurityAnalysis(reporter, network.getId());

Stopwatch stopwatch = Stopwatch.createStarted();

LoadFlowParameters lfParameters = securityAnalysisParameters.getLoadFlowParameters();
Expand All @@ -74,10 +77,10 @@ SecurityAnalysisReport runSync(String workingVariantId, SecurityAnalysisParamete
lfParameters.isShuntCompensatorVoltageControlOn(), lfParameters.getBalanceType() == LoadFlowParameters.BalanceType.PROPORTIONAL_TO_CONFORM_LOAD,
lfParameters.isHvdcAcEmulation());

AcLoadFlowParameters acParameters = OpenLoadFlowParameters.createAcParameters(network, lfParameters, lfParametersExt, matrixFactory, connectivityFactory, Reporter.NO_OP, true, false);
AcLoadFlowParameters acParameters = OpenLoadFlowParameters.createAcParameters(network, lfParameters, lfParametersExt, matrixFactory, connectivityFactory, saReporter, true, false);

// create networks including all necessary switches
List<LfNetwork> lfNetworks = createNetworks(allSwitchesToOpen, acParameters.getNetworkParameters());
List<LfNetwork> lfNetworks = createNetworks(allSwitchesToOpen, acParameters.getNetworkParameters(), saReporter);

// run simulation on largest network
SecurityAnalysisResult result;
Expand All @@ -99,15 +102,16 @@ SecurityAnalysisReport runSync(String workingVariantId, SecurityAnalysisParamete
return new SecurityAnalysisReport(result);
}

List<LfNetwork> createNetworks(Set<Switch> allSwitchesToOpen, LfNetworkParameters networkParameters) {
List<LfNetwork> createNetworks(Set<Switch> allSwitchesToOpen, LfNetworkParameters networkParameters,
Reporter saReporter) {
List<LfNetwork> lfNetworks;
String tmpVariantId = "olf-tmp-" + UUID.randomUUID();
network.getVariantManager().cloneVariant(network.getVariantManager().getWorkingVariantId(), tmpVariantId);
try {
network.getSwitchStream().filter(sw -> sw.getVoltageLevel().getTopologyKind() == TopologyKind.NODE_BREAKER)
.forEach(sw -> sw.setRetained(false));
allSwitchesToOpen.forEach(sw -> sw.setRetained(true));
lfNetworks = Networks.load(network, networkParameters, Reporter.NO_OP);
lfNetworks = Networks.load(network, networkParameters, saReporter);
} finally {
network.getVariantManager().removeVariant(tmpVariantId);
}
Expand All @@ -130,6 +134,10 @@ private SecurityAnalysisResult runSimulations(LfNetwork network, List<Propagated
boolean createResultExtension = openSecurityAnalysisParameters.isCreateResultExtension();

try (AcLoadFlowContext context = new AcLoadFlowContext(network, acParameters)) {
Reporter networkReporter = network.getReporter();
Reporter preContSimReporter = Reports.createPreContingencySimulation(networkReporter);
network.setReporter(preContSimReporter);

// run pre-contingency simulation
AcLoadFlowResult preContingencyLoadFlowResult = new AcloadFlowEngine(context)
.run();
Expand All @@ -156,6 +164,9 @@ private SecurityAnalysisResult runSimulations(LfNetwork network, List<Propagated
PropagatedContingency propagatedContingency = contingencyIt.next();
propagatedContingency.toLfContingency(network, true)
.ifPresent(lfContingency -> { // only process contingencies that impact the network
Reporter postContSimReporter = Reports.createPostContingencySimulation(networkReporter, lfContingency.getId());
network.setReporter(postContSimReporter);

lfContingency.apply(loadFlowParameters.getBalanceType());

distributedMismatch(network, lfContingency.getActivePowerLoss(), loadFlowParameters, openLoadFlowParameters);
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/com/powsybl/openloadflow/sa/DcSecurityAnalysis.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@
public class DcSecurityAnalysis extends AbstractSecurityAnalysis {

protected DcSecurityAnalysis(Network network, MatrixFactory matrixFactory, GraphDecrementalConnectivityFactory<LfBus, LfBranch> connectivityFactory,
List<StateMonitor> stateMonitors) {
super(network, matrixFactory, connectivityFactory, stateMonitors);
List<StateMonitor> stateMonitors, Reporter reporter) {
super(network, matrixFactory, connectivityFactory, stateMonitors, reporter);
}

@Override
Expand Down
5 changes: 3 additions & 2 deletions src/main/java/com/powsybl/openloadflow/sa/OpenSecurityAnalysisProvider.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,12 +62,13 @@ public CompletableFuture<SecurityAnalysisReport> run(Network network, String wor
Objects.requireNonNull(securityAnalysisParameters);
Objects.requireNonNull(contingenciesProvider);
Objects.requireNonNull(stateMonitors);
Objects.requireNonNull(reporter);

AbstractSecurityAnalysis securityAnalysis;
if (securityAnalysisParameters.getLoadFlowParameters().isDc()) {
securityAnalysis = new DcSecurityAnalysis(network, matrixFactory, connectivityFactory, stateMonitors);
securityAnalysis = new DcSecurityAnalysis(network, matrixFactory, connectivityFactory, stateMonitors, reporter);
} else {
securityAnalysis = new AcSecurityAnalysis(network, matrixFactory, connectivityFactory, stateMonitors);
securityAnalysis = new AcSecurityAnalysis(network, matrixFactory, connectivityFactory, stateMonitors, reporter);
}

return securityAnalysis.run(workingVariantId, securityAnalysisParameters, contingenciesProvider, computationManager);
Expand Down
47 changes: 32 additions & 15 deletions src/main/java/com/powsybl/openloadflow/util/Reports.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,40 +21,35 @@ public final class Reports {
private static final String NETWORK_NUM_CC = "networkNumCc";
private static final String NETWORK_NUM_SC = "networkNumSc";
private static final String ITERATION = "iteration";
private static final String NETWORK_ID = "networkId";

private Reports() {
}

public static void reportNetworkSize(Reporter reporter, int networkNumCc, int networkNumSc, int busCount, int branchCount) {
public static void reportNetworkSize(Reporter reporter, int busCount, int branchCount) {
reporter.report(Report.builder()
.withKey("networkSize")
.withDefaultMessage("Network CC${networkNumCc} SC${networkNumSc} has ${busCount} buses and ${branchCount} branches")
.withValue(NETWORK_NUM_CC, networkNumCc)
.withValue(NETWORK_NUM_SC, networkNumSc)
.withDefaultMessage("Network has ${busCount} buses and ${branchCount} branches")
.withValue("busCount", busCount)
.withValue("branchCount", branchCount)
.build());
}

public static void reportNetworkBalance(Reporter reporter, int networkNumCc, int networkNumSc, double activeGeneration, double activeLoad, double reactiveGeneration, double reactiveLoad) {
public static void reportNetworkBalance(Reporter reporter, double activeGeneration, double activeLoad, double reactiveGeneration, double reactiveLoad) {
reporter.report(Report.builder()
.withKey("networkBalance")
.withDefaultMessage("Network CC${networkNumCc} SC${networkNumSc} balance: active generation=${activeGeneration} MW, active load=${activeLoad} MW, reactive generation=${reactiveGeneration} MVar, reactive load=${reactiveLoad} MVar")
.withValue(NETWORK_NUM_CC, networkNumCc)
.withValue(NETWORK_NUM_SC, networkNumSc)
.withDefaultMessage("Network balance: active generation=${activeGeneration} MW, active load=${activeLoad} MW, reactive generation=${reactiveGeneration} MVar, reactive load=${reactiveLoad} MVar")
.withValue("activeGeneration", activeGeneration)
.withValue("activeLoad", activeLoad)
.withValue("reactiveGeneration", reactiveGeneration)
.withValue("reactiveLoad", reactiveLoad)
.build());
}

public static void reportNetworkMustHaveAtLeastOneBusVoltageControlled(Reporter reporter, int networkNumCc, int networkNumSc) {
public static void reportNetworkMustHaveAtLeastOneBusVoltageControlled(Reporter reporter) {
reporter.report(Report.builder()
.withKey("networkMustHaveAtLeastOneBusVoltageControlled")
.withDefaultMessage("Network CC${networkNumCc} SC${networkNumSc} must have at least one bus voltage controlled")
.withValue(NETWORK_NUM_CC, networkNumCc)
.withValue(NETWORK_NUM_SC, networkNumSc)
.withDefaultMessage("Network must have at least one bus voltage controlled")
.build());
}

Expand Down Expand Up @@ -142,9 +137,17 @@ public static void reportGeneratorsDiscardedFromVoltageControlBecauseMaxReactive
.build());
}

public static void reportAcLfComplete(Reporter reporter, String nrStatus) {
reporter.report(Report.builder()
.withKey("acLfComplete")
.withDefaultMessage("AC load flow complete with NR status '${nrStatus}'")
.withValue("nrStatus", nrStatus)
.build());
}

public static Reporter createLoadFlowReporter(Reporter reporter, String networkId) {
return reporter.createSubReporter("loadFlow", "Load flow on network ${networkId}",
"networkId", networkId);
return reporter.createSubReporter("loadFlow", "Load flow on network '${networkId}'",
NETWORK_ID, networkId);
}

public static Reporter createLfNetworkReporter(Reporter reporter, int networkNumCc, int networkNumSc) {
Expand All @@ -163,6 +166,20 @@ public static Reporter createOuterLoopReporter(Reporter reporter, String outerLo

public static Reporter createSensitivityAnalysis(Reporter reporter, String networkId) {
return reporter.createSubReporter("sensitivityAnalysis",
"Sensitivity analysis on network ${networkId}", "networkId", networkId);
"Sensitivity analysis on network '${networkId}'", NETWORK_ID, networkId);
}

public static Reporter createAcSecurityAnalysis(Reporter reporter, String networkId) {
return reporter.createSubReporter("acSecurityAnalysis",
"AC security analysis on network '${networkId}'", NETWORK_ID, networkId);
}

public static Reporter createPreContingencySimulation(Reporter reporter) {
return reporter.createSubReporter("preContingencySimulation", "Pre-contingency simulation");
}

public static Reporter createPostContingencySimulation(Reporter reporter, String contingencyId) {
return reporter.createSubReporter("postContingencySimulation", "Post-contingency simulation '${contingencyId}'",
"contingencyId", contingencyId);
}
}
2 changes: 1 addition & 1 deletion src/test/java/com/powsybl/openloadflow/ac/AcLoadFlowEurostagTutorialExample1Test.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,7 @@ void noGeneratorTest() {
ReporterModel postLoadingReporter = createNetworkReporter.getSubReporters().get(0);
assertEquals("postLoadingProcessing", postLoadingReporter.getTaskKey());
assertEquals(1, postLoadingReporter.getReports().size());
assertEquals("Network CC${networkNumCc} SC${networkNumSc} must have at least one bus voltage controlled",
assertEquals("Network must have at least one bus voltage controlled",
postLoadingReporter.getReports().iterator().next().getDefaultMessage());
}

Expand Down
7 changes: 4 additions & 3 deletions src/test/java/com/powsybl/openloadflow/sa/LfContingencyTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

import com.powsybl.commons.AbstractConverterTest;
import com.powsybl.commons.PowsyblException;
import com.powsybl.commons.reporter.Reporter;
import com.powsybl.contingency.BranchContingency;
import com.powsybl.contingency.Contingency;
import com.powsybl.contingency.GeneratorContingency;
Expand Down Expand Up @@ -65,7 +66,7 @@ void test() throws IOException {
LfNetwork mainNetwork = lfNetworks.get(0);
assertEquals(2, lfNetworks.size());

new AcSecurityAnalysis(network, new DenseMatrixFactory(), connectivityFactory, Collections.emptyList());
new AcSecurityAnalysis(network, new DenseMatrixFactory(), connectivityFactory, Collections.emptyList(), Reporter.NO_OP);

String branchId = "LINE_S3S4";
Contingency contingency = new Contingency(branchId, new BranchContingency(branchId));
Expand Down Expand Up @@ -94,7 +95,7 @@ void testGeneratorNotFound() {
assertEquals(2, lfNetworks.size());

GraphDecrementalConnectivityFactory<LfBus, LfBranch> connectivityFactory = new EvenShiloachGraphDecrementalConnectivityFactory<>();
new AcSecurityAnalysis(network, new DenseMatrixFactory(), connectivityFactory, Collections.emptyList());
new AcSecurityAnalysis(network, new DenseMatrixFactory(), connectivityFactory, Collections.emptyList(), Reporter.NO_OP);

String generatorId = "GEN";
Contingency contingency = new Contingency(generatorId, new GeneratorContingency(generatorId));
Expand All @@ -110,7 +111,7 @@ void testLoadNotFound() {
assertEquals(2, lfNetworks.size());

GraphDecrementalConnectivityFactory<LfBus, LfBranch> connectivityFactory = new EvenShiloachGraphDecrementalConnectivityFactory<>();
new AcSecurityAnalysis(network, new DenseMatrixFactory(), connectivityFactory, Collections.emptyList());
new AcSecurityAnalysis(network, new DenseMatrixFactory(), connectivityFactory, Collections.emptyList(), Reporter.NO_OP);

String loadId = "LOAD";
Contingency contingency = new Contingency(loadId, new LoadContingency(loadId));
Expand Down
4 changes: 2 additions & 2 deletions src/test/java/com/powsybl/openloadflow/sa/OpenSecurityAnalysisGraphTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ private void printResult(List<List<LfContingency>> result) {
List<List<LfContingency>> getLoadFlowContingencies(GraphDecrementalConnectivityFactory<LfBus, LfBranch> connectivityFactory) {

var matrixFactory = new DenseMatrixFactory();
AcSecurityAnalysis securityAnalysis = new AcSecurityAnalysis(network, matrixFactory, connectivityFactory, Collections.emptyList());
AcSecurityAnalysis securityAnalysis = new AcSecurityAnalysis(network, matrixFactory, connectivityFactory, Collections.emptyList(), Reporter.NO_OP);

LoadFlowParameters lfParameters = securityAnalysisParameters.getLoadFlowParameters();
OpenLoadFlowParameters lfParametersExt = OpenLoadFlowParameters.get(securityAnalysisParameters.getLoadFlowParameters());
Expand All @@ -157,7 +157,7 @@ List<List<LfContingency>> getLoadFlowContingencies(GraphDecrementalConnectivityF
lfParameters, lfParametersExt, matrixFactory, connectivityFactory, Reporter.NO_OP, true, false);

// create networks including all necessary switches
List<LfNetwork> lfNetworks = securityAnalysis.createNetworks(allSwitchesToOpen, acParameters.getNetworkParameters());
List<LfNetwork> lfNetworks = securityAnalysis.createNetworks(allSwitchesToOpen, acParameters.getNetworkParameters(), Reporter.NO_OP);

// run simulation on each network
start = System.currentTimeMillis();
Expand Down
Loading