Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Beta version of security analysis #135

Merged
merged 86 commits into from
Oct 13, 2020
Merged

Beta version of security analysis #135

merged 86 commits into from
Oct 13, 2020

Conversation

geofjamg
Copy link
Member

@geofjamg geofjamg commented Aug 2, 2020

Please check if the PR fulfills these requirements (please use '[x]' to check the checkboxes, or submit the PR and then click the checkboxes)

  • The commit message follows our guidelines
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)

Does this PR already have an issue describing the problem ? If so, link to this issue using '#XXX' and skip the rest

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

What is the current behavior? (You can also link to an open issue here)

What is the new behavior (if this is a feature change)?

Does this PR introduce a breaking change or deprecate an API? If yes, check the following:

  • The Breaking Change or Deprecated label has been added
  • The migration guide has been updated in the github wiki (What changes might users need to make in their application due to this PR?)

Other information:

(if any of the questions/checkboxes don't apply, please delete them entirely)

geofjamg added 4 commits June 18, 2020 14:20
Signed-off-by: Geoffroy Jamgotchian <geoffroy.jamgotchian@rte-france.com>
Signed-off-by: Geoffroy Jamgotchian <geoffroy.jamgotchian@rte-france.com>
Signed-off-by: Geoffroy Jamgotchian <geoffroy.jamgotchian@rte-france.com>
Signed-off-by: Geoffroy Jamgotchian <geoffroy.jamgotchian@rte-france.com>
@geofjamg geofjamg changed the title Security analysis [WIP] Security analysis Aug 4, 2020
geofjamg and others added 11 commits August 4, 2020 14:00
Signed-off-by: Geoffroy Jamgotchian <geoffroy.jamgotchian@rte-france.com>
Signed-off-by: Geoffroy Jamgotchian <geoffroy.jamgotchian@rte-france.com>
Signed-off-by: Geoffroy Jamgotchian <geoffroy.jamgotchian@rte-france.com>
Signed-off-by: Geoffroy Jamgotchian <geoffroy.jamgotchian@rte-france.com>
Signed-off-by: Geoffroy Jamgotchian <geoffroy.jamgotchian@gmail.com>
Signed-off-by: Geoffroy Jamgotchian <geoffroy.jamgotchian@gmail.com>
Signed-off-by: Geoffroy Jamgotchian <geoffroy.jamgotchian@gmail.com>
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
- using BiconnectivityInspector
- using ConnectivityInspector

Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
@flo-dup
Copy link
Contributor

flo-dup commented Aug 13, 2020

Graph calculations are needed in this PR to calculate the connected components; for each contingency we check if connected component is split or not by the equipments removal. As for each contingency we remove only a few edges from the original graph, we might optimize these calculations by implementing a dynamic graph algorithm. A dynamic graph algorithm is an algorithm which is maintaining a result after each graph modification (here edge deletion) in order to avoid unnecessary computations.

The algorithm suggested in the related issue #129 is Even-Shiloach'81: complexity of graph update O(n) / of requests O(1), where n is the number of vertices. It has been implemented in previous commit. It is based on 2 processes working simultanously after each edge deletion, the first process which halts stops the other.

  • a classical process A detects if a new connected component is created
  • a more complex process B detects if the connected components remain intact

Drawbacks:

  • only supports edge deletion, hence to deal with contingencies one after the other we cannot use edge insertion to undo the deletion. In current implementation I saved the results of the original graph and cloned them when a new contingency is considered - this might be improved though.
  • when an edge is deleted and it doesn't create a new connected component, all the changes done by process B have to be tediously undone (I still need to implement that, thus so far it does not support several edge deletions in a row)

A few other algorithms exist for this matter:

  • Frederickson'84: complexity of graph update 0(sqrt(m)) / of requests O(1) (m: number of edges), which is based on a minimum spanning tree maintained through the changes of the graph
  • Eppstein et al'97: complexity of graph update 0(sqrt(n)) / of requests O(1), which is based on a "sparsification" of previous algorithm; we might want to avoid the introduced complexity as this is of interest for dense graphs only
  • and others: since then more complex algorithms have been introduced which have better update complexity, but the request cost is not in O(1), and most of them are randomized algorithm. For instance, the (deterministic) algorithm Holm et al'01 has an update cost of O(log²n), and each request has O(log n/log log n) complexity.

I'm wondering if we need to go towards those more complex algorithms?

Note that jgrapht ConnectivityInspector now supports dynamic graphs... nonetheless, in current version, when a edge is removed all the calculations are reinitialized (with following comment: "for now invalidate cached results, in the future need to amend them")

@flo-dup
Copy link
Contributor

flo-dup commented Aug 14, 2020

To optimize these algorithms, we can use a bridge detection pre-calculation, which will be useful if many contingencies are tested (a bridge is an edge which creates a new connected component if removed). Indeed, that allows not to compute the connected component if we know that the edge removed is not a bridge. AND if only one edge is removed (no fault propagation...). Computing bridges in a graph is a classical problem (see e.g. Hopcroft & Tarjan's algorithm implemented in in jgrapht BiconnectivityInspector) and is done in a single graph scan. Nonetheless I don't know of any algorithm for dynamic graphs supporting edge deletion, so it's only for the first edge deletion.

This is of interest if we're dealing with graphs which don't have many bridges. In the network I used for testing, there is about 25% of bridges. Note that among those bridges there are only 15% which produce a new connected component which is bigger than just one node.

@flo-dup
Copy link
Contributor

flo-dup commented Aug 14, 2020

In BridgesTest I used algorithms computing connected components to find bridges in a basic network. Each branch of the network is deleted and I test if the two buses of the branch are still in the same connected component after that deletion. It doesn't correspond exactly to the use case but it already gives an idea of the algorithms performance. On a more complex network of 10000 branches the results are the following on my laptop:

  • Naive implementation through jgrapht ConnectivityInspector (calculation done from scratch after each deletion): ~40s
  • Connectivity calculation based on graphstream library: ~15s
  • Connectivity calculation based on Even-Shiloach algorithm: ~10s

With that same network, the bridges calculation is done in about 5ms. Therefore, the previous execution times are divided

  • by about 4 if the calculation is skipped for non-bridges edge deletion (about 25% of bridges in that network)
  • by about 20 if the calculation is also skipped for bridges which produce a new connected component of size 1 (in that test network, 5% of the edges are bridges producing a new connected component of size greater than 1)

flo-dup and others added 6 commits August 17, 2020 11:10
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
Signed-off-by: Geoffroy Jamgotchian <geoffroy.jamgotchian@gmail.com>
Signed-off-by: Geoffroy Jamgotchian <geoffroy.jamgotchian@gmail.com>
flo-dup and others added 5 commits October 2, 2020 23:48
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
- now supports internal connections
- now supports branching of several switches at an end node
- isLineBeforeSwitch removed: lines are never before a switch in traverser (except if the line is the origin of the contingency)

Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
@annetill annetill changed the title [WIP] Security analysis Beta version of security analysis Oct 8, 2020
annetill and others added 4 commits October 8, 2020 08:28
- Add the active power loss of a LfContingency.

Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>
Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
Corresponding package: com.powsybl.openloadflow.sa

Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
flo-dup and others added 3 commits October 12, 2020 11:43
Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>
The traverser can add all terminals of the traversed connectable

Signed-off-by: Florian Dupuy <florian.dupuy@rte-france.com>
Some TODO et FIXME.

Signed-off-by: Anne Tilloy <anne.tilloy@rte-france.com>
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities (and Security Hotspot 0 Security Hotspots to review)
Code Smell A 14 Code Smells

90.2% 90.2% Coverage
1.0% 1.0% Duplication

@annetill
Copy link
Member

@geofjamg @floriand-e2r this PR is for me ready to be merged. @geofjamg do you agree?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants