Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

README Updates #91

Merged
merged 31 commits into from
May 30, 2023
Merged
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
8418fa1
Update intro
sajith Jan 11, 2023
19d7129
Reformat text
sajith Jan 11, 2023
3e20a17
Link to Atlanticwave-SDX URL
sajith Jan 11, 2023
1949864
Reword intro
sajith Jan 11, 2023
ff2408f
Add PCE CI badge
sajith Jan 11, 2023
0e75dcf
Add PCE test coverage badge
sajith Jan 11, 2023
2967344
Use right comment syntax
sajith Jan 11, 2023
83d382d
Update topology description
sajith Jan 11, 2023
0b333cc
Update input
sajith Jan 11, 2023
23bbf2b
Remove reference to connection.json
sajith Jan 11, 2023
47ba678
Update connection request description
sajith Jan 11, 2023
e471da8
Mention random connection and topology generators
sajith Jan 11, 2023
0a449da
Remove outdated testing related notes
sajith Jan 11, 2023
6215874
Merge remote-tracking branch 'origin/main' into 41.update-readme
sajith May 30, 2023
f557d0c
Trim whitespace
sajith May 30, 2023
0ab1ef5
Update connection request description
sajith May 30, 2023
e759632
Remove outdated solver descriptions
sajith May 30, 2023
301d278
Add an example
sajith May 30, 2023
556817c
Move example up
sajith May 30, 2023
52f43cd
Update description of input and intermediate steps
sajith May 30, 2023
dac4d25
Add link to networkx
sajith May 30, 2023
5bc959a
Add link to OR-Tools
sajith May 30, 2023
8b8d226
Update usage description
sajith May 30, 2023
4154983
Upgrade pip and setuptools during venv setup
sajith May 30, 2023
937edf8
Use correct grammar
sajith May 30, 2023
292f9ca
Add link to test data
sajith May 30, 2023
46cac31
Correct link
sajith May 30, 2023
3f6dbf1
Update usage description
sajith May 30, 2023
0e614da
Remove outdated description
sajith May 30, 2023
c1d8500
Update general description
sajith May 30, 2023
2931bb3
Update general description
sajith May 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 73 additions & 64 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,85 +1,83 @@
# PCE
## Run with python

### Input
Input includes two main parts: Network Topology and Connections\
\
Both input json file is stored under `./tests/data`
#### Network Topology
%The connection will be a list of list stored in /test/data/connection.json.
\
The Network Topology should be generated in the format of NetworkX graph. For each link, three attributes need to be assigned: cost, bandwidth and latency. The current unit of each attribute is abstract, but the unit in the Network Topology should be consistent with the unit in Connections.\
\
Currently, the graph is randomly generated by using NetworkX and all the attributes are randomly assigned based on the users setting.
#### Connection
The connection will be a list of list stored in `./tests/data/connection.json`. \
\
The format of the input connection is `[Source node, Destination node, Bandwidth required, Latency required]`. \
Here is an example of a random connection.json:
```
[[1,10,8,20],[2,9,10,15],[15,10,6,22]]
```
There are three queries in this connection.json, the first one is `[1,10,15,20]`.\
It means this connection query is to route traffic from Node 1 to Node 10, requring a bandwith of 8 and maximum latency of 20.\
\
For testing, a random connection generator is located at `Utility/RandomConnectionGen.py`. It can randomly generate one or multiple queries in one connection.json.
# Path Computation Element

### Connection Generation (use bullet just for random generation)
Random connection generator is located at Utility/RandomConnectionGen.py.
```
RandomConnectionGenerator(nodes, querynum, bw, latencylimit)
```
This will randomly generate one or multiple queries in one connection.json stored in `tests/data/connection.json`.
### Where to Input
Connection is called at the beginning of lbnxgraphgenerator() using GetConnection() in `/LoadBalancing/RandomTopologyGenerator`
\
[![pce-ci-badge]][pce-ci] [![pce-cov-badge]][pce-cov]

g will be the input NetworkX format topology. The current random topology is get from GetNetworkToplogy() .
Path Computation Element, also called PCE, is a component of
[Atlanticwave SDX][aw-sdx] project.

### Output
To get the optimal path and related objective value:
#### MC_Solver (how I split the function and call the input)
For Minimizing the total path cost, do `from LoadBalancing.MC_Solver import runMC_Solver`:
```
runMC_Solver()
```
This will output a tuple of 2 elements: (Path List Dict, Total Cost).\
For example, using the previous connection.json, the output will be:
```
({1: [[1, 13], [13, 19], [19, 8], [8, 2], [2, 10]], 2: [[2, 10], [10, 9]], 3: [[15, 10]]}, 27401516.0)
```
The first element is a dict which has the connection# as key and path list as value, for the first connection, the traffic will be routed from Node 1 to Node 10, the path will be 1->13->19->8->2->10, and the total cost for applying all the three connections will be 27401516 in total.\
The problem PCE aims to solve is this: given a network topology and a
set of connection requests between some nodes in the topology that
must satisfy some requirements (regarding bandwidth, latency, number
of hops, packet loss, etc.) how do we find the right path between the
given nodes?

#### LB_Solver
For Minimizing the total path utilization, do `from LoadBalancing.LB_Utilization_Solver import runLB_UT_Solver`:
```
runLB_UT_Solver()
## Using PCE

PCE's API is still evolving. With that caveat, and omitting some
details, the general usage is like this:

```python
from sdx.pce.load_balancing.te_solver import TESolver
from sdx.pce.topology.temanager import TEManager

temanager = TEManager(initial_topology, connection_request)
for topology in topologies:
temanager.add_topology(topology)

graph = temanager.generate_graph_te()
traffic_matrix = temanager.generate_connection_te()

solution = TESolver(graph, traffic_matrix).solve()
```
This will output a tuple of 2 elements: (Path List Dict, Total Utilization).\
Still using the previous connection.json, this time output will be:

Note that PCE requires two inputs: network topology and connection
requests. For testing, a random topology generator and a random
connection request generator is available.

In the intermediate steps, network topology is generated by [NetworkX]
and the traffic matrix computation is executed by Google [OR-Tools]
Solver.


### Network Topology

The Network Topology should be in the format of NetworkX graph. For
each link, three attributes need to be assigned: cost, bandwidth and
latency. The current unit of each attribute is abstract, but the unit
in the Network Topology should be consistent with the unit in
Connections.


### Connection Requests

Format of a connection request is in the form of [[Source node,
Destination node, Bandwidth required, Latency required],..]. Here is
an example of a random connection request:

```
({1: [[1, 14], [14, 10]], 2: [[2, 9]], 3: [[15, 10]]}, 0.08040263457045238)
[[1,10,8,20],[2,9,10,15],[15,10,6,22]]
```
The first element is same as in MC_Solver, but the second element will be the total utilization of all links which is around 0.08.

There are three queries in this connection request. The first one is
`[1,10,8,20]`, and it means this connection query is to route traffic
from Node 1 to Node 10, requring a bandwith of 8 and maximum latency
of 20.

## Working with PCE code

The network topology is generated by NetworkX and the traffic matrix
computation is executed by Google OrTools Solver.
## Working with PCE code

Working with PCE in a virtual environment is a good idea, with a
workflow like this:

```console
$ git clone https://github.com/atlanticwave-sdx/pce.git
$ cd pce
$ python3 -m venv venv
$ python3 -m venv venv --upgrade-deps
$ source venv/bin/activate
$ pip install .[test]
```

Please note that editable installs does not work currently, due to the
Please note that editable installs do not work currently, due to the
shared top-level `sdx` module in datamodel.

PCE can read topology data from Graphviz dot files, if the optional
Expand Down Expand Up @@ -115,9 +113,20 @@ with:
$ tox -e extras
```

Test data is stored in `test/data` as JSON files. You might want to
be in the top-level directory when running tests.
Test data is stored in [tests/data](./tests/data) as JSON files.


<!-- URLs -->

[aw-sdx]: https://www.atlanticwave-sdx.net/ (Atlanticwave-SDX)

[pce-ci-badge]: https://github.com/atlanticwave-sdx/pce/actions/workflows/test.yml/badge.svg
[pce-ci]: https://github.com/atlanticwave-sdx/pce/actions/workflows/test.yml

[pce-cov-badge]: https://coveralls.io/repos/github/atlanticwave-sdx/pce/badge.svg?branch=main (Coverage Status)
[pce-cov]: https://coveralls.io/github/atlanticwave-sdx/pce?branch=main

<!-- URLS -->
[NetworkX]: https://networkx.org/
[OR-Tools]: https://developers.google.com/optimization/

[tox]: https://tox.wiki/en/latest/index.html