Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/network monitor #369

Merged
merged 64 commits into from
Oct 9, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
f863f5a
WIP commit
futurechimp Sep 9, 2020
346b2ee
Merge branch 'develop' into feature/network-monitor
futurechimp Sep 11, 2020
50b6e09
Further into errors (WIP)
futurechimp Sep 15, 2020
2a58625
WIP commit, more stuff compiling, but now we switch to sneaky Jedrzej…
futurechimp Sep 16, 2020
642f350
Websocket connections starting to work
futurechimp Sep 16, 2020
6019c16
WIP
futurechimp Sep 16, 2020
4e4f3cb
Constructing the socket stream in one go
futurechimp Sep 16, 2020
09f56b0
Nicer connections. Getting topology now works.
futurechimp Sep 16, 2020
0b7800e
Fixing startup message
futurechimp Sep 16, 2020
920c153
Injecting directory uri
futurechimp Sep 16, 2020
3c4d0fd
Injecting good mixndodes
futurechimp Sep 16, 2020
5992714
Deleting old healthcheck
futurechimp Sep 18, 2020
bf507a6
Starting to wrap gateway client
futurechimp Sep 18, 2020
4cd50f0
Merge branch 'develop' into feature/network-monitor
futurechimp Sep 18, 2020
ed999b6
Splitting out good topology contstruction
futurechimp Sep 18, 2020
64bb1f9
Starting to breathe finally
futurechimp Sep 18, 2020
03ed989
Fixed layer assignment error
futurechimp Sep 18, 2020
a550cf9
Starting gateway client in a better spot
futurechimp Sep 18, 2020
7d2ee84
Cleanup
futurechimp Sep 18, 2020
c899132
Renamed construct() to new()
futurechimp Sep 21, 2020
f2ca087
Injected channels, removed websockets
futurechimp Sep 21, 2020
342dfd5
Added a MixnetListener to deal with returned packet traffic
futurechimp Sep 22, 2020
e5e1c7d
Simplification
futurechimp Sep 22, 2020
90064f7
Renaming and commenting a few things
futurechimp Sep 22, 2020
b5aeb9b
Renamed temp variables
futurechimp Sep 22, 2020
8968e16
Fixed variable names
futurechimp Sep 22, 2020
1dcdf0c
Made errors a bit more explicit on message reconstruction
futurechimp Sep 22, 2020
28e8f76
Merge branch 'develop' into feature/network-monitor
futurechimp Sep 30, 2020
ae37579
Added the mixmining route / object to the directory server client
futurechimp Sep 30, 2020
5f2ab83
Recipient is always "me", a bit easier to understand what's going on
futurechimp Sep 30, 2020
28f0e17
Subbing in test nodes works
futurechimp Sep 30, 2020
d88aaed
Using QA directory
futurechimp Oct 1, 2020
41fa7d2
Noting which directory server we're using at monitor start
futurechimp Oct 1, 2020
b0f7e30
Adding Debug to MixStatus so we can print it more easily
futurechimp Oct 1, 2020
2187264
Prettification of startup messages
futurechimp Oct 1, 2020
e565a06
Notifying directory of successful packet receipt
futurechimp Oct 1, 2020
cbc33a4
All nodes now being tested
futurechimp Oct 1, 2020
a54270a
Now able to do timer-based test runs and detect the last node tested
futurechimp Oct 1, 2020
6fc4e86
Renamed mixnet_listener to notifier
futurechimp Oct 1, 2020
5414f89
Moved message chunking into its own home
futurechimp Oct 1, 2020
1e97054
Starting to pull out packet sends
futurechimp Oct 1, 2020
4885bbb
Basic functionality working!
futurechimp Oct 2, 2020
18c26e9
Enabled timer runs forever
futurechimp Oct 2, 2020
54da8f2
License notices
jstuczyn Oct 2, 2020
f0b4a56
Depdendency cleanup
jstuczyn Oct 2, 2020
fcf322a
Import cleanup
jstuczyn Oct 2, 2020
856259d
Moved to implicit tokio runtime
jstuczyn Oct 2, 2020
5ee798f
Removed some unnecessary data clones
jstuczyn Oct 2, 2020
35400fe
Fixed monitor doing two runs at startup rather than one
jstuczyn Oct 2, 2020
baa9238
dalek version updates
jstuczyn Oct 5, 2020
eab4cb2
This should have been split but got carried away
jstuczyn Oct 5, 2020
83a3d5f
Initial ipv6 support
jstuczyn Oct 6, 2020
381f911
Don't test outdated nodes
jstuczyn Oct 6, 2020
70f6a89
Added network monitor to default workspace members
jstuczyn Oct 6, 2020
7cedea0
Actually sending reports
jstuczyn Oct 6, 2020
8a4c04f
Batch-sending mix status
jstuczyn Oct 8, 2020
cf0d44a
Cargo fmt
jstuczyn Oct 9, 2020
41c5866
Minor cleanup
jstuczyn Oct 9, 2020
ac49736
Merge branch 'develop' into feature/network-monitor
jstuczyn Oct 9, 2020
f6680a7
Malformed Cargo.lock
jstuczyn Oct 9, 2020
9ff8181
Post-merge fixes
jstuczyn Oct 9, 2020
66d1503
Defined constant in main.rs for enabling detailed reporting
jstuczyn Oct 9, 2020
3e51dd3
Updated package versions of the system
jstuczyn Oct 9, 2020
7c54d61
Checking version compatibility on startup
jstuczyn Oct 9, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 28 additions & 10 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ members = [
"gateway/gateway-requests",
"service-providers/sphinx-socks",
"mixnode",
"network-monitor",
"validator",
]

Expand All @@ -51,4 +52,5 @@ default-members = [
"gateway",
"mixnode",
"validator",
"network-monitor",
]
2 changes: 1 addition & 1 deletion clients/native/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[package]
build = "build.rs"
name = "nym-client"
version = "0.8.1"
version = "0.9.0-dev"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
edition = "2018"

Expand Down
2 changes: 1 addition & 1 deletion clients/socks5/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[package]
build = "build.rs"
name = "nym-socks5-client"
version = "0.8.1"
version = "0.9.0-dev"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
edition = "2018"

Expand Down
2 changes: 1 addition & 1 deletion clients/webassembly/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
build = "build.rs"
name = "nym-client-wasm"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jedrzej Stuczynski <andrew@nymtech.net>"]
version = "0.8.1"
version = "0.9.0-dev"
edition = "2018"
keywords = ["nym", "sphinx", "wasm", "webassembly", "privacy", "client"]
license = "Apache-2.0"
Expand Down
1 change: 1 addition & 0 deletions common/client-libs/directory-client/models/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,5 @@
// limitations under the License.

pub mod metrics;
pub mod mixmining;
pub mod presence;
19 changes: 19 additions & 0 deletions common/client-libs/directory-client/models/src/mixmining.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
use serde::{Deserialize, Serialize};

#[derive(Debug, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
/// A notification sent to the validators to let them know whether a given mix is
/// currently up or down (based on whether it's mixing packets)
pub struct MixStatus {
pub pub_key: String,
pub ip_version: String,
pub up: bool,
}

#[derive(Debug, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
/// A notification sent to the validators to let them know whether a given set of mixes is
/// currently up or down (based on whether it's mixing packets)
pub struct BatchMixStatus {
pub status: Vec<MixStatus>,
}
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ impl From<io::Error> for ConversionError {
}
}

#[derive(Clone, Debug, Deserialize, Serialize)]
#[derive(Clone, Debug, Deserialize, PartialEq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct MixNodePresence {
pub location: String,
Expand Down
22 changes: 21 additions & 1 deletion common/client-libs/directory-client/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@
use crate::requests::health_check_get::Request as HealthCheckRequest;
use crate::requests::metrics_mixes_get::Request as MetricsMixRequest;
use crate::requests::metrics_mixes_post::Request as MetricsMixPost;
use crate::requests::mix_mining_batch_status_post::Request as MixMiningBatchStatusPost;
use crate::requests::mix_mining_status_post::Request as MixMiningStatusPost;
use crate::requests::presence_coconodes_post::Request as PresenceCocoNodesPost;
use crate::requests::presence_gateways_post::Request as PresenceGatewayPost;
use crate::requests::presence_mixnodes_post::Request as PresenceMixNodesPost;
Expand All @@ -25,10 +27,12 @@ use directory_client_models::presence::{
coconodes::CocoPresence, gateways::GatewayPresence, mixnodes::MixNodePresence,
providers::MixProviderPresence,
};
use mixmining::MixStatus;
use requests::{health_check_get::HealthCheckResponse, DirectoryGetRequest, DirectoryPostRequest};

use directory_client_models::mixmining::BatchMixStatus;
pub use directory_client_models::{
metrics,
metrics, mixmining,
presence::{self, Topology},
};

Expand Down Expand Up @@ -123,6 +127,22 @@ impl Client {
self.post(req).await
}

pub async fn post_mixmining_status(
&self,
status: MixStatus,
) -> reqwest::Result<reqwest::Response> {
let req = MixMiningStatusPost::new(&self.base_url, status);
self.post(req).await
}

pub async fn post_batch_mixmining_status(
&self,
batch_status: BatchMixStatus,
) -> reqwest::Result<reqwest::Response> {
let req = MixMiningBatchStatusPost::new(&self.base_url, batch_status);
self.post(req).await
}

// this should be soft-deprecated as the whole concept of provider will
// be removed in the next topology rework
pub async fn post_provider_presence(
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
use super::{DirectoryPostRequest, DirectoryRequest};
use crate::mixmining::BatchMixStatus;

const PATH: &str = "/api/mixmining/batch";

pub struct Request {
base_url: String,
path: String,
payload: BatchMixStatus,
}

impl DirectoryRequest for Request {
fn url(&self) -> String {
format!("{}{}", self.base_url, self.path)
}
}

impl DirectoryPostRequest for Request {
type Payload = BatchMixStatus;
fn new(base_url: &str, payload: Self::Payload) -> Self {
Request {
base_url: base_url.to_string(),
path: PATH.to_string(),
payload,
}
}

fn json_payload(&self) -> &BatchMixStatus {
&self.payload
}
}

#[cfg(test)]
mod batch_mix_status_post_request {
use super::*;
use crate::client_test_fixture;
use mockito::mock;

#[cfg(test)]
mod on_a_400_status {
use super::*;

#[tokio::test]
async fn it_returns_an_error() {
let _m = mock("POST", PATH).with_status(400).create();
let client = client_test_fixture(&mockito::server_url());
let result = client
.post_batch_mixmining_status(fixtures::new_status())
.await;
assert_eq!(400, result.unwrap().status());
_m.assert();
}
}

#[cfg(test)]
mod on_a_201 {
use super::*;

#[tokio::test]
async fn it_returns_a_response_with_201() {
let json = r#"{
"ok": true
}"#;
let _m = mock("POST", "/api/mixmining/batch")
.with_status(201)
.with_body(json)
.create();
let client = client_test_fixture(&mockito::server_url());
let result = client
.post_batch_mixmining_status(fixtures::new_status())
.await;
assert!(result.is_ok());
_m.assert();
}
}

#[cfg(test)]
mod fixtures {
use crate::mixmining::{BatchMixStatus, MixStatus};

pub fn new_status() -> BatchMixStatus {
BatchMixStatus {
status: vec![MixStatus {
pub_key: "abc".to_string(),
ip_version: "4".to_string(),
up: true,
}],
}
}
}
}
Loading