Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch p2p messages in one round instead of flushing immediately #127

Open
wants to merge 2 commits into
base: m
Choose a base branch
from
+48 −16
Open

Batch p2p messages in one round instead of flushing immediately #127

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7cd49d4
Batch p2p messages in one round instead of flushing immediately
maurges Dec 20, 2024
eeba7c2
Fix nostd build with updated deps
maurges Dec 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ rand_core = { version = "0.6", default-features = false }
rand_hash = { version = "0.1" }
rand_dev = "0.1"

futures = "0.3"
futures = { version = "0.3", default-features = false }

anyhow = "1"
thiserror = "1"
Expand Down
2 changes: 2 additions & 0 deletions cggmp21-keygen/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ serde = { workspace = true, features = ["derive"] }
serde_with = { workspace = true }
hex = { workspace = true, features = ["serde"] }

futures = { workspace = true }

displaydoc = { workspace = true }
thiserror = { workspace = true, optional = true }

Expand Down
13 changes: 7 additions & 6 deletions cggmp21-keygen/src/threshold.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -277,15 +277,16 @@ where
.await
.map_err(IoError::send_message)?;

for j in utils::iter_peers(i, n) {
let messages = utils::iter_peers(i, n).map(|j| {
let message = MsgRound2Uni {
sigma: sigmas[usize::from(j)],
};
outgoings
.send(Outgoing::p2p(j, Msg::Round2Uni(message)))
.await
.map_err(IoError::send_message)?;
}
Outgoing::p2p(j, Msg::Round2Uni(message))
});
outgoings
.send_all(&mut futures::stream::iter(messages.map(Ok)))
.await
.map_err(IoError::send_message)?;
Comment on lines +286 to +289
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it why you added futures dep, to do futures::stream::iter? You can use futures-util crate instead, i.e. futures_util::stream::iter, futures crate just re-exports this function from futures-util

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, you could remove new (direct) dependency by using chain of outgoings.feed(msg).await inside of loop, followed by outgoings.flush().await.

I would prefer this option, though using futures_util::stream::iter does have an advantage that we can't forget to call .flush().await.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that in other places of the protocol, you are using feed/flush chain, so you can do that here as well and remove the dependency

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use send_all where it doesn't change existing code too much, and feed+flush where send_all would. I think send_all is better, but I have a bias to not change existing lines too much

tracer.msg_sent();

// Round 3
Expand Down
15 changes: 6 additions & 9 deletions cggmp21/src/signing.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -715,13 +715,12 @@ where
)
.map_err(|e| Bug::PiEnc(BugSource::psi0, e))?;

tracer.send_msg();
outgoings
.send(Outgoing::p2p(j, Msg::Round1b(MsgRound1b { psi0 })))
.feed(Outgoing::p2p(j, Msg::Round1b(MsgRound1b { psi0 })))
.await
.map_err(IoError::send_message)?;
tracer.msg_sent();
}
outgoings.flush().await.map_err(IoError::send_message)?;
Comment on lines -718 to +723
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should keep tracer.send_msg()/tracer.msg_sent() so the time we spend on sending messages gets counted

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can rename send_msg/msg_sent function names so the naming doesn't imply that a message has to be sent between these calls

Copy link
Contributor

@survived survived Jan 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, the only implementation of the trace trait we have does not assume that a single message actually gets sent between these calls. The time between these calls just get accounted as spent on sending messages, no matter how many were sent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, the only implementation of the trace trait we have does not assume that a single message actually gets sent between these calls

Ugh, ok, that's a weird metric then. But I'll put the traces back in this case

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Measuring how much time we spend on sending messages in a weird metric to you?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good metric. But we're not measuring it here, what we'll be measuring is time spent adding messages to a queue. I don't find it very useful, especially with a misleading name. Maybe it makes sense to add some trace to flushing though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will be measuring time spent on adding to queue + time spent on flushing items from the queue. Two of them makes the time we spend on sending messages.


// Round 2
tracer.round_begins();
Expand Down Expand Up @@ -950,9 +949,8 @@ where
.map_err(|e| Bug::PiLog(BugSource::psi_prime, e))?;
runtime.yield_now().await;

tracer.send_msg();
outgoings
.send(Outgoing::p2p(
.feed(Outgoing::p2p(
j,
Msg::Round2(MsgRound2 {
Gamma: Gamma_i,
Expand All @@ -967,8 +965,8 @@ where
))
.await
.map_err(IoError::send_message)?;
tracer.msg_sent();
}
outgoings.flush().await.map_err(IoError::send_message)?;

// Round 3
tracer.round_begins();
Expand Down Expand Up @@ -1124,9 +1122,8 @@ where
)
.map_err(|e| Bug::PiLog(BugSource::psi_prime_prime, e))?;

tracer.send_msg();
outgoings
.send(Outgoing::p2p(
.feed(Outgoing::p2p(
j,
Msg::Round3(MsgRound3 {
delta: delta_i,
Expand All @@ -1136,8 +1133,8 @@ where
))
.await
.map_err(IoError::send_message)?;
tracer.msg_sent();
}
outgoings.flush().await.map_err(IoError::send_message)?;

// Output
tracer.named_round_begins("Presig output");
Expand Down
31 changes: 31 additions & 0 deletions wasm/no_std/Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading