[2.x.x] Use blocking IO in P2P to reduce CPU load #2855
Conversation
| time::Duration::from_secs(10), | ||
| true, | ||
| )?; | ||
| self.stream.read_exact(&mut buf[..])?; |
There was a problem hiding this comment.
Is this now controlled by IO_TIMEOUT (i.e. 1 second) ? if the buf can't be fully filled in 1 second, read_exact will return a TIMEOUT error. So, now the minimum speed is 8kByte per second, instead of old 800 Byte/s.
Is my understanding correct?
There was a problem hiding this comment.
Not exactly, read (syscall) timeout occurs when we we can't read a single byte in 1 second. In our read_exact we didn't have a timeout for a single read, but for the entire operation (filling a buffer). In Rust read_exact we don't have any high-level timeouts but can control the timeout for a single read.
So it's quite opposite, the previous impl was more demanding for connection speed, a new one will work for any connection faster than 1 byte/sec. A drawback that a peer's thread can be blocked by read_exact for a while on a slow connection, that what we can experience during shutdown.
There was a problem hiding this comment.
read (syscall) timeout occurs when we we can't read a single byte in 1 second.
In the case of a peer's sending thread doesn't get the CPU resources for 1 second, does this read_exact will return a timeout error with part of buff filled?
There was a problem hiding this comment.
@hashmap remind above question in case you missed it :-)
There was a problem hiding this comment.
Ah, thanks! Well, this is timeout for a single read operation which is basically syscall, I'm not an expert in preemptive scheduling but I think it's safe to assume that this situation is very unlikely.
| @@ -160,14 +152,14 @@ pub fn read_item<T: Readable>(stream: &mut dyn Read) -> Result<(T, u64), Error> | |||
| /// until we have a result (or timeout). | |||
| pub fn read_body<T: Readable>(h: &MsgHeader, stream: &mut dyn Read) -> Result<T, Error> { | |||
| let mut body = vec![0u8; h.msg_len as usize]; | |||
| read_exact(stream, &mut body, time::Duration::from_secs(20), true)?; | |||
| stream.read_exact(&mut body)?; | |||
There was a problem hiding this comment.
Similar question here, what's the biggest possible length for a body? perhaps a Block is the biggest? then it's maximum possible value 1.5MB for example.
The timeout is 1 second and minimum speed is 1.5MB/s ?
There was a problem hiding this comment.
See my previous comment, the minimum speed is still 1 byte/sec
|
I like the performance improvement 👍 |
There was a problem hiding this comment.
Yeah. Let's do this.
I was not convinced (entirely) before as I think we should be moving a more async approach and not a less async approach.
But that said - this does simplify things quite a bit and cleans a bunch of custom code that worked around our "partially async but not really" approach.
👍
I'm still doing some testing locally on this branch just to try and break it.
antiochp
changed the title
|
I updated this PR to merge onto |
|
Synced from the scratch, no issues, on my machine it consumes 10% CPU with ~30 peers vs 25% on master |
|
CPU consumption without TUI master (22.9%): |
Instead we create 2 threads - one for reading, one for writing, both block for 1 second then check stop flag, if it is not set the process continue.
The current master consumes 20-25% CPU on my machine with ~ 20 peers, this version 9-12% with the same number of peers (with TUI enabled).
Also we can use all higher level reader/writer methods like read_string, read_exact etc