-
Notifications
You must be signed in to change notification settings - Fork 56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implemented NetworkConditioner #141
Conversation
Codecov Report
@@ Coverage Diff @@
## master #141 +/- ##
==========================================
+ Coverage 83.16% 83.74% +0.57%
==========================================
Files 26 28 +2
Lines 1740 1882 +142
==========================================
+ Hits 1447 1576 +129
- Misses 190 197 +7
- Partials 103 109 +6
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
215bc56
to
299835f
Compare
NetworkConditioner allows to replicate in a predictable way network condition, such as a 3G or LTE network. There is a list of pre-defined NetworkConditionerPreset, or the user can provided their own.
299835f
to
2dfbb94
Compare
Before I look closely into the code, are we really sure we'd want to introduce network impairment tool per instance of Net (NIC) when we already have the same at Router level? RouterConfig:
See: https://pkg.go.dev/github.com/pion/transport@v0.12.3/vnet Notes:
I am just wondering if this is worth the effort / maintenance cost for the increased complexity just for the simulation/testing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe you'd need to implement queues (outbound & inbound), just like Router does in order to overcome the unwanted blocking operations I pointed out. Please let me know if I did not correctly capture your intention. Also, please consider what I suggested earlier (if the existing solution really cannot help your problems at all, etc).
|
||
// Apply constant latency if required | ||
if sleepNeeded := time.Duration(atomic.LoadInt64((*int64)(&n.UpLink.Latency))) - bandwidthLimiterSleep; sleepNeeded > 0 { | ||
time.Sleep(sleepNeeded) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is blocking the calling thread. WriteTo() would block when the send-buffer is full (which does not happen much with UDP) and I believe this is not what you are trying to simulate with this network conditioner.
|
||
// Apply constant latency if required | ||
if sleepNeeded := time.Duration(atomic.LoadInt64((*int64)(&n.DownLink.Latency))) - bandwidthLimiterSleep; sleepNeeded > 0 { | ||
time.Sleep(sleepNeeded) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I'm not mistaken (it's been a while since I wrote this code.. :P), you are blocking the calling thread which is the processChunks
routine from a router. During this blockage, the route cannot process other chunks unrelated to this NIC. I believe this is not what you intended...
Thanks a lot for your comments @enobufs Totally right about the blocked With the current PR, things in a typical scenario looks like this: wan, err := vnet.NewRouter(&vnet.RouterConfig{
CIDR: "0.0.0.0/0",
LoggerFactory: logging.NewDefaultLoggerFactory(),
})
assert.NoError(t, err)
vnetA := vnet.NewNet(&vnet.NetConfig{
NetworkConditioner: vnet.NewNetworkConditioner(vnet.NetworkConditionerPresetNone),
})
assert.NoError(t, wan.AddNet(vnetA))
assert.NoError(t, wan.Start())
defer func() { _ = wan.Stop() }()
// Later, the user can dynamically change the NetworkConditioner, to simulate network disconnection for example
vnetA.SetNetworkConditioner(vnet.NewNetworkConditioner(vnet.NetworkConditionerPresetFullLoss)) By implementing on top of the Router, we'd get something like this: wan, err := vnet.NewRouter(&vnet.RouterConfig{
CIDR: "0.0.0.0/0",
LoggerFactory: logging.NewDefaultLoggerFactory(),
})
assert.NoError(t, err)
subnetA, err := vnet.NewRouter(&vnet.RouterConfig{
CIDR: "1.0.0.0/0",
LoggerFactory: logging.NewDefaultLoggerFactory(),
NetworkConditioner: vnet.NewNetworkConditioner(vnet.NetworkConditionerPresetNone),
})
assert.NoError(t, err)
assert.NoError(t, wan.AddRouter(vnetA))
vnetA := vnet.NewNet(&vnet.NetConfig{})
assert.NoError(t, subnetA.AddNet(vnetA))
assert.NoError(t, wan.Start())
defer func() { _ = wan.Stop() }() I think we could introduce a helper function which would wrap the Router+Net creation, but then we'd have to handle CIDR (or let the user provide it, but I feel that's unnecessary complexity), since a subrouter can't have the same CIDR as its parent (or am I wrong on this?) A bit out of the scope of this PR, but I've been wanting to simulate network changes, to test behavior in mobile-like situations (wifi<->lte switches). With this PR, we can change the behavior of the network, but that's still an inaccurate simulation since the address of the client doesn't change. I think it's something that could be handled at the router's level, and that would make sense to handle network condition on the same level in this situation. If I'm not mistaken, the current implementation of The way I see it, there are 3 possibilities:
( I'd also love your thoughts on this @Sean-Der ) |
My only concern is if per NIC delay is that important for this level of testing. Primary goal of vnet was to emulate NAT so that we could test with various types of NAT. Network impairment was introduced later on, on the router because it already had a queue, so I agree, it was a shortcut and not truly representing real world's complex scenario, but that was good enough to find most of bugs with race conditions. I used to use WAN emulators such as this. This helped me to evaluate overall performance and to be reassured before shipping a product to the real world, but most of the bugs were found just by introducing simple network impairment between point A and point B, with a tool similar to comcast (using iptables/netem). The vnet does this simple stuff and even more quickly because everything is on the test code to achieve the same. Having said that, I understand what you are try to do (introducing more controls and helping user side complexity with presets also), and I am not too strong about the above thoughts. Maybe I am too minimalistic.. |
I still think there's value in enabling users to simulate network conditions via unit tests:
I also agree we should not overcomplicate stuff. I feel the parameters I added to I get your point though, and the solutions I proposed might not be best ways to solve this. I dislike having network impairment controls on both the |
After wrapping my head against all of this again, I think the best solution is to keep the impairment logic at the
The only downside I see with this approach is we introduce breaking changes. But on the other hand, I don't think a lot of code depends on I've been debating between this, and the If someone has a better solution, I'm all for it :) |
If we were to emulate network more resembles to the real network, then we should keep the network impairment on the router and improve them for better. "Network conditioner" settings are from the aspect of one endpoint. It is a special case and does not cover scenarios like having multiple endpoints sharing the same subscriber (i.e. Comcast cable, AT&T fiber, etc) line. The subscriber line is in between your residential router and the internet (other "WAN" routers), while the network speed within the LAN is very fast, etc. Mac's Network Link Conditioner is an alternative solution (things are happening only on your PC) when you don't have a way to introduce network impairment elsewhere. I use Comcast and most of the bottle neck comes from the subscriber line, particularly the upload speed is very limited with Comcast cable. I'm sure lots of packet loss is happening behind my cable modem (a residential router). What's missing in the Net class (~= NIC) is the socket buffer (currently not emulated). Write()/WriteTo() call would block if the send (socket) buffer becomes full (until the deadline expires). But if that is UDP, this blockage rarely happen. Also, if it is wifi, there would be relatively large packet loss due to radio signal interference (microwave in the kitchen, etc). So, if I were to improvement vnet further, I'd look into:
|
Hey @Antonito, are you still interested in getting this merged? Thanks :) |
Hey @stv0g! I'm still interested in getting this merged, however I don't have the bandwidth to work on this. |
Thanks @Antonito, as a maintainer, I need somebody who feels responsible for getting this PR merged and bring it forward. As this seems to be the case, I would like to close the PR until we find somebody who wants to work on it to get it merged. Please feel free to reopen the PR if you have the bandwidth :) |
This PR adds a
NetworkConditioner
onvnet.Net
– which behaves in addition to the Router's settings.A
NetworkConditioner
can be provided atvnet.Net
creation time, or later.The change in itself is not atomic, rather each value of the struct is updated atomically. It shouldn't cause any issue though (maybe invalid behavior on 1 or 2 packets at most) and permits to avoid any per-packet locking mechanism – which would hurt performances.
A user can use one of the preset (the values are taken from Apple's NetworkLinkConditioner), or provide their own custom one.
This is useful in situation such as when you need to unit test behavior when a specific user drops their connection.
I'm not 100% sure the
handleDownLink
andhandleUpLink
calls are at the best place nor return the correct values?