Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ndt7: AppInfo should be mandatory for counterflow messages during upload tests #358

Open
robertodauria opened this issue May 17, 2022 · 3 comments

Comments

@robertodauria
Copy link
Contributor

robertodauria commented May 17, 2022

This is a proposal to change the ndt7 protocol specification. It has been discussed with @bassosimone (original author of the ndt7 spec) at m-lab/ndt7-client-go#75.

Currently, both TCPInfo and AppInfo are optional fields for a Measurement object. When the client is uploading, the sender-side rate is overestimated due to buffers and data still "in transit" at the end of the measurement period. To get an accurate rate, the client must rely on Measurement messages periodically sent by the server. However, an ndt7 server today can be compliant without implementing TCPInfo nor AppInfo, which would make the clients blind to how much data has been actually received by the server and unable to show a meaningful rate to the user.

ndt-server currently sends TCPInfo only, which provides network-level byte count, and that's what our clients use. However, there is a significant asymmetry in what clients show: when downloading, the rate is computed at the application level -- (size of WS payloads / elapsed time) while when uploading the rate is computed at the network level -- (bytes received by the server / elapsed time). Clients cannot generally access TCPInfo metrics (most of our measurements come from Javascript-based implementations) so using a network-level rate for the download is not an option, but we can be consistent by showing application-level measurements in both tests.

Making AppInfo mandatory in the spec for counterflow messages (and adding it to ndt-server) would solve both these problems, guaranteeing that the clients get at least one source of accurate measurements and that we're not mixing apples and oranges by providing rates computed at different levels of the OSI model.

@mattmathis
Copy link

It should probably specify a frequency (range) as well.

@stephen-soltesz
Copy link
Contributor

My understanding is that the use case we are seeking to address is archival storage of goodput measurements.

Please be precise about the source of the following information:

  • Client visible goodput, upload and download
  • Server visible goodput, upload and download
  • Client visible network throughput, upload and download
  • Server visible network throughput, upload and download

This summary may be helpful for updates to the ndt7 spec or to inform the MSAK design.

I know not all are available. So, which will we regard as authoritative, advisory, part of presentation only or part of archival data? Can we meet the use case? If so, what will we change? If not, what do we need to do instead?

@robertodauria
Copy link
Contributor Author

My understanding is that the use case we are seeking to address is archival storage of goodput measurements.

The original intention was to provide the client with a reliable source of application-level goodput on upload tests, which we currently lack.

* Client visible goodput, upload and download
  • Upload: AppInfo.NumBytes / AppInfo.ElapsedTime from server measurements (currently missing)
  • Download: NumBytes / ElapsedTime from client's own measurements
* Server visible goodput, upload and download
  • Upload: NumBytes / ElapsedTime from server's own measurements
  • Download: AppInfo.NumBytes / AppInfo.ElapsedTime from client measurements (currently missing, advisory, can't trust the client)
* Client visible network throughput, upload and download
  • Upload: TCPInfo.BytesReceived / TCPInfo.ElapsedTime from server measurements
  • Download: TCPInfo.BytesAcked / TCPInfo.ElapsedTime from server measurements
* Server visible network throughput, upload and download
  • Upload: TCPInfo.BytesReceived / TCPInfo.ElapsedTime from server's own measurements
  • Download: TCPInfo.BytesAcked / TCPInfo.ElapsedTime from server's own measurements

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants