Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generic Send Offload (GSO) Support #1024

Merged
merged 11 commits into from
Feb 22, 2021
Merged

Generic Send Offload (GSO) Support #1024

merged 11 commits into from
Feb 22, 2021

Commits on Feb 17, 2021

  1. Allow to query for more data than "mtu" in the pacing module

    With GSO, we need to ask whether we can send 2 additional datagrams.
    Matthias247 authored and Matthias Einwag committed Feb 17, 2021
    Configuration menu
    Copy the full SHA
    9167f61 View commit details
    Browse the repository at this point in the history
  2. Update CMSG_LEN

    Wit GSO activated, encoding cmsgs paniced since not enough
    space was available. We need a minimum of 88 bytes.
    Matthias247 authored and Matthias Einwag committed Feb 17, 2021
    Configuration menu
    Copy the full SHA
    ad6e127 View commit details
    Browse the repository at this point in the history
  3. Move the check for sendable data in a space

    `space_can_send` is now a separate function which returns whether there
    is sendable data in a certain space.
    Matthias Einwag committed Feb 17, 2021
    Configuration menu
    Copy the full SHA
    f0ad120 View commit details
    Browse the repository at this point in the history
  4. Update congestion_blocked function for multiple datagrams

    With GSO we need to be able to check whether multiple datagrams
    are sendable.
    Matthias Einwag committed Feb 17, 2021
    Configuration menu
    Copy the full SHA
    085fe9a View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2021

  1. Store tag_len and ack_eliciting in PacketBuilder

    This allows easy access the information later on.
    Matthias Einwag authored and Matthias247 committed Feb 22, 2021
    Configuration menu
    Copy the full SHA
    59ae00d View commit details
    Browse the repository at this point in the history
  2. Store the transmit buffer outside of PacketBuilder

    With GSO storing the buffer inside the builder does not work
    due to lifetime issues - we need to hold the buffer across various packets.
    This keeps the buffer outside of the builder and just references the
    start of a packet via an offset.
    This also removes the lifetime from `PacketBuilder`.
    Matthias Einwag authored and Matthias247 committed Feb 22, 2021
    Configuration menu
    Copy the full SHA
    09394bf View commit details
    Browse the repository at this point in the history
  3. Move packet finalization into a seperate function

    `finish_and_track_packet` will now take information from the builder
    when finalizing the packet.
    Matthias247 committed Feb 22, 2021
    Configuration menu
    Copy the full SHA
    292d025 View commit details
    Browse the repository at this point in the history
  4. Update stats for GSO

    This adds a new stat for the number of transmit calls
    and fixes the metrics for path challenges.
    Matthias Einwag authored and Matthias247 committed Feb 22, 2021
    Configuration menu
    Copy the full SHA
    dedd95d View commit details
    Browse the repository at this point in the history
  5. Set segment_size based on the amount of sent datagrams

    The option only needs to be sent for GSO (multiple datagrams
    in a buffer).
    Matthias Einwag authored and Matthias247 committed Feb 22, 2021
    Configuration menu
    Copy the full SHA
    ca4faa3 View commit details
    Browse the repository at this point in the history
  6. Change padding logic

    This change defers padding packets as a runtime decision
    while the packet is populated.
    
    For individual packets, we store in `SentFrames` whether a padding is
    required. If padding is required a `min_datagram_size` variable is
    increased, which will later on assure that only the last packet in a
    datagram gets padded.
    Matthias247 committed Feb 22, 2021
    Configuration menu
    Copy the full SHA
    463ed09 View commit details
    Browse the repository at this point in the history
  7. Generic Send Offload (GSO) Support

    This change implements an initial version of GSO support [1][2][3]
    for Linux, which improves the effiency of sending data.
    
    The approach taken in this change is to create a buffer which contains
    multiple datagrams in `Connection::poll_transmit`. This was picked
    over trying to merge packets in the endpoint task, since packets
    seem to need to be padded to a common segment size in order to
    make them sendable via GSO.
    
    In order to to this in an efficient fashion, the `poll_transmit`
    method was restructred. Instead of selecting spaces upfront, it
    will now loop through all possible packet spaces, check if there is
    pending data to send and create packets and datagrams out of this.
    
    The last packet which was written to a datagram buffer is not
    finalized until it is clear whether follow-up packets need to be
    written, since we need to know whether this packet should get padded
    to MTU length or not.
    
    This change doesn't enable GSO yet, since it will only produce a
    single datagram. I will create a separate change for this.
    
    Performance measurements:
    
    **No GSO:**
    
    ```
    Sent 1073741824 bytes on 1 streams in 4.31s (237.66 MiB/s)
    ```
    
    **With GSO (up to 8 packets):**
    
    ```
    Sent 1073741824 bytes on 1 streams in 3.02s (339.18 MiB/s)
    ```
    
    [1] http://vger.kernel.org/lpc_net2018_talks/willemdebruijn-lpc2018-udpgso-paper-DRAFT-1.pdf
    [2] http://vger.kernel.org/lpc_net2018_talks/willemdebruijn-lpc2018-udpgso-presentation-20181104.pdf
    [3] https://lwn.net/Articles/752956/
    Matthias247 committed Feb 22, 2021
    Configuration menu
    Copy the full SHA
    88aed6f View commit details
    Browse the repository at this point in the history