Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Buffer Occupancy #55

Merged
merged 20 commits into from
Dec 8, 2018
50 changes: 44 additions & 6 deletions telemetry/specs/INT.mdk
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Title : In-band Network Telemetry (INT) Dataplane Specification
Title Note : Working draft. Note: consider using tagged versions for implementation.
Title Footer: 2018-05-08
Author : The P4.org Applications Working Group. Contributions from
Affiliation : *Alibaba, Arista, Barefoot Networks, Dell, Intel, Marvell, Netronome, VMware*
Affiliation : *Alibaba, Arista, Barefoot Networks, Cisco Systems, Dell, Intel, Marvell, Netronome, VMware*
Heading depth: 5
Pdf Latex: xelatex
Document Class: [11pt]article
Expand Down Expand Up @@ -316,7 +316,16 @@ simply leaves those decisions to device vendors.

* Queue occupancy
- The build-up of traffic in the queue (in bytes, cells, or packets) that the
INT packet observes in the device while being forwarded.
INT packet observes in the device while being forwarded. The format of this 4-octet
metadata field is switch implementation specific and the YANG model shall describe
the format and units of this metadata field in the metadata stack.

* Buffer occupancy
- The build-up of traffic in the buffer (in bytes, cells, or packets) that the
INT packet observes in the device while being forwarded. Use case is when the buffer
is shared between multiple queue. The format of this 4-octet metadata field is switch
implementation specific and the YANG model shall describe the format and units of this metadata
field in the metadata stack.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps remove the word "shared". Reporting both queue occupancy and buffer occupancy makes sense only if the buffer is shared, but technically the buffer could be a per-queue buffer, just that in this case, both values reported will be the same. INT source does not know whether downstream switches have shared buffers or not. It can set both bits. Switches that have per-queue buffers (unlikely, switches typically use shared buffers) will just report the same value for both instructions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be word this differently, to say this instruction is to get buffer occupancy. Use-case is when buffer is shared.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the reference to shared.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor editorial rewording:
The use case is when the buffer is shared between multiple queues.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the minor editorial change suggested.

# INT Headers

Expand Down Expand Up @@ -752,12 +761,12 @@ hop-by-hop INT header must fit in a single Geneve option.
In this section, we define the format for INT hop-by-hop metadata headers,
and the metadata itself.

INT Metadata Header and Metadata Stack:
INT Metadata Header and Metadata Stack (Version = 1):
`
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver |Rep|C|E|M| Reserved | Hop ML |RemainingHopCnt|
|Ver = 1|Rep|C|E|M| Reserved | Hop ML |RemainingHopCnt|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Instruction Bitmap | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Expand Down Expand Up @@ -821,7 +830,7 @@ The original packet must have C bit set to 0.
switch(es) set the M bit based on knowledge of the network topology
and "Switch ID, Ingress port ID, Egress port ID" tuples in the INT
metadata stack.
- R: Reserved bits.
- R (10b): Reserved bits.
- Hop ML (5b): Per-hop Metadata Length, the length of metadata in 4-Byte words
to be inserted at each INT hop.
- While the largest value of Per-hop Metadata Length is 31, an INT-capable
Expand Down Expand Up @@ -854,6 +863,7 @@ each bit corresponds to a specific standard metadata as specified in Section 3.
- bit5: Egress timestamp
- bit6: Level 2 Ingress Port ID + Egress Port ID (4 bytes each)
- bit7: Egress port Tx utilization
- bit8: Buffer ID (8 bits) + Buffer Occupancy (24 bits)
- bit15: Checksum Complement
- The remaining bits are reserved.
Each instruction requests 4 bytes of metadata to be inserted at each hop,
Expand Down Expand Up @@ -1219,6 +1229,10 @@ header int_egress_port_tx_util_t {
bit<32> egress_port_tx_util;
}

header int_b_occupancy_t {
bit<32> q_occupancy;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make it consistent with the (example) layout provided in line 866, which has 8b buffer id.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let us discuss the semantics of Queue and Buffer occupancy. I will make it consistent after the discussion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added Buffer ID (8 bits) and Buffer Occupancy (24 bits)

}

/* standard ethernet/ip/tcp headers */
header ethernet_t {
bit<48> dstAddr;
Expand Down Expand Up @@ -1279,6 +1293,7 @@ struct headers {
int_egress_tstamp_t int_egress_tstamp;
int_level2_port_ids_t int_level2_port_ids;
int_egress_port_tx_util_t int_egress_port_tx_util;
int_b_occupancy_t int_b_occupancy;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: fix indent

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}

struct empty_metadata_t {
Expand Down Expand Up @@ -1610,6 +1625,11 @@ control EgressDeparserImpl(packet_out packet,
ck.add({hdr.int_egress_port_tx_util.egress_port_tx_util});
}

if (hdr.int_b_occupancy.isValid()) {
ck.add({
hdr.int_b_occupancy.b_occupancy
});
}
if (hdr.tcp.isValid()) {
ck.add({
hdr.tcp.srcPort,
Expand Down Expand Up @@ -1656,6 +1676,7 @@ control EgressDeparserImpl(packet_out packet,
packet.emit(hdr.int_egress_tstamp);
packet.emit(hdr.int_level2_port_ids);
packet.emit(hdr.int_egress_port_tx_util);
packet.emit(hdr.int_b_occupancy);
}
}

Expand Down Expand Up @@ -2021,6 +2042,23 @@ specific.
* Queue drop count
- Total # of packets dropped from the queue


The buffer occupancy INT packet observes in the device while being forwarded. Use case is when buffer is
shared between multiple queue.

* Buffer id
- The id of the buffer the device used to serve the INT packet.
* Instantaneous buffer occupancy
- The instantaneous value (in bytes, or cells) of the buffer occupancy the INT
packet has observed in the device while being forwarded. The units used need
not be consistent across an INT domain, but care must be taken to ensure that
there is a known, consistent mapping of {device, buffer} values to their
respective unit {bytes, cells}.
* Average buffer occupancy
- The average value (in bytes or cells) of the buffer occupancy
the INT packet was observed. The calculation mechanism of this value is device
specific.

## Miscellaneous

* Checksum Complement
Expand Down