LS1028ardb set PCP field #80

GJim01 · 2020-12-02T10:40:26Z

Hello,
I am working on a TSN demo set up using two LS1028ardb running on OpenIL. I have camera images that have to be transmitted from one of the board to the other. This is the priority traffic that i need to send. It is my understanding that the PCP – Priority Code Point in 802.1 Q tag, is to be set to assign the priority to the image data. Could you please let me know if there is a way to set the PCP flag? Would I need an application that uses ‘raw socket’ connection to set the PCP flag or is there any tsntool configuration that I can use to set the priority and gate mapping on the eno0 port for the image data?

The idea is to set PCP = say 5 for the image buffer, PCP = 3 for PTP frames etc.

Thanks in advance for your help.

vladimiroltean · 2020-12-02T10:53:25Z

You could create an 8021q interface on top of eno0, and create a one-to-one mapping between skb->priority and VLAN PCP as follows:

ip link add link eno0 name eno0.100 type vlan id 100 egress-qos-map 0:0 1:1 2:2 3:3 4:4 5:5 6:6 7:7

You need to open your sockets in ptp4l and in your custom application on top of this VLAN interface (example: ptp4l -i eno0.100).

Then, how you set the packet priority is up to you. Generally there are two options.

You can use the SO_PRIORITY API from your application, and enqueue to a certain traffic class directly. At least ptp4l understands the SO_PRIORITY API, you just need to use the socket_priority option in the configuration file (see man ptp4l.8). I don't know about your custom application.
You can use custom classification rules in the queuing discipline for the interface, via tc filters. This would make your applications unaware of priority. Example:

tc qdisc add dev eno0.100 clsact
tc filter add dev eno0.100 egress protocol 0x88f7 flower action skbedit priority 3
tc filter add dev eno0.100 egress protocol ipv4 flower ${insert classification rule for image traffic} action skbedit priority 5

I need to mention that QoS classification via the VLAN PCP is not the only option. At least the LS1028A has the option to do QoS classification based on other frame headers as well.

vladimiroltean · 2020-12-02T11:06:33Z

gate mapping

This suggests that you are interested in IEEE 802.1Qbv as well. Very well.
IEEE 802.1Qbv can be used in conjunction with what I said above.
You would need the taprio queuing discipline installed on the physical port.

	tc qdisc add dev eno0 root taprio num_tc 8 \
		map 0 1 2 3 4 5 6 7 \
		queues 1@0 1@1 1@2 1@3 1@4 1@5 1@6 1@7 \
		base-time 0 \
		sched-entry S 08  10000 \
		sched-entry S 20 10000 \
		flags 2

Leo1726 · 2021-04-28T12:17:34Z

Sorry for brothering you，vladimiroltean！When I run the code you mentioned, one error occurs:

Error: Specified qdisc not found.

vladimiroltean · 2021-04-28T13:11:22Z

What is the output of "zcat /proc/config.gz | grep NET_SCH_INGRESS" on your system?
Is the tc binary coming from an Ubuntu package or is it compiled from iproute2 using Buildroot? It needs to be compiled using Buildroot.

Leo1726 · 2021-04-28T13:16:57Z

I guess it was not compiled?

Leo1726 · 2021-04-28T13:24:14Z

Also, I tried to follow the method mentioned in #38 to set PCP on ports by adding a script text file (say felix.sh).
It also failed by:

How could I know what's wrong on it?
Or, is there any other methods to tag different PCP on Vlan frames creating from the CPU(LS1028Board) or frames through swp ports?

vladimiroltean · 2021-04-28T13:33:59Z

How could I know what's wrong on it?

Try "#!/bin/bash" instead of "#!/bin/sh", I think "-o pipefail" is a bashism.

vladimiroltean · 2021-04-28T13:35:18Z

Or, is there any other methods to tag different PCP on Vlan frames creating from the CPU(LS1028Board) or frames through swp ports?

The LS1028A switch will automatically set a one-to-one mapping between VLAN PCP and traffic class:
https://github.com/openil/linux/blob/linux-5.4.y/drivers/net/dsa/ocelot/felix.c#L351

Leo1726 · 2021-04-28T13:39:08Z

How could I know what's wrong on it?

Try "#!/bin/bash" instead of "#!/bin/sh", I think "-o pipefail" is a bashism.

Error is disappeared, but nothing change...

vladimiroltean · 2021-04-28T13:49:33Z

Yes, the output of "cat /proc/iomem" has changed between kernels. This line from the script no longer matches:

	local es_ana_bar=$(cat /proc/iomem | awk -F- '/: ana$/ { gsub(/ /, "", $0); print "0x"$1 }')

I've updated that script and fixed both issues, can you please try again?

Leo1726 · 2021-04-28T14:01:31Z

Sorry, where is the update? How to change this line?

vladimiroltean · 2021-04-28T14:07:47Z

The updated script is in the thread you linked. Just remove the leading ":" from ": ana" to become " ana".

Leo1726 · 2021-04-28T14:22:22Z

I changed it:

But, still nothing happened.

vladimiroltean · 2021-04-28T14:29:38Z

Can you show the output of "cat /proc/iomem"? It looks like this for me (extracted only the portion for PCIe PF 5, the Felix switch):

1fc000000-1fc3fffff : pcie@1f0000000
  1fc000000-1fc3fffff : 0000:00:00.5
    1fc010000-1fc01ffff : 0000:00:00.5 sys
    1fc030000-1fc03ffff : 0000:00:00.5 rew
    1fc040000-1fc0403ff : 0000:00:00.5 s0
    1fc050000-1fc0503ff : 0000:00:00.5 s1
    1fc060000-1fc0603ff : 0000:00:00.5 s2
    1fc070000-1fc0701ff : 0000:00:00.5 devcpu_gcb
    1fc080000-1fc0800ff : 0000:00:00.5 qs
    1fc090000-1fc0900cb : 0000:00:00.5 ptp
    1fc100000-1fc10ffff : 0000:00:00.5 port0
    1fc110000-1fc11ffff : 0000:00:00.5 port1
    1fc120000-1fc12ffff : 0000:00:00.5 port2
    1fc130000-1fc13ffff : 0000:00:00.5 port3
    1fc140000-1fc14ffff : 0000:00:00.5 port4
    1fc150000-1fc15ffff : 0000:00:00.5 port5
    1fc200000-1fc21ffff : 0000:00:00.5 qsys
    1fc280000-1fc28ffff : 0000:00:00.5 ana

It's possible that you don't have this patch applied?
https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=35bd8c07db2ce8fd2834ef866240613a4ef982e7

Leo1726 · 2021-04-28T14:33:17Z

Like this, no port names...

vladimiroltean · 2021-04-28T14:38:08Z

They aren't port names, they are memory region names. This is why it doesn't work, it searches for the "ANA" region (which stands for Analyzer). You need the linked kernel patch.

Leo1726 · 2021-04-28T14:46:43Z

Got it! I will try later.
Did this patch includes in OpenIL v1.8/v1.9?
(I'm in a hurry now. I would give you feedback as soon as possible.)

Btw,

Or, is there any other methods to tag different PCP on Vlan frames creating from the CPU(LS1028Board) or frames through swp ports?

The LS1028A switch will automatically set a one-to-one mapping between VLAN PCP and traffic class:
https://github.com/openil/linux/blob/linux-5.4.y/drivers/net/dsa/ocelot/felix.c#L351

So, How could I use this? What's the mapping rule, such as what kinds of traffic class map which PCP number?
Could you give me an example?

vladimiroltean · 2021-04-28T14:55:41Z

Did this patch includes in OpenIL v1.9?

No, I don't think so.

So, How could I use this? What's the mapping rule, such as what kinds of traffic class map which PCP number?

It depends if the packet is injected into the switch from the CPU or from the outside world.
For traffic injected from the CPU (meaning: packets sent through a socket opened on swp0 etc), it is sufficient to create a qdisc such as mqprio or taprio with 8 traffic classes and one queue per traffic class (the example in this thread should work), and then a set of tc filters on the egress qdisc which select the skb priority (again, the examples in this thread should work). The kernel driver looks at skb->priority and sets the appropriate QoS class in the DSA frame header when transmitting to the switch.
https://github.com/openil/linux/blob/linux-5.4.y/net/dsa/tag_ocelot.c#L160

For packets received from the outside world, it is sufficient for them to contain a VLAN PCP value i (between 0 and 7), and they will be classified to traffic class i inside the switch.

You can look at the following port counters:

ethtool -S swp0 | grep x_green_prio
     rx_green_prio_0: 0
     rx_green_prio_1: 0
     rx_green_prio_2: 0
     rx_green_prio_3: 0
     rx_green_prio_4: 0
     rx_green_prio_5: 0
     rx_green_prio_6: 0
     rx_green_prio_7: 0
     tx_green_prio_0: 0
     tx_green_prio_1: 0
     tx_green_prio_2: 0
     tx_green_prio_3: 0
     tx_green_prio_4: 0
     tx_green_prio_5: 0
     tx_green_prio_6: 0
     tx_green_prio_7: 0

These are per-traffic class counters, you can check that the proper one is incrementing when receiving a VLAN-tagged packet.

The felix.sh switch only affects the traffic class in case the received packets do not have a VLAN PCP.

Leo1726 · 2021-04-29T09:43:37Z

Hi, vladimiroltean. I have enable NET_SCH_INGRESS and could use tc qdisc.

I want to ask how to set PCP in the following situation.

One LS1028 borad sends any message with certain VID and PCP(e.g.VID=1, PCP=7) to other nodes through swp0-3.

I guess this is quite similar to what you said, "the packet is injected into the switch from the CPU."
How could I use “tc filter ... flower‘’ command to do this? Should I add tc filters on swp0-3 or on switch?

This is my config for swp0-3.

ifconfig eno2 up
ip link add name switch type bridge
ip link set switch up
ip link set swp0 master switch && ip link set swp0 up
ip link set swp1 master switch && ip link set swp1 up
ip link set swp2 master switch && ip link set swp2 up
ip link set swp3 master switch && ip link set swp3 up
ifconfig switch 192.168.200.103/24

vladimiroltean · 2021-04-29T10:27:10Z

So, as mentioned, if you send packets through the "switch" bridge interface, and eno2 is configured as the DSA master, they will exit the system using the DSA injection mechanism (switch -> swp0 -> eno2). This means that the frame analyzer will be bypassed, so the local switch will not look at the VLAN PCP at all, it will just enqueue into the traffic class specified in the QOS_CLASS field of the DSA header (which the driver inserts automatically into the packet based on skb->priority). That is to say, you can still send VLAN-tagged packets, and the VLAN PCP will be interpreted by other nodes in the network, but for the local device, what you need is to set the skb->priority properly. This can be done using the "skbedit priority" tc action, put in the software egress qdisc either of the bridge interface, or of individual switch interfaces:

tc qdisc add dev swp0 clsact
tc filter add dev swp0 egress protocol ipv4 flower dst_ip 192.168.100.1 action skbedit priority 7

You can of course also use the SO_PRIORITY kernel API from the application directly such that all packets sent from a specific socket are treated by the kernel with a given skb->priority, and this will translate into the correct QOS_CLASS field of the Felix switch DSA header.

Leo1726 · 2021-04-29T10:38:15Z

So, as mentioned, if you send packets through the "switch" bridge interface, and eno2 is configured as the DSA master, they will exit the system using the DSA injection mechanism (switch -> swp0 -> eno2).

I got little confuse about it. The messages should flow from (LS1028)cpu -->eno2 -->switch -->swp0 --> node2(ip:192.168.100.1, outside).

vladimiroltean · 2021-04-29T10:48:04Z

I am talking about the skb path through software interfaces. Your "switch" name for the bridge interface is what is confusing. From the software's perspective, the physical transmission is always done by eno2, so it makes sense that this is the last element in the path I mentioned. The first element in the path is the bridge interface, because that is where the user socket is opened. And the bridge interface selects a lower bridge port interface for transmission (in this case swp0) by looking up its software FDB for that destination MAC address, or performs flooding if an FDB entry is not found.

Leo1726 · 2021-04-30T02:52:24Z

If I want to send with PCP=7, VID=1, should I enable vlan_filtering first?

tc qdisc add dev swp0 clsact
tc filter add dev swp0 egress protocol ipv4 flower dst_ip 192.168.100.1 action skbedit priority 7

ip link set dev switch type bridge vlan_filtering 1
bridge vlan add dev switch vid 1 pvid self
bridge vlan add dev swp0 vid 1 pvid

Leo1726 · 2021-04-30T03:31:26Z

I got no VID or PCP in ptp4l test.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LS1028ardb set PCP field #80

LS1028ardb set PCP field #80

GJim01 commented Dec 2, 2020

vladimiroltean commented Dec 2, 2020 •

edited

Loading

vladimiroltean commented Dec 2, 2020

Leo1726 commented Apr 28, 2021

vladimiroltean commented Apr 28, 2021 •

edited

Loading

Leo1726 commented Apr 28, 2021

Leo1726 commented Apr 28, 2021

vladimiroltean commented Apr 28, 2021

vladimiroltean commented Apr 28, 2021

Leo1726 commented Apr 28, 2021

vladimiroltean commented Apr 28, 2021

Leo1726 commented Apr 28, 2021

vladimiroltean commented Apr 28, 2021 •

edited

Loading

Leo1726 commented Apr 28, 2021

vladimiroltean commented Apr 28, 2021

Leo1726 commented Apr 28, 2021

vladimiroltean commented Apr 28, 2021 •

edited

Loading

Leo1726 commented Apr 28, 2021 •

edited

Loading

vladimiroltean commented Apr 28, 2021 •

edited

Loading

Leo1726 commented Apr 29, 2021 •

edited

Loading

vladimiroltean commented Apr 29, 2021 •

edited

Loading

Leo1726 commented Apr 29, 2021

vladimiroltean commented Apr 29, 2021

Leo1726 commented Apr 30, 2021

Leo1726 commented Apr 30, 2021 •

edited

Loading

LS1028ardb set PCP field #80

LS1028ardb set PCP field #80

Comments

GJim01 commented Dec 2, 2020

vladimiroltean commented Dec 2, 2020 • edited Loading

vladimiroltean commented Dec 2, 2020

Leo1726 commented Apr 28, 2021

vladimiroltean commented Apr 28, 2021 • edited Loading

Leo1726 commented Apr 28, 2021

Leo1726 commented Apr 28, 2021

vladimiroltean commented Apr 28, 2021

vladimiroltean commented Apr 28, 2021

Leo1726 commented Apr 28, 2021

vladimiroltean commented Apr 28, 2021

Leo1726 commented Apr 28, 2021

vladimiroltean commented Apr 28, 2021 • edited Loading

Leo1726 commented Apr 28, 2021

vladimiroltean commented Apr 28, 2021

Leo1726 commented Apr 28, 2021

vladimiroltean commented Apr 28, 2021 • edited Loading

Leo1726 commented Apr 28, 2021 • edited Loading

vladimiroltean commented Apr 28, 2021 • edited Loading

Leo1726 commented Apr 29, 2021 • edited Loading

vladimiroltean commented Apr 29, 2021 • edited Loading

Leo1726 commented Apr 29, 2021

vladimiroltean commented Apr 29, 2021

Leo1726 commented Apr 30, 2021

Leo1726 commented Apr 30, 2021 • edited Loading

vladimiroltean commented Dec 2, 2020 •

edited

Loading

vladimiroltean commented Apr 28, 2021 •

edited

Loading

vladimiroltean commented Apr 28, 2021 •

edited

Loading

vladimiroltean commented Apr 28, 2021 •

edited

Loading

Leo1726 commented Apr 28, 2021 •

edited

Loading

vladimiroltean commented Apr 28, 2021 •

edited

Loading

Leo1726 commented Apr 29, 2021 •

edited

Loading

vladimiroltean commented Apr 29, 2021 •

edited

Loading

Leo1726 commented Apr 30, 2021 •

edited

Loading