Unify qos json by using qos_config.j2 template #2023

wendani · 2018-09-06T04:07:53Z

- What I did
Unify qos json by using qos_config.j2 template.

All devices use the same WRED profile. This changes the corresponding parameters on all Mellanox devices.

- How I did it

- How to verify it

Verified on 7050

Verified by qos subtest in test_j2files.py

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Wenda <wenni@microsoft.com>

Signed-off-by: Wenda <wenni@microsoft.com> modified: device/arista/x86_64-arista_7050_qx32/Arista-7050-QX32/qos.json.j2 modified: device/arista/x86_64-arista_7050_qx32s/Arista-7050-QX-32S/qos.json.j2

Signed-off-by: Wenda <wenni@microsoft.com>

Signed-off-by: Wenda <wenni@microsoft.com> modified: ../../../mellanox/x86_64-mlnx_msn2100-r0/ACS-MSN2100/qos.json.j2 modified: ../../../mellanox/x86_64-mlnx_msn2410-r0/ACS-MSN2410/qos.json.j2 modified: ../../../mellanox/x86_64-mlnx_msn2700-r0/ACS-MSN2700/qos.json.j2 modified: ../../../mellanox/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-D48C8/qos.json.j2

Signed-off-by: Wenda <wenni@microsoft.com> modified: barefoot/x86_64-accton_wedge100bf_32x-r0/montara/qos.json.j2 modified: barefoot/x86_64-accton_wedge100bf_65x-r0/mavericks/qos.json.j2

Signed-off-by: Wenda <wenni@microsoft.com> modified: accton/x86_64-accton_as7212_54x-r0/AS7212-54x/qos.json.j2

Signed-off-by: Wenda <wenni@microsoft.com>

wendani · 2018-09-06T04:08:54Z

@simonJi2018 please check the accton devices

wendani · 2018-09-06T04:09:14Z

@mkbalani please check the barefoot devices

wendani · 2018-09-06T04:10:08Z

@stepanblyschak please check the mlnx devices

SAI-shared-user

Can qos.j2 be merged into buffers.j2?

in buffers.j2. currently, we only configure the 'active' ports (ports with neighbor defined). This new qos.j2 didn't take consistent actions.

If merging these 2 files is not feasible, can we at least abstract the port list builder code out and share between the 2?

wendani · 2018-09-06T18:00:05Z

It is using active ports. Which part do you think is not consistent? buffers and qos are different entities. We definitely can merge, but this is not within the scope of this pr.

#2023 (review)

andriymoroz-mlnx · 2018-09-07T06:49:25Z

WRED_PROFILE is defined in global template (files/build_templates/qos_config.j2) but the numbers in it are platform specific. I suggest to mote it to device//.../qos.json.j2
Also I guess you missed section where WRED_PROFILE is used, e.g:

 "QUEUE": {	
        "Ethernet0,...|3-4" : {	
            "scheduler"     :   "[SCHEDULER|scheduler.0]",	
            "wred_profile"  :   "[WRED_PROFILE|AZURE_LOSSLESS]"

lguohan · 2018-09-07T08:04:45Z

device/accton/x86_64-accton_as7212_54x-r0/AS7212-54x/qos.json.j2

+ {% for port_idx in range(0, 32) %}
+ {% if PORT_ALL.append("Ethernet%d" % (port_idx * 4)) %}{% endif %}
+ {% endfor %}
+{%- endmacro %}


why not use PORT?

sonic-cfggen -d -v PORT

It is using PORT as higher precedence. https://github.com/Azure/sonic-buildimage/pull/2023/files#diff-786c596a58cef95307fd598ac8204e35R3

This is a layer underneath PORT that provides the default port name and index in case PORT is not defined. This logic exists before unification, and is inherited here.

PORT is always defined.

wendani · 2018-09-07T16:48:22Z

@andriymoroz-mlnx the missing WRED_PROFILE is done on purpose, because the external team does not want the ecn to be applied on queues by default.

#2023 (comment)

wendani · 2018-09-07T18:10:13Z

@andriymoroz-mlnx Sorry I did not point out the WRED parameter changes on MLNX platforms. Just added in the head description. For our use, the WRED (or more general, QoS) parameters are not platform specific but global. This is the drive for the unification, and hence the WRED parameter changes as you observe as a result.

If other users want to have customized parameters for WRED or other QoS parameters, they can later put them in the local device//.../qos.json.j2, and add a hook in the qos_config.j2 template to dictate the precedence. Such changes would be straightforward.

#2023 (comment)

pavel-shirshov · 2018-09-07T20:05:08Z

device/arista/x86_64-arista_7050_qx32/Arista-7050-QX32/qos.json.j2

+ {% for port_idx in range(0, 32) %}
+ {% if PORT_ALL.append("Ethernet%d" % (port_idx * 4)) %}{% endif %}
+ {% endfor %}
+{%- endmacro %}


I'd suggest to make the generate_port_lists macro the same for all platforms.We can introduce port range, port_multiplier and exception ports as parameters. It will be more readable.

this has been removed completely.

…the generate_port_lists macro, because PORT is always defined Signed-off-by: Wenda <wenni@microsoft.com>

lguohan · 2018-09-08T22:55:12Z

device/accton/x86_64-accton_as7212_54x-r0/AS7212-54x/qos.json.j2

- }
- }
-}
+{%- set pfc_to_pg_map = true %}


some of the files do not have this line, {%- set pfc_to_pg_map = true %}

for example, the qos.json.j2 for arista 7050 qx32 do no have, why?

it is better to hide it from the user since it is asic specific

files/build_templates/qos_config.j2

SAI-shared-user

Sorry, Wenda,

I somehow missed the active port part of the change. You are right. This change is consistent with buffers configuration.

Cheers,
Ying

/etc/sonic/sonic_version.yml rather than specifying per hwsku Signed-off-by: Wenda Ni <wenni@microsoft.com>

mkbalani · 2018-09-10T18:36:47Z

files/build_templates/qos_config.j2

+
+
+{
+ "TC_TO_PRIORITY_GROUP_MAP": {


These values would not work in barefoot platforms. Is there a way a platform can overwrite these values?

Hi @mkbalani, This is actually the question I have for you. I see TC_TO_PRIORITY_GROUP_MAP to have only 3 and 4 on barefoot. We cannot include other mappings: "0": "0", "1": "1", "2": "2", "5": "5", "6": "6", "7": "7" here? Can you elaborate the reasons?

"TC_TO_PRIORITY_GROUP_MAP": { "AZURE": { "3": "3", "4": "4" } },

I have customized TC_TO_PRIORITY_GROUP_MAP and PFC_PRIORITY_TO_PRIORITY_GROUP_MAP for barefoot devices. Can you check again?

I do not understand why customization is needed for barefoot platform. @mkbalani , can you elaborate? @wendani , we should not do customization unless there is legitimate reason, otherwise it is too difficult the manage the complexity here.

mkbalani · 2018-09-10T18:38:27Z

files/build_templates/qos_config.j2

+ }
+ },
+{% if pfc_to_pg_map is defined and pfc_to_pg_map %}
+ "PFC_PRIORITY_TO_PRIORITY_GROUP_MAP": {


Same comment here. Can this be overwritten?

I do not understand why customization is needed for barefoot platform. @mkbalani , can you elaborate? @wendani , we should not do customization unless there is legitimate reason, otherwise it is too difficult the manage the complexity here.

PFC_PRIORITY_TO_PRIORITY_GROUP_MAP for barefoot Signed-off-by: Wenda <wenni@microsoft.com>

lguohan · 2018-09-11T11:03:26Z

can you update the sonic-utilities submodule in the PR, i think that is needed.

wendani · 2018-09-11T19:40:07Z

Will update the sonic-utilities submodule reference point once that is merged

#2023 (comment)

pavel-shirshov · 2018-09-12T19:15:35Z

device/barefoot/x86_64-accton_wedge100bf_32x-r0/montara/qos.json.j2

- "weight": "20"
- }
- },
+{%- endmacro %}


Is it possible to define this macro in qos_config.j2?

Is this question for @wendani or me?

these two pgs do not generate PFC frames. Signed-off-by: Wenda <wenni@microsoft.com>

bb0733a [aclorch] Add ACL_TABLE_TYPE configuration (sonic-net#1982) 59cab5d Support for setting switch level DSCP to TC QoS map (sonic-net#2023) da21172 [aclorch] add generic AclOrch::updateAclRule() method (sonic-net#1993) 4f6cb05 [Reclaiming buffer] Support reclaiming buffer in traditional model (sonic-net#2011) 32d7a69 [Reclaiming buffer] Common code update (sonic-net#1996) b91d8ba [swss] L2 Forwarding Enhancements (sonic-net#1716) 797dab4 [muxorch] Bind all ports to drop ACL table (sonic-net#2027) 99929cd [lgtm.yml] add libgmock-dev (sonic-net#2035) 8727ae5 [flex counter] Flex counter threads consume too much CPU resources sonic-net#9202 (sonic-net#2031) 103fdf0 Remove redundant calls to get child scheduler group during initialization (sonic-net#1965) 18ea840 [macsec]: MACsec statistics support (sonic-net#1867) 0c46242 [orchagent] Flush pipeline every 1 second, not only when select will timeout (sonic-net#2003) 339101c [cbf] Add class-based forwarding support (sonic-net#1963) 24a615b Fix issue: accumulative headroom can exceed limit in rare scenario (sonic-net#2020) 708e232 Test divide by zero processing path (sonic-net#2028) 8f1d035 [macsecmgr]: Wait for port up before enabling macsec (sonic-net#2032) 4912a77 Remove buffer drop counter when port is removed (sonic-net#1860) f9462c4 [Dynamic buffer] [Mellanox] Calculate the peer response time according to the speed (sonic-net#1930) 8b5a401 Routed subinterface enhancements (sonic-net#2017) cdea5e9 Fix next hop compilation (sonic-net#2025) 37c197d [SRV6] Sonic-swss changes for SRV6 (sonic-net#1964) f502c32 [vnetorch] Add ECMP support for vnet tunnel routes (sonic-net#1960) Signed-off-by: Stephen Sun <stephens@nvidia.com>

bb0733a [aclorch] Add ACL_TABLE_TYPE configuration (#1982) 59cab5d Support for setting switch level DSCP to TC QoS map (#2023) da21172 [aclorch] add generic AclOrch::updateAclRule() method (#1993) 4f6cb05 [Reclaiming buffer] Support reclaiming buffer in traditional model (#2011) 32d7a69 [Reclaiming buffer] Common code update (#1996) b91d8ba [swss] L2 Forwarding Enhancements (#1716) 797dab4 [muxorch] Bind all ports to drop ACL table (#2027) 99929cd [lgtm.yml] add libgmock-dev (#2035) 8727ae5 [flex counter] Flex counter threads consume too much CPU resources #9202 (#2031) 103fdf0 Remove redundant calls to get child scheduler group during initialization (#1965) 18ea840 [macsec]: MACsec statistics support (#1867) 0c46242 [orchagent] Flush pipeline every 1 second, not only when select will timeout (#2003) 339101c [cbf] Add class-based forwarding support (#1963) 24a615b Fix issue: accumulative headroom can exceed limit in rare scenario (#2020) 708e232 Test divide by zero processing path (#2028) 8f1d035 [macsecmgr]: Wait for port up before enabling macsec (#2032) 4912a77 Remove buffer drop counter when port is removed (#1860) f9462c4 [Dynamic buffer] [Mellanox] Calculate the peer response time according to the speed (#1930) 8b5a401 Routed subinterface enhancements (#2017) cdea5e9 Fix next hop compilation (#2025) 37c197d [SRV6] Sonic-swss changes for SRV6 (#1964) f502c32 [vnetorch] Add ECMP support for vnet tunnel routes (#1960) Signed-off-by: Stephen Sun <stephens@nvidia.com>

Signed-off-by: Vineet Mittal <vineetmittal@microsoft.com>

wendani added 11 commits August 24, 2018 23:36

Unify qos config with qos_config.j2 template

5e31afb

Signed-off-by: Wenda <wenni@microsoft.com>

Change 7050 to use qos config template

0b62f8c

Signed-off-by: Wenda <wenni@microsoft.com> modified: device/arista/x86_64-arista_7050_qx32/Arista-7050-QX32/qos.json.j2 modified: device/arista/x86_64-arista_7050_qx32s/Arista-7050-QX-32S/qos.json.j2

Change a7060, a7260, s6000, s6100, z9100 to use qos config template

5cba1fe

Signed-off-by: Wenda <wenni@microsoft.com>

Merge remote-tracking branch 'public/master'

90fce33

Change barefoot devices to use qos config template

827b53d

Signed-off-by: Wenda <wenni@microsoft.com> modified: barefoot/x86_64-accton_wedge100bf_32x-r0/montara/qos.json.j2 modified: barefoot/x86_64-accton_wedge100bf_65x-r0/mavericks/qos.json.j2

Change accton as7212 to use qos config template

393c396

Signed-off-by: Wenda <wenni@microsoft.com> modified: accton/x86_64-accton_as7212_54x-r0/AS7212-54x/qos.json.j2

Apply PORT_QOS_MAP to active ports only

cae2712

Signed-off-by: Wenda <wenni@microsoft.com>

Update qos config test with qos_config.j2 template

b826770

Signed-off-by: Wenda <wenni@microsoft.com>

Merge remote-tracking branch 'origin/one_qos_json' into HEAD

74cc2c4

Update sample output of qos-dell6100.json

7cff8ce

Signed-off-by: Wenda <wenni@microsoft.com>

wendani requested review from lguohan, pavel-shirshov, yxieca and andriymoroz-mlnx September 6, 2018 04:07

SAI-shared-user suggested changes Sep 6, 2018

View reviewed changes

lguohan reviewed Sep 7, 2018

View reviewed changes

pavel-shirshov suggested changes Sep 7, 2018

View reviewed changes

wendani added 2 commits September 8, 2018 02:31

Merge remote-tracking branch 'public/master'

d45af41

Remove generating the default port name and index list, i.e., remove …

f6001b2

…the generate_port_lists macro, because PORT is always defined Signed-off-by: Wenda <wenni@microsoft.com>

lguohan reviewed Sep 8, 2018

View reviewed changes

andriymoroz-mlnx suggested changes Sep 10, 2018

View reviewed changes

files/build_templates/qos_config.j2 Show resolved Hide resolved

SAI-shared-user reviewed Sep 10, 2018

View reviewed changes

Include pfc_to_pg_map according to platform asic type obtained from

f99d928

/etc/sonic/sonic_version.yml rather than specifying per hwsku Signed-off-by: Wenda Ni <wenni@microsoft.com>

mkbalani reviewed Sep 10, 2018

View reviewed changes

Customize TC_TO_PRIORITY_GROUP_MAP and

ca3573a

PFC_PRIORITY_TO_PRIORITY_GROUP_MAP for barefoot Signed-off-by: Wenda <wenni@microsoft.com>

stcheng added the Enhancement ➕ label Sep 12, 2018

pavel-shirshov reviewed Sep 12, 2018

View reviewed changes

wendani added 2 commits September 21, 2018 17:26

Merge remote-tracking branch 'public/master'

d6289de

Unify PFC_PRIORITY_TO_PRIORITY_GROUP_MAP: remove "0":"0", "1":"1" as

9455b0d

these two pgs do not generate PFC frames. Signed-off-by: Wenda <wenni@microsoft.com>

lguohan approved these changes Oct 9, 2018

View reviewed changes

pavel-shirshov approved these changes Oct 10, 2018

View reviewed changes

Merge remote-tracking branch 'public/master'

d0b0a63

andriymoroz-mlnx approved these changes Oct 17, 2018

View reviewed changes

lguohan merged commit 77652c5 into sonic-net:master Oct 17, 2018

andriymoroz-mlnx mentioned this pull request Mar 22, 2019

Add support of HwSKU Mellanox-SN2700-C28D8 #2690

Merged

theasianpianist pushed a commit to theasianpianist/sonic-buildimage that referenced this pull request Feb 5, 2022

Support for setting switch level DSCP to TC QoS map (sonic-net#2023)

59cab5d

Signed-off-by: Vineet Mittal <vineetmittal@microsoft.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unify qos json by using qos_config.j2 template #2023

Unify qos json by using qos_config.j2 template #2023

wendani commented Sep 6, 2018 •

edited

Loading

wendani commented Sep 6, 2018

wendani commented Sep 6, 2018

wendani commented Sep 6, 2018 •

edited

Loading

SAI-shared-user left a comment

wendani commented Sep 6, 2018 •

edited

Loading

andriymoroz-mlnx commented Sep 7, 2018

lguohan Sep 7, 2018

wendani Sep 7, 2018

lguohan Sep 7, 2018

wendani commented Sep 7, 2018 •

edited

Loading

wendani commented Sep 7, 2018 •

edited

Loading

pavel-shirshov Sep 7, 2018

lguohan Sep 8, 2018

lguohan Sep 8, 2018

lguohan Sep 9, 2018

SAI-shared-user left a comment

mkbalani Sep 10, 2018

wendani Sep 10, 2018

wendani Sep 11, 2018

lguohan Sep 11, 2018

mkbalani Sep 10, 2018

lguohan Sep 11, 2018

lguohan commented Sep 11, 2018

wendani commented Sep 11, 2018 •

edited

Loading

pavel-shirshov Sep 12, 2018

mkbalani Sep 14, 2018

Unify qos json by using qos_config.j2 template #2023

Unify qos json by using qos_config.j2 template #2023

Conversation

wendani commented Sep 6, 2018 • edited Loading

wendani commented Sep 6, 2018

wendani commented Sep 6, 2018

wendani commented Sep 6, 2018 • edited Loading

SAI-shared-user left a comment

Choose a reason for hiding this comment

wendani commented Sep 6, 2018 • edited Loading

andriymoroz-mlnx commented Sep 7, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wendani commented Sep 7, 2018 • edited Loading

wendani commented Sep 7, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SAI-shared-user left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lguohan commented Sep 11, 2018

wendani commented Sep 11, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wendani commented Sep 6, 2018 •

edited

Loading

wendani commented Sep 6, 2018 •

edited

Loading

wendani commented Sep 6, 2018 •

edited

Loading

wendani commented Sep 7, 2018 •

edited

Loading

wendani commented Sep 7, 2018 •

edited

Loading

wendani commented Sep 11, 2018 •

edited

Loading