From 90d51076de214a69490cd627cc0e70ced24f5622 Mon Sep 17 00:00:00 2001 From: trvanduy Date: Fri, 11 Dec 2020 16:18:53 -0800 Subject: [PATCH 1/6] Created initial version on SNMP migration from snmp.yml to Redis ConfigDB --- doc/snmp/snmp-configdb-migration-hld.md | 216 ++++++++++++++++++++++++ 1 file changed, 216 insertions(+) create mode 100644 doc/snmp/snmp-configdb-migration-hld.md diff --git a/doc/snmp/snmp-configdb-migration-hld.md b/doc/snmp/snmp-configdb-migration-hld.md new file mode 100644 index 0000000000..e83be2b855 --- /dev/null +++ b/doc/snmp/snmp-configdb-migration-hld.md @@ -0,0 +1,216 @@ +# SNMP Migration from snmp.yml to ConfigDB +# High Level Design Document +### Rev 0.1 + +###### Revision +| Rev | Date | Author | Change Description | +|:---:|:-----------:|:------------------:|-----------------------------------| +| 0.1 | | Travis Van Duyn | Initial version | + +# About this Manual +This document provides general information about the migration of SNMP community information from snmp.yml file to the ConfigDB. +# Scope +Currently we are using the /etc/sonic/snmp.yml file to populate the SNMP communities in the /etc/snmp/snmpd.conf file. +The goal of this update is to move away from the snmp.yml file and move towards the ConfigDB for better supportibility and ease of use. + + +# Config DB +## SNMP SCHEMA +Some new "SNMP" tables should be added to ConfigDB in order to store SNMP related configuration. +https://github.com/Azure/SONiC/blob/master/doc/snmp/snmp-schema-addition.md + +The new SNMP tables are: +SNMP +SNMP_COMMUNITY +SNMP_USER + + +# SNMP Table +``` +SNMP|LOCATION +SNMP|CONTACT + +admin@str-s6000-acs-11:~$ redis-cli -n 4 hgetall "SNMP|LOCATION" +1) "LOCATION" +2) "Redmond" + +admin@str-s6000-acs-11:~$ redis-cli -n 4 hgetall "SNMP|CONTACT" +1) "joe" +2) "joe@microsoft.com" +``` + +# SNMP_COMMUNITY Table +``` +SNMP_COMMUNITY| + +admin@str-s6000-acs-11:~$ redis-cli -n 4 hgetall "SNMP_COMMUNITY|Jack" +1) "TYPE" +2) "RW" + +``` + +# SNMP_USER Table +``` +SNMP_USER| + +admin@str-s6000-acs-11:~$ redis-cli -n 4 hgetall "SNMP_USER|Travis" + 1) "SNMP_USER_ENCRYPTION_TYPE" + 2) "AES" + 3) "SNMP_USER_AUTH_TYPE" + 4) "SHA" + 5) "SNMP_USER_ENCRYPTION_PASSWORD" + 6) "TravisEncryptPass" + 7) "SNMP_USER_AUTH_PASSWORD" + 8) "TravisAuthPass" + 9) "SNMP_USER_TYPE" +10) "Priv" +11) "SNMP_USER_PERMISSION" +12) "RO" +``` + + + +# New SNMP CLI Commands +# Show commands +``` +admin@str-s6000-acs-11:~$ show run snmp -h +Usage: show run snmp [OPTIONS] COMMAND [ARGS]... + + Show SNMP information + +Options: + -?, -h, --help Show this message and exit. + +Commands: + community show runningconfiguration snmp community + contact show runningconfiguration snmp contact + location show runningconfiguration snmp location + users show runningconfiguration snmp users +admin@str-s6000-acs-11:~$ +``` + +show run snmp community +``` +admin@str-s6000-acs-11:~$ show run snmp community +Community String Community Type +------------------ ---------------- +Qi RO +Travis RO +Bill RO +Jack RW +public RO +Joker RW +admin@str-s6000-acs-11:~$ +``` + +show run snmp contact +``` +admin@str-s6000-acs-11:~$ show run snmp contact +Contact Contact Email +--------- ----------------- +Joe joe@microsoft.com +admin@str-s6000-acs-11:~$ +``` + +show run snmp location +``` +admin@str-s6000-acs-11:~$ show run snmp location +Location +---------- +Redmond +admin@str-s6000-acs-11:~$ +``` + +show run snmp users +``` +admin@str-s6000-acs-11:~$ show run snmp users +User Type Auth Type Auth Password Encryption Type Encryption Password +------ ------ ----------- --------------- ----------------- --------------------- +Travis Priv SHA TravisAuthPass AES TravisEncryptPass +admin@str-s6000-acs-11:~$ +``` + + +# Config Commands +``` +admin@str-s6000-acs-11:~$ sudo config snmp -h +Usage: config snmp [OPTIONS] COMMAND [ARGS]... + + SNMP configuration tasks + +Options: + -?, -h, --help Show this message and exit. + +Commands: + community + contact + location + user +admin@str-s6000-acs-11:~$ +``` + +sudo config snmp community +``` +admin@str-s6000-acs-11:~$ sudo config snmp community -h +Usage: config snmp community [OPTIONS] COMMAND [ARGS]... + +Options: + -?, -h, --help Show this message and exit. + +Commands: + add Add snmp community + del Add snmp community +admin@str-s6000-acs-11:~$ +``` + +sudo config snmp contact +``` +admin@str-s6000-acs-11:~$ sudo config snmp contact -h +Usage: config snmp contact [OPTIONS] COMMAND [ARGS]... + +Options: + -?, -h, --help Show this message and exit. + +Commands: + modify Modify snmp contact +admin@str-s6000-acs-11:~$ +``` + +sudo config snmp location +``` +admin@str-s6000-acs-11:~$ sudo config snmp location -h +Usage: config snmp location [OPTIONS] COMMAND [ARGS]... + +Options: + -?, -h, --help Show this message and exit. + +Commands: + modify Modify snmp location +admin@str-s6000-acs-11:~$ +``` + +sudo config snmp user +``` +admin@str-s6000-acs-11:~$ sudo config snmp user -h +Usage: config snmp user [OPTIONS] COMMAND [ARGS]... + +Options: + -?, -h, --help Show this message and exit. + +Commands: + add Add snmp user + del Add snmp user +admin@str-s6000-acs-11:~$ +``` + +# Migration Plan +In order to move from using the snmp.yml file to using the Redis ConfigDB here are the things the need to be done to move this in a way that is backward compatible. +1. Create a python conversion script to take input from snmp.yml file and convert it to Redis ConfigDB format from above schema. +2. Update Dockerfile.j2 in docker-snmp container to add a line to copy over new python conversion script to "/usr/bin/" +3. Update the snmpd.conf.j2 jinja template to pull SNMP information from only the Redis ConfigDB +4. Update start.sh in docker-snmp container to add a line above sonic-cfggen to run python conversion script (which will run everytime this docker container starts) and then comment out the "-y /etc/sonic/snmp.yml" file as this is not needed since we grabbed the snmp.yml info in the python conversion script and have the information available in the Redis ConfigDB. +5. Create new docker-snmp container with all these updates so that we will eventually be able to remove the snmp.yml file and only use the Redis ConfigDB after we socalize the update. + +If we do the migration in this way then when we rollout a new docker-snmp container to the existing devices we will still support the information in the snmp.yml file but we'll also be able to get the information from the Redis ConfigDB for all the new show and config commands. + +After this update is rolled out to the fleet we will stablize for some time and work with the NDM and HWProxy teams to migration over to using ConfigDB for their configurations. From 747d1c4d0187f578f638f3a1646643adda49f87f Mon Sep 17 00:00:00 2001 From: trvanduy Date: Fri, 11 Dec 2020 17:06:52 -0800 Subject: [PATCH 2/6] updated per comments in PR --- doc/snmp/snmp-configdb-migration-hld.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/doc/snmp/snmp-configdb-migration-hld.md b/doc/snmp/snmp-configdb-migration-hld.md index e83be2b855..326b744d33 100644 --- a/doc/snmp/snmp-configdb-migration-hld.md +++ b/doc/snmp/snmp-configdb-migration-hld.md @@ -205,11 +205,16 @@ admin@str-s6000-acs-11:~$ # Migration Plan In order to move from using the snmp.yml file to using the Redis ConfigDB here are the things the need to be done to move this in a way that is backward compatible. -1. Create a python conversion script to take input from snmp.yml file and convert it to Redis ConfigDB format from above schema. +1. Create a python conversion script to parse the data in snmp.yml file and store it in ConfigDB using the above schema. 2. Update Dockerfile.j2 in docker-snmp container to add a line to copy over new python conversion script to "/usr/bin/" 3. Update the snmpd.conf.j2 jinja template to pull SNMP information from only the Redis ConfigDB -4. Update start.sh in docker-snmp container to add a line above sonic-cfggen to run python conversion script (which will run everytime this docker container starts) and then comment out the "-y /etc/sonic/snmp.yml" file as this is not needed since we grabbed the snmp.yml info in the python conversion script and have the information available in the Redis ConfigDB. -5. Create new docker-snmp container with all these updates so that we will eventually be able to remove the snmp.yml file and only use the Redis ConfigDB after we socalize the update. +4. Update start.sh in docker-snmp container as follows: + - Add a line above the `sonic-cfggen` call to run the python conversion script from step 1 (which will run every time this docker container starts) + - Remove the `-y /etc/sonic/snmp.yml` argument from the existing call to sonic-cfggen as it will no longer needed because the python conversion script above will have already loaded that data into Config DB + + +# Notes: +A new docker-snmp container with all the above updates will be created so that we will eventually be able to remove the snmp.yml file and only use the Redis ConfigDB after we socalize the update. If we do the migration in this way then when we rollout a new docker-snmp container to the existing devices we will still support the information in the snmp.yml file but we'll also be able to get the information from the Redis ConfigDB for all the new show and config commands. From 1edcdef1dc3b1279fe5f8cd10aede46d694aa5c1 Mon Sep 17 00:00:00 2001 From: trvanduy Date: Tue, 15 Dec 2020 10:29:13 -0800 Subject: [PATCH 3/6] updated per comments --- doc/snmp/snmp-configdb-migration-hld.md | 121 +++++++++++++++--------- 1 file changed, 78 insertions(+), 43 deletions(-) diff --git a/doc/snmp/snmp-configdb-migration-hld.md b/doc/snmp/snmp-configdb-migration-hld.md index 326b744d33..2875db31a1 100644 --- a/doc/snmp/snmp-configdb-migration-hld.md +++ b/doc/snmp/snmp-configdb-migration-hld.md @@ -1,4 +1,4 @@ -# SNMP Migration from snmp.yml to ConfigDB +# SNMP Migration from snmp.yml to ConfigDB # High Level Design Document ### Rev 0.1 @@ -16,10 +16,10 @@ The goal of this update is to move away from the snmp.yml file and move towards # Config DB ## SNMP SCHEMA -Some new "SNMP" tables should be added to ConfigDB in order to store SNMP related configuration. +Some new "SNMP" tables should be added to ConfigDB in order to store SNMP related configuration. https://github.com/Azure/SONiC/blob/master/doc/snmp/snmp-schema-addition.md -The new SNMP tables are: +The new SNMP tables are: SNMP SNMP_COMMUNITY SNMP_USER @@ -30,30 +30,30 @@ SNMP_USER SNMP|LOCATION SNMP|CONTACT -admin@str-s6000-acs-11:~$ redis-cli -n 4 hgetall "SNMP|LOCATION" +admin@switch1:~$ redis-cli -n 4 hgetall "SNMP|LOCATION" 1) "LOCATION" -2) "Redmond" +2) "Emerald City" -admin@str-s6000-acs-11:~$ redis-cli -n 4 hgetall "SNMP|CONTACT" +admin@switch1:~$ redis-cli -n 4 hgetall "SNMP|CONTACT" 1) "joe" -2) "joe@microsoft.com" +2) "joe@contoso.com" ``` # SNMP_COMMUNITY Table ``` SNMP_COMMUNITY| -admin@str-s6000-acs-11:~$ redis-cli -n 4 hgetall "SNMP_COMMUNITY|Jack" +admin@switch1:~$ redis-cli -n 4 hgetall "SNMP_COMMUNITY|Jack" 1) "TYPE" 2) "RW" ``` -# SNMP_USER Table +# SNMP_USER Table ``` SNMP_USER| -admin@str-s6000-acs-11:~$ redis-cli -n 4 hgetall "SNMP_USER|Travis" +admin@switch1:~$ redis-cli -n 4 hgetall "SNMP_USER|Travis" 1) "SNMP_USER_ENCRYPTION_TYPE" 2) "AES" 3) "SNMP_USER_AUTH_TYPE" @@ -73,25 +73,25 @@ admin@str-s6000-acs-11:~$ redis-cli -n 4 hgetall "SNMP_USER|Travis" # New SNMP CLI Commands # Show commands ``` -admin@str-s6000-acs-11:~$ show run snmp -h +admin@switch1:~$ show run snmp -h Usage: show run snmp [OPTIONS] COMMAND [ARGS]... - Show SNMP information + Show SNMP running configuration Options: -?, -h, --help Show this message and exit. Commands: - community show runningconfiguration snmp community - contact show runningconfiguration snmp contact - location show runningconfiguration snmp location - users show runningconfiguration snmp users -admin@str-s6000-acs-11:~$ + community show running configuration snmp community + contact show running configuration snmp contact + location show running configuration snmp location + users show running configuration snmp users +admin@switch1:~$ ``` show run snmp community ``` -admin@str-s6000-acs-11:~$ show run snmp community +admin@switch1:~$ show run snmp community Community String Community Type ------------------ ---------------- Qi RO @@ -100,40 +100,40 @@ Bill RO Jack RW public RO Joker RW -admin@str-s6000-acs-11:~$ +admin@switch1:~$ ``` show run snmp contact ``` -admin@str-s6000-acs-11:~$ show run snmp contact +admin@switch1:~$ show run snmp contact Contact Contact Email --------- ----------------- Joe joe@microsoft.com -admin@str-s6000-acs-11:~$ +admin@switch1:~$ ``` show run snmp location ``` -admin@str-s6000-acs-11:~$ show run snmp location +admin@switch1:~$ show run snmp location Location ---------- Redmond -admin@str-s6000-acs-11:~$ +admin@switch1:~$ ``` show run snmp users ``` -admin@str-s6000-acs-11:~$ show run snmp users +admin@switch1:~$ show run snmp users User Type Auth Type Auth Password Encryption Type Encryption Password ------ ------ ----------- --------------- ----------------- --------------------- Travis Priv SHA TravisAuthPass AES TravisEncryptPass -admin@str-s6000-acs-11:~$ +admin@switch1:~$ ``` # Config Commands ``` -admin@str-s6000-acs-11:~$ sudo config snmp -h +admin@switch1:~$ sudo config snmp -h Usage: config snmp [OPTIONS] COMMAND [ARGS]... SNMP configuration tasks @@ -146,12 +146,12 @@ Commands: contact location user -admin@str-s6000-acs-11:~$ +admin@switch1:~$ ``` -sudo config snmp community +sudo config snmp community ``` -admin@str-s6000-acs-11:~$ sudo config snmp community -h +admin@switch1:~$ sudo config snmp community -h Usage: config snmp community [OPTIONS] COMMAND [ARGS]... Options: @@ -159,13 +159,30 @@ Options: Commands: add Add snmp community - del Add snmp community -admin@str-s6000-acs-11:~$ + del Delete snmp community +admin@switch1:~$ +admin@switch1:~$ sudo config snmp community add -h +Usage: config snmp community add [OPTIONS] + + Add snmp community + +Options: + -?, -h, --help Show this message and exit. +admin@switch1:~$ +admin@switch1:~$ +admin@switch1:~$ sudo config snmp community del -h +Usage: config snmp community del [OPTIONS] + + Delete snmp community + +Options: + -?, -h, --help Show this message and exit. +admin@switch1:~$ ``` -sudo config snmp contact +sudo config snmp contact ``` -admin@str-s6000-acs-11:~$ sudo config snmp contact -h +admin@switch1:~$ sudo config snmp contact -h Usage: config snmp contact [OPTIONS] COMMAND [ARGS]... Options: @@ -173,12 +190,12 @@ Options: Commands: modify Modify snmp contact -admin@str-s6000-acs-11:~$ +admin@switch1:~$ ``` -sudo config snmp location +sudo config snmp location ``` -admin@str-s6000-acs-11:~$ sudo config snmp location -h +admin@switch1:~$ sudo config snmp location -h Usage: config snmp location [OPTIONS] COMMAND [ARGS]... Options: @@ -186,12 +203,12 @@ Options: Commands: modify Modify snmp location -admin@str-s6000-acs-11:~$ +admin@switch1:~$ ``` -sudo config snmp user +sudo config snmp user ``` -admin@str-s6000-acs-11:~$ sudo config snmp user -h +admin@switch1:~$ sudo config snmp user -h Usage: config snmp user [OPTIONS] COMMAND [ARGS]... Options: @@ -199,12 +216,30 @@ Options: Commands: add Add snmp user - del Add snmp user + del Delete snmp user +admin@switch1:~$ +admin@str-s6000-acs-11:~$ sudo config snmp user add -h +Usage: config snmp user add [OPTIONS] + + + + Add snmp user + +Options: + -?, -h, --help Show this message and exit. +admin@str-s6000-acs-11:~$ sudo config snmp user del -h +Usage: config snmp user del [OPTIONS] + + Delete snmp user + +Options: + -?, -h, --help Show this message and exit. admin@str-s6000-acs-11:~$ ``` # Migration Plan -In order to move from using the snmp.yml file to using the Redis ConfigDB here are the things the need to be done to move this in a way that is backward compatible. +In order to move from using the snmp.yml file to using the Redis ConfigDB here are the things the need to be done to move this in a way that is backward compatible. 1. Create a python conversion script to parse the data in snmp.yml file and store it in ConfigDB using the above schema. 2. Update Dockerfile.j2 in docker-snmp container to add a line to copy over new python conversion script to "/usr/bin/" 3. Update the snmpd.conf.j2 jinja template to pull SNMP information from only the Redis ConfigDB @@ -216,6 +251,6 @@ In order to move from using the snmp.yml file to using the Redis ConfigDB here a # Notes: A new docker-snmp container with all the above updates will be created so that we will eventually be able to remove the snmp.yml file and only use the Redis ConfigDB after we socalize the update. -If we do the migration in this way then when we rollout a new docker-snmp container to the existing devices we will still support the information in the snmp.yml file but we'll also be able to get the information from the Redis ConfigDB for all the new show and config commands. +If we do the migration in this way then when we rollout a new docker-snmp container to the existing devices we will still support the information in the snmp.yml file but we'll also be able to get the information from the Redis ConfigDB for all the new show and config commands. -After this update is rolled out to the fleet we will stablize for some time and work with the NDM and HWProxy teams to migration over to using ConfigDB for their configurations. +After this update is rolled out to the fleet we will stablize for some time and work with the NDM and HWProxy teams to migration over to using ConfigDB for their configurations. From b0e9a1abf42e7a2971c4651f92b26e2d37635756 Mon Sep 17 00:00:00 2001 From: trvanduy Date: Thu, 17 Dec 2020 11:31:55 -0800 Subject: [PATCH 4/6] updated PR --- doc/snmp/snmp-configdb-migration-hld.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/doc/snmp/snmp-configdb-migration-hld.md b/doc/snmp/snmp-configdb-migration-hld.md index 2875db31a1..0d9004e3b4 100644 --- a/doc/snmp/snmp-configdb-migration-hld.md +++ b/doc/snmp/snmp-configdb-migration-hld.md @@ -252,5 +252,3 @@ In order to move from using the snmp.yml file to using the Redis ConfigDB here a A new docker-snmp container with all the above updates will be created so that we will eventually be able to remove the snmp.yml file and only use the Redis ConfigDB after we socalize the update. If we do the migration in this way then when we rollout a new docker-snmp container to the existing devices we will still support the information in the snmp.yml file but we'll also be able to get the information from the Redis ConfigDB for all the new show and config commands. - -After this update is rolled out to the fleet we will stablize for some time and work with the NDM and HWProxy teams to migration over to using ConfigDB for their configurations. From 88b5723c9745d1dc4c9afb93767ece4a7b1a1563 Mon Sep 17 00:00:00 2001 From: trvanduy Date: Fri, 18 Dec 2020 14:27:21 -0800 Subject: [PATCH 5/6] added output for config snmp community replace command --- doc/snmp/snmp-configdb-migration-hld.md | 27 ++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/doc/snmp/snmp-configdb-migration-hld.md b/doc/snmp/snmp-configdb-migration-hld.md index 0d9004e3b4..febdabadce 100644 --- a/doc/snmp/snmp-configdb-migration-hld.md +++ b/doc/snmp/snmp-configdb-migration-hld.md @@ -175,6 +175,27 @@ Usage: config snmp community del [OPTIONS] Delete snmp community +Options: + -?, -h, --help Show this message and exit. +admin@switch1:~$ + + +admin@switch1:~$ sudo config snmp community -h +Usage: config snmp community [OPTIONS] COMMAND [ARGS]... + +Options: + -?, -h, --help Show this message and exit. + +Commands: + add Add snmp community string + del Delete snmp community string + replace Replace snmp community string +admin@switch1:~$ sudo config snmp community replace -h +Usage: config snmp community replace [OPTIONS] + + + Replace snmp community string + Options: -?, -h, --help Show this message and exit. admin@switch1:~$ @@ -218,7 +239,7 @@ Commands: add Add snmp user del Delete snmp user admin@switch1:~$ -admin@str-s6000-acs-11:~$ sudo config snmp user add -h +admin@switch1:~$ sudo config snmp user add -h Usage: config snmp user add [OPTIONS] @@ -228,14 +249,14 @@ Usage: config snmp user add [OPTIONS] Options: -?, -h, --help Show this message and exit. -admin@str-s6000-acs-11:~$ sudo config snmp user del -h +admin@switch1:~$ sudo config snmp user del -h Usage: config snmp user del [OPTIONS] Delete snmp user Options: -?, -h, --help Show this message and exit. -admin@str-s6000-acs-11:~$ +admin@switch1:~$ ``` # Migration Plan From ab8294ba2144c4730aa4f00bf04c865a93303328 Mon Sep 17 00:00:00 2001 From: trvanduy Date: Fri, 18 Dec 2020 14:33:07 -0800 Subject: [PATCH 6/6] fixed contoso.com email --- doc/snmp/snmp-configdb-migration-hld.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/snmp/snmp-configdb-migration-hld.md b/doc/snmp/snmp-configdb-migration-hld.md index febdabadce..8cfc397f8b 100644 --- a/doc/snmp/snmp-configdb-migration-hld.md +++ b/doc/snmp/snmp-configdb-migration-hld.md @@ -108,7 +108,7 @@ show run snmp contact admin@switch1:~$ show run snmp contact Contact Contact Email --------- ----------------- -Joe joe@microsoft.com +Joe joe@contoso.com admin@switch1:~$ ```