-
Notifications
You must be signed in to change notification settings - Fork 270
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[VOQ][saidump] To move saidump.sh from the sonic-buildimage repo to the sairedis repo #1298
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
• Saidump for DNX-SAI sonic-net/sonic-buildimage#13561 Solution and modification: To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host. (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. (3) Add a new script file: files/scripts/saidump.sh, to do the below steps For each ASIC0, such as ASIC0, #1. Save the Redis data. sudo sonic-db-cli -n asic$1 SAVE > /dev/null #2. Move dump files to /var/run/redisX/ docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/" sonic-net#3. Run rdb command to convert the dump files into JSON files sudo python /usr/local/bin/rdb --command json /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null sonic-net#4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output. docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json" sonic-net#5. clear sudo rm -f /var/run/redis$1/dump.rdb sudo rm -f /var/run/redis$1/dump.json (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
• Saidump for DNX-SAI sonic-net/sonic-buildimage#13561 Solution and modification: To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host. (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. (3) Add a new script file: files/scripts/saidump.sh, to do the below steps For each ASIC0, such as ASIC0, 1. Save the Redis data. sudo sonic-db-cli -n asic$1 SAVE > /dev/null 2. Move dump files to /var/run/redisX/ docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/" 3. Run rdb command to convert the dump files into JSON files sudo python /usr/local/bin/rdb --command json /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null 4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output. docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json" 5. clear sudo rm -f /var/run/redis$1/dump.rdb sudo rm -f /var/run/redis$1/dump.json (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
• Saidump for DNX-SAI sonic-net/sonic-buildimage#13561 Solution and modification: To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host. (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. (3) Add a new script file: files/scripts/saidump.sh, to do the below steps For each ASIC0, such as ASIC0, 1. Save the Redis data. sudo sonic-db-cli -n asic$1 SAVE > /dev/null 2. Move dump files to /var/run/redisX/ docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/" 3. Run rdb command to convert the dump files into JSON files sudo python /usr/local/bin/rdb --command json /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null 4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output. docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json" 5. clear sudo rm -f /var/run/redis$1/dump.rdb sudo rm -f /var/run/redis$1/dump.json (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
• Saidump for DNX-SAI sonic-net/sonic-buildimage#13561 Solution and modification: To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host. (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. (3) Add a new script file: files/scripts/saidump.sh, to do the below steps For each ASIC0, such as ASIC0, 1. Save the Redis data. sudo sonic-db-cli -n asic$1 SAVE > /dev/null 2. Move dump files to /var/run/redisX/ docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/" 3. Run rdb command to convert the dump files into JSON files sudo python /usr/local/bin/rdb --command json /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null 4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output. docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json" 5. clear sudo rm -f /var/run/redis$1/dump.rdb sudo rm -f /var/run/redis$1/dump.json (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
• Saidump for DNX-SAI sonic-net/sonic-buildimage#13561 Solution and modification: To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host. (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. (3) Add a new script file: files/scripts/saidump.sh, to do the below steps For each ASIC0, such as ASIC0, 1. Save the Redis data. sudo sonic-db-cli -n asic$1 SAVE > /dev/null 2. Move dump files to /var/run/redisX/ docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/" 3. Run rdb command to convert the dump files into JSON files sudo python /usr/local/bin/rdb --command json /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null 4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output. docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json" 5. clear sudo rm -f /var/run/redis$1/dump.rdb sudo rm -f /var/run/redis$1/dump.json (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
• Saidump for DNX-SAI sonic-net/sonic-buildimage#13561 Solution and modification: To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host. (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. (3) Add a new script file: files/scripts/saidump.sh, to do the below steps For each ASIC0, such as ASIC0, 1. Save the Redis data. sudo sonic-db-cli -n asic$1 SAVE > /dev/null 2. Move dump files to /var/run/redisX/ docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/" 3. Run rdb command to convert the dump files into JSON files sudo python /usr/local/bin/rdb --command json /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null 4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output. docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json" 5. clear sudo rm -f /var/run/redis$1/dump.rdb sudo rm -f /var/run/redis$1/dump.json (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
• Saidump for DNX-SAI sonic-net/sonic-buildimage#13561 Solution and modification: To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host. (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. (3) Add a new script file: files/scripts/saidump.sh, to do the below steps For each ASIC0, such as ASIC0, 1. Save the Redis data. sudo sonic-db-cli -n asic$1 SAVE > /dev/null 2. Move dump files to /var/run/redisX/ docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/" 3. Run rdb command to convert the dump files into JSON files sudo python /usr/local/bin/rdb --command json /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null 4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output. docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json" 5. clear sudo rm -f /var/run/redis$1/dump.rdb sudo rm -f /var/run/redis$1/dump.json (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
• Saidump for DNX-SAI sonic-net/sonic-buildimage#13561 Solution and modification: To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host. (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. (3) Add a new script file: files/scripts/saidump.sh, to do the below steps For each ASIC0, such as ASIC0, 1. Save the Redis data. sudo sonic-db-cli -n asic$1 SAVE > /dev/null 2. Move dump files to /var/run/redisX/ docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/" 3. Run rdb command to convert the dump files into JSON files sudo python /usr/local/bin/rdb --command json /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null 4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output. docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json" 5. clear sudo rm -f /var/run/redis$1/dump.rdb sudo rm -f /var/run/redis$1/dump.json (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
• Saidump for DNX-SAI sonic-net/sonic-buildimage#13561 Solution and modification: To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host. (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. (3) Add a new script file: files/scripts/saidump.sh, to do the below steps For each ASIC0, such as ASIC0, 1. Save the Redis data. sudo sonic-db-cli -n asic$1 SAVE > /dev/null 2. Move dump files to /var/run/redisX/ docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/" 3. Run rdb command to convert the dump files into JSON files sudo python /usr/local/bin/rdb --command json /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null 4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output. docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json" 5. clear sudo rm -f /var/run/redis$1/dump.rdb sudo rm -f /var/run/redis$1/dump.json (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
• Saidump for DNX-SAI sonic-net/sonic-buildimage#13561 Solution and modification: To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host. (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. (3) Add a new script file: files/scripts/saidump.sh, to do the below steps For each ASIC0, such as ASIC0, 1. Save the Redis data. sudo sonic-db-cli -n asic$1 SAVE > /dev/null 2. Move dump files to /var/run/redisX/ docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/" 3. Run rdb command to convert the dump files into JSON files sudo python /usr/local/bin/rdb --command json /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null 4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output. docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json" 5. clear sudo rm -f /var/run/redis$1/dump.rdb sudo rm -f /var/run/redis$1/dump.json (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
• Saidump for DNX-SAI sonic-net/sonic-buildimage#13561 Solution and modification: To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host. (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. (3) Add a new script file: files/scripts/saidump.sh, to do the below steps For each ASIC0, such as ASIC0, 1. Save the Redis data. sudo sonic-db-cli -n asic$1 SAVE > /dev/null 2. Move dump files to /var/run/redisX/ docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/" 3. Run rdb command to convert the dump files into JSON files sudo python /usr/local/bin/rdb --command json /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null 4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output. docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json" 5. clear sudo rm -f /var/run/redis$1/dump.rdb sudo rm -f /var/run/redis$1/dump.json
* [saidump] • Saidump for DNX-SAI sonic-net/sonic-buildimage#13561 Solution and modification: To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host. (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. (3) Add a new script file: files/scripts/saidump.sh, to do the below steps For each ASIC0, such as ASIC0, 1. Save the Redis data. sudo sonic-db-cli -n asic$1 SAVE > /dev/null 2. Move dump files to /var/run/redisX/ docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/" 3. Run rdb command to convert the dump files into JSON files sudo python /usr/local/bin/rdb --command json /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null 4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output. docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json" 5. clear sudo rm -f /var/run/redis$1/dump.rdb sudo rm -f /var/run/redis$1/dump.json
• Saidump for DNX-SAI sonic-net/sonic-buildimage#13561 Solution and modification: To use the redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated sonic-buildimage/build_debian.sh, to install Python library rdbtools into the host. (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. (3) Add a new script file: files/scripts/saidump.sh, to do the below steps For each ASIC0, such as ASIC0, 1. Save the Redis data. sudo sonic-db-cli -n asic$1 SAVE > /dev/null 2. Move dump files to /var/run/redisX/ docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/" 3. Run rdb command to convert the dump files into JSON files sudo python /usr/local/bin/rdb --command json /var/run/redis$1/dump.rdb | sudo tee /var/run/redis$1/dump.json > /dev/null 4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output. docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json" 5. clear sudo rm -f /var/run/redis$1/dump.rdb sudo rm -f /var/run/redis$1/dump.json (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, replace saidump with saidump.sh
• Saidump for DNX-SAI sonic-net/sonic-buildimage#13561 Solution and modification: To use the Redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. (1) Updated platform/broadcom/docker-syncd-brcm-dnx/Dockerfile.j2, install Python library rdbtools into the syncd containter. (2) Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. (3) Updated sonic-buildimage/build_debian.sh, to add a new script file: files/scripts/saidump.sh into the host. This shell file does the below steps: For each ASIC0, such as ASIC0, 1. Save the Redis data. sudo sonic-db-cli -n asic$1 SAVE > /dev/null 2. Move dump files to /var/run/redisX/ docker exec database$1 sh -c "mv /var/lib/redis/dump.rdb /var/run/redis$1/" 3. Run rdb command to convert the dump files into JSON files docker exec syncd$1 sh -c "rdb --command json /var/run/redis$1/dump.rdb | tee /var/run/redis$1/dump.json > /dev/null" 4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump result in standard output. docker exec syncd$1 sh -c "saidump -r /var/run/redis$1/dump.json -m 100" 5. clear sudo rm -f /var/run/redis$1/dump.rdb sudo rm -f /var/run/redis$1/dump.json (4) Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump, to check the asic db size and if it is larger than xxx entries, then do with REDIS SAVE, otherwise, to do with old method: looping through each entry of Redis DB.
To move saidump.sh from this repo to sairedis repo.
JunhongMao
changed the title
[VOQ][saidump]
[VOQ][saidump] To move saidump.sh from the sonic-buildimage repo to the sairedis repo
Oct 3, 2023
kcudnik
requested changes
Oct 3, 2023
JunhongMao
added a commit
to JunhongMao/sonic-sairedis
that referenced
this pull request
Oct 3, 2023
kcudnik
approved these changes
Oct 4, 2023
@kcudnik , please help to merge this PR. Thanks. |
lguohan
pushed a commit
to sonic-net/sonic-buildimage
that referenced
this pull request
Nov 8, 2023
…rs. (#16466) Fix #13561 The existing saidump use https://github.com/sonic-net/sonic-swss-common/blob/master/common/table_dump.lua script which loops the ASIC_DB more than 5 seconds and blocks other processes access. This solution uses the Redis SAVE command to save the snapshot of DB each time and recover later, instead of looping through each entry in the table. Related PRs: sonic-net/sonic-utilities#2972 sonic-net/sonic-sairedis#1288 sonic-net/sonic-sairedis#1298 How did I do it? To use the Redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. 1. Updated dockers/docker-base-bullseye/Dockerfile.j2, install Python library rdbtools into the all the docker-base-bullseye containers. 2. Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. 3. To add a new script file: syncd/scripts/saidump.sh into the sairedis repo. This shell script does the following steps: For each ASIC, such as ASIC0, 3.1. Config Redis consistency directory. redis-cli -h $hostname -p $port CONFIG SET dir $redis_dir > /dev/null 3.2. Save the Redis data. redis-cli -h $hostname -p $port SAVE > /dev/null 3.3. Run rdb command to convert the dump files into JSON files rdb --command json $redis_dir/dump.rdb | tee $redis_dir/dump.json > /dev/null 3.4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump's result in standard output." saidump -r $redis_dir/dump.json -m 100 3.5. Clear the temporary files. rm -f $redis_dir/dump.rdb rm -f $redis_dir/dump.json 4. Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump. To check the asic db size and if it is larger than ROUTE_TAB_LIMIT_DIRECT_ITERATION (with default value 24000) entries, then do with REDIS SAVE, otherwise, to do with old method: looping through each entry of Redis DB. How to verify it On T2 setup with more than 96K routes, execute CLI command -- generate_dump No error should be shown Download the generate_dump result and verify the saidump file after unpacking it.
@StormLiangMS , @yxieca , please review/approve for 202205 and 202305. |
10 tasks
mssonicbld
pushed a commit
to mssonicbld/sonic-buildimage
that referenced
this pull request
Nov 19, 2023
…rs. (sonic-net#16466) Fix sonic-net#13561 The existing saidump use https://github.com/sonic-net/sonic-swss-common/blob/master/common/table_dump.lua script which loops the ASIC_DB more than 5 seconds and blocks other processes access. This solution uses the Redis SAVE command to save the snapshot of DB each time and recover later, instead of looping through each entry in the table. Related PRs: sonic-net/sonic-utilities#2972 sonic-net/sonic-sairedis#1288 sonic-net/sonic-sairedis#1298 How did I do it? To use the Redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. 1. Updated dockers/docker-base-bullseye/Dockerfile.j2, install Python library rdbtools into the all the docker-base-bullseye containers. 2. Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. 3. To add a new script file: syncd/scripts/saidump.sh into the sairedis repo. This shell script does the following steps: For each ASIC, such as ASIC0, 3.1. Config Redis consistency directory. redis-cli -h $hostname -p $port CONFIG SET dir $redis_dir > /dev/null 3.2. Save the Redis data. redis-cli -h $hostname -p $port SAVE > /dev/null 3.3. Run rdb command to convert the dump files into JSON files rdb --command json $redis_dir/dump.rdb | tee $redis_dir/dump.json > /dev/null 3.4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump's result in standard output." saidump -r $redis_dir/dump.json -m 100 3.5. Clear the temporary files. rm -f $redis_dir/dump.rdb rm -f $redis_dir/dump.json 4. Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump. To check the asic db size and if it is larger than ROUTE_TAB_LIMIT_DIRECT_ITERATION (with default value 24000) entries, then do with REDIS SAVE, otherwise, to do with old method: looping through each entry of Redis DB. How to verify it On T2 setup with more than 96K routes, execute CLI command -- generate_dump No error should be shown Download the generate_dump result and verify the saidump file after unpacking it.
StormLiangMS
pushed a commit
that referenced
this pull request
Nov 19, 2023
…he sairedis repo (#1298) To fix the issue: sonic-net/sonic-buildimage#13561 The existing saidump use https://github.com/sonic-net/sonic-swss-common/blob/master/common/table_dump.lua script which loops the ASIC_DB more than 5 seconds and blocks other processes access. This solution uses the Redis SAVE command to save the snapshot of DB each time and recover later, instead of looping through each entry in the table.
10 tasks
mssonicbld
pushed a commit
to mssonicbld/sonic-buildimage
that referenced
this pull request
Nov 21, 2023
…rs. (sonic-net#16466) Fix sonic-net#13561 The existing saidump use https://github.com/sonic-net/sonic-swss-common/blob/master/common/table_dump.lua script which loops the ASIC_DB more than 5 seconds and blocks other processes access. This solution uses the Redis SAVE command to save the snapshot of DB each time and recover later, instead of looping through each entry in the table. Related PRs: sonic-net/sonic-utilities#2972 sonic-net/sonic-sairedis#1288 sonic-net/sonic-sairedis#1298 How did I do it? To use the Redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. 1. Updated dockers/docker-base-bullseye/Dockerfile.j2, install Python library rdbtools into the all the docker-base-bullseye containers. 2. Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. 3. To add a new script file: syncd/scripts/saidump.sh into the sairedis repo. This shell script does the following steps: For each ASIC, such as ASIC0, 3.1. Config Redis consistency directory. redis-cli -h $hostname -p $port CONFIG SET dir $redis_dir > /dev/null 3.2. Save the Redis data. redis-cli -h $hostname -p $port SAVE > /dev/null 3.3. Run rdb command to convert the dump files into JSON files rdb --command json $redis_dir/dump.rdb | tee $redis_dir/dump.json > /dev/null 3.4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump's result in standard output." saidump -r $redis_dir/dump.json -m 100 3.5. Clear the temporary files. rm -f $redis_dir/dump.rdb rm -f $redis_dir/dump.json 4. Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump. To check the asic db size and if it is larger than ROUTE_TAB_LIMIT_DIRECT_ITERATION (with default value 24000) entries, then do with REDIS SAVE, otherwise, to do with old method: looping through each entry of Redis DB. How to verify it On T2 setup with more than 96K routes, execute CLI command -- generate_dump No error should be shown Download the generate_dump result and verify the saidump file after unpacking it.
mssonicbld
pushed a commit
to mssonicbld/sonic-buildimage
that referenced
this pull request
Nov 21, 2023
…rs. (sonic-net#16466) Fix sonic-net#13561 The existing saidump use https://github.com/sonic-net/sonic-swss-common/blob/master/common/table_dump.lua script which loops the ASIC_DB more than 5 seconds and blocks other processes access. This solution uses the Redis SAVE command to save the snapshot of DB each time and recover later, instead of looping through each entry in the table. Related PRs: sonic-net/sonic-utilities#2972 sonic-net/sonic-sairedis#1288 sonic-net/sonic-sairedis#1298 How did I do it? To use the Redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. 1. Updated dockers/docker-base-bullseye/Dockerfile.j2, install Python library rdbtools into the all the docker-base-bullseye containers. 2. Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. 3. To add a new script file: syncd/scripts/saidump.sh into the sairedis repo. This shell script does the following steps: For each ASIC, such as ASIC0, 3.1. Config Redis consistency directory. redis-cli -h $hostname -p $port CONFIG SET dir $redis_dir > /dev/null 3.2. Save the Redis data. redis-cli -h $hostname -p $port SAVE > /dev/null 3.3. Run rdb command to convert the dump files into JSON files rdb --command json $redis_dir/dump.rdb | tee $redis_dir/dump.json > /dev/null 3.4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump's result in standard output." saidump -r $redis_dir/dump.json -m 100 3.5. Clear the temporary files. rm -f $redis_dir/dump.rdb rm -f $redis_dir/dump.json 4. Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump. To check the asic db size and if it is larger than ROUTE_TAB_LIMIT_DIRECT_ITERATION (with default value 24000) entries, then do with REDIS SAVE, otherwise, to do with old method: looping through each entry of Redis DB. How to verify it On T2 setup with more than 96K routes, execute CLI command -- generate_dump No error should be shown Download the generate_dump result and verify the saidump file after unpacking it.
10 tasks
mssonicbld
pushed a commit
to sonic-net/sonic-buildimage
that referenced
this pull request
Nov 21, 2023
…rs. (#16466) Fix #13561 The existing saidump use https://github.com/sonic-net/sonic-swss-common/blob/master/common/table_dump.lua script which loops the ASIC_DB more than 5 seconds and blocks other processes access. This solution uses the Redis SAVE command to save the snapshot of DB each time and recover later, instead of looping through each entry in the table. Related PRs: sonic-net/sonic-utilities#2972 sonic-net/sonic-sairedis#1288 sonic-net/sonic-sairedis#1298 How did I do it? To use the Redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. 1. Updated dockers/docker-base-bullseye/Dockerfile.j2, install Python library rdbtools into the all the docker-base-bullseye containers. 2. Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. 3. To add a new script file: syncd/scripts/saidump.sh into the sairedis repo. This shell script does the following steps: For each ASIC, such as ASIC0, 3.1. Config Redis consistency directory. redis-cli -h $hostname -p $port CONFIG SET dir $redis_dir > /dev/null 3.2. Save the Redis data. redis-cli -h $hostname -p $port SAVE > /dev/null 3.3. Run rdb command to convert the dump files into JSON files rdb --command json $redis_dir/dump.rdb | tee $redis_dir/dump.json > /dev/null 3.4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump's result in standard output." saidump -r $redis_dir/dump.json -m 100 3.5. Clear the temporary files. rm -f $redis_dir/dump.rdb rm -f $redis_dir/dump.json 4. Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump. To check the asic db size and if it is larger than ROUTE_TAB_LIMIT_DIRECT_ITERATION (with default value 24000) entries, then do with REDIS SAVE, otherwise, to do with old method: looping through each entry of Redis DB. How to verify it On T2 setup with more than 96K routes, execute CLI command -- generate_dump No error should be shown Download the generate_dump result and verify the saidump file after unpacking it.
mssonicbld
pushed a commit
to sonic-net/sonic-buildimage
that referenced
this pull request
Nov 21, 2023
…rs. (#16466) Fix #13561 The existing saidump use https://github.com/sonic-net/sonic-swss-common/blob/master/common/table_dump.lua script which loops the ASIC_DB more than 5 seconds and blocks other processes access. This solution uses the Redis SAVE command to save the snapshot of DB each time and recover later, instead of looping through each entry in the table. Related PRs: sonic-net/sonic-utilities#2972 sonic-net/sonic-sairedis#1288 sonic-net/sonic-sairedis#1298 How did I do it? To use the Redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it. 1. Updated dockers/docker-base-bullseye/Dockerfile.j2, install Python library rdbtools into the all the docker-base-bullseye containers. 2. Updated sonic-buildimage/src/sonic-sairedis/saidump/saidump.cpp, add a new option -r, which updates the rdbtools's output-JSON files' format. 3. To add a new script file: syncd/scripts/saidump.sh into the sairedis repo. This shell script does the following steps: For each ASIC, such as ASIC0, 3.1. Config Redis consistency directory. redis-cli -h $hostname -p $port CONFIG SET dir $redis_dir > /dev/null 3.2. Save the Redis data. redis-cli -h $hostname -p $port SAVE > /dev/null 3.3. Run rdb command to convert the dump files into JSON files rdb --command json $redis_dir/dump.rdb | tee $redis_dir/dump.json > /dev/null 3.4. Run saidump -r to update the JSON files' format as same as the saidump before. Then we can get the saidump's result in standard output." saidump -r $redis_dir/dump.json -m 100 3.5. Clear the temporary files. rm -f $redis_dir/dump.rdb rm -f $redis_dir/dump.json 4. Update sonic-buildimage/src/sonic-utilities/scripts/generate_dump. To check the asic db size and if it is larger than ROUTE_TAB_LIMIT_DIRECT_ITERATION (with default value 24000) entries, then do with REDIS SAVE, otherwise, to do with old method: looping through each entry of Redis DB. How to verify it On T2 setup with more than 96K routes, execute CLI command -- generate_dump No error should be shown Download the generate_dump result and verify the saidump file after unpacking it.
JunhongMao
added a commit
to JunhongMao/sonic-utilities
that referenced
this pull request
Dec 13, 2023
sonic-net#2972 added two below functions into scripts/generate_dump. get_route_table_size_by_asic_id_and_ipver save_saidump_by_route_size The unittest scripts need to be added. Related PRs: sonic-net#2972 sonic-net/sonic-buildimage#16466 sonic-net/sonic-sairedis#1288 sonic-net/sonic-sairedis#1298 Microsoft ADO (25892277): Add two scripts: tests/saidump_test.py tests/saidump_test.sh To use below 6 test cases to verify the functionality of get_route_table_size_by_asic_id_and_ipver and save_saidump_by_route_size behave correctly. ``` saidump test list format: [ACIS number, ipv4 and ipv6 route table size, expected function save_cmd arguments] saidump_test_list = [ [1, 10000, "docker exec syncd saidump saidump"], [1, 12000, "docker exec syncd saidump saidump"], [1, 12001, "docker exec syncd saidump.sh saidump"], [1, 20000, "docker exec syncd saidump.sh saidump"], [2, 10000, "docker exec syncd0 saidump saidump0\ndocker exec syncd1 saidump saidump1"], [2, 12000, "docker exec syncd0 saidump saidump0\ndocker exec syncd1 saidump saidump1"], [2, 12001, "docker exec syncd0 saidump.sh saidump0\ndocker exec syncd1 saidump.sh saidump1"], [2, 20000, "docker exec syncd0 saidump.sh saidump0\ndocker exec syncd1 saidump.sh saidump1"] ] ``` During the compiling stage, run the below command to check if it's PASSED. jumao@1b1ffba5949a:/sonic/src/sonic-utilities$ time python3 setup.py test tests/saidump_test.py::test_saidump PASSED
JunhongMao
added a commit
to JunhongMao/sonic-utilities
that referenced
this pull request
Dec 13, 2023
sonic-net#2972 added two below functions into scripts/generate_dump. get_route_table_size_by_asic_id_and_ipver save_saidump_by_route_size The unittest scripts need to be added. Related PRs: sonic-net#2972 sonic-net/sonic-buildimage#16466 sonic-net/sonic-sairedis#1288 sonic-net/sonic-sairedis#1298 Microsoft ADO (25892277): Add two scripts: tests/saidump_test.py tests/saidump_test.sh To use below 6 test cases to verify the functionality of get_route_table_size_by_asic_id_and_ipver and save_saidump_by_route_size behave correctly. ``` saidump test list format: [ACIS number, ipv4 and ipv6 route table size, expected function save_cmd arguments] saidump_test_list = [ [1, 10000, "docker exec syncd saidump saidump"], [1, 12000, "docker exec syncd saidump saidump"], [1, 12001, "docker exec syncd saidump.sh saidump"], [1, 20000, "docker exec syncd saidump.sh saidump"], [2, 10000, "docker exec syncd0 saidump saidump0\ndocker exec syncd1 saidump saidump1"], [2, 12000, "docker exec syncd0 saidump saidump0\ndocker exec syncd1 saidump saidump1"], [2, 12001, "docker exec syncd0 saidump.sh saidump0\ndocker exec syncd1 saidump.sh saidump1"], [2, 20000, "docker exec syncd0 saidump.sh saidump0\ndocker exec syncd1 saidump.sh saidump1"] ] ``` During the compiling stage, run the below command to check if it's PASSED. jumao@1b1ffba5949a:/sonic/src/sonic-utilities$ time python3 setup.py test tests/saidump_test.py::test_saidump PASSED
#1298 depend on #1288, I reverted both, will cherry pick this one after #1288 get separate PR merged to 202305. @JunhongMao @gechiang |
10 tasks
This was referenced Jun 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why did I do it?
To fix the issue: sonic-net/sonic-buildimage#13561
The existing saidump use https://github.com/sonic-net/sonic-swss-common/blob/master/common/table_dump.lua script which loops the ASIC_DB more than 5 seconds and blocks other processes access.
This solution uses the Redis SAVE command to save the snapshot of DB each time and recover later, instead of looping through each entry in the table.
Related PRs:
sonic-net/sonic-utilities#2972
sonic-net/sonic-buildimage#16466
#1288
In addition to the PR:
#1288
This PR is to address the below review comments:
To move saidump.sh from the sonic-buildimage repo to the sairedis repo.
sonic-net/sonic-buildimage#16466 (comment)
Work item tracking
Microsoft ADO (25892380):
How did I do it?
To use the Redis-db SAVE option to save the snapshot of DB each time and recover later, instead of looping through each entry in the table and saving it.
How to verify it
Which release branch to backport (provide reason below if selected)
Tested branch (Please provide the tested image version)