Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle exception to avoid lldp_syncd crash #65

Merged
merged 3 commits into from
May 23, 2024

Conversation

ZhaohuiS
Copy link
Contributor

@ZhaohuiS ZhaohuiS commented May 21, 2024

Issue description

When there is invalid string in chassis name or other fields, json.load could throw exception and return None in function _scrap_output, it could cause TypeError: 'NoneType' object does not support item assignment when running the following code:

lldp_json['lldp_loc_chassis'] = self._scrap_output(cmd_local

Traceback looks like below and cause lldp_syncd crash:

May  6 21:31:23.136172 docker ps ERR lldp#lldp-syncd [lldp_syncd] ERROR: Failed to parse lldpctl output#012Traceback (most recent call last):#012  File "/usr/local/lib/python3.9/dist-packages/lldp_syncd/daemon.py", line 180, in _scrap_output#012    lldpctl_json = json.loads(lldpctl_output)#012  File "/usr/lib/python3.9/json/__init__.py", line 346, in loads#012    return _default_decoder.decode(s)#012  File "/usr/lib/python3.9/json/decoder.py", line 337, in decode#012    obj, end = self.raw_decode(s, idx=_w(s, 0).end())#012  File "/usr/lib/python3.9/json/decoder.py", line 353, in raw_decode#012    obj, end = self.scan_once(s, idx)#012json.decoder.JSONDecodeError: Invalid \escape: line 130 column 14 (char 3552)
May  6 21:31:23.147700 LON21-0101-0105-12T1 INFO lldp#supervisord: lldp-syncd Exception in thread LldpSyncDaemon:
May  6 21:31:23.147700 LON21-0101-0105-12T1 INFO lldp#supervisord: lldp-syncd Traceback (most recent call last):
May  6 21:31:23.147700 LON21-0101-0105-12T1 INFO lldp#supervisord: lldp-syncd   File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
May  6 21:31:23.148327 LON21-0101-0105-12T1 INFO lldp#supervisord: lldp-syncd     self.run()
May  6 21:31:23.148327 LON21-0101-0105-12T1 INFO lldp#supervisord: lldp-syncd   File "/usr/local/lib/python3.9/dist-packages/sonic_syncd/interface.py", line 43, in run
May  6 21:31:23.148725 LON21-0101-0105-12T1 INFO lldp#supervisord: lldp-syncd     update_obj = self.source_update()
May  6 21:31:23.148763 LON21-0101-0105-12T1 INFO lldp#supervisord: lldp-syncd   File "/usr/local/lib/python3.9/dist-packages/lldp_syncd/daemon.py", line 197, in source_update
May  6 21:31:23.148782 LON21-0101-0105-12T1 INFO lldp#supervisord: lldp-syncd     lldp_json['lldp_loc_chassis'] = self._scrap_output(cmd_local)
May  6 21:31:23.148800 LON21-0101-0105-12T1 INFO lldp#supervisord: lldp-syncd TypeError: 'NoneType' object does not support item assignment
May  6 21:31:23.183356 LON21-0101-0105-12T1 INFO lldp#supervisord 2024-05-06 21:31:23,182 INFO exited: lldp-syncd (exit status 0; expected)

How to fix

Handle None returned value in source_update and return None if can't parse the lldpctl json output.
This can avoid lldp_syncd crash.

The expected syslog for invalid output in lldpctl command:
May 21 07:11:35.135275 hostname WARNING lldp#lldp-syncd [sonic_syncd] WARNING: No source information returned during last update. Skipping sync.

workitem

28004915

Signed-off-by: Zhaohui Sun <zhaohuisun@microsoft.com>
@ZhaohuiS ZhaohuiS requested a review from bingwang-ms May 21, 2024 07:42
@bingwang-ms
Copy link

LGTM. Thanks for the fix!

@ZhaohuiS
Copy link
Contributor Author

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

ZhaohuiS and others added 2 commits May 22, 2024 12:50
@ZhaohuiS ZhaohuiS merged commit d5fa793 into sonic-net:master May 23, 2024
5 checks passed
@StormLiangMS
Copy link

hi @yxieca need cherry pick for 202311.

@StormLiangMS
Copy link

hi @ZhaohuiS could you update the ADO in this PR? And also the test result with 202305 and 202311?

mssonicbld pushed a commit to mssonicbld/sonic-dbsyncd that referenced this pull request Aug 2, 2024
* Handle exception smooth to avoid lldp_syncd crash

Signed-off-by: Zhaohui Sun <zhaohuisun@microsoft.com>
@mssonicbld
Copy link

Cherry-pick PR to 202311: #69

mssonicbld pushed a commit that referenced this pull request Aug 2, 2024
* Handle exception smooth to avoid lldp_syncd crash

Signed-off-by: Zhaohui Sun <zhaohuisun@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants