Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vppapi event takes a long time #464

Closed
glazychev-art opened this issue Nov 29, 2021 · 3 comments
Closed

vppapi event takes a long time #464

glazychev-art opened this issue Nov 29, 2021 · 3 comments

Comments

@glazychev-art
Copy link
Contributor

Description

Looking through the logs, it was noticed that the event from vpp about upping memif interface can take a long time.
See: https://github.com/networkservicemesh/sdk-vpp/blob/main/pkg/networkservice/up/common.go#L102-L118

For example, we were supposed to get an event here:

...
{"log":"Nov 24 21:55:25.958\u001b[37m [DEBU] [id:15381945-9fe7-4afa-8b2b-f0e6347af950] [name:forwarder-vpp-4rq6w] [details.Flags:IF_STATUS_API_FLAG_ADMIN_UP] [duration:257.36µs] [swIfIndex:4] [type:NetworkService] [vppapi:SwInterfaceDump] \u001b[0m(48.2)                                                  completed span=7e3a66648206cc0f:28c2658fae6b3450:705fd3ed70c65f2b:1\n","stream":"stderr","time":"2021-11-24T21:55:25.959022416Z"}
{"log":"Nov 24 21:55:33.330\u001b[37m [TRAC] [id:15381945-9fe7-4afa-8b2b-f0e6347af950] [name:forwarder-vpp-4rq6w] [type:NetworkService] \u001b[0m(48.3)                                                  close={...}
...

Later we get an event, but it takes a long time (see duration):

...
{"log":"Nov 24 21:56:03.055\u001b[37m [DEBU] [id:ecbdebf2-c694-41e0-9e02-51ebaacde72b] [name:forwarder-vpp-4rq6w] [duration:9.059329171s] [msg.Flags:IF_STATUS_API_FLAG_LINK_UP] [swIfIndex:4] [type:NetworkService] [vppapi:SwInterfaceEvent] \u001b[0m(48.3) 
...

Logs

forwarder21.zip

Related issue

networkservicemesh/deployments-k8s#2381

@glazychev-art
Copy link
Contributor Author

Hi @jkossak

We looked at the logs from nse-composition case when ping did not work for you (forwarder log is attached in description)
Could you please clarify how often this problem is? Do you have logs from other runs?

@jkossak
Copy link

jkossak commented Dec 1, 2021

Hi @glazychev-art, it rather happened on my system when I was testing certain commit. When I turned back to try previous commmits the ping was working again. I returned to the same one and ping was not working again.

Now I'm on commit b8988399c800a966cf8148d0e42762e54266f6c9 and I do not not see issues with ping test.

The commit that I used when ping did not work was f04abcdc94c1a96652886c1d84f6e29601431ea4. From that commit I removed all cpu and mem limits and requests for exclude-prefixes to avoid restarts of exclude-prefixes container.

@denis-tingaikin
Copy link
Member

Seems like the issue outdated and was fixed by f04abcdc94c1a96652886c1d84f6e29601431ea4.

@jkossak Be free to reopen this if you see any issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

No branches or pull requests

3 participants