Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix broken --filter-trace-tc and --filter-trace-xdp #424

Merged
merged 2 commits into from
Sep 26, 2024

Conversation

Asphaltt
Copy link
Contributor

When I want to troubleshoot a network issue with --filter-trace-tc, it cannot to get the address of bpf prog.

After fixing --filter-trace-tc issue, I find it fails to run with --filter-trace-xdp too. So, I fix the broken --filter-trace-xdp BTW.

Signed-off-by: Leon Hwang hffilwlqm@gmail.com

@Asphaltt Asphaltt requested a review from a team as a code owner September 24, 2024 15:46
@Asphaltt Asphaltt requested review from brb and removed request for a team September 24, 2024 15:46
@brb brb requested review from jschwinger233 and removed request for brb September 25, 2024 08:44
Copy link
Member

@jschwinger233 jschwinger233 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--track-bpf-helpers prefers the raw ksym such as bpf_prog_71868b7d3ecebc37_tail_ipv6_ct_ingress[bpf] that clearly tells it's a bpf prog. The tag shows which exact bpf prog is triggered, considered useful during cilium upgrade when there are multiple bpf prog with a same name but different tags.

That's why ParseKallsyms was changed to use full ksym without stripping any parts.

Do you think we can make --trace-tc and --trace-xdp also use the raw ksym? For me it's also valuable to show tag in case there are xdp/tc bpf with duplicate names.

Also asking @brb 's opinion.

internal/pwru/ksym.go Outdated Show resolved Hide resolved
@Asphaltt
Copy link
Contributor Author

to show tag in case there are xdp/tc bpf with duplicate names.

Aha, previously, --filter-trace-{tc,xdp} does not care about the tag, and focuses on prog name only.

However, it's acceptable to show the raw ksym including tag for --filter-trace-{tc,xdp}.

I'll update the commit later.

It fails to run with --filter-trace-tc:

```bash
2024/09/24 12:59:02 Attaching tc-bpf progs...
2024/09/24 12:59:02 failed to trace TC progs: failed to trace bpf progs: failed to find address for function entry2 of bpf prog SchedCLS(entry2)cilium#28
```

Then, I fix it by matching prog ksym with prog tag.

And avoid `addr - 1` for `--filter-trace-{tc,xdp}`.

```bash
2024/09/25 13:57:00 Attaching tc-bpf progs...
2024/09/25 13:57:00 Attaching kprobes (via kprobe-multi)...
49 / 49 [--------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
2024/09/25 13:57:00 Attached (ignored 0)
2024/09/25 13:57:00 Listening for events..
SKB                CPU PROCESS          NETNS      MARK/x        IFACE       PROTO  MTU   LEN   TUPLE FUNC
0xffff8e151d39be00 6   <empty>:0        4026531840 0            ens33:2      0x0800 1500  98    192.168.241.1:0->192.168.241.133:0(icmp) bpf_prog_e20a685998fd41c8_entry1[bpf](tc)
0xffff8e151d39a500 6   <empty>:0        4026531840 0            ens33:2      0x0800 1500  98    192.168.241.1:0->192.168.241.133:0(icmp) bpf_prog_e20a685998fd41c8_entry1[bpf](tc)
0xffff8e151d39bd00 6   <empty>:0        4026531840 0            ens33:2      0x0800 1500  98    192.168.241.1:0->192.168.241.133:0(icmp) bpf_prog_e20a685998fd41c8_entry1[bpf](tc)
0xffff8e151d39a200 6   <empty>:0        4026531840 0            ens33:2      0x0800 1500  98    192.168.241.1:0->192.168.241.133:0(icmp) bpf_prog_e20a685998fd41c8_entry1[bpf](tc)
0xffff8e151d39a800 6   <empty>:0        4026531840 0            ens33:2      0x0800 1500  98    192.168.241.1:0->192.168.241.133:0(icmp) bpf_prog_e20a685998fd41c8_entry1[bpf](tc)
^C2024/09/25 14:02:16 Received signal, exiting program..
2024/09/25 14:02:16 Detaching kprobes...
3 / 3 [---------------------------------------------------------------------------------------------------------------------------------------] 100.00% 50 p/s
```

Signed-off-by: Leon Hwang <hffilwlqm@gmail.com>
It fails to run with --filter-trace-xdp:

```bash
2024/09/24 15:38:48 Attaching xdp progs...
2024/09/24 15:38:48 failed to trace XDP progs: failed to trace bpf progs: failed to load objects:
program fexit_xdp: attach Tracing/TraceFExit: fexit xdp not supported
```

Signed-off-by: Leon Hwang <hffilwlqm@gmail.com>
Copy link
Member

@jschwinger233 jschwinger233 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@brb brb merged commit 1de15ff into cilium:main Sep 26, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants