[BPF] printk gets fmt from .rodata instead of stack #9417
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
marvin-tigera
added
release-note-required
sridhartigera
approved these changes
Oct 29, 2024
tomastigera
added
docs-not-required
Instead of creating the string on stack before every printk call, declaring it static const places is in .rodata where from kernel 5.2+ can get it. That saves some instructions for each printk call and reduces binary size. Based on this blog: https://nakryiko.com/posts/bpf-tips-printk/ Before: 0000000000000000 <calico_tc_v6>: 0: bf 19 00 00 00 00 00 00 r9 = r1 1: b7 01 00 00 61 6d 0a 00 r1 = 683361 2: 63 1a e8 ff 00 00 00 00 *(u32 *)(r10 - 24) = r1 3: 18 01 00 00 75 65 20 70 00 00 00 00 72 6f 67 72 r1 = 8243680180223108469 ll 5: 7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1 6: 18 01 00 00 36 20 70 72 00 00 00 00 6f 6c 6f 67 r1 = 7453295134238580790 ll 8: 7b 1a d8 ff 00 00 00 00 *(u64 *)(r10 - 40) = r1 9: 18 01 00 00 72 69 6e 67 00 00 00 00 20 49 50 76 r1 = 8525394498135419250 ll 11: 7b 1a d0 ff 00 00 00 00 *(u64 *)(r10 - 48) = r1 12: 18 01 00 00 2d 49 3a 20 00 00 00 00 45 6e 74 65 r1 = 7310589338301647149 ll 14: 7b 1a c8 ff 00 00 00 00 *(u64 *)(r10 - 56) = r1 15: 18 06 00 00 43 41 4c 49 00 00 00 00 43 4f 4c 4f r6 = 5714029177637454147 ll 17: 7b 6a c0 ff 00 00 00 00 *(u64 *)(r10 - 64) = r6 18: bf a1 00 00 00 00 00 00 r1 = r10 19: 07 01 00 00 c0 ff ff ff r1 += -64 20: b7 02 00 00 2c 00 00 00 r2 = 44 21: 85 00 00 00 06 00 00 00 call 6 22: b7 08 00 00 00 00 00 00 r8 = 0 23: 63 8a c0 ff 00 00 00 00 *(u32 *)(r10 - 64) = r8 24: bf a2 00 00 00 00 00 00 r2 = r10 25: 07 02 00 00 c0 ff ff ff r2 += -64 26: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll 28: 85 00 00 00 01 00 00 00 call 1 29: bf 07 00 00 00 00 00 00 r7 = r0 30: 63 8a c0 ff 00 00 00 00 *(u32 *)(r10 - 64) = r8 31: bf a2 00 00 00 00 00 00 r2 = r10 32: 07 02 00 00 c0 ff ff ff r2 += -64 33: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll 35: 85 00 00 00 01 00 00 00 call 1 36: 55 07 14 00 00 00 00 00 if r7 != 0 goto +20 <LBB0_2> 37: 73 8a ea ff 00 00 00 00 *(u8 *)(r10 - 22) = r8 38: b7 01 00 00 50 0a 00 00 r1 = 2640 39: 6b 1a e8 ff 00 00 00 00 *(u16 *)(r10 - 24) = r1 40: 18 01 00 00 6c 65 64 3a 00 00 00 00 20 44 52 4f r1 = 5715705782295881068 ll 42: 7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1 43: 18 01 00 00 6f 6b 75 70 00 00 00 00 20 66 61 69 r1 = 7593462736235096943 ll 45: 7b 1a d8 ff 00 00 00 00 *(u64 *)(r10 - 40) = r1 46: 18 01 00 00 65 20 6d 61 00 00 00 00 70 20 6c 6f r1 = 8028827902737719397 ll 48: 7b 1a d0 ff 00 00 00 00 *(u64 *)(r10 - 48) = r1 49: 18 01 00 00 2d 49 3a 20 00 00 00 00 53 74 61 74 r1 = 8386111881512372525 ll 51: 7b 1a c8 ff 00 00 00 00 *(u64 *)(r10 - 56) = r1 52: 7b 6a c0 ff 00 00 00 00 *(u64 *)(r10 - 64) = r6 53: bf a1 00 00 00 00 00 00 r1 = r10 54: 07 01 00 00 c0 ff ff ff r1 += -64 55: b7 02 00 00 2b 00 00 00 r2 = 43 56: 05 00 6f 00 00 00 00 00 goto +111 <LBB0_10> After: 0000000000000000 <calico_tc_v6>: 0: bf 16 00 00 00 00 00 00 r6 = r1 1: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll 3: b7 02 00 00 2c 00 00 00 r2 = 44 4: 85 00 00 00 06 00 00 00 call 6 5: b7 08 00 00 00 00 00 00 r8 = 0 6: 63 8a f8 ff 00 00 00 00 *(u32 *)(r10 - 8) = r8 7: bf a2 00 00 00 00 00 00 r2 = r10 8: 07 02 00 00 f8 ff ff ff r2 += -8 9: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll 11: 85 00 00 00 01 00 00 00 call 1 12: bf 07 00 00 00 00 00 00 r7 = r0 13: 63 8a f8 ff 00 00 00 00 *(u32 *)(r10 - 8) = r8 14: bf a2 00 00 00 00 00 00 r2 = r10 15: 07 02 00 00 f8 ff ff ff r2 += -8 16: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll 18: 85 00 00 00 01 00 00 00 call 1 19: bf 08 00 00 00 00 00 00 r8 = r0 20: 55 07 04 00 00 00 00 00 if r7 != 0 goto +4 <LBB0_2> 21: 18 01 00 00 2c 00 00 00 00 00 00 00 00 00 00 00 r1 = 44 ll 23: b7 02 00 00 2b 00 00 00 r2 = 43 24: 05 00 2d 00 00 00 00 00 goto +45 <LBB0_11>
tomastigera
force-pushed
the
tomas-bpf-prints
branch
from
fdc27fb
to
24ce05f
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Instead of creating the string on stack before every printk call, declaring it static const places is in .rodata where from kernel 5.2+ can get it. That saves some instructions for each printk call and reduces binary size.
Based on this blog: https://nakryiko.com/posts/bpf-tips-printk/
Before:
0000000000000000 <calico_tc_v6>:
0: bf 19 00 00 00 00 00 00 r9 = r1
1: b7 01 00 00 61 6d 0a 00 r1 = 683361
2: 63 1a e8 ff 00 00 00 00 *(u32 *)(r10 - 24) = r1
3: 18 01 00 00 75 65 20 70 00 00 00 00 72 6f 67 72 r1 = 8243680180223108469 ll
5: 7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1
6: 18 01 00 00 36 20 70 72 00 00 00 00 6f 6c 6f 67 r1 = 7453295134238580790 ll
8: 7b 1a d8 ff 00 00 00 00 *(u64 *)(r10 - 40) = r1
9: 18 01 00 00 72 69 6e 67 00 00 00 00 20 49 50 76 r1 = 8525394498135419250 ll
11: 7b 1a d0 ff 00 00 00 00 *(u64 *)(r10 - 48) = r1
12: 18 01 00 00 2d 49 3a 20 00 00 00 00 45 6e 74 65 r1 = 7310589338301647149 ll
14: 7b 1a c8 ff 00 00 00 00 *(u64 *)(r10 - 56) = r1
15: 18 06 00 00 43 41 4c 49 00 00 00 00 43 4f 4c 4f r6 = 5714029177637454147 ll
17: 7b 6a c0 ff 00 00 00 00 *(u64 *)(r10 - 64) = r6
18: bf a1 00 00 00 00 00 00 r1 = r10
19: 07 01 00 00 c0 ff ff ff r1 += -64
20: b7 02 00 00 2c 00 00 00 r2 = 44
21: 85 00 00 00 06 00 00 00 call 6
22: b7 08 00 00 00 00 00 00 r8 = 0
23: 63 8a c0 ff 00 00 00 00 *(u32 *)(r10 - 64) = r8
24: bf a2 00 00 00 00 00 00 r2 = r10
25: 07 02 00 00 c0 ff ff ff r2 += -64
26: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll
28: 85 00 00 00 01 00 00 00 call 1
29: bf 07 00 00 00 00 00 00 r7 = r0
30: 63 8a c0 ff 00 00 00 00 *(u32 *)(r10 - 64) = r8
31: bf a2 00 00 00 00 00 00 r2 = r10
32: 07 02 00 00 c0 ff ff ff r2 += -64
33: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll
35: 85 00 00 00 01 00 00 00 call 1
36: 55 07 14 00 00 00 00 00 if r7 != 0 goto +20 <LBB0_2>
37: 73 8a ea ff 00 00 00 00 *(u8 *)(r10 - 22) = r8
38: b7 01 00 00 50 0a 00 00 r1 = 2640
39: 6b 1a e8 ff 00 00 00 00 *(u16 *)(r10 - 24) = r1
40: 18 01 00 00 6c 65 64 3a 00 00 00 00 20 44 52 4f r1 = 5715705782295881068 ll
42: 7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1
43: 18 01 00 00 6f 6b 75 70 00 00 00 00 20 66 61 69 r1 = 7593462736235096943 ll
45: 7b 1a d8 ff 00 00 00 00 *(u64 *)(r10 - 40) = r1
46: 18 01 00 00 65 20 6d 61 00 00 00 00 70 20 6c 6f r1 = 8028827902737719397 ll
48: 7b 1a d0 ff 00 00 00 00 *(u64 *)(r10 - 48) = r1
49: 18 01 00 00 2d 49 3a 20 00 00 00 00 53 74 61 74 r1 = 8386111881512372525 ll
51: 7b 1a c8 ff 00 00 00 00 *(u64 *)(r10 - 56) = r1
52: 7b 6a c0 ff 00 00 00 00 *(u64 *)(r10 - 64) = r6
53: bf a1 00 00 00 00 00 00 r1 = r10
54: 07 01 00 00 c0 ff ff ff r1 += -64
55: b7 02 00 00 2b 00 00 00 r2 = 43
56: 05 00 6f 00 00 00 00 00 goto +111 <LBB0_10>
After:
0000000000000000 <calico_tc_v6>:
0: bf 16 00 00 00 00 00 00 r6 = r1
1: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll
3: b7 02 00 00 2c 00 00 00 r2 = 44
4: 85 00 00 00 06 00 00 00 call 6
5: b7 08 00 00 00 00 00 00 r8 = 0
6: 63 8a f8 ff 00 00 00 00 *(u32 *)(r10 - 8) = r8
7: bf a2 00 00 00 00 00 00 r2 = r10
8: 07 02 00 00 f8 ff ff ff r2 += -8
9: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll
11: 85 00 00 00 01 00 00 00 call 1
12: bf 07 00 00 00 00 00 00 r7 = r0
13: 63 8a f8 ff 00 00 00 00 *(u32 *)(r10 - 8) = r8
14: bf a2 00 00 00 00 00 00 r2 = r10
15: 07 02 00 00 f8 ff ff ff r2 += -8
16: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll
18: 85 00 00 00 01 00 00 00 call 1
19: bf 08 00 00 00 00 00 00 r8 = r0
20: 55 07 04 00 00 00 00 00 if r7 != 0 goto +4 <LBB0_2>
21: 18 01 00 00 2c 00 00 00 00 00 00 00 00 00 00 00 r1 = 44 ll
23: b7 02 00 00 2b 00 00 00 r2 = 43
24: 05 00 2d 00 00 00 00 00 goto +45 <LBB0_11>
Description
Related issues/PRs
Todos
Release Note
Reminder for the reviewer
Make sure that this PR has the correct labels and milestone set.
Every PR needs one
docs-*label.docs-pr-required: This change requires a change to the documentation that has not been completed yet.docs-completed: This change has all necessary documentation completed.docs-not-required: This change has no user-facing impact and requires no docs.Every PR needs one
release-note-*label.release-note-required: This PR has user-facing changes. Most PRs should have this label.release-note-not-required: This PR has no user-facing changes.Other optional labels:
cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.