Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: CI for bash completion #2

Open
wants to merge 103 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
103 commits
Select commit Hold shift + click to select a range
f566717
sync: Update libbpf submodule
qmonnet Oct 19, 2023
6982b3e
bpf: Add ability to pin bpf timer to calling CPU
Byte-Lab Oct 4, 2023
e8b7df5
bpftool: Align output skeleton ELF code
captain5050 Oct 7, 2023
f12f538
bpftool: Align bpf_load_and_run_opts insns and data
captain5050 Oct 7, 2023
0a9ad5d
bpf: Derive source IP addr via bpf_*_fib_lookup()
brb Oct 7, 2023
e4ddc81
bpf: Implement cgroup sockaddr hooks for unix sockets
DaanDeMeyer Oct 11, 2023
087d22a
bpftool: Add support for cgroup unix socket address hooks
DaanDeMeyer Oct 11, 2023
8485b9f
sync: Pull latest bpftool changes from kernel
qmonnet Oct 19, 2023
1485934
sync: Update libbpf submodule
qmonnet Nov 14, 2023
5978b98
bpftool: Fix printing of pointer value
chantra Oct 18, 2023
b9530de
bpftool: Wrap struct_ops dump in an array
chantra Oct 18, 2023
0946be6
netkit, bpf: Add bpf programmable net device
borkmann Oct 24, 2023
c59bb37
tools: Sync if_link uapi header
borkmann Oct 24, 2023
0fcd8de
bpftool: Implement link show support for netkit
borkmann Oct 24, 2023
d712a3e
bpftool: Extend net dump with netkit progs
borkmann Oct 24, 2023
fa46ebb
bpftool: Fix prog object type in manpage
sm00th Nov 3, 2023
b5aaf97
bpf: Use named fields for certain bpf uapi structs
Nov 4, 2023
7842164
sync: Pull latest bpftool changes from kernel
qmonnet Nov 14, 2023
9429d83
sync: Update libbpf submodule
qmonnet Nov 23, 2023
0664481
bpf: Add crosstask check to __bpf_get_stack
jordalgo Nov 8, 2023
d4ac6ea
bpf: add register bounds sanity checks and sanitization
anakryiko Nov 12, 2023
62edbc3
bpf: rename BPF_F_TEST_SANITY_STRICT to BPF_F_TEST_REG_INVARIANTS
anakryiko Nov 17, 2023
687e7f0
sync: Pull latest bpftool changes from kernel
qmonnet Nov 23, 2023
515739f
mirror: Fix CLANG_BPF_CO_RE_PROBE_CMD
dylanorz Dec 5, 2023
806bbc5
sync: Update libbpf submodule
qmonnet Jan 5, 2024
5b0a3a4
bpftool: mark orphaned programs during prog show
fomichev Nov 27, 2023
e5aae2e
bpf: Add link_info support for uprobe multi link
olsajiri Nov 25, 2023
dd0b761
bpftool: Add support to display uprobe_multi links
olsajiri Nov 25, 2023
c03dd58
bpf: introduce BPF token object
anakryiko Nov 30, 2023
21eccab
bpf: add BPF token support to BPF_MAP_CREATE command
anakryiko Nov 30, 2023
6384c2f
bpf: add BPF token support to BPF_BTF_LOAD command
anakryiko Nov 30, 2023
ac0294b
bpf: add BPF token support to BPF_PROG_LOAD command
anakryiko Nov 30, 2023
bc61d0d
bpf: rename MAX_BPF_LINK_TYPE into __MAX_BPF_LINK_TYPE for consistency
anakryiko Dec 6, 2023
da013dc
Revert BPF token-related functionality
anakryiko Dec 19, 2023
570698f
net/sched: Remove uapi support for rsvp classifier
jhsmt Dec 23, 2023
1b6407c
net/sched: Remove uapi support for tcindex classifier
jhsmt Dec 23, 2023
67a1ca1
net/sched: Remove uapi support for dsmark qdisc
jhsmt Dec 23, 2023
98c8c08
net/sched: Remove uapi support for ATM qdisc
jhsmt Dec 23, 2023
5ba3bba
net/sched: Remove uapi support for CBQ qdisc
jhsmt Dec 23, 2023
e725e62
bpfilter: remove bpfilter
qdeslandes Dec 26, 2023
b0e69ac
sync: Pull latest bpftool changes from kernel
qmonnet Jan 5, 2024
184ca6d
sync: Update libbpf submodule
qmonnet Jan 29, 2024
c4cc180
bpftool: Silence build warning about calloc()
Jan 16, 2024
1e82718
bpf: Sync uapi bpf.h header for the tooling infra
borkmann Jan 17, 2024
bb4dd70
bpf: Add cookie to perf_event bpf_link_info records
olsajiri Jan 19, 2024
39b14f3
bpf: Store cookies in kprobe_multi bpf_link_info data
olsajiri Jan 19, 2024
fb85c68
bpftool: Fix wrong free call in do_show_link
olsajiri Jan 19, 2024
25df69b
bpftool: Display cookie for perf event link probes
olsajiri Jan 19, 2024
4263cc0
bpftool: Display cookie for kprobe multi link
olsajiri Jan 19, 2024
c2a7c94
bpf: pass btf object id in bpf_map_info.
ThinkerYzu1 Jan 19, 2024
23baad8
bpf: pass attached BTF to the bpf_struct_ops subsystem
ThinkerYzu1 Jan 19, 2024
1ca16b5
bpf: Introduce BPF token object
anakryiko Jan 24, 2024
7892e21
bpf: Add BPF token support to BPF_MAP_CREATE command
anakryiko Jan 24, 2024
f29d89d
bpf: Add BPF token support to BPF_BTF_LOAD command
anakryiko Jan 24, 2024
1554ad5
bpf: Add BPF token support to BPF_PROG_LOAD command
anakryiko Jan 24, 2024
a04db04
sync: Pull latest bpftool changes from kernel
qmonnet Jan 29, 2024
1c79dbd
sync: Update libbpf submodule
qmonnet Feb 6, 2024
43cbd7a
tools headers UAPI: Sync include/uapi/linux/perf_event.h header with …
Oct 25, 2023
6e0d7d0
bpftool: Be more portable by using POSIX's basename()
acmel Jan 29, 2024
22621e3
sync: Pull latest bpftool changes from kernel
qmonnet Feb 6, 2024
e93d117
sync: Update libbpf submodule
qmonnet Mar 7, 2024
16e9ae8
bonding: Add independent control state machine
Feb 2, 2024
5aee3c0
bpf: Clarify batch lookup/lookup_and_delete semantics
Feb 21, 2024
99a3e05
bpf: Replace bpf_lpm_trie_key 0-length array with flexible array
kees Feb 22, 2024
fc0ad76
bpftool: Generated shadow variables for struct_ops maps.
ThinkerYzu1 Feb 29, 2024
579d6b0
bpftool: Add an example for struct_ops map and shadow type.
ThinkerYzu1 Feb 29, 2024
8328f37
bpf: Introduce may_goto instruction
Mar 6, 2024
3e6f814
sync: Pull latest bpftool changes from kernel
qmonnet Mar 7, 2024
779cba7
mirror: Add u16 definition to types.h
qmonnet Mar 26, 2024
a655360
sync: Update libbpf submodule
qmonnet Mar 26, 2024
bd1e316
bpftool: rename is_internal_mmapable_map into is_mmapable_map
anakryiko Mar 7, 2024
ee84172
bpf: Introduce bpf_arena.
Mar 8, 2024
c305ebf
bpf: Disasm support for addr_space_cast instruction.
Mar 8, 2024
a09e203
bpftool: Recognize arena map type
Mar 8, 2024
73a2c7e
libbpf: Recognize __arena global variables.
anakryiko Mar 8, 2024
89d84b3
bpftool: Cast pointers for shadow types explicitly.
ThinkerYzu1 Mar 12, 2024
3214350
bpftool: Fix missing pids during link show
Mar 12, 2024
d616707
bpf: support BPF cookie in raw tracepoint (raw_tp, tp_btf) programs
anakryiko Mar 19, 2024
0cb4aaf
bpftool: Enable libbpf logs when loading pid_iter in debug mode
qmonnet Mar 20, 2024
3dc1ac6
bpftool: Remove unnecessary source files from bootstrap version
qmonnet Mar 20, 2024
c1adf4a
bpftool: Clean up HOST_CFLAGS, HOST_LDFLAGS for bootstrap bpftool
qmonnet Mar 20, 2024
b79c0dc
bpf: Sync uapi bpf.h to tools directory
Mar 25, 2024
0bac781
libbpf, selftests/bpf: Adjust libbpf, bpftool, selftests to match LLVM
Mar 15, 2024
a668a13
sync: Pull latest bpftool changes from kernel
qmonnet Mar 26, 2024
20ce693
mirror: Update expected diff with kernel sources
qmonnet Mar 26, 2024
1641a50
sync: Update libbpf submodule
qmonnet Apr 2, 2024
91360ab
bpf: Add support for passing mark with bpf_fib_lookup
aspsk Mar 26, 2024
9a40292
bpf: improve error message for unsupported helper
mykyta5 Mar 25, 2024
0451298
bpftool: Use simpler indentation in source rST for documentation
rameezrehman408 Mar 31, 2024
214eb3e
bpftool: Remove useless emphasis on command description in man pages
rameezrehman408 Mar 31, 2024
4ce420d
bpftool: Clean-up typos, punctuation, list formatting in docs
rameezrehman408 Mar 31, 2024
5e805cd
bpf: Fix typo in uapi doc comments
dlech Mar 29, 2024
9fd551e
bpftool: Use __typeof__() instead of typeof() in BPF skeleton
anakryiko Apr 1, 2024
0b5a81c
sync: Pull latest bpftool changes from kernel
qmonnet Apr 2, 2024
06c61ec
mirror: Set DESTDIR to empty when building libbpf
dylanorz Apr 7, 2024
31e6f7c
WIP: CI for bash completion
qmonnet Apr 12, 2024
4e574a4
WIP - bpftool bash comp fixes
qmonnet Apr 12, 2024
5c4570f
WIP - test without prog/map in workflow
qmonnet Apr 12, 2024
48ae360
wip - move files instead of copying them
qmonnet Apr 12, 2024
55e5805
WIP - struct_ops support
qmonnet Apr 12, 2024
13943a0
fixup! WIP: CI for bash completion
qmonnet Apr 12, 2024
c7484a0
WIP: Actions Update in workflow
qmonnet Apr 12, 2024
128c423
WIP: Add workflow_dispatch
qmonnet Apr 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 109 additions & 0 deletions .github/workflows/bash-completion.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
name: bash-completion

on:
workflow_dispatch:
inputs:
SHA:
description: "SHA under test"
required: true
pull_request:
paths:
- '.github/workflows/bash-completion.yaml'
- 'bash-completion/bpftool'
- 'libbpf/**'
- 'src/**'
push:
branches:
- master

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.event.after }}
cancel-in-progress: true

jobs:
bash-completion:
name: Bash completion
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
path: 'bpftool'
submodules: recursive
ref: ${{ inputs.SHA || github.sha }}

- name: Checkout bash-completion repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
repository: 'scop/bash-completion'
ref: '81b0f8c4dfafd6219c95ffa1defad7ff34394b0f' # v2.13.0
path: 'bash-completion'

- name: Install dependencies
run: |
sudo apt-get update
sudo apt-get install -y \
libbpf-dev libelf-dev llvm

- name: Set up Python
uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0
with:
python-version: '3.10'

- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
sudo pip install pytest pytest-xdist psutil

- name: Build and install bpftool
working-directory: 'bpftool'
run: |
sudo CLANG='/usr/bin/false' LLVM_CONFIG="$CLANG" \
make -j -C src install

#- name: Build and load sample BPF program, map
# working-directory: '/tmp'
# run: |
# cat > test.c << EOF
# #include <linux/bpf.h>
# #include <bpf/bpf_helpers.h>

# struct {
# __uint(type, BPF_MAP_TYPE_ARRAY);
# __type(key, __u32);
# __type(value, __u32);
# __uint(max_entries, 1);
# } bash_comp_map SEC(".maps");

# int bash_comp_test(__attribute__((unused)) void *ctx)
# {
# __u32 key = 0;
# __u32 *value;

# value = bpf_map_lookup_elem(&bash_comp_map, &key);
# if (!value)
# return 0;

# return *value;
# }
# EOF
# cat test.c
# clang -g -O2 -emit-llvm -c test.c -o - | \
# llc -march=bpf -filetype=obj -o test.o
# sudo bpftool prog load test.o /sys/fs/bpf/bash-completion-test type xdp
# sudo bpftool prog list
# sudo bpftool map list

- name: Move Bash completion files to bash-completion repository
run: |
mv -t bash-completion/completions/ bpftool/bash-completion/bpftool
mv -t bash-completion/test/t/ bpftool/scripts/test_bpftool.py
# Shorten timeout (defaults to 30), too long to debug CI
sed -i '/bash.expect(/,/]$/ s/]$/] , timeout=3/' \
bash-completion/test/t/conftest.py

- name: Run Bash completion tests
working-directory: 'bash-completion'
run: |
sudo PYTHONDONTWRITEBYTECODE=1 \
pytest -n auto --color=yes -vv test/t/test_bpftool.py
2 changes: 1 addition & 1 deletion BPF-CHECKPOINT-COMMIT
Original file line number Diff line number Diff line change
@@ -1 +1 @@
9077fc228f09c9f975c498c55f5d2e882cd0da59
443574b033876c85a35de4c65c14f7fe092222b2
2 changes: 1 addition & 1 deletion CHECKPOINT-COMMIT
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2147c8d07e1abc8dfc3433ca18eed5295e230ede
2a24e2485722b0e12e17a2bd473bd15c9e420bdb
75 changes: 32 additions & 43 deletions bash-completion/bpftool
Original file line number Diff line number Diff line change
Expand Up @@ -106,19 +106,19 @@ _bpftool_get_link_ids()

_bpftool_get_obj_map_names()
{
local obj
local obj maps

obj=$1

maps=$(objdump -j maps -t $obj 2>/dev/null | \
command awk '/g . maps/ {print $NF}')
maps=$(objdump -j .maps -t $obj 2>/dev/null | \
command awk '/g . .maps/ {print $NF}')

COMPREPLY+=( $( compgen -W "$maps" -- "$cur" ) )
}

_bpftool_get_obj_map_idxs()
{
local obj
local obj nmaps

obj=$1

Expand All @@ -136,7 +136,7 @@ _sysfs_get_netdevs()
# Retrieve type of the map that we are operating on.
_bpftool_map_guess_map_type()
{
local keyword ref
local keyword idx ref=""
for (( idx=3; idx < ${#words[@]}-1; idx++ )); do
case "${words[$((idx-2))]}" in
lookup|update)
Expand Down Expand Up @@ -255,8 +255,9 @@ _bpftool_map_update_get_name()

_bpftool()
{
local cur prev words objword json=0
_init_completion || return
local cur prev words cword comp_args
local json=0
_init_completion -- "$@" || return

# Deal with options
if [[ ${words[cword]} == -* ]]; then
Expand Down Expand Up @@ -293,7 +294,7 @@ _bpftool()
esac

# Remove all options so completions don't have to deal with them.
local i
local i pprev
for (( i=1; i < ${#words[@]}; )); do
if [[ ${words[i]::1} == - ]] &&
[[ ${words[i]} != "-B" ]] && [[ ${words[i]} != "--base-btf" ]]; then
Expand All @@ -307,7 +308,7 @@ _bpftool()
prev=${words[cword - 1]}
pprev=${words[cword - 2]}

local object=${words[1]} command=${words[2]}
local object=${words[1]}

if [[ -z $object || $cword -eq 1 ]]; then
case $cur in
Expand All @@ -324,8 +325,12 @@ _bpftool()
esac
fi

local command=${words[2]}
[[ $command == help ]] && return 0

local MAP_TYPE='id pinned name'
local PROG_TYPE='id pinned tag name'

# Completion depends on object and command in use
case $object in
prog)
Expand All @@ -346,8 +351,6 @@ _bpftool()
;;
esac

local PROG_TYPE='id pinned tag name'
local MAP_TYPE='id pinned name'
local METRIC_TYPE='cycles instructions l1d_loads llc_misses \
itlb_misses dtlb_misses'
case $command in
Expand Down Expand Up @@ -457,7 +460,7 @@ _bpftool()
obj=${words[3]}

if [[ ${words[-4]} == "map" ]]; then
COMPREPLY=( $( compgen -W "id pinned" -- "$cur" ) )
COMPREPLY=( $( compgen -W "$MAP_TYPE" -- "$cur" ) )
return 0
fi
if [[ ${words[-3]} == "map" ]]; then
Expand All @@ -480,13 +483,13 @@ _bpftool()
action tracepoint raw_tracepoint \
xdp perf_event cgroup/skb cgroup/sock \
cgroup/dev lwt_in lwt_out lwt_xmit \
lwt_seg6local sockops sk_skb sk_msg \
lirc_mode2 cgroup/bind4 cgroup/bind6 \
cgroup/connect4 cgroup/connect6 \
cgroup/getpeername4 cgroup/getpeername6 \
cgroup/getsockname4 cgroup/getsockname6 \
cgroup/sendmsg4 cgroup/sendmsg6 \
cgroup/recvmsg4 cgroup/recvmsg6 \
lwt_seg6local sockops sk_skb sk_msg lirc_mode2 \
cgroup/bind4 cgroup/bind6 \
cgroup/connect4 cgroup/connect6 cgroup/connect_unix \
cgroup/getpeername4 cgroup/getpeername6 cgroup/getpeername_unix \
cgroup/getsockname4 cgroup/getsockname6 cgroup/getsockname_unix \
cgroup/sendmsg4 cgroup/sendmsg6 cgroup/sendmsg_unix \
cgroup/recvmsg4 cgroup/recvmsg6 cgroup/recvmsg_unix \
cgroup/post_bind4 cgroup/post_bind6 \
cgroup/sysctl cgroup/getsockopt \
cgroup/setsockopt cgroup/sock_release struct_ops \
Expand Down Expand Up @@ -541,20 +544,9 @@ _bpftool()
COMPREPLY=( $( compgen -W "$METRIC_TYPE duration" -- "$cur" ) )
return 0
;;
6)
case $prev in
duration)
return 0
;;
*)
COMPREPLY=( $( compgen -W "$METRIC_TYPE" -- "$cur" ) )
return 0
;;
esac
return 0
;;
*)
COMPREPLY=( $( compgen -W "$METRIC_TYPE" -- "$cur" ) )
[[ $prev == duration ]] && return 0
_bpftool_once_attr "$METRIC_TYPE"
return 0
;;
esac
Expand Down Expand Up @@ -612,7 +604,7 @@ _bpftool()
return 0
;;
register)
_filedir
[[ $prev == $command ]] && _filedir
return 0
;;
*)
Expand All @@ -638,9 +630,12 @@ _bpftool()
pinned)
_filedir
;;
*)
map)
_bpftool_one_of_list $MAP_TYPE
;;
*)
_bpftool_once_attr 'map'
;;
esac
return 0
;;
Expand All @@ -652,7 +647,6 @@ _bpftool()
esac
;;
map)
local MAP_TYPE='id pinned name'
case $command in
show|list|dump|peek|pop|dequeue|freeze)
case $prev in
Expand Down Expand Up @@ -793,13 +787,11 @@ _bpftool()
# map, depending on the type of the map to update.
case "$(_bpftool_map_guess_map_type)" in
array_of_maps|hash_of_maps)
local MAP_TYPE='id pinned name'
COMPREPLY+=( $( compgen -W "$MAP_TYPE" \
-- "$cur" ) )
return 0
;;
prog_array)
local PROG_TYPE='id pinned tag name'
COMPREPLY+=( $( compgen -W "$PROG_TYPE" \
-- "$cur" ) )
return 0
Expand All @@ -821,7 +813,7 @@ _bpftool()
esac

_bpftool_once_attr 'key'
local UPDATE_FLAGS='any exist noexist'
local UPDATE_FLAGS='any exist noexist' idx
for (( idx=3; idx < ${#words[@]}-1; idx++ )); do
if [[ ${words[idx]} == 'value' ]]; then
# 'value' is present, but is not the last
Expand Down Expand Up @@ -893,7 +885,6 @@ _bpftool()
esac
;;
btf)
local PROG_TYPE='id pinned tag name'
local MAP_TYPE='id pinned name'
case $command in
dump)
Expand Down Expand Up @@ -1033,7 +1024,6 @@ _bpftool()
local BPFTOOL_CGROUP_ATTACH_TYPES="$(bpftool feature list_builtins attach_types 2>/dev/null | \
grep '^cgroup_')"
local ATTACH_FLAGS='multi override'
local PROG_TYPE='id pinned tag name'
# Check for $prev = $command first
if [ $prev = $command ]; then
_filedir
Expand Down Expand Up @@ -1086,7 +1076,6 @@ _bpftool()
esac
;;
net)
local PROG_TYPE='id pinned tag name'
local ATTACH_TYPES='xdp xdpgeneric xdpdrv xdpoffload'
case $command in
show|list)
Expand Down Expand Up @@ -1193,14 +1182,14 @@ _bpftool()
pin|detach)
if [[ $prev == "$command" ]]; then
COMPREPLY=( $( compgen -W "$LINK_TYPE" -- "$cur" ) )
else
elif [[ $pprev == "$command" ]]; then
_filedir
fi
return 0
;;
*)
[[ $prev == $object ]] && \
COMPREPLY=( $( compgen -W 'help pin show list' -- "$cur" ) )
COMPREPLY=( $( compgen -W 'help pin detach show list' -- "$cur" ) )
;;
esac
;;
Expand Down
6 changes: 3 additions & 3 deletions docs/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@ see_also = $(subst " ",, \
"\n" \
"SEE ALSO\n" \
"========\n" \
"\t**bpf**\ (2),\n" \
"\t**bpf-helpers**\\ (7)" \
$(foreach page,$(call list_pages,$(1)),",\n\t**$(page)**\\ (8)") \
"**bpf**\ (2),\n" \
"**bpf-helpers**\\ (7)" \
$(foreach page,$(call list_pages,$(1)),",\n**$(page)**\\ (8)") \
"\n")

$(OUTPUT)%.8: %.rst
Expand Down
Loading
Loading