make check-xdp failed

[KekeZheng]

i have been install p4c and xdp,now i want to try p4c-xdp test,but when i input make check-xdp. it failed,i don't know why? Is the version not match?

[rst0git]

What error message do you see?

[KekeZheng]

This is the error message when i input check-xdp, I only passed two tests when I typed this this afternoon, but most of them pass now

coco@ubuntu20:~/p4c/build$ make check-xdp
[100%] Running tests for tag xdp
Test project /home/coco/p4c/build
Start 2: xdp/extensions/p4c-xdp/tests/xdp.p4
Start 3: xdp/extensions/p4c-xdp/tests/xdp1.p4
1/18 Test #2: xdp/extensions/p4c-xdp/tests/xdp.p4 ..... Passed 0.22 sec
Start 4: xdp/extensions/p4c-xdp/tests/xdp10.p4
2/18 Test #3: xdp/extensions/p4c-xdp/tests/xdp1.p4 .... Passed 0.27 sec
Start 5: xdp/extensions/p4c-xdp/tests/xdp11.p4
3/18 Test #4: xdp/extensions/p4c-xdp/tests/xdp10.p4 ... Passed 0.30 sec
Start 7: xdp/extensions/p4c-xdp/tests/xdp13.p4
4/18 Test #5: xdp/extensions/p4c-xdp/tests/xdp11.p4 ... Passed 0.27 sec
Start 8: xdp/extensions/p4c-xdp/tests/xdp14.p4
5/18 Test #7: xdp/extensions/p4c-xdp/tests/xdp13.p4 ... Passed 0.29 sec
Start 9: xdp/extensions/p4c-xdp/tests/xdp15.p4
6/18 Test #8: xdp/extensions/p4c-xdp/tests/xdp14.p4 ... Passed 0.30 sec
Start 10: xdp/extensions/p4c-xdp/tests/xdp16.p4
7/18 Test #9: xdp/extensions/p4c-xdp/tests/xdp15.p4 ... Passed 0.24 sec
Start 11: xdp/extensions/p4c-xdp/tests/xdp17.p4
8/18 Test #10: xdp/extensions/p4c-xdp/tests/xdp16.p4 ... Passed 0.31 sec
Start 12: xdp/extensions/p4c-xdp/tests/xdp2.p4
9/18 Test #11: xdp/extensions/p4c-xdp/tests/xdp17.p4 ...***Failed 0.42 sec
Start 13: xdp/extensions/p4c-xdp/tests/xdp3.p4
10/18 Test #12: xdp/extensions/p4c-xdp/tests/xdp2.p4 .... Passed 0.47 sec
Start 14: xdp/extensions/p4c-xdp/tests/xdp4.p4
11/18 Test #13: xdp/extensions/p4c-xdp/tests/xdp3.p4 .... Passed 0.38 sec
Start 15: xdp/extensions/p4c-xdp/tests/xdp5.p4
12/18 Test #14: xdp/extensions/p4c-xdp/tests/xdp4.p4 .... Passed 0.34 sec
Start 16: xdp/extensions/p4c-xdp/tests/xdp6.p4
13/18 Test #15: xdp/extensions/p4c-xdp/tests/xdp5.p4 .... Passed 0.28 sec
Start 18: xdp/extensions/p4c-xdp/tests/xdp8.p4
14/18 Test #16: xdp/extensions/p4c-xdp/tests/xdp6.p4 .... Passed 0.30 sec
Start 19: xdp/extensions/p4c-xdp/tests/xdp9.p4
15/18 Test #18: xdp/extensions/p4c-xdp/tests/xdp8.p4 .... Passed 0.40 sec
Start 17: xdp/extensions/p4c-xdp/tests/xdp7.p4
16/18 Test #19: xdp/extensions/p4c-xdp/tests/xdp9.p4 .... Passed 0.46 sec
Start 6: xdp/extensions/p4c-xdp/tests/xdp12.p4
17/18 Test #17: xdp/extensions/p4c-xdp/tests/xdp7.p4 .... Passed 0.61 sec
18/18 Test #6: xdp/extensions/p4c-xdp/tests/xdp12.p4 ... Passed 0.45 sec

94% tests passed, 1 tests failed out of 18

Label Time Summary:
XFAIL = 1.06 secproc (2 tests)
xdp = 6.32 secproc (18 tests)

Total Test time (real) = 3.24 sec

The following tests FAILED:
11 - xdp/extensions/p4c-xdp/tests/xdp17.p4 (Failed)
Errors while running CTest
make[3]: *** [CMakeFiles/check-xdp.dir/build.make:60: CMakeFiles/check-xdp] Error 8
make[2]: *** [CMakeFiles/Makefile2:1070: CMakeFiles/check-xdp.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:1077: CMakeFiles/check-xdp.dir/rule] Error 2
make: *** [Makefile:496: check-xdp] Error 2

---

This is the error message when i input make -j4 check

root@ubuntu20:/home/coco/p4c/build# make -j4 check
[100%] Running tests for tag all
Test project /home/coco/p4c/build
1/2222 Test #3: xdp/extensions/p4c-xdp/tests/xdp1.p4 ...........................................................***Failed 9.40 sec
Start 4: xdp/extensions/p4c-xdp/tests/xdp10.p4
2/2222 Test #2: xdp/extensions/p4c-xdp/tests/xdp.p4 ............................................................***Failed 9.42 sec
Start 5: xdp/extensions/p4c-xdp/tests/xdp11.p4
...
2221/2222 Test #1988: err/testdata/p4_16_errors/template_e.p4 ........................................................ Passed 0.06 sec
2222/2222 Test #1974: err/testdata/p4_16_errors/structured-annotation-e.p4 ........................................... Passed 0.05 sec

98% tests passed, 38 tests failed out of 2222

Label Time Summary:
XFAIL = 23.73 secproc (39 tests)
bmv2 = 1446.72 secproc (540 tests)
bmv2-parser-inline-opt-disabled = 34.88 secproc (13 tests)
bmv2-parser-inline-opt-enabled = 34.83 secproc (13 tests)
cpplint = 19.28 secproc (1 test)
dpdk = 74.96 secproc (109 tests)
ebpf = 16.83 secproc (23 tests)
ebpf-bcc = 4.10 secproc (21 tests)
ebpf-kernel = 10.86 secproc (23 tests)
err = 26.66 secproc (250 tests)
gtest = 31.78 secproc (1 test)
p14_to_16 = 247.68 secproc (192 tests)
p4 = 595.66 secproc (993 tests)
p4unroll = 5.21 secproc (7 tests)
ubpf = 8.58 secproc (18 tests)
xdp = 26.97 secproc (18 tests)
The following tests FAILED:
2 - xdp/extensions/p4c-xdp/tests/xdp.p4 (Failed)
3 - xdp/extensions/p4c-xdp/tests/xdp1.p4 (Failed)
4 - xdp/extensions/p4c-xdp/tests/xdp10.p4 (Failed)
5 - xdp/extensions/p4c-xdp/tests/xdp11.p4 (Failed)
7 - xdp/extensions/p4c-xdp/tests/xdp13.p4 (Failed)
8 - xdp/extensions/p4c-xdp/tests/xdp14.p4 (Failed)
9 - xdp/extensions/p4c-xdp/tests/xdp15.p4 (Failed)
10 - xdp/extensions/p4c-xdp/tests/xdp16.p4 (Failed)
11 - xdp/extensions/p4c-xdp/tests/xdp17.p4 (Failed)
12 - xdp/extensions/p4c-xdp/tests/xdp2.p4 (Failed)
13 - xdp/extensions/p4c-xdp/tests/xdp3.p4 (Failed)
14 - xdp/extensions/p4c-xdp/tests/xdp4.p4 (Failed)
15 - xdp/extensions/p4c-xdp/tests/xdp5.p4 (Failed)
16 - xdp/extensions/p4c-xdp/tests/xdp6.p4 (Failed)
18 - xdp/extensions/p4c-xdp/tests/xdp8.p4 (Failed)
19 - xdp/extensions/p4c-xdp/tests/xdp9.p4 (Failed)
695 - ebpf-kernel/testdata/p4_16_samples/action_call_ebpf.p4 (Failed)
696 - ebpf-kernel/testdata/p4_16_samples/action_call_table_ebpf.p4 (Failed)
697 - ebpf-kernel/testdata/p4_16_samples/bool_ebpf.p4 (Failed)
698 - ebpf-kernel/testdata/p4_16_samples/count_add_ebpf.p4 (Failed)
699 - ebpf-kernel/testdata/p4_16_samples/count_ebpf.p4 (Failed)
700 - ebpf-kernel/testdata/p4_16_samples/hit_ebpf.p4 (Failed)
701 - ebpf-kernel/testdata/p4_16_samples/init_ebpf.p4 (Failed)
702 - ebpf-kernel/testdata/p4_16_samples/issue2791_ebpf.p4 (Failed)
703 - ebpf-kernel/testdata/p4_16_samples/issue2793_ebpf.p4 (Failed)
704 - ebpf-kernel/testdata/p4_16_samples/issue2797_ebpf.p4 (Failed)
705 - ebpf-kernel/testdata/p4_16_samples/issue2816-1_ebpf.p4 (Failed)
706 - ebpf-kernel/testdata/p4_16_samples/issue2816_ebpf.p4 (Failed)
707 - ebpf-kernel/testdata/p4_16_samples/issue870_ebpf.p4 (Failed)
708 - ebpf-kernel/testdata/p4_16_samples/key-issue-1020_ebpf.p4 (Failed)
709 - ebpf-kernel/testdata/p4_16_samples/key_ebpf.p4 (Failed)
711 - ebpf-kernel/testdata/p4_16_samples/stack_ebpf.p4 (Failed)
712 - ebpf-kernel/testdata/p4_16_samples/switch_ebpf.p4 (Failed)
713 - ebpf-kernel/testdata/p4_16_samples/test_ebpf.p4 (Failed)
714 - ebpf-kernel/testdata/p4_16_samples/two_ebpf.p4 (Failed)
715 - ebpf-kernel/testdata/p4_16_samples/valid_ebpf.p4 (Failed)
716 - ebpf-kernel/testdata/p4_16_samples/ebpf_conntrack_extern.p4 (Failed)
717 - ebpf-kernel/testdata/p4_16_samples/ebpf_checksum_extern.p4 (Failed)
Errors while running CTest
make[3]: *** [CMakeFiles/check-all.dir/build.make:60: CMakeFiles/check-all] Error 8
make[2]: *** [CMakeFiles/Makefile2:746: CMakeFiles/check-all.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:510: CMakeFiles/check.dir/rule] Error 2
make: *** [Makefile:223: check] Error 2

I'm not sure if I made my point clearly, may I ask if this is because some dependency files are not downloaded?

[mihaibudiu]

There are several different things that you are reporting here.
The first one is indeed a bug in the p4c-xdp compiler, and should be reported as an issue in that repository: https://github.com/vmware/p4c-xdp
The second one is that your ebpf kernel-based tests are failing.
There are certain requirements for the ebpf tests to run on your machine, these have been tested only with some specific versions of the Linux kernel. @fruffy do you remember what kernels we tested this on?

[mihaibudiu]

So test17 from xdp was indeed a bug in the ebpf back-end (which is extended by the xdp backend).
I have submitted a fix which should take care of xdp17.
This should have been caught sooner.

[fruffy]

We were using Ubuntu 16.04 and were testing these on the 4.15 kernel. It might make sense to update the XDP CI, too, now that p4c runs with Ubuntu 20.04.

[mihaibudiu]

There are some checks in the makefiles that will skip the kernel tests if the versions are not known to work.
I don't know if the tests will work with newer kernels.
The main problem is the ebpf header file which we hardwired and which may not match the kernel version.
I think there are better ways to do this now.

[fruffy]

Are you referring to the ebpf_kernel.h file? I remember switching to libbpf to be largely independent of the distro-provided header files. But there are still some version issues left. For example the version if iproute2 matters because we have to load our programs with it (see #2407). I should revisit this.

[KekeZheng]

Thanks for answering my questions,as i was busy in having lesson today,so i didn't reply you for a long time,i will try to run it in ubuntu16.04 tomorrow.