R? min value is negative when calling ipv4_l4_csum

[sbernard31]

Hi,

I played with bpf/xdp to prototype an UDP loadbalancer.
For UDP checksum, I reused your ipv4_l4_csum function, but I fall in a verifier issue :

R? min value is negative, either use unsigned or 'var &= const' 

We currently discussing about this tough issue on bcc mailing list.
I would like to know if you already faced this kind of issue and you have some input about that ?

Thx

[tehnerd]

from comments in bcc thread looks like this is because of reg spilling (thanks @yonghong-song for debugging). usually on old kernels (w/o support for non-inline function calls) some moving code around helps. but you have mentioned that you are flexible w/ kernel rebuild - in your case i'd try on some more modern kernel (instead of 4.15 from ubuntu). bassically the one which do supports non-inlining for functions, and retry w/ removing "inline" modifier w/ this function

[yonghong-song]

The bcc does not have a good support for func-to-func calls. Something like what libbpf did (reloc text from one place to another) will be needed in bcc to support func-to-func calls...

[sbernard31]

@tehnerd,

in your case i'd try on some more modern kernel (instead of 4.15 from ubuntu)

I'm using a 4.19.16. Is it recent enough ? I tried to remove (inline or __attribute__((__always_inline__)) or both) without any success. I also tried to inline the function myself :p.

@yonghong-song,

The bcc does not have a good support for func-to-func calls

There is something which is not clear to me.
There is several tools which let us attach/load xdp program. (bcc, iproute2, ...) In my mind all those tools used the same underlying library (e.g. libbpf to do that). But your remark confuse me a little. Loading an xdp program with bcc is not the same as loading it with iproute2 ?

This bring me to a second question : compilation. Bpf code is generally build with clang(maybe gcc too?). When I use bcc, which version of clang does it use ? I look at debian package dependency and I see nothing about clang. But looking at the bcc code, I pretty sure bcc uses clang API and not call clang command line. I don't think clang is build statically with bcc. So maybe clang, is a kind of optional dependencies ? but in my case I have 2 clang/llvm version installed : v3.8 and v6.0. Which one is used ? Does bcc generate same bpf bytecode than if I directly used clang ? I mean is there any chance that my programs works if I directly use clang to compile and iproute2 to attach it ?
(I know those last 2 questions are a bit out of topic, I can ask it on bcc github or mailing list if you prefer)

[yonghong-song]

libbpf did some relocation if functions are in different sections. The bcc did not do this yet. It is certainly doable.

bpf code is built with clang/llvm only. which clang is used dependence your path, you can check bcc cmake file to find more. Yes, bcc generates different byte codes then clang. bcc does some rewriting, using bpf_pseudo etc. Feel free to use bcc debug flags to find out more about bcc internals. bcc won't produce an elf file, you cannot load bcc generated binary with iproute2 utility.

[tehnerd]

I’d add that In my opinion bcc is well suited for tracing related progs. But for networking I’d prefer direct use of clang over bcc any day

[sbernard31]

@tehnerd I will try to rewrite my prototype without bcc.

• my bpf program will be written in C. (pretty much like I did without bcc helpers)
• for my userspace program used to configure MAP or get some statistics. Is there any way to use a script language (without writing binding myself) ? (based on my research, I guess there is nothing like this...)

@yonghong-song about clang version, I didn't build bcc myself. I use the debian package and after more investigation I believe that clang is statically in libbpfcc and so I didn't know which version of clang is used to generate my bpf bytecode...

Thx guys for your time.

[tehnerd]

@sbernard31 yeah, i'm not aware about any scripting lang for bpf which wont use bcc, unfortunately