Skip to content

Commit 75c2a67

Browse files
yonghong-songyonghong-song
committed
[BPF] add new intrinsics preserve_{array,union,struct}_access_index
For background of BPF CO-RE project, please refer to http://vger.kernel.org/bpfconf2019.html In summary, BPF CO-RE intends to compile bpf programs adjustable on struct/union layout change so the same program can run on multiple kernels with adjustment before loading based on native kernel structures. In order to do this, we need keep track of GEP(getelementptr) instruction base and result debuginfo types, so we can adjust on the host based on kernel BTF info. Capturing such information as an IR optimization is hard as various optimization may have tweaked GEP and also union is replaced by structure it is impossible to track fieldindex for union member accesses. Three intrinsic functions, preserve_{array,union,struct}_access_index, are introducted. addr = preserve_array_access_index(base, index, dimension) addr = preserve_union_access_index(base, di_index) addr = preserve_struct_access_index(base, gep_index, di_index) here, base: the base pointer for the array/union/struct access. index: the last access index for array, the same for IR/DebugInfo layout. dimension: the array dimension. gep_index: the access index based on IR layout. di_index: the access index based on user/debuginfo types. For example, for the following example, $ cat test.c struct sk_buff { int i; int b1:1; int b2:2; union { struct { int o1; int o2; } o; struct { char flags; char dev_id; } dev; int netid; } u[10]; }; static int (*bpf_probe_read)(void *dst, int size, const void *unsafe_ptr) = (void *) 4; #define _(x) (__builtin_preserve_access_index(x)) int bpf_prog(struct sk_buff *ctx) { char dev_id; bpf_probe_read(&dev_id, sizeof(char), _(&ctx->u[5].dev.dev_id)); return dev_id; } $ clang -target bpf -O2 -g -emit-llvm -S -mllvm -print-before-all \ test.c >& log The generated IR looks like below: ... define dso_local i32 @bpf_prog(%struct.sk_buff*) #0 !dbg !15 { %2 = alloca %struct.sk_buff*, align 8 %3 = alloca i8, align 1 store %struct.sk_buff* %0, %struct.sk_buff** %2, align 8, !tbaa !45 call void @llvm.dbg.declare(metadata %struct.sk_buff** %2, metadata !43, metadata !DIExpression()), !dbg !49 call void @llvm.lifetime.start.p0i8(i64 1, i8* %3) #4, !dbg !50 call void @llvm.dbg.declare(metadata i8* %3, metadata !44, metadata !DIExpression()), !dbg !51 %4 = load i32 (i8*, i32, i8*)*, i32 (i8*, i32, i8*)** @bpf_probe_read, align 8, !dbg !52, !tbaa !45 %5 = load %struct.sk_buff*, %struct.sk_buff** %2, align 8, !dbg !53, !tbaa !45 %6 = call [10 x %union.anon]* @llvm.preserve.struct.access.index.p0a10s_union.anons.p0s_struct.sk_buffs( %struct.sk_buff* %5, i32 2, i32 3), !dbg !53, !llvm.preserve.access.index !19 %7 = call %union.anon* @llvm.preserve.array.access.index.p0s_union.anons.p0a10s_union.anons( [10 x %union.anon]* %6, i32 1, i32 5), !dbg !53 %8 = call %union.anon* @llvm.preserve.union.access.index.p0s_union.anons.p0s_union.anons( %union.anon* %7, i32 1), !dbg !53, !llvm.preserve.access.index !26 %9 = bitcast %union.anon* %8 to %struct.anon.0*, !dbg !53 %10 = call i8* @llvm.preserve.struct.access.index.p0i8.p0s_struct.anon.0s( %struct.anon.0* %9, i32 1, i32 1), !dbg !53, !llvm.preserve.access.index !34 %11 = call i32 %4(i8* %3, i32 1, i8* %10), !dbg !52 %12 = load i8, i8* %3, align 1, !dbg !54, !tbaa !55 %13 = sext i8 %12 to i32, !dbg !54 call void @llvm.lifetime.end.p0i8(i64 1, i8* %3) #4, !dbg !56 ret i32 %13, !dbg !57 } !19 = distinct !DICompositeType(tag: DW_TAG_structure_type, name: "sk_buff", file: !3, line: 1, size: 704, elements: !20) !26 = distinct !DICompositeType(tag: DW_TAG_union_type, scope: !19, file: !3, line: 5, size: 64, elements: !27) !34 = distinct !DICompositeType(tag: DW_TAG_structure_type, scope: !26, file: !3, line: 10, size: 16, elements: !35) Note that @llvm.preserve.{struct,union}.access.index calls have metadata llvm.preserve.access.index attached to instructions to provide struct/union debuginfo type information. For &ctx->u[5].dev.dev_id, . The "%6 = ..." represents struct member "u" with index 2 for IR layout and index 3 for DI layout. . The "%7 = ..." represents array subscript "5". . The "%8 = ..." represents union member "dev" with index 1 for DI layout. . The "%10 = ..." represents struct member "dev_id" with index 1 for both IR and DI layout. Basically, traversing the use-def chain recursively for the 3rd argument of bpf_probe_read() and examining all preserve_*_access_index calls, the debuginfo struct/union/array access index can be achieved. The intrinsics also contain enough information to regenerate codes for IR layout. For array and structure intrinsics, the proper GEP can be constructed. For union intrinsics, replacing all uses of "addr" with "base" should be enough. Signed-off-by: Yonghong Song <yhs@fb.com> Differential Revision: https://reviews.llvm.org/D61810 llvm-svn: 365352
1 parent 81db9f5 commit 75c2a67

File tree

5 files changed

+189
-1
lines changed

5 files changed

+189
-1
lines changed

‎llvm/docs/LangRef.rst

+103
Original file line numberDiff line numberDiff line change
@@ -17304,3 +17304,106 @@ Lowering:
1730417304
"""""""""
1730517305

1730617306
Lowers to a call to `objc_storeWeak <https://clang.llvm.org/docs/AutomaticReferenceCounting.html#arc-runtime-objc-storeweak>`_.
17307+
17308+
Preserving Debug Information Intrinsics
17309+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
17310+
17311+
These intrinsics are used to carry certain debuginfo together with
17312+
IR-level operations. For example, it may be desirable to
17313+
know the structure/union name and the original user-level field
17314+
indices. Such information got lost in IR GetElementPtr instruction
17315+
since the IR types are different from debugInfo types and unions
17316+
are converted to structs in IR.
17317+
17318+
'``llvm.preserve.array.access.index``' Intrinsic
17319+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
17320+
17321+
Syntax:
17322+
"""""""
17323+
::
17324+
17325+
declare <type2>
17326+
@llvm.preserve.array.access.index.p0s_union.anons.p0a10s_union.anons(<type> base,
17327+
i32 dim,
17328+
i32 index)
17329+
17330+
Overview:
17331+
"""""""""
17332+
17333+
The '``llvm.preserve.array.access.index``' intrinsic returns the getelementptr address
17334+
based on array base ``base``, array dimension ``dim`` and the last access index ``index``
17335+
into the array.
17336+
17337+
Arguments:
17338+
""""""""""
17339+
17340+
The ``base`` is the array base address. The ``dim`` is the array dimension.
17341+
The ``base`` is a pointer if ``dim`` equals 0.
17342+
The ``index`` is the last access index into the array or pointer.
17343+
17344+
Semantics:
17345+
""""""""""
17346+
17347+
The '``llvm.preserve.array.access.index``' intrinsic produces the same result
17348+
as a getelementptr with base ``base`` and access operands ``{dim's 0's, index}``.
17349+
17350+
'``llvm.preserve.union.access.index``' Intrinsic
17351+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
17352+
17353+
Syntax:
17354+
"""""""
17355+
::
17356+
17357+
declare <type>
17358+
@llvm.preserve.union.access.index.p0s_union.anons.p0s_union.anons(<type> base,
17359+
i32 di_index)
17360+
17361+
Overview:
17362+
"""""""""
17363+
17364+
The '``llvm.preserve.union.access.index``' intrinsic carries the debuginfo field index
17365+
``di_index`` and returns the ``base`` address.
17366+
The ``llvm.preserve.access.index`` type of metadata is attached to this call instruction
17367+
to provide union debuginfo type.
17368+
17369+
Arguments:
17370+
""""""""""
17371+
17372+
The ``base`` is the union base address. The ``di_index`` is the field index in debuginfo.
17373+
17374+
Semantics:
17375+
""""""""""
17376+
17377+
The '``llvm.preserve.union.access.index``' intrinsic returns the ``base`` address.
17378+
17379+
'``llvm.preserve.struct.access.index``' Intrinsic
17380+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
17381+
17382+
Syntax:
17383+
"""""""
17384+
::
17385+
17386+
declare <type2>
17387+
@llvm.preserve.struct.access.index.p0i8.p0s_struct.anon.0s(<type> base,
17388+
i32 gep_index,
17389+
i32 di_index)
17390+
17391+
Overview:
17392+
"""""""""
17393+
17394+
The '``llvm.preserve.struct.access.index``' intrinsic returns the getelementptr address
17395+
based on struct base ``base`` and IR struct member index ``gep_index``.
17396+
The ``llvm.preserve.access.index`` type of metadata is attached to this call instruction
17397+
to provide struct debuginfo type.
17398+
17399+
Arguments:
17400+
""""""""""
17401+
17402+
The ``base`` is the structure base address. The ``gep_index`` is the struct member index
17403+
based on IR structures. The ``di_index`` is the struct member index based on debuginfo.
17404+
17405+
Semantics:
17406+
""""""""""
17407+
17408+
The '``llvm.preserve.struct.access.index``' intrinsic produces the same result
17409+
as a getelementptr with base ``base`` and access operands ``{0, gep_index}``.

‎llvm/include/llvm/IR/IRBuilder.h

+68
Original file line numberDiff line numberDiff line change
@@ -2453,6 +2453,74 @@ class IRBuilder : public IRBuilderBase, public Inserter {
24532453
return V;
24542454
}
24552455

2456+
Value *CreatePreserveArrayAccessIndex(Value *Base, unsigned Dimension,
2457+
unsigned LastIndex) {
2458+
assert(isa<PointerType>(Base->getType()) &&
2459+
"Invalid Base ptr type for preserve.array.access.index.");
2460+
auto *BaseType = Base->getType();
2461+
2462+
Value *LastIndexV = getInt32(LastIndex);
2463+
Constant *Zero = ConstantInt::get(Type::getInt32Ty(Context), 0);
2464+
SmallVector<Value *, 4> IdxList;
2465+
for (unsigned I = 0; I < Dimension; ++I)
2466+
IdxList.push_back(Zero);
2467+
IdxList.push_back(LastIndexV);
2468+
2469+
Type *ResultType =
2470+
GetElementPtrInst::getGEPReturnType(Base, IdxList);
2471+
2472+
Module *M = BB->getParent()->getParent();
2473+
Function *FnPreserveArrayAccessIndex = Intrinsic::getDeclaration(
2474+
M, Intrinsic::preserve_array_access_index, {ResultType, BaseType});
2475+
2476+
Value *DimV = getInt32(Dimension);
2477+
CallInst *Fn =
2478+
CreateCall(FnPreserveArrayAccessIndex, {Base, DimV, LastIndexV});
2479+
2480+
return Fn;
2481+
}
2482+
2483+
Value *CreatePreserveUnionAccessIndex(Value *Base, unsigned FieldIndex,
2484+
MDNode *DbgInfo) {
2485+
assert(isa<PointerType>(Base->getType()) &&
2486+
"Invalid Base ptr type for preserve.union.access.index.");
2487+
auto *BaseType = Base->getType();
2488+
2489+
Module *M = BB->getParent()->getParent();
2490+
Function *FnPreserveUnionAccessIndex = Intrinsic::getDeclaration(
2491+
M, Intrinsic::preserve_union_access_index, {BaseType, BaseType});
2492+
2493+
Value *DIIndex = getInt32(FieldIndex);
2494+
CallInst *Fn =
2495+
CreateCall(FnPreserveUnionAccessIndex, {Base, DIIndex});
2496+
Fn->setMetadata(LLVMContext::MD_preserve_access_index, DbgInfo);
2497+
2498+
return Fn;
2499+
}
2500+
2501+
Value *CreatePreserveStructAccessIndex(Value *Base, unsigned Index,
2502+
unsigned FieldIndex, MDNode *DbgInfo) {
2503+
assert(isa<PointerType>(Base->getType()) &&
2504+
"Invalid Base ptr type for preserve.struct.access.index.");
2505+
auto *BaseType = Base->getType();
2506+
2507+
Value *GEPIndex = getInt32(Index);
2508+
Constant *Zero = ConstantInt::get(Type::getInt32Ty(Context), 0);
2509+
Type *ResultType =
2510+
GetElementPtrInst::getGEPReturnType(Base, {Zero, GEPIndex});
2511+
2512+
Module *M = BB->getParent()->getParent();
2513+
Function *FnPreserveStructAccessIndex = Intrinsic::getDeclaration(
2514+
M, Intrinsic::preserve_struct_access_index, {ResultType, BaseType});
2515+
2516+
Value *DIIndex = getInt32(FieldIndex);
2517+
CallInst *Fn = CreateCall(FnPreserveStructAccessIndex,
2518+
{Base, GEPIndex, DIIndex});
2519+
Fn->setMetadata(LLVMContext::MD_preserve_access_index, DbgInfo);
2520+
2521+
return Fn;
2522+
}
2523+
24562524
private:
24572525
/// Helper function that creates an assume intrinsic call that
24582526
/// represents an alignment assumption on the provided Ptr, Mask, Type

‎llvm/include/llvm/IR/Intrinsics.td

+16-1
Original file line numberDiff line numberDiff line change
@@ -1040,7 +1040,6 @@ def int_clear_cache : Intrinsic<[], [llvm_ptr_ty, llvm_ptr_ty],
10401040
// Intrinsic to detect whether its argument is a constant.
10411041
def int_is_constant : Intrinsic<[llvm_i1_ty], [llvm_any_ty], [IntrNoMem], "llvm.is.constant">;
10421042

1043-
10441043
//===-------------------------- Masked Intrinsics -------------------------===//
10451044
//
10461045
def int_masked_store : Intrinsic<[], [llvm_anyvector_ty,
@@ -1214,6 +1213,22 @@ def int_loop_decrement_reg :
12141213

12151214
def int_ssa_copy : Intrinsic<[llvm_any_ty], [LLVMMatchType<0>],
12161215
[IntrNoMem, Returned<0>]>;
1216+
1217+
//===------- Intrinsics that are used to preserve debug information -------===//
1218+
1219+
def int_preserve_array_access_index : Intrinsic<[llvm_anyptr_ty],
1220+
[llvm_anyptr_ty, llvm_i32_ty,
1221+
llvm_i32_ty],
1222+
[IntrNoMem, ImmArg<1>, ImmArg<2>]>;
1223+
def int_preserve_union_access_index : Intrinsic<[llvm_anyptr_ty],
1224+
[llvm_anyptr_ty, llvm_i32_ty],
1225+
[IntrNoMem, ImmArg<1>]>;
1226+
def int_preserve_struct_access_index : Intrinsic<[llvm_anyptr_ty],
1227+
[llvm_anyptr_ty, llvm_i32_ty,
1228+
llvm_i32_ty],
1229+
[IntrNoMem, ImmArg<1>,
1230+
ImmArg<2>]>;
1231+
12171232
//===----------------------------------------------------------------------===//
12181233
// Target-specific intrinsics
12191234
//===----------------------------------------------------------------------===//

‎llvm/include/llvm/IR/LLVMContext.h

+1
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,7 @@ class LLVMContext {
9999
MD_irr_loop = 24, // "irr_loop"
100100
MD_access_group = 25, // "llvm.access.group"
101101
MD_callback = 26, // "callback"
102+
MD_preserve_access_index = 27, // "llvm.preserve.*.access.index"
102103
};
103104

104105
/// Known operand bundle tag IDs, which always have the same value. All

‎llvm/lib/IR/LLVMContext.cpp

+1
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@ LLVMContext::LLVMContext() : pImpl(new LLVMContextImpl(*this)) {
6363
{MD_irr_loop, "irr_loop"},
6464
{MD_access_group, "llvm.access.group"},
6565
{MD_callback, "callback"},
66+
{MD_preserve_access_index, "llvm.preserve.access.index"},
6667
};
6768

6869
for (auto &MDKind : MDKinds) {

0 commit comments

Comments
 (0)