Support `execve` exit events and `clone` child exit events on `ARM64` #416

Andreagit97 · 2022-06-21T13:42:16Z

What type of PR is this?

/kind bug

/kind feature

Any specific area of the project related to this PR?

/area driver-kmod

/area driver-bpf

/area libscap-engine-bpf

What this PR does / why we need it:

This PR wants to fix a known problem regarding ARM64 architecture, described here in this thread of the kernel mailing list. In a nutshell the exeve exit event is not traced by the syscalls:sys_exit tracepoint and same thing for the clone child exit event. These 2 events are really important for the state of the so we implemented this workaround to catch this 2 missing events. We have instrumented 2 new tracepoints:

sched_proc_exec to catch the execve exit events. This tracepoint sends the same event PPME_SYSCALL_EXECVE_19_X to userspace, so the consumer won't notice any difference. This new tracepoint calls a new filler that is really similar to our actual proc_startupdate filler.
sched_proc_fork to catch the clone child exit events. Again we send the same event PPME_SYSCALL_CLONE_20_X and we have a new filler. The only change seen by userspace is that the clone child exit event will be received before the clone parent exit event, but this shouldn't be a problem since libsinsp already has the code to manage this situation.

Some important notes:

This solution is only for ARM64 and it is completely compiled out on other supported architectures (x86)
The only pieces of information we lose are the followings:
- the clone child exit event has no flags information so param number 16 is always 0.
- the clone child exit event gets euid and guid directly from the task struct without relating them to the user namespace (param numbers 17 and 18)
- even if the system throws a clone3 the child exit event will be of type clone and not clone3 because we are not able to distinguish kernel side which syscall has been called. In the end, this doesn't change so much since the 2 events follow the same path both kernel side and userspace side.
fork and vfork are not defined in ARM64 so we don't lose information.
In case the execve or clone syscalls fail, we catch the exit events with our sys_exit tracepoint as usual, so we don't lose information at all in this case.

Compatibility:

BPF: we need kernel version >=4.17 since we need bpf raw_tracepoints programs to catch this new information. We lose 3 supported versions (x86 support BPF starting from 4.14) but we have no alternatives with this approach. If we want to support them we have to use kprobes.
KERNEL MODULE: we need kernel versions >=3.4 since sched/sched_process_exec is available only kernels >= 3.4

Kernel engineers are working to find solution as you can see here , in the meanwhile this could be a possible solution, let me know what to you think about that.

Which issue(s) this PR fixes:

Special notes for your reviewer:

Next steps:

add further comments
test it on more machines (any help is well accepted).
remove the test commits since they are only useful to test the new patch (I removed the DCO from them, so we cannot merge the PR)
add logic to catch clone child flags.
remove the [WIP] tag.

--> One last note
We compiled out also the page_fault tracepoints both from BPF and kernel module since they are not supported on ARM64

Does this PR introduce a user-facing change?:

fix: enable `execve` exit events and `clone` child exit events on `ARM64` architecture

Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it> Co-authored-by: Federico Di Pierro <nierro92@gmail.com>

We implemented this tracepoint to catch `execve` exit events on `arm64` Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

Removing this check, we can compile out fillers that are not used in that specific architecture Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

This tracepoint catches `clone` child exit events on `arm64` Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

This tracepoint catches `execve` exit events on `arm64` Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

This tracepoint catches `clone` child exit events on `arm64` Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

We are not interested in catching information about kernel threads Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

Andreagit97 · 2022-06-21T13:45:09Z

I removed the DCO from the last two commits since we have to remove them before merging the PR

driver/main.c

userspace/libscap/scap-int.h

FedeDP · 2022-06-22T09:18:18Z

What about adding a README section with supported architectures and their kernel versions requirements for probe and kmod?
Thank you!

Andreagit97 · 2022-06-22T10:31:22Z

What about adding a README section with supported architectures and their kernel versions requirements for probe and kmod? Thank you!

I will open another PR, with also a Cmake check on supported architectures, if this is ok for you :)

Andreagit97 · 2022-06-22T14:20:36Z

I will open another PR, with also a Cmake check on supported architectures, if this is ok for you :)

This is the related pull request -> #421

FedeDP

/approve

poiana · 2022-06-27T07:10:18Z

LGTM label has been added.

Git tree hash: a3b9f13fa4f2464e93f57d41d72e82d8f63d146d

Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

FedeDP · 2022-06-28T07:07:44Z

/test build-libs-bundled-deps

FedeDP

/approve

poiana · 2022-06-28T07:09:11Z

LGTM label has been added.

Git tree hash: 813e46e1d98b6cb7a22dcdfbe0eddc61d0608360

Andreagit97 · 2022-06-28T07:53:39Z

driver/bpf/fillers.h

+	if(pidns_level != 0)
+	{
+		flags |= PPM_CL_CHILD_IN_PIDNS;
+	}


In the end, we want to detect every child that is spawned in a PID namespace different from the init one so we can keep only this logic in the if statement

Andreagit97 · 2022-06-28T20:01:20Z

Hi @gnosek, are these changes ok for you?

deepskyblue86 · 2022-06-29T16:11:56Z

CC @jcpittman144, PTAL

leogr

LGTM

gnosek · 2022-06-30T14:45:38Z

/approve

poiana · 2022-06-30T14:45:46Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Andreagit97, FedeDP, gnosek

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [Andreagit97,FedeDP,gnosek]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…falcosecurity#93) - Falcosecurity/libs falcosecurity#416: Support execve exit and clone child exit events on ARM64 - Falcosecurity/libs falcosecurity#418: Enable 64BIT_ARGS_SINGLE_REGISER on ARM64 - Also, disable userspace workarounds ARM, which attempted to compensate for the missing execve/clone exit events

Andreagit97 and others added 11 commits June 21, 2022 13:34

new(bpf): add support for raw_tracepoint in ARM64

3a92cb0

Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it> Co-authored-by: Federico Di Pierro <nierro92@gmail.com>

new(bpf): add sched_process_exec raw tracepoint

be0774c

We implemented this tracepoint to catch `execve` exit events on `arm64` Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

cleanup(scap_bpf_engine): remove useless check on fillers implementation

a185cca

Removing this check, we can compile out fillers that are not used in that specific architecture Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

new(bpf): add sched_process_fork raw tracepoint

90fc567

This tracepoint catches `clone` child exit events on `arm64` Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

new(kmod): add sched_process_exec tracepoint

00d1650

This tracepoint catches `execve` exit events on `arm64` Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

new(kmod): add sched_process_fork tracepoint

272c3d8

This tracepoint catches `clone` child exit events on `arm64` Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

update(bpf, kmod): ignore kernel threads

a10b23e

We are not interested in catching information about kernel threads Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

cleanup(kmod): use kernel config CONFIG_ARM64 to compile out

46309ed

Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

cleanup(bpf): use clang pre-processor macros to compile out

8f94ab9

Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

cleanup(scap_bpf_engine): remove duplicate definitions of BPF macros

0c7fa56

Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

update(kmod): compile out page_fault tracepoints in ARM64

865a9a4

Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

poiana added do-not-merge/work-in-progress release-note kind/bug Something isn't working dco-signoff: no kind/feature New feature or request area/driver-kmod area/driver-bpf area/libscap-engine-bpf size/XXL labels Jun 21, 2022

poiana requested review from gnosek and mstemm June 21, 2022 13:43

poiana added the approved label Jun 21, 2022

FedeDP reviewed Jun 21, 2022

View reviewed changes

driver/main.c Show resolved Hide resolved

FedeDP reviewed Jun 21, 2022

View reviewed changes

driver/main.c Outdated Show resolved Hide resolved

FedeDP reviewed Jun 21, 2022

View reviewed changes

userspace/libscap/scap-int.h Show resolved Hide resolved

Andreagit97 changed the title ~~[WIP] Support execve exit events and clone child exit events on ARM64~~ Support execve exit events and clone child exit events on ARM64 Jun 24, 2022

poiana removed the do-not-merge/work-in-progress label Jun 24, 2022

FedeDP previously approved these changes Jun 27, 2022

View reviewed changes

poiana assigned FedeDP Jun 27, 2022

poiana added the lgtm label Jun 27, 2022

Andreagit97 added 2 commits June 27, 2022 11:57

refactor(bpf,kmod): rename child struct into child

f38eb4b

Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

update(bpf,kmod): check pidns!=init_ns

9154d36

Signed-off-by: Andrea Terzolo <andrea.terzolo@polito.it>

Andreagit97 dismissed FedeDP’s stale review via 9154d36 June 27, 2022 20:41

poiana removed the lgtm label Jun 27, 2022

poiana requested a review from FedeDP June 27, 2022 20:41

FedeDP approved these changes Jun 28, 2022

View reviewed changes

poiana added the lgtm label Jun 28, 2022

Andreagit97 commented Jun 28, 2022

View reviewed changes

FedeDP mentioned this pull request Jun 28, 2022

new(falco): add gVisor support falcosecurity/falco#2078

Merged

leogr approved these changes Jun 30, 2022

View reviewed changes

poiana assigned leogr Jun 30, 2022

poiana merged commit 010bf3c into master Jun 30, 2022

poiana deleted the fix_arm_registers branch June 30, 2022 15:47

FedeDP mentioned this pull request Jul 1, 2022

fix(test): fixed arm64 tests: eBPF probe requirement is now kernel >= 4.17 falcosecurity/driverkit#178

Merged

Andreagit97 mentioned this pull request Jul 4, 2022

cleanup(drivers): introduce CAPTURE_SCHED_PROC_EXEC and CAPTURE_SCHED_PROC_FORK feature gates #452

Merged

Andreagit97 mentioned this pull request Jul 23, 2022

fix(kmod): support clone child exit events on s390x #500

Merged

incertum mentioned this pull request Sep 20, 2022

[FEATURE] Options available for tapping into "linux_binprm" that holds args used when loading binaries #621

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support `execve` exit events and `clone` child exit events on `ARM64` #416

Support `execve` exit events and `clone` child exit events on `ARM64` #416

Andreagit97 commented Jun 21, 2022 •

edited

Loading

Andreagit97 commented Jun 21, 2022

FedeDP commented Jun 22, 2022

Andreagit97 commented Jun 22, 2022

Andreagit97 commented Jun 22, 2022

FedeDP left a comment

poiana commented Jun 27, 2022

FedeDP commented Jun 28, 2022

FedeDP left a comment

poiana commented Jun 28, 2022

Andreagit97 Jun 28, 2022

Andreagit97 commented Jun 28, 2022

deepskyblue86 commented Jun 29, 2022

leogr left a comment

gnosek commented Jun 30, 2022

poiana commented Jun 30, 2022

Support execve exit events and clone child exit events on ARM64 #416

Support execve exit events and clone child exit events on ARM64 #416

Conversation

Andreagit97 commented Jun 21, 2022 • edited Loading

Andreagit97 commented Jun 21, 2022

FedeDP commented Jun 22, 2022

Andreagit97 commented Jun 22, 2022

Andreagit97 commented Jun 22, 2022

FedeDP left a comment

Choose a reason for hiding this comment

poiana commented Jun 27, 2022

FedeDP commented Jun 28, 2022

FedeDP left a comment

Choose a reason for hiding this comment

poiana commented Jun 28, 2022

Andreagit97 Jun 28, 2022

Choose a reason for hiding this comment

Andreagit97 commented Jun 28, 2022

deepskyblue86 commented Jun 29, 2022

leogr left a comment

Choose a reason for hiding this comment

gnosek commented Jun 30, 2022

poiana commented Jun 30, 2022

Support `execve` exit events and `clone` child exit events on `ARM64` #416

Support `execve` exit events and `clone` child exit events on `ARM64` #416

Andreagit97 commented Jun 21, 2022 •

edited

Loading