[Auditbeat] Unable to start; unable to guess one or more required parameters: guess_ip_local_out failed #18755
Comments
botelastic
bot
added
the
needs_team
|
Pinging @elastic/siem (Team:SIEM) |
botelastic
bot
removed
the
needs_team
|
to finally be able to test auditbeat on my Fedora 32 machine I had to disable "socket"-stuff completely by setting |
|
I can confirm similar behaviour on Ubuntu 20.04 Server LTS. |
|
Still broken in 7.8 |
|
I'm running into this as well on CentOS 8.1, 5.6 kernel. |
|
Same on Kali Linux 2020.3 (Debian) using Auditbeat 7.9.2. |
|
same on centos 7.8 with auditbeat 7.8.1 and 7.9.2 |
|
Same on Kali 2020.3 with Auditbeat 7.9.3 |
|
Thanks all for your reports. Can you please provide the kernel version ( Also, does it always fail for the same "guess", or do you get different guess names in the error:
Does adding - module: system
datasets:
- host # General host information, e.g. uptime, IPs
- login # User logins, logouts, and system boots.
- process # Started and stopped processes
- socket # Opened and closed sockets
- user # User information
# How often datasets send state updates with the
# current state of the system (e.g. all currently
# running processes, all open sockets).
state.period: 12h
# Enabled by default. Auditbeat will read password fields in
# /etc/passwd and /etc/shadow and store a hash locally to
# detect any changes.
user.detect_password_changes: true
# File patterns of the login record files.
login.wtmp_file_pattern: /var/log/wtmp*
login.btmp_file_pattern: /var/log/btmp*
+ socket.guess_timeout: 1m |
5.8.3-arch-1-1 Added in socket.guess_timeout: 1m |
|
Uninstall old Packeage with dpkg -P auditbeat. Then reinstall works for me! Then the service start. root@hostname:/etc/auditbeat# auditbeat test config With Kernel |
|
There's a few things that can be going wrong:
|
|
Same error on Fedora 33, kernel 5.8.16-300.fc33.x86_64 and auditbeat 7.9.3. I also added |
cat /sys/kernel/debug/kprobes/enabled system/socket is being run from auditbeat.yml |
|
@adriansr And under Auditbeat 7.7.0 it crashed immediately: However under 7.10.0 it didn't crash: |
|
@legoguy1000 @Aqualie @xennn |
|
Finished upgrading both kernel and auditbeat still having the same error:
sysctl if it makes a difference: |
|
Also to add this machine has multiple vlan's with a bridged interface (same network configuration I've had since using auditbeat since version 6) |
|
@Aqualie And last but not least it might be helpful to understand how this network config looks like: |
|
Had to modify the output abit as I'm using zsh and directory ownership is different: |
|
@Aqualie |
|
pacman -Q | grep linux cat /etc/sysctl.d/99-sysctl.conf | grep ipv6 ip -d addr show |
|
Thanks @adriansr this works perfectly and I can see all the socket events in ES! |
4 tasks
|
Nice to hear, that it's working for you now @Aqualie . |
|
Here's an updated snapshot build with an additional fix: https://drive.google.com/drive/u/0/folders/1HJVrTF2iG45vYqSdBDcSynZjiiGSm31w It includes:
|
I haven't done anything with it in a while. The project I was working on was finished and we just left it out. |
|
I give you next week a update. I'm out of office for this week |
adriansr
added a commit
to adriansr/beats
that referenced
this issue
Dec 2, 2020
Auditbeat's system/socket dataset needs to install kprobes on all online CPUs. Previously, it was using runtime.NumCPU() to determine the CPUs in the system, and monitoring CPUs 0 to NumCPU. This was a mistake that lead to startup failures or loss of events in any of the following scenarios: - When Auditbeat is started with a CPU affinity mask that excludes some CPUs - When there are offline CPUs in the system. This patch updates the tracing library in Auditbeat to fetch the list of online CPUs from /sys/devices/system/cpu/online so that it can install kprobes in all of them regardless of its own affinity mask, and correctly skipping offline CPUs. Related elastic#18755
adriansr
added a commit
that referenced
this issue
Dec 2, 2020
This commit adds a new function alternative, `__ip_local_out` for selecting a proper ip_local_out function, and fixes `guess_ip_local_out` logic in order to account for this new function. The new order of precedence is: - ip_local_out_sk (kernels before 3.16) - __ip_local_out (for kernels where ip_local_out calls are inlined) - ip_local_out (all others). Relates #18755
adriansr
added a commit
that referenced
this issue
Dec 2, 2020
Auditbeat's system/socket dataset needs to install kprobes on all online CPUs. Previously, it was using runtime.NumCPU() to determine the CPUs in the system, and monitoring CPUs 0 to NumCPU. This was a mistake that lead to startup failures or loss of events in any of the following scenarios: - When Auditbeat is started with a CPU affinity mask that excludes some CPUs - When there are offline or isolated CPUs in the system. This patch updates the tracing library in Auditbeat to fetch the list of online CPUs from /sys/devices/system/cpu/online so that it can install kprobes in all of them regardless of its own affinity mask, and correctly skipping offline CPUs. Related #18755
3 tasks
adriansr
added a commit
to adriansr/beats
that referenced
this issue
Dec 2, 2020
This commit adds a new function alternative, `__ip_local_out` for selecting a proper ip_local_out function, and fixes `guess_ip_local_out` logic in order to account for this new function. The new order of precedence is: - ip_local_out_sk (kernels before 3.16) - __ip_local_out (for kernels where ip_local_out calls are inlined) - ip_local_out (all others). Relates elastic#18755 (cherry picked from commit b627fb7)
3 tasks
adriansr
added a commit
to adriansr/beats
that referenced
this issue
Dec 2, 2020
This commit adds a new function alternative, `__ip_local_out` for selecting a proper ip_local_out function, and fixes `guess_ip_local_out` logic in order to account for this new function. The new order of precedence is: - ip_local_out_sk (kernels before 3.16) - __ip_local_out (for kernels where ip_local_out calls are inlined) - ip_local_out (all others). Relates elastic#18755 (cherry picked from commit b627fb7)
adriansr
added a commit
to adriansr/beats
that referenced
this issue
Dec 2, 2020
Auditbeat's system/socket dataset needs to install kprobes on all online CPUs. Previously, it was using runtime.NumCPU() to determine the CPUs in the system, and monitoring CPUs 0 to NumCPU. This was a mistake that lead to startup failures or loss of events in any of the following scenarios: - When Auditbeat is started with a CPU affinity mask that excludes some CPUs - When there are offline or isolated CPUs in the system. This patch updates the tracing library in Auditbeat to fetch the list of online CPUs from /sys/devices/system/cpu/online so that it can install kprobes in all of them regardless of its own affinity mask, and correctly skipping offline CPUs. Related elastic#18755 (cherry picked from commit 6356887)
4 tasks
adriansr
added a commit
to adriansr/beats
that referenced
this issue
Dec 2, 2020
Auditbeat's system/socket dataset needs to install kprobes on all online CPUs. Previously, it was using runtime.NumCPU() to determine the CPUs in the system, and monitoring CPUs 0 to NumCPU. This was a mistake that lead to startup failures or loss of events in any of the following scenarios: - When Auditbeat is started with a CPU affinity mask that excludes some CPUs - When there are offline or isolated CPUs in the system. This patch updates the tracing library in Auditbeat to fetch the list of online CPUs from /sys/devices/system/cpu/online so that it can install kprobes in all of them regardless of its own affinity mask, and correctly skipping offline CPUs. Related elastic#18755 (cherry picked from commit 6356887)
4 tasks
adriansr
added a commit
that referenced
this issue
Dec 3, 2020
#22869) * system/socket: Add ip_local_out alternative (#22787) This commit adds a new function alternative, `__ip_local_out` for selecting a proper ip_local_out function, and fixes `guess_ip_local_out` logic in order to account for this new function. The new order of precedence is: - ip_local_out_sk (kernels before 3.16) - __ip_local_out (for kernels where ip_local_out calls are inlined) - ip_local_out (all others). Relates #18755 (cherry picked from commit b627fb7)
adriansr
added a commit
that referenced
this issue
Dec 3, 2020
…ve (#22870) * system/socket: Add ip_local_out alternative (#22787) This commit adds a new function alternative, `__ip_local_out` for selecting a proper ip_local_out function, and fixes `guess_ip_local_out` logic in order to account for this new function. The new order of precedence is: - ip_local_out_sk (kernels before 3.16) - __ip_local_out (for kernels where ip_local_out calls are inlined) - ip_local_out (all others). Relates #18755 (cherry picked from commit b627fb7) * Changelog entry
adriansr
added a commit
that referenced
this issue
Dec 3, 2020
Auditbeat's system/socket dataset needs to install kprobes on all online CPUs. Previously, it was using runtime.NumCPU() to determine the CPUs in the system, and monitoring CPUs 0 to NumCPU. This was a mistake that lead to startup failures or loss of events in any of the following scenarios: - When Auditbeat is started with a CPU affinity mask that excludes some CPUs - When there are offline or isolated CPUs in the system. This patch updates the tracing library in Auditbeat to fetch the list of online CPUs from /sys/devices/system/cpu/online so that it can install kprobes in all of them regardless of its own affinity mask, and correctly skipping offline CPUs. Related #18755 (cherry picked from commit 6356887)
adriansr
added a commit
that referenced
this issue
Dec 3, 2020
Auditbeat's system/socket dataset needs to install kprobes on all online CPUs. Previously, it was using runtime.NumCPU() to determine the CPUs in the system, and monitoring CPUs 0 to NumCPU. This was a mistake that lead to startup failures or loss of events in any of the following scenarios: - When Auditbeat is started with a CPU affinity mask that excludes some CPUs - When there are offline or isolated CPUs in the system. This patch updates the tracing library in Auditbeat to fetch the list of online CPUs from /sys/devices/system/cpu/online so that it can install kprobes in all of them regardless of its own affinity mask, and correctly skipping offline CPUs. Related #18755 (cherry picked from commit 6356887)
|
Its working for me. |
|
I've started to upgrade my servers as this issue appeared to be fixed however after upgrading both the kernel and auditbeat to 7.10.1 which has the fix I'm still receiving the error. This host has IPV6 enabled with IPV6 addresses assigned and is running the linux-hardened kernel instead. Below is the debug file as requested before: |
|
@Aqualie |
|
This release fixed the problem it's working now, thank you. |
|
That's nice to hear, that this fixed it for you. |
|
Issue fixed |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Linux 5.6.14-arch1-1
Auditbeat 7.7.0
The text was updated successfully, but these errors were encountered: