removes ignore everything else audit rule

[angrieralien]

Removes ignore everything else rule

The ignore everything else rule in /etc/audit/rules.d/99-default.rules ignores all audit rules added before the ignore everything else rule.

Once a rule triggers, the event then goes to the exclude filter for processing. So, if you told it to drop all events, you will not see any events.

How to use

Add a rule to audit rules and trigger the rule. You should see the log entry in /var/log/audit/audit.log
e.g. auditctl -A exit,always -S all

Testing done

Removed the ignore everything else rule and the audit rules started logging to audit.log.

Posted issue in auditd-userspace repo. linux-audit/audit-userspace#302

• Changelog entries added in the respective changelog/ directory (user-facing change, bug fix, security fix, update)
• Inspected CI output for image differences: /boot and /usr size, packages, list files for any missing binaries, kernel modules, config files, kernel modules, etc.

[pothos]

Thanks!

Add a rule to audit rules and trigger the rule. You should see the log entry in /var/log/audit/audit.log

Can you give the exact file name you used and what the rule was/how to trigger it?

[angrieralien]

Sure @pothos.

setup

Start with an empty /etc/audit/rules.d

cd /etc/audit
mv rules.d rules.d.old
mkdir rules.d

add a file in /etc/audit/rules.d/00-my-test.rules.

cat << EOF > /etc/audit/rules.d/00-my-test.rules
# First rule - delete all
# This is to clear out old rules, so we don't append to them.
-D
# log on every execve 
-a always,exit -F arch=b64 -S execve -F auid=500 -F key=auditcmd
-a always,exit -F arch=b32 -S execve -F auid=500 -F key=auditcmd
EOF

Restart the rules.

systemctl restart audit-rules
auditctl -l

Output:

-a always,exit -F arch=b64 -S execve -F auid=500 -F key=auditcmd
-a always,exit -F arch=b32 -S execve -F auid=500 -F key=auditcmd

open second terminal and run:

tail -f /var/log/audit/audit.log

run any command in the first terminal:

ls
crictl
cat /etc/shadow

Result without ignore everything else rule:

logs are being written to audit.log

Add in ignore everything else rule:

add in 99-default.rules

cat << EOF > /etc/audit/rules.d/99-default.rules
# Always report changes to the audit subsystem itself.
-a exclude,never -F msgtype=CONFIG_CHANGE

# Ignore everything else.
-a exclude,always -F msgtype>0
EOF

systemctl restart audit-rules
auditctl -l

Output:

auditctl -l
-a always,exit -F arch=b64 -S execve -F auid=500 -F key=auditcmd
-a always,exit -F arch=b32 -S execve -F auid=500 -F key=auditcmd
-a never,exclude -F msgtype=CONFIG_CHANGE
-a always,exclude -F msgtype>0

hit enter a few times in your tail terminal

run any command:

ls
crictl
cat /etc/shadow

Result

no output to your audit.log file.

Fix

Comment out rule:

cat << EOF > /etc/audit/rules.d/99-default.rules
# Always report changes to the audit subsystem itself.
-a exclude,never -F msgtype=CONFIG_CHANGE

# Ignore everything else.
# -a exclude,always -F msgtype>0
EOF

systemctl restart audit-rules
auditctl -l

Output:

-a always,exit -F arch=b64 -S execve -F auid=500 -F key=auditcmd
-a always,exit -F arch=b32 -S execve -F auid=500 -F key=auditcmd
-a never,exclude -F msgtype=CONFIG_CHANGE

Now look at the tail terminal. Logs start showing up.

Summary

This is the intended behavior of the ignore everything rule because once a rule triggers, the event then goes to the exclude filter for processing. Since the ignore everything rule is excluding everything no rules are being triggered.

[pothos]

Suggested change

	-a exclude,never -F msgtype=CONFIG_CHANGE
	-a exclude,never -F msgtype=CONFIG_CHANGE

A newline here would be good since these files are concatenated and one may want to run cat rules.d/*

[pothos]

Thanks, so the issue is that one would need to add an explicit rule to "not exclude this and that", making it harder/error prone to use, or?
The change looks good, I've started a test build, just to be sure.
Can you also add a changelog entry in changelog/bugfixes/2023-03-22-audit-rules.md?
The format is markdown, so, e.g., - Removed the audit subsystem exclusion rule that caused non-exclusion rules to be ignored if they didn't have an explicit rule to be not excluded ([coreos-overlay#2545](https://github.com/flatcar/coreos-overlay/pull/2545))
Here again the last part of my sentence is based on my understanding that some exclude,never entry would have been needed as workaround for each of the rules you wanted to add.

[pothos]

As @jepio pointed out, the rule was there because by default too much gets logged. That's also what I see when removing only this rule.
In the end I think this means that we need a docs entry in https://www.flatcar.org/docs/latest/setup/security/ instead of removing it here.

[angrieralien]

Hey @pothos. Thanks for looking into this. The -a always,exit -F arch=b64 -S execve -F auid=500 -F key=auditcmd is a very verbose rule. With no audit rules you only get the hard coded audit rules e.g. in PAM module. Not as much gets logged.

To clarify, the 99-default.rules excludes all rules. Even if you had rules before the -a exclude,always -F msgtype>0 rule, those rules would be triggered but immediately excluded by the -a exclude,always -F msgtype>0 rule. Really the ignore everything else rule is really excluding all rules. This is because of the order the rules are executed in auditd. Once a rule triggers, the event then goes to the exclude filter for processing.

With no rules (the rules.d directory is empty) it seems we just get the hard coded audit rules in auditd and the linux kernel. I think the hard coded rules are the most important audit rules. auditd is disabled by default in flatcar IIRC. If users enables auditd and want to exclude specific audit rules then those users can modify the audit rules to their use cases.

I'd suggest removing the ignore everything else rule.

If that is not an option then I'd suggest renaming the rule to ignore all rules and rename the file to 99-ignore-all.rules

Thoughts?

[pothos]

Yes, it's not that many log entries but still it creates a couple of lines in dmesg for every container start and one sudo true creates 5 lines in dmesg. I'm not totally against removing this rule but would also appreciate more input.

As for docs, I think this Butane(→Ignition) config should work to overwrite the default rules:

variant: flatcar
version: 1.0.0
storage:
  files:
    - path: /etc/audit/rules.d/99-default.rules
      overwrite: true
      contents:
        inline: |
          # new content

[pothos]

Ok, I've prepared a docs PR: flatcar-archive/flatcar-docs#296

But I'm still interested in having better defaults – e.g., could we maybe tweak the ignore filter to only suppress the PAM rules while letting your example rule through?

[angrieralien]

@pothos i'll take a look at tweaking the ignore filter.

[angrieralien]

@pothos the 99-default.rules come back after update. since the ignition file runs only one time at first boot the updates place the original 99-default.rules back on disk . This does seem to be a bug. I think a solution is to move the 99-default.rules to a different directory that does not get overwritten on update. We are open for other suggestion too.

[pothos]

Thanks, for reaching out, I've commented in flatcar/Flatcar#1023 (comment)
(On Flatcar Stable you have to create an empty file or with just comments in it, but in Flatcar Beta/Alpha it will work to delete the file and it won't be recreated.)