Add mod audit logs

[Zabuzard]

Overview

Implements #222 .

Logs moderative actions to a dedicated channel.

Supported actions

The current implementation supports logging of

• ban
• unban
• kick
• user muted
• user unmuted
• message deleted by another user

Config changes

This changes the config file and adds two new entries:

"modAuditLogChannelPattern": "mod_audit_log",
"mutedRolePattern": "Muted"

The entries are REGEX patterns used to identify the corresponding channel and role, based on their name.

Implementation

The system is a routine which is executed automatically on a specific schedule. Due to hard rate limits on viewing the audit log, right now it is executed only three times per day, on fixed times:

• 04:00 UTC
• 12:00 UTC
• 20:00 UTC

There is no proper "system support" for routines yet, hence there are a bunch of TODOs linking to the corresponding GH issue #235 .

The system maintains a database table where it remembers for each guild a timestamp when it checked the logs last time. Whenever it executes, it retrieves all audit log messages after that specific date and logs them to the dedicated channel.

Limitations (Discussion)

I see 3 approaches to implement this and unfortunately neither is really optimal:

1. Use ListenerAdapter to react on the events exposed by JDA
2. Check the Discord Audit Log on a certain interval (guild.retrieveAuditLogs())
3. Let the bot commands log actions themselve (maybe offer a central point for them to report to)

Combinations are of course also an option, if necessary.

Approach 2. is the one currently implemented by this PR. Approach 1. was implemented by this PR in the past.

extend ListenerAdapter

This works and is reliable. Unfortunately, the information that the API exposes is extremely limited and sparse. For example, an user-ban-event only exposes the guild and the banned user. It does not tell who banned the user.

But more ciritical is that neither kick nor message deletion can be retrieved.

There is no onGuildMemberKicked event, only onGuildMemberRemove​ but this, unfortunately, includes when a member left on their own and there is no way to differentiate them.

For message deletion, there is only onGuildMessageDelete but this also includes when an user deleted their own messages (which is not an action that should be logged) and there is no way to differentiate them.

Check Discord Audit Log

The Discord Audit Log exposes a lot of useful information. Everything expect showing actual message content can be implemented by reading the log.

However, it seems that reading the audit log is heavily rate limited. The rate limit seems to be something around 5 reads per day (maybe less, but still). And unfortunately there is no way to "subscribe" to the event of new actions being added.

So the audit log is barely usable and could, if at all, only be used to log events much after they actually happened. Extensive logs that appear only once or twice per day might still be useful though.

Let commands log themselves

This is the most flexible approach and offers the most details, also not subject to any rate limiting.

However, we can only control our own commands. That means, it is not possible to log any actions executed via other bot commands (e.g. Dyno) or via the Discord UI.

Which makes this option barely useful.

[marko-radosavljevic]

but not

• user kicked
• user deleted a message from another user

Those are the most used moderative actions tho, especially deleting a message. I really dislike incomplete solution for a mod log.

I think the third option is the best.

We would instruct all staff members to use exclusively bot commands for their actions. And we can use discord audit log daily, to check if everyone is using the commands properly. Maybe bot could send a warning to mods for every undesirable behaviour detected in discord audit log.

For message deletion, there is only onGuildMessageDelete but this also includes when an user deleted their own messages (which is not an action that should be logged) and there is no way to differentiate them.

For this, we will keep using dyno? We still want to be able to see if something got deleted, or edited by the user.

[Zabuzard]

but not

* user kicked

* user deleted a message from another user

Those are the most used moderative actions tho, especially deleting a message. I really dislike incomplete solution for a mod log.

Totally agree to this, which is why I am leaning back towards 2. now.

I think the third option is the best.

We would instruct all staff members to use exclusively bot commands for their actions. And we can use discord audit log daily, to check if everyone is using the commands properly. Maybe bot could send a warning to mods for every undesirable behaviour detected in discord audit log.

I think going with approach 3. is the worst "general" option. Maybe a good option if coupled with another option though. Its impossible and impractiable to force users to not use the Discord UI for message deletion, banning or kicking. They will eventually do it and then it slips through. Its just too convenient and easy to right click a bad message and click on delete. They wont do right click, copy id, /delete-message ... (and we also dont have such a command yet).

Similar with banning or kicking. In fact, Doppey told us to use Dyno only but I still see a lot of actions happening through the regular Discord UI occasionally, hence why we lack visibility in the first place. If any action would go through a command, there would be no visibility issue in the first place, cause everyone can see if a command was used.

So the motivation for this is kind of because people dont consistently use commands and I do not think that this will completely change with the slash commands. It will improve though, thats for sure.

For message deletion, there is only onGuildMessageDelete but this also includes when an user deleted their own messages (which is not an action that should be logged) and there is no way to differentiate them.

For this, we will keep using dyno? We still want to be able to see if something got deleted, or edited by the user.

Yeah, I think the #logs provided by Dyno are really helpful. I want to keep them. But its just too verbose to use as a main source, its more of a lookup source.

---

Personally, my main motivation for this is not necessarily to audit other mods but to audit the staff assistants.

[marko-radosavljevic]

Sure, I agree it's a bit cumbersome solution.

Personally, my main motivation for this is not necessarily to audit other mods but to audit the staff assistants.

But staff assistants almost exclusively kick and delete messages, so the first option is almost useless for this purpose?

[Zabuzard]

But staff assistants almost exclusively kick and delete messages, so the first option is almost useless for this purpose?

Precisely.

[marko-radosavljevic]

Everything expect showing actual message content can be implemented by reading the log.

So, even if we combine the first and second solution, we still won't have all information we need?

For example, in case of a staff assistant deleting a message, we won't be able to see the actual message that has been deleted?

@Tais993
Do you know anything that could help us here? ^^

[Tais993]

Discord doesn't provide a way.

Popular bots cache message content for .. days or weeks to solve this

[marko-radosavljevic]

Popular bots cache message content for .. days or weeks to solve this

And the issue with this approach is? We have to take care of privacy policy, and encrypt messages. Anything else?

Seems like something that would be useful for other commands too, so it's maybe worth considering.

[Tais993]

Encrypt content yes

[Zabuzard]

Everything expect showing actual message content can be implemented by reading the log.

So, even if we combine the first and second solution, we still won't have all information we need?

For example, in case of a staff assistant deleting a message, we won't be able to see the actual message that has been deleted?

@Tais993 Do you know anything that could help us here? ^^

Regarding that, I just vote for keeping Dyno. I would just keep #logs and you can always check it if your "audit log" tells you that a message was deleted that you want to check.

I would like to avoid having to save message content (even encrypted). Its like opening pandoras box in regards to complexity and legal and ToS and GDPR and whatever...

Maybe in a second iteration far in the future 😆

[illuminator3]

Don't forget that you'll also have to update the example config on the wiki.

[Zabuzard]

Don't forget that you'll also have to update the example config on the wiki.

yikes

Stale, requested changes are not relevant anymore

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
3 Code Smells

0.0% Coverage
0.0% Duplication

Fixed

[marko-radosavljevic]

Short one, but tricky to get right. ^^

All questions I had have been answered in the code, which is neat. A bit heavier lambdas/streams on moments, but that's alright. All in all, clean. ☺️

Since this is an isolated and easily replaceable piece of code, and also important for moderation, feel free to merge it.