Customizing Log Entries Display in the Admin Panel

[SathishPR18]

On the admin page, the Log Entries section currently displays all the data recorded so far, including every action or change logged in the system. However, I would like to know if there’s a way to customize or filter this data to display only specific records based on certain criteria. For instance, instead of showing all the entries, I’m only interested in viewing the details from the previous month. Is there an option or feature in the admin panel to apply such filters or customize the data display to meet this requirement?

Additionally, instead of deleting data one by one, is there an option to delete a group of data all at once? If such an option exists, where can it be found, and is it possible to perform this action in the admin panel?

Priority
[High.]

[horilla-opensource]

Hi @SathishPR18 ,

By default there is no such feature currently available.

With Regards,
Team Horilla

[horilla-opensource]

Hi @SathishPR18 ,

Update the LogEntryAdmin class in the virtual environment:
In the Horilla project, modify the LogEntryAdmin class located in the file path:
Lib > site-packages > auditlog > admin.py.
Replace the existing class with the following updated code:

class LogEntryAdmin(admin.ModelAdmin):
    list_display = ('timestamp', 'action', 'content_type', 'object_repr', 'actor')
    actions = ['delete_selected']

    def get_queryset(self, request):
        queryset = super().get_queryset(request)
        date = datetime.today()
        first_day_of_prev_month = (date.replace(day=1) - timedelta(days=1)).replace(day=1).date()
        return queryset.filter(timestamp__date__gte=first_day_of_prev_month)

Please try this out and let us know if you face any issues.

With Regards,
Team Horilla

[SathishPR18]

Hi @horilla-opensource ,

First of all, thank you for your response. However, we are currently facing an issue related to deleting log entries.

We have around 800,000 (8 lakh) records in the log entry table. When we try to select all the records and delete them at once, the process doesn’t work as expected. Instead, the result shown is the same as the picture below.

Could you please look into this issue and suggest a solution?

[horilla-opensource]

Hi @SathishPR18 ,

Thank you for reaching out and providing detailed information about the issue you're facing. Based on your description and the large number of records in the LogEntry table (800,000+), the error seems to be caused by Django's limitations in handling a massive number of form fields or records in a single request. Here's how you can resolve this:

**Root Cause: **
The error occurs because the number of fields being sent during the delete operation exceeds Django's DATA_UPLOAD_MAX_NUMBER_FIELDS setting, and the sheer size of the dataset adds additional overhead to the operation.

Proposed Solutions

1. Increase DATA_UPLOAD_MAX_NUMBER_FIELDS Setting
   Update your settings.py to handle a larger number of fields:
   DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000000
   # Set this to a value higher than the number of fields involved

Restart your server after making the change.

2. Use Bulk Deletion with Scripts
   Given the scale of your data, performing a bulk delete operation in the Django Admin UI is not efficient. Instead, use the Django shell or write a management command for this. Here's an example using the Django shell:

python manage.py shell

     from auditlog.models import LogEntry
     LogEntry.objects.all().delete()

3. Paginate the Deletion Process

If deleting all records at once still causes issues, process the records in smaller batches:

     from auditlog.models import LogEntry
     batch_size = 1000
     while LogEntry.objects.exists():
         LogEntry.objects.all()[:batch_size].delete()

This approach reduces the memory load and avoids exceeding request limits.

Please try these solutions, starting with the bulk deletion script or paginated deletion method. If you continue to face issues, feel free to share more details, and we’ll be happy to assist further.

With Regards,
Team Horilla

[SathishPR18]

Hi @horilla-opensource
The solution provided is working as we were able to clear the data. Thanks a lot.

Now I am worried about the volume of data that will grow exponentially in next couple of months. Is there any option for us not capture the log entries for specific fields. This will be of great help to us in managing the log data

[horilla-opensource]

Hi @SathishPR18,

To exclude specific fields or models from audit logging in Horilla, you can follow these steps:

1. Excluding Fields:

   • Add the fields you want to exclude to the AUDITLOG_EXCLUDE_TRACKING_FIELDS variable in horilla_apps.py.

   AUDITLOG_EXCLUDE_TRACKING_FIELDS = (
       "created",
       "modified",
       # Add other fields you want to exclude
   )

   • After defining this variable, register it using the setattr function:

   setattr(settings, "AUDITLOG_EXCLUDE_TRACKING_FIELDS", AUDITLOG_EXCLUDE_TRACKING_FIELDS)

2. Excluding Models:

   • Use the AUDITLOG_EXCLUDE_TRACKING_MODELS variable to exclude specific apps or models from audit logging.
   • To exclude an entire app (e.g., the base app), define it like this:

   AUDITLOG_EXCLUDE_TRACKING_MODELS = (
       "base",
       # Add other apps or models as needed
   )

   • To exclude a specific model, add the model in the app_name.model_name format:

   AUDITLOG_EXCLUDE_TRACKING_MODELS = (
       "base.WorkTypeRequest",
       # Add other models as needed
   )

Make sure to define these variables in your horilla_apps.py file and register them accordingly to ensure they take effect.

Let me know if you need any further assistance!

With Regards,
Team Horilla