Creation of tests before putting out new releases

[appiekap653]

With the latest release going from 2.0.4 to 2.1.0 some existing code involving Plex users rule variables were altered.

with this altercation variables like:

- Users that saw all available episodes
- Users that watch the show/season/episode

stopped working because it returned empty.

In this case it wasn't that big of a problem because it would just NOT delete shows while it should have deleted them.

But what if the next change to existing code has an effect the other way around?
What if it decides that every item in your whole Plex server is eligible for removal, and your settings are set to delete immediately?

When dealing with software that has removal of files on your hardware as it's main function, I would strongly recommend to implement strict and robust testing of all functions that involve the decision making of removal and all functions that execute the removal of media.

I don't know if this can be done correctly with just unit testing or that you need to have some testing setup with a Plex server that has all the different possibilities in it like: a server with one user and a server with multiple users, some movies just set as watched, some actually watched, Overseerr with multiple users etc. I think you get my point.

Maybe use something like a VM that you can recreate for every test.

Hope to hear your thoughts about this.

[jorenn92]

A more robust testing setup would be beneficial, but I can't do this on my own.

The time between releases is already quite long, so I can test on my own server for a while before releasing. However, since I don't use every rule, mistakes might still slip through.

There are a few unit tests, but not many. There's significant room for improvement in this area.

It might also help if more people used the less stable 'main' or 'develop' images, so issues could be caught earlier.

Ultimately, the main challenge is that I'm the only one working on this project, and I no longer have as much time to dedicate to it due to other life commitments.