WebHost: add startup option to clean datapackages

[Berserker66]

What is this fixing or adding?

Allows cleaning datapackage entries that have the wrong checksum. They're deleted entirely, which should be fine for the server, though some clients may break. This is a very small may though, as it should only affect clients that relied on a miscontructed apworld.

How was this tested?

copied the checksum from one row and the data from another row, then watched how only that row disappeared on startup.

If this makes graphical changes, please attach screenshots.

[ThePhar]

Do you think this should also be added to docs/webhost configuration sample.yaml?

[Berserker66]

No idea, if you think it should, feel free to PR it in.

[ThePhar]

Works as advertised.

[ThePhar]

Additional clarification was made in Discord, so going to post it here so it's not lost.

Essentially found out this check will not work for the issue described in #3206 because upload.py ends up overriding the checksum on both the column and in the data package of the uploaded file with the re-calculated value so it's essentially impossible to look for if the rest of the data package is technically valid.

Leaving my approval, but unless someone can think of another reason this could be useful (maybe there was another way data packages could be corrupt?), I don't think this will be necessary.

[black-sliver]

I feel like this should be a click instead, so you can just run flask cleanup-database in a separate process in the same venv without impacting the running instance.

[black-sliver]

As for what to clean out, I think it should (also) search for unreferenced game packages, so that when seeds are deleted we also delete the corresponding games packages on the next cleanup. (Since this needs to load all rooms and create a massive set, I don't think that can be done on startup, but needs to run in a separate process.)

[ThePhar]

As for what to clean out, I think it should (also) search for unreferenced game packages, so that when seeds are deleted we also delete the corresponding games packages on the next cleanup. (Since this needs to load all rooms and create a massive set, I don't think that can be done on startup, but needs to run in a separate process.)

Yeah, I think that should be doable.