Derpibooru csv and colors update

[Siberpone]

Up-to-date Derpibooru tags as of 2024-07-04 neatly sorted by category and image count + added tag aliases + default tag coloration updated to match the booru's dark and light themes respectively. Only includes tags with at least 10 images. "Spoiler" category was not included, as there was only ~300 entries total and I can not imagine a scenario where it can be useful for prompting.

[DominikDoom]

The reason for the strange key numbering here was that back when derpibooru tags were first added, they started category numbering at 1 and used 2 for general tags. I'm not sure if this was actually the APIs fault or just a mistake of the user that submitted them, but I changed all tags of cat 2 to 0 to match danboorus general tags (color-wise). Since the other numbers seemed to match the API output, I didn't shift them back to fill the empty spot left by 2.

This might not seem like an issue since you used the correct numbers in your updated CSV. But the color changes here are only the option default. Without resetting them manually, users would suddenly see a lot of wrong colors after updating, since their options are already applied and keep the old values. That means all tags you currently have marked with cat 2 will show as invalid for existing users, with the -1 color being used.

Sadly there isn't a way in place to inform the user about changed defaults at this time, I think the webui itself dealt with this by just renaming the option so that the old saved values get ignored. But I'm not too happy with this approach, as it would also undo all custom colors people might have set up for themselves (although I don't know if many did so). What do you think?

[Siberpone]

Well, the problem with derpi categories is that they don't really have ids, they're just stored in the "tags" table as plain text with NULL meaning "general" category. I based the numbering just off of alphabetical sorting. So, I've reviewed the old csv and currently the mapping looks like this:

category	old csv	new csv
NULL (general)	0	0
body-type	6	1
character	7	2
content-fanmade	11	3
content-official	1	4
error	9	5
oc	4	6
origin	8	7
rating	5	8
species	3	9
spoiler	10	-

Which, as you explained, will mess up the coloring with no obvious way for the user to fix it. So, what I'm thinking is:

• renumber new csv categories to match the old ones
• adjust default color definitions accordingly

I'm not insisting on adjusting colors, btw. Just thought it would be a nice touch :) At least for the new or more attentive users. Let me know if you want me to revert this.

[DominikDoom]

Well, the problem with derpi categories is that they don't really have ids, they're just stored in the "tags" table as plain text with NULL meaning "general" category. I based the numbering just off of alphabetical sorting

Yeah I'm aware, and derpibooru itself seems to just order alphabetically too:

By the way, the current colors were already picked to be similar to the derpibooru category colors, just adjusted for readability in the default webui theme as the exact colors were too low contrast in my tests. So it shouldn't be too different by just renumbering the CSV to the old values like you suggested.

And I actually misremembered, it was danbooru that skips the 2:

So while the original explanation is still true, index 2 wasn't filled mainly because of that. I think it would be best to mirror it in your updated CSV.

[Siberpone]

Alright, that should take care of that. Categories renumbered and color changes removed.

[DominikDoom]

Sorry to keep bothering you, one more thing:
I noticed your CSV is only ordered by post count per category, not overall.
E.g. "oc" in category 4 with a post count of 963353 should be rank 3 overall but only comes after 56k+ other tags from categories 0,1 and 3.
Or in other words, your categories currently aren't mixed.

Since normal tags don't get re-sorted in most cases, that would push a lot of matches further down or even outside of the max cap, so that is pretty important to fix.

[Siberpone]

No problem. Should be ordered by tag count now.