AniDB anime-titles.xml not cached

[Pixelz]

Describe the bug
It seems like the AniDB anime titles database gets downloaded every time a request is made to AnbiDB. This results in their servers automatically placing a ban if, for instance, you view the fansub group settings of too many different shows in rapid succession.

To Reproduce
Steps to reproduce the behavior:

• Add a new anime series
• Go to the series and select edit
• Click the format tab
• Do this enough times in rapid succession

Expected behavior

• Not being banned on AniDB

Additional context
AniDB's rules regarding anime-titles.xml.gz can be found at https://wiki.anidb.net/w/API#Anime_Titles and states that it should not be downloaded more than once per day.

[p0psicles]

That strange. I do know that the location where the two files are cached is not ideal.
Can you take a look at the folder: medusa/lib/adba folder. There should be two files: anime-list.xml and animetitles.xml.

Could you check if the data modified is changing each time you edit a show? It should stay the same for one day. And the file should not be updated more then once a day.

Also which os are you using?

[Pixelz]

I can't verify at the moment. Since I'm currently banned the edit page won't even load. There's nothing interesting in the debug logs either. Is there any way to get more helpful debug messages? Currently it's not even mentioning AniDB when I edit a show.

This is using the linuxserver.io docker image on RancherOS.

[p0psicles]

Do you maybe only have the cache folder mounted in your container? So you get a new medusa application files each time you restart it?

[p0psicles]

You should check out the lib/abda folder i mentioned. And check the modification dates on the files.

[Pixelz]

I was able to verify using tcpdump that it indeed only tries to access the UDP API and that the xml file seems to be cached properly. I'm not sure what caused me to get banned but it definitely happened after doing what I described previously. Maybe it's triggering some flood protection in their UDP API.

I've restarted the container a few times to verify and the file doesn't get removed. It should only get wiped if I re-created the container.

[p0psicles]

Okay.. Well ill move the cached files to the medusa cache folder. Then you should never loose them. Also not when recreating the container.

I have another pr ready where we will cache the anidb udp release names. That should also help.

[Pixelz]

I was unbanned for a while and then hit the ban again trying to debug this. The only thing going on was medusa requesting fansub groups from the UDP API so that has got to be it. Here's the ban message from the logs:

DEBUG    Thread_12 :: [f5973b2] anidb: handling(1-2) command GROUPSTATUS
DEBUG    Thread_12 :: [f5973b2] anidb: NetIO > 'GROUPSTATUS aid=7251&s=KU2PF&tag=T001'
DEBUG    Thread-17 :: [f5973b2] anidb: NetIO < '555 BANNED\n'
WARNING  Thread_12 :: [f5973b2] Unable to retreive Fansub Groups from AniDB. Error:User is banned

[p0psicles]

Ill try to implement the caching next days.

[p0psicles]

This should be fixed in the develop branch. Can you test it?

[Pixelz]

Using latest develop I was able to verify that it now only sends 2 requests to the UDP API when adding a new show and never anything else, so that seems to be working.

I was not able to get it to update any shows that were already added through various attempts. Perhaps doing a "Force Full Update" should also request the release groups again.

[p0psicles]

Ill look into that

[p0psicles]

Your new request is about invalidating the cache. We'll pick that up in next release.