[ FEATURE ] Improve Postprocessing

[p0psicles]

I want to look into queuing of the PP jobs.
I think we should implement this. IMO we have the following solution alternatives:

1. Implement queuing in Medusa. nzbToMedia will deliver a job with medusa, it's queued, and we need to create some sort of callback to nzbToMedia, to update it's db.
2. Implement queing in nzbToMedia, and make it send jobs to medusa synchronous.
3. Integrate nzbToMedia in medusa, and let download client communicate directly with medusa. This will be sort of a solution like number, with the difference that we won't need to make callbacks.
   @pymedusa/developers what do you guys think?

IMO the main benefits of queuing would be:

• Throttling of medusa's cpu/mem usage.
• Preventing file locks
• ?

I tend to solution 2, because that's the least amount of effort (I guess), and would offload some cpu/mem from medusa.
Just had a chat with @duramato. But the only issue we're having is when you AutoPP and Manual PP the same file at the same time. Which IMO is pretty rare?!

This change will make sure PP jobs can be run in parallel again: #290

I propose that we collect all issues in this FR, and try to collect as much knowledge as possible.

[OmgImAlexis]

But the only issue we're having is when you AutoPP and Manual PP the same file at the same time.

Maybe add a queue directory in the PP directory and don't allow anything to PP that "automatically", so only the thing that moved it there can handle it. Kind of like a "protected zone".

[duramato]

@OmgImAlexis I wouldnt say just dont allow the file itself becouse imagine
Top folder is Downloads you have Show1.xyz and Show2.xyz there and Folder/Show3.xyz
auto PP is PPing Show1.xyz dont allow any other PP thread to pp it beeing manual PP , or a third party via api, since thats what causes the issues.
IMO

[p0psicles]

Sorry @duramato i didn't understand a single word of that comment.
@OmgImAlexis, can't say I understand your comment. But my gut feeling tells me to mess as little with the filesystem as possible. So i'd prefer to not use all kinds of staging directories. And keep it as simple as possible.

[p0psicles]

Maybe it's a good idea to create a list of requirements which PP should be compliant to? So what do we expect from PP?

[fernandog]

@p0psicles
I would like to PP to checks if in the PP folder we have a Show1.S01E01.mkv and Show1.S01E01.PROPER.mkv so if deletes the non-proper and only PP the "proper"

[duramato]

What i meant is to simply check if the same file trying to be PPed already beeing PPed since thats the only time issues arive ,no need to queue it

[p0psicles]

I guess we can adapt the decorator mii created to do just that? But what then? Return a failure to nzbToMedia, or a notification if it's a manual pp job?

[duramato]

Just return a "File is already beeing PPed" if its the same file, like it did with multiple threads

[NicoLeOca]

Hi,
could PP integrate the subs handling more properly?
I think about:
parsing files into PP temporary folder to detect embedded subs (most web-dl have embedded and get stuck in this folder because they are not detected)
enable auto PP for shows that don't require subs and also "delay PP if no subs" for show that need them (you can't have both settings at the same time at the moment).

[NicoLeOca]

@fernandog following the few discussions we had about it, you might be willing to comment

[fernandog]

@NicoLeOca problem is embedded subs need the lang tag inside it. without it there is no way to know the lang.

[fernandog]

@NicoLeOca i can remove the "auto PP" disabling when postpone if no subs. that will fix

[NicoLeOca]

@fernandog
please, that would fix one

about the embedded, the language could be detect by the name of the subs track, when the language flag is missing.
I know SR doesn't at the moment, but Plex does it's very successful.
I undestand that the files which have neither of the language name or the flag are impossible to detect though.

[duramato]

Plex uses the track lang, if there isnt any it defaults to english

[NicoLeOca]

thanks @duramato
I just made a few tests and you are right
let's forget it then

[labrys]

Added to master feature request list - discussion for feature will continue here even though issue is closed.