Understanding Recognize's Behavior

[bsaggy]

Hello,

I'm having some difficulty understanding how Recognize is supposed to behave, so I have a few questions. If this is basic stuff and there are answers elsewhere, please point me to that resource; I haven't found anything that answers these questions.

For some context, my configuration is:

System Configuration

• Ubuntu 20.04 VM
• 32 GB RAM (I increased memory & vCPU in hopes of speeding things up to no avail)
• 8 vCPU
• NC v25.0.1
• Recognize v3.1.2

Recognize Configuration:

• Face Recognition is enabled
• 6 Cores allowed for use
• WASM mode is enabled

I think Recognize runs in the background via occ cron. It seems to run every 30 minutes.

1. How many photos does Recognize analyze on each run?
2. What's the difference between Recognize running in the background on a fresh install where it has never run before vs. clicking "Rescan all files" vs. running occ recognize:recrawl?
3. I have ~34k jpgs across my entire installation (not counting appdata) - jpg type accounts for the most likely photo file type on my system. Recognize currently shows ~51k files queued and increasing. Where would this discrepancy come from, and is there a listing of queued files somewhere that I could view?
4. Is there a way to bypass cron running every 30 minutes and run it manually until all files have been classified? At the current rate things are going, it will take many weeks to finish classification on each image. I'd rather go hard for a couple days straight to finish initial classification.

Thanks for your hard work on this project!

Brian

[illnesse]

It would definitely help if there was an occ command that just does that: a full crawl using like 75% of the cores without any weird undocumented intransparent job scheduling, just to run it as initial scan.
That would be a big win, and if the user had more control over intervals, batch sizes of the different jobs and other settings which are all pretty self-explanatory,, or at least if there was an explanation somewhere in the readme as to what to expect and how the app is supposed to work, as clearly the majority of people in the issues on GitHub are confused over the erratic behavior of recognize (if they manage to get it to run at all)

~140k files here, i had to increase batch size and lower the interval timing in the code, then ran cron.php using watch -n 1 in a loop, after days of debugging to get it to actually run. If you're just interested in the faces and you're willing to tinker around a bit during install, you might want to give https://github.com/matiasdelellis/facerecognition a try too, seems more straightforward with better results according to tests and has a more useful ui.

https://github.com/pulsejet/memories is definitely a must have too, it will have support for facerecognition too, soon. This app should be the default photos app in my opinion. Very responsive dev too

[marcelklehr]

It would definitely help if there was an occ command that just does that: a full crawl using like 75% of the cores without any weird undocumented intransparent job scheduling, just to run it as initial scan.

Sadly, without weird job scheduling it doesn't scale. I understand the need for a simple command that takes care of an initial scan. v2 did have that after all and I do intend to add that again.

Meanwhile, I've started a write-up of how things are supposed to work behind the scenes, in case anyone would like to contribute or add questions that aren't answered by the article yet: github.com/nextcloud/recognize/wiki/Behind-the-scenes

[illnesse]

Awesome, really enjoying the app btw, I know it's probably a lot of work on the side, but we will help as good as possible 👍🏻

[bsaggy]

~140k files here, i had to increase batch size and lower the interval timing in the code, then ran cron.php using watch -n 1 in a loop, after days of debugging to get it to actually run.

Can you point out the files and changes you made to speed things up? How long did it take for your classifications to finish with 140k files and your tweaks?

If you're just interested in the faces and you're willing to tinker around a bit during install, you might want to give https://github.com/matiasdelellis/facerecognition a try too, seems more straightforward with better results according to tests and has a more useful ui.

I've considered trying Face Recognition, but I haven't given up on Recognize yet. Face Recognition also doesn't have a stable release on NC 25.x yet.

https://github.com/pulsejet/memories is definitely a must have too, it will have support for facerecognition too, soon. This app should be the default photos app in my opinion. Very responsive dev too

I was initially using Photos but that proved to be buggy with People. Memories works much better.

[illnesse]

Can you point out the files and changes you made to speed things up? How long did it take for your classifications to finish with 140k files and your tweaks?

i believe it was ClassifierJob and ClassifyFacesJob, but i'm pretty sure @marcelklehr will put those two in the settings soon

[marcelklehr]

Yes, that's in the works :)