[Feature] Face recognition

[bpatrik]

Building on #56 and using face-api.js, PiGallery2 will detect and recognize faces.

[nishantbb]

New user. Is this a tool that can be implemented from the gallery, or support for existing metadata? If the former, is there a link with any basic guidance you can provide?

[bpatrik]

The app cannot recognize faces at the moment, but can extract from exif data if the files contains it (i.e: it was set with a different app, like Adobe lightroom)

[nishantbb]

Thanks for the response.

[utkarshjain00]

Please consider integrating with https://docs.deepstack.cc/face-recognition/

[lakano]

+1 for the suggestion of @utkarshjain00 and it's now possible to make DeepStack run on small ARM64 devices like the Raspberry Pi 4 ( 1GB minimum but 8GB is preferred) https://docs.deepstack.cc/arm64/index.html
With their REST API, I suppose this could help a lot.

[bpatrik]

Thanks. It looks good.

I will def. do a research before getting into implementing this. I expect this to be a huge chunk of work and I don't think it will be part of the app in 2023.

[rhatguy]

I think the killer use case would be if this could be implemented such that a web based user could do it easily. There are a lot of desktop computer based tools that can generate face metadata (like Digikam), but being opening up tagging to my family from their phone while they're sitting on the couch would be nice. That being said, there are already a lot of tools focusing on detecting/adding faces. PiGallery2 does what it does well. I hope adding face detection doesn't take away from PiGallery2's main focus.

[dima-ser]

That being said, there are already a lot of tools focusing on detecting/adding faces. PiGallery2 does what it does well. I hope adding face detection doesn't take away from PiGallery2's main focus.

Completely agree. I love that it's currently read-only and is focused on consuming data rather that producing it. We should keep it this way. Perhaps develop a fork for those who wish to add face editing to keep this one focused on what it does well.

[bpatrik]

My vision on the feature would be that:

• App will do face detection and recognision (and probably more like categoriezing)
• If you already tagged some faces, the app would use that as reference to categize people
• There will be a new page (probably a tab on the faces page) where admins can confirm a person if the app is not sure
• App tagged faces will be saved to the db first.
• Depending on the speed and the accuracy of the tagging it might also save these data to an xmp file in the temp folder. That the user can copy back to the original gallery if they want
  • This ofcourse needs xmp file support first
• Very maybe I will optionally enable writer back to the images

The main goal would be that minimal input is needed from the user and tagging is determistic and can be regenerated in case of a db or temp folder loss.

[bpatrik]

I need to push this feature even further. So wont be part of the next release.

[grasdk]

Some comments on the plans for adding face detection and recognition:

The algorithms / ML / AI are getting better and better, but there is still a long way to go.

• My experience with a few tools is that the workflow to approving, rejecting and deleting / ignoring suggestions from especially the recoginition is important.
• • Rejections (wrong face) and deletions (non-face) should be remembered so you don't have to re-do it...
• • • but with an option to cancel a rejection / deletion if it was a mistake.
• • Some photos, e.g. dark / low res / over exposed make recognition worse
• • • so until the algorithms can detect such things reliably, there should also be a workflow for marking photos / face tags that the algorithm should not use for a particular person, even if it is the right person.
• • Bulk handling (1000's maybe even 100's) of photos requires a big screen to have overview.
• • • DigiKam does it ok, but it currently lacks some of the features I mentioned above. But hey! it's open source like PiGallery :)

All in all, I think mobile-enabled tagging would be cool as an extension feature, perhaps with a scope of showing a face and swipe right to accept and left to reject (and up to delete?). But as some posters above indicate, it seems to be an app different from PiGallery.

[ericnotthered]

I would personally prefer that the Pigallery team focus on other issues. There are existing tools (i.e., Digikam) that support face tagging. PiGallery2 already reads and handles this data well.

I'd rather see the ability to update keywords (specifically, the Rating value).