Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

export dependencies to seperate container #690

Closed
KoMa1012 opened this issue Aug 8, 2023 · 3 comments
Closed

export dependencies to seperate container #690

KoMa1012 opened this issue Aug 8, 2023 · 3 comments

Comments

@KoMa1012
Copy link

KoMa1012 commented Aug 8, 2023

Every once in a while there is a breaking update to the nextcloud docker container which requires new dependencie versions which might even not be available. This would be prevented if there is a seperate container for the face recognition and the app in nextcloud is just handing over pictures to this container and getting back all the information it requires (faces etc..). Also, it would be possible to run the face recognition on e.g. your gaming rig at home which would speed up the recognition speed dramatically.
I think requiring the dependencies on the same machine or container as nextcloud doesn't seem to be the optimal choice to me.

Create a docker container which can be used to run all the "AI jobs", something like this: https://github.com/matiasdelellis/facerecognition-external-model

Create an app which does not require all the external dependencies (pdlib etc.) which can only run on model 5 (aka external model).

And for whoever doesn't like docker, you still can install this locally.

@muebau
Copy link

muebau commented Aug 8, 2023

Well this idea came up before:

#210
exadel-inc/CompreFace#554
nextcloud/recognize#680

So it is the main thing to offload the work and dependencies to some other entity (could be local or remote).

The app recognize (https://github.com/nextcloud/recognize) did a smart thing to use a JavaScript (WebAssembly) version if the underlying platform does not support the execution format.

I think it should be possible to use an interface like this to run the code in JavaScript (WebAssembly) in the browser with GPU (WebGL) acceleration (https://github.com/justadudewhohacks/face-api.js).

This could be a variant where a explicit "web runner' window could be opened to support the process with the GPU of some client. The "external model" interface would be used by multiple helpers (eg. docker container, WebGL client).

matiasdelellis added a commit that referenced this issue Aug 22, 2023
To the happiness of many (Issue #690, #688, #687, 685, #649, #632,
extension, but it goes without saying that its use is still highly
recommended.

You will understand that it is slower, however I must admit that with
JIT enabled, it is quite acceptable, and this is the only reason why
decided to publish it.

It is still experimental, and it works, but it has problems such as it
seems not to converge in stable clusters. When I can fix this, it will
probably be even slower.
matiasdelellis added a commit that referenced this issue Aug 22, 2023
To the happiness of many (Issue #690, #688, #687, $685, #649, #632,
extension, but it goes without saying that its use is still highly
recommended.

You will understand that it is slower, however I must admit that with
JIT enabled, it is quite acceptable, and this is the only reason why
decided to publish it.

It is still experimental, and it works, but it has problems such as it
seems not to converge in stable clusters. When I can fix this, it will
probably be even slower.
matiasdelellis added a commit that referenced this issue Aug 22, 2023
To the happiness of many (Issue #690, #688, #687, #685, #649, #632,
extension, but it goes without saying that its use is still highly
recommended.

You will understand that it is slower, however I must admit that with
JIT enabled, it is quite acceptable, and this is the only reason why
decided to publish it.

It is still experimental, and it works, but it has problems such as it
seems not to converge in stable clusters. When I can fix this, it will
probably be even slower.
matiasdelellis added a commit that referenced this issue Aug 22, 2023
To the happiness of many (Issue #690, #688, #687, #685, #649, #632, #627, #625, etc),
this means that we do not depend on the pdlib extension, but it goes
without saying that its use is still highly recommended.

You will understand that it is slower, however I must admit that with
JIT enabled, it is quite acceptable, and this is the only reason why
decided to publish it.

It is still experimental, and it works, but it has problems such as it
seems not to converge in stable clusters. When I can fix this, it will
probably be even slower.
matiasdelellis added a commit that referenced this issue Aug 23, 2023
To the happiness of many (Issues #690, #688, #687, #685, #649, #632
, #627, #625, etc..?) this means that we do not depend on the pdlib
extension, but it goes without saying that its use is still highly
recommended.

You will understand that it is slower, however I must admit that with
JIT enabled, it is quite acceptable, and this is the only reason why
decided to publish it.
@matiasdelellis
Copy link
Owner

Hi,
I invite you to try the external model, using the latest version released.

Open a new issue to see how we continue. 😬

@KoMa1012
Copy link
Author

@matiasdelellis, thank you! This is freaking awesome! Finally I can do the initial recognition on my Gaming setup with a proper gpu.

I’m running the first tests with your docker container and I am starting to set up my gaming PC so that it can run the clustering, thanks to your documentation this shouldn‘t be to hard.

Do you want positive feedback or only if something fails in the process?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants