The ordeal of installing the Nvidia runtime Using Docker, Windows 10 and WSL #1084

calloatti · 2023-06-08T17:03:08Z

calloatti
Jun 8, 2023

Scenario:

Windows 10 machine, Docker Desktop 4.19.0, WSL installed and configured.

I have never used Docker Desktop before.
I have never used Linux for more that 2 minutes at a time.
I have no idea what CUDA stands for.
I managed to install and use the custom build SubCenter-ArcFace-r100, everything works, it takes about 9 seconds to detect a face in an image.

Edit: I should state that I barely know what I am doing here, I am like a monkey pressing keys and hoping that something will come out working at the end.

Trying to use the custom build SubCenter-ArcFace-r100-gpu:

Installed Nvidia GPU: NVIDIA GeForce GTX 650 Ti BOOST
Installed Nvidia Windows Drivers 30.0.14.7430, downloaded from Nvidia

Now, it seems that I have to install the "nvidia runtime" in docker, spent days looking for info on how to do this using Docker on Windows+WSL, all the info I can find is about using Linux as the base OS and not Windows.

Every single thing I find about installing the nvidia runtime on Windows seems outdated, does not work, or is not clear enough.

Somehow it seems I have to install Ubuntu under WSL, run Ubuntu, install Docker inside Ubuntu, install the nvidia runtime in docker running on Ubuntu, that is running on WSL, that is integrated into Docker Desktop running on Windows. How does this work? No idea, Followed lots of blog posts, nvidia docs, cannot understand how this inception like mess ends up with the nvidia runtime installed and configured on the top level Docker Desktop on Windows, gave up with that route.

Then came across https://hub.docker.com/r/nvidia/cuda

Is this the nvidia runtime that I have to install? No idea, let's try. Oh I have to specify a tag for it. Lets go with this:

docker pull nvidia/cuda:11.4.0-runtime

OK, I got something:

How do I test if this is working? (insert half a day of research and 200 open tabs in Chrome)

Lets create a docker-compose.yml file with this inside:

Now lets do docker-compose up

Nice? Are we done? Is that all? No idea.

Now lets try to set up SubCenter-ArcFace-r100-gpu:

Seems logical, how will it know that the image nvidia/cuda is the one with the nvidia runtime.

Cue another half a day of research: It seems I have to somehow edit daemon.json and add the info of where the nvidia runtime is located?

Where is this daemon,json file located on Docker Desktop Windows? Success! found it at C:\Users\username.docker\daemon.json

Now all the examples I can find seem to be using Linux as the main OS, could not find one example of doing this using Windows 10

Seems I can edit this file from inside Docker Desktop?

Too scared to proceed: "This can prevent Docker from starting"

Even if I wanted to, I have no idea what to add to it, found some info, for example:

$ cat /etc/docker/daemon.json
{

    "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    }
}

How does that translate to my scenario? No idea.

Sidequests:

Found some info about replacing "runtime=nvidia" with "--gpus all" No idea how I can use that.

Then found this, seems interesting: https://docs.docker.com/compose/gpu-support/

Lets do some experiments then, edit SubCenter-ArcFace-r100-gpu\docker-compose.yml:

remove the runtime: nvidia line from the compreface-core section, then add this at the end:

  deploy:
     resources:
       reservations:
         devices:
           - driver: nvidia
             count: 1
             capabilities: [gpu]

Save, try docker-compose up -d

Well, is it working?, at least it creates the container and images, and they seem to be up and running:

Lets set up a detection service and try it out:

compreface-core log:

2023-06-08 13:56:55 [uWSGI] getting INI configuration from uwsgi.ini
2023-06-08 13:56:55 *** Starting uWSGI 2.0.19 (64bit) on [Thu Jun 8 16:56:55 2023] ***
2023-06-08 13:56:55 compiled with version: 9.4.0 on 07 October 2022 10:41:31
2023-06-08 13:56:55 os: Linux-5.15.90.1-microsoft-standard-WSL2 #1 SMP Fri Jan 27 02:56:13 UTC 2023
2023-06-08 13:56:55 nodename: c5c28882c0f4
2023-06-08 13:56:55 machine: x86_64
2023-06-08 13:56:55 clock source: unix
2023-06-08 13:56:55 detected number of CPU cores: 8
2023-06-08 13:56:55 current working directory: /app/ml
2023-06-08 13:56:55 detected binary path: /usr/local/bin/uwsgi
2023-06-08 13:56:55 !!! no internal routing support, rebuild with pcre support !!!
2023-06-08 13:56:55 setgid() to 33
2023-06-08 13:56:55 setuid() to 33
2023-06-08 13:56:55 your memory page size is 4096 bytes
2023-06-08 13:56:55 detected max file descriptor number: 1048576
2023-06-08 13:56:55 lock engine: pthread robust mutexes
2023-06-08 13:56:55 thunder lock: disabled (you can enable it with --thunder-lock)
2023-06-08 13:56:55 uwsgi socket 0 bound to TCP address 0.0.0.0:3000 fd 3
2023-06-08 13:56:55 Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0]
2023-06-08 13:56:55 Python main interpreter initialized at 0x56179bc86660
2023-06-08 13:56:55 python threads support enabled
2023-06-08 13:56:55 your server socket listen backlog is limited to 100 connections
2023-06-08 13:56:55 your mercy for graceful operations on workers is 60 seconds
2023-06-08 13:56:55 mapped 145808 bytes (142 KB) for 1 cores
2023-06-08 13:56:55 *** Operational MODE: single process ***
2023-06-08 13:56:55 {"severity": "INFO", "message": "{'BUILD_VERSION': 'dev', 'CALCULATION_PLUGIN': 'insightface.Calculator@arcface-r100-msfdrop75', 'EXTRA_PLUGINS': ['insightface.LandmarksDetector', 'insightface.GenderDetector', 'insightface.AgeDetector', 'insightface.facemask.MaskDetector', 'insightface.PoseEstimator'], 'FACE_DETECTION_PLUGIN': 'insightface.FaceDetector@retinaface_r50_v1', 'GPU_IDX': 0, 'IMG_LENGTH_LIMIT': 640, 'INTEL_OPTIMIZATION': False, 'IS_DEV_ENV': False, 'LOGGING_LEVEL_NAME': 'DEBUG', 'ML_PORT': 3000, 'RUN_MODE': True}", "request": null, "logger": "src.app", "module": "app", "traceback": null, "build_version": "dev"}
2023-06-08 13:56:55 {"severity": "DEBUG", "message": "Creating new app for WSGI", "request": null, "logger": "src.app", "module": "app", "traceback": null, "build_version": "dev"}
2023-06-08 13:56:55 WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x56179bc86660 pid: 1 (default app)
2023-06-08 13:56:55 *** uWSGI is running in multiple interpreter mode ***
2023-06-08 13:56:55 spawned uWSGI master process (pid: 1)
2023-06-08 13:56:55 spawned uWSGI worker 1 (pid: 22, cores: 1)
2023-06-08 13:58:14 [16:58:14] ../src/nnvm/legacy_json_util.cc:208: Loading symbol saved by previous version v1.3.0. Attempting to upgrade...
2023-06-08 13:58:14 [16:58:14] ../src/nnvm/legacy_json_util.cc:216: Symbol successfully upgraded!
2023-06-08 13:58:14 [16:58:14] ../src/engine/engine.cc:54: MXNet start using engine: ThreadedEnginePerDevice
2023-06-08 13:58:15 [16:58:15] ../src/base.cc:79: cuDNN lib mismatch: linked-against version 8101 != compiled-against version 8100. Set MXNET_CUDNN_LIB_CHECKING=0 to quiet this warning.
2023-06-08 13:58:15 terminate called after throwing an instance of 'dmlc::Error'
2023-06-08 13:58:15 what(): [16:58:15] ../include/mshadow/./stream_gpu-inl.h:133: Check failed: err == CUBLAS_STATUS_SUCCESS (3 vs. 0) : Create cublas handle failed
2023-06-08 13:58:15
2023-06-08 13:58:15
2023-06-08 13:58:15 DAMN ! worker 1 (pid: 22) died, killed by signal 6 :( trying respawn ...
2023-06-08 13:58:15 Respawned uWSGI worker 1 (new pid: 37)

Well, I guess it does not work. Why? Is it that the GPU I managed to get my hands on is too old? Is it that everything I did makes no sense at all? No idea.

I would love to test with a GeForce 3060, but cannot justify spending USD 500 on a video card just to see if I can make this work. I guess I will stick with 9 second detection times until I can get some help figuring this out.

Answered by calloatti

Jul 10, 2023

Well, it works, the xtimer column is the time it takes to do recognition of a face using SubCenter-ArcFace-r100, around 5 seconds:

And using SubCenter-ArcFace-r100-gpu with RTX 3060 12GB:

All I did was install the video card drivers, and then edited the docker-compose.yml file like this:

  compreface-core:
    image: ${registry}compreface-core:${CORE_VERSION}
    restart: always
    container_name: "compreface-core"
    environment:
      - ML_PORT=3000
      - UWSGI_PROCESSES=${uwsgi_processes:-2}
      - UWSGI_THREADS=${uwsgi_threads:-1}
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:…

View full answer

calloatti · 2023-06-08T17:05:14Z

calloatti
Jun 8, 2023
Author

My main question is: Is there anyone out there that managed to make this work under Windows 10 + Docker Desktop + WSL2?

If you are out there, could you share the steps and incantations required to install a working copy of the nvidia runtime?

0 replies

calloatti · 2023-06-08T17:21:11Z

calloatti
Jun 8, 2023
Author

More anecdotal info: I have 150000 640x480 images with (generally) one face on it, from an attendance system I developed using android tablets, that have been in use now for a couple of years. Right now there is no automated recognition of subjects, just the stored images.

There are 250 employees in total, what I am doing right now is processing the 150k images to extract the 5 best ones for each subject to use as samples to upload for each subject and then be able to do recognition.

At 9 seconds on average per image, I have been doing this for about a week now, and still not done.

This is my quick and dirty form for doing this:

I calculate the face box area, the distance of the center of the box from the center of the image, and with box_probability will use those parameters to select the 5 best images for each subject, for example:

1 reply

pospielov Jun 16, 2023
Maintainer

Looks great.
Just an idea - did you try to use MobileNet version? It should run much faster on old machines. And the accuracy shouldn't be so bad, as you take the top 5 similarities.

pospielov · 2023-06-16T16:41:18Z

pospielov
Jun 16, 2023
Maintainer

I don't have a clear answer on how to install CompreFace with GPU for Windows.
I know that it's not as easy a process as on Linux.
What I know, is that NVIDIA GeForce GTX 650 Ti is not supported from CompreFace 1.0.1.
So first thing, you need to downgrade your CompreFace to the 1.0.0 version.

0 replies

calloatti · 2023-06-16T23:37:20Z

calloatti
Jun 16, 2023
Author

First of all, thank you for replying. I solved my problem, I am using now the compreface-core:1.1.0-arcface-r100 build, no GPU required.

The reason for the 10 seconds detection, was simply that I was using det_prob_threshold=0.0 thinking that it was a sane default, and also using limit=1, so basically the limit of 1 was masking the problem with setting the threshold to zero.

I made a request with limit=0, and then I realized that the API was detecting about 6k faces in each picture, so basically that was the reason for the 10 second detection times.

Now I am using a sane value of det_prob_threshold=0.5 and limit=2, and the detection time is on average 0.3 seconds per image.

I use limit 2 to find out what pictures cannot be used as samples.

Everything is working fine for now, I processed all 150k images thru detection, wrote some code to select the best 5 to use as samples for each subject, uploaded the samples, and I am happy to say that the recognition rate for proper pictures is 100% accurate.

For recognition I am using prediction_count=5 and wrote some code to select the best candidate based on how many votes each subject gets on the prediction set, and the similarity value, still have to refine that for edge cases (When the image has half a face for example, or its very blurry)

Each recognition takes about 3.5 seconds

With these results I just mentioned, there is no point for me to invest in a video card to get more performance and accuracy, these numbers are good enough for me.

TLDR for the using GPU/CUDA on Docker Desktop on Windows, I still have to find one acount in the internet of someone that managed to pull that out succesfully.

1 reply

Anatolii-R Jun 20, 2023
Collaborator

I successfully started SubCenter-ArcFace-r100-gpu on Windows 10.
I used first 5 points from these instructions to prepare Windows 10 for usung CompreFace

These instructions is partly outdated, for installing CUDA, please use this link

I used the following commands to install Docker on Ubuntu due to the Nvidia runtime error, as I am using Windows Docker Desktop:
In Ubuntu terminal (WSL) type -

curl https://get.docker.com | sh
sudo usermod -aG docker $USER

##### NVIDIA Container Toolkit
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
curl -s -L https://nvidia.github.io/libnvidia-container/experimental/$distribution/libnvidia-container-experimental.list | sudo tee /etc/apt/sources.list.d/libnvidia-container-experimental.list

sudo apt-get update
sudo apt-get install -y nvidia-docker2
sudo service docker restart
sudo service docker start

calloatti · 2023-06-20T14:13:32Z

calloatti
Jun 20, 2023
Author

I successfully started SubCenter-ArcFace-r100-gpu on Windows 10.

Thank you so much!

0 replies

calloatti · 2023-07-10T18:39:18Z

calloatti
Jul 10, 2023
Author

Update: I bought an RTX 3060 12GB, did everything as explained, still it does not work.

C:\CompreFace_1.1.0\custom-builds\SubCenter-ArcFace-r100-gpu>docker compose up -d
[+] Running 0/0
 - Container compreface-core   Creating                                                                            0.0s
 - Container compreface-admin  Creating                                                                            0.0s
Error response from daemon: unknown or invalid runtime name: nvidia

C:\CompreFace_1.1.0\custom-builds\SubCenter-ArcFace-r100-gpu>

EDIT: I think I managed to make it work, will post solution after checking that everything works.

0 replies

calloatti · 2023-07-10T20:00:04Z

calloatti
Jul 10, 2023
Author

Well, it works, the xtimer column is the time it takes to do recognition of a face using SubCenter-ArcFace-r100, around 5 seconds:

And using SubCenter-ArcFace-r100-gpu with RTX 3060 12GB:

All I did was install the video card drivers, and then edited the docker-compose.yml file like this:

  compreface-core:
    image: ${registry}compreface-core:${CORE_VERSION}
    restart: always
    container_name: "compreface-core"
    environment:
      - ML_PORT=3000
      - UWSGI_PROCESSES=${uwsgi_processes:-2}
      - UWSGI_THREADS=${uwsgi_threads:-1}
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

Removed the nvidia runtime line and added the deploy:.... part

See https://docs.docker.com/compose/gpu-support/

It was a very simple solution after all.

1 reply

antonbek89 Oct 20, 2023

Hi
Did you can share the full docker-compose.yml file?
Include all necessary files

I want download it and build it with "docker compose up" command

calloatti · 2023-10-23T15:32:12Z

calloatti
Oct 23, 2023
Author

The file I edited is "CompreFace_1.1.0\custom-builds\SubCenter-ArcFace-r100-gpu\docker-compose.yml"

Just edit that file:

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The ordeal of installing the Nvidia runtime Using Docker, Windows 10 and WSL #1084

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 8 comments 3 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

Select a reply

The ordeal of installing the Nvidia runtime Using Docker, Windows 10 and WSL #1084

calloatti Jun 8, 2023

Replies: 8 comments · 3 replies

calloatti Jun 8, 2023 Author

calloatti Jun 8, 2023 Author

pospielov Jun 16, 2023 Maintainer

pospielov Jun 16, 2023 Maintainer

calloatti Jun 16, 2023 Author

Anatolii-R Jun 20, 2023 Collaborator

calloatti Jun 20, 2023 Author

calloatti Jul 10, 2023 Author

calloatti Jul 10, 2023 Author

antonbek89 Oct 20, 2023

calloatti Oct 23, 2023 Author

calloatti
Jun 8, 2023

Replies: 8 comments 3 replies

calloatti
Jun 8, 2023
Author

calloatti
Jun 8, 2023
Author

pospielov Jun 16, 2023
Maintainer

pospielov
Jun 16, 2023
Maintainer

calloatti
Jun 16, 2023
Author

Anatolii-R Jun 20, 2023
Collaborator

calloatti
Jun 20, 2023
Author

calloatti
Jul 10, 2023
Author

calloatti
Jul 10, 2023
Author

calloatti
Oct 23, 2023
Author