How does IR face recognition work with a model trained on RGB?

[SimonBrandner]

Sorry, this might not strictly qualify as an issue but it feels like the documentation is a bit lacking in this context and I don't see any other place to ask...

I've been trying to make sense out of how Howdy works on the inside but I am struggling a bit.

As far as I can tell the DLIB model used by howdy has been trained on RGB images of people (https://github.com/davisking/dlib-models#dlib_face_recognition_resnet_model_v1datbz2) but howdy is supposed to use images taken with an IR webcam. The VideoCapture class seems to return both an RGB frame as well as a grayscale frame (https://github.com/boltgolt/howdy/blob/beta/howdy/src/recorders/video_capture.py#L97) which is confusing since OBS is giving me what I would call grayscale when using my IR camera (where is the RGB coming from?). Howdy then proceeds to use the grayscale frame for face detection (https://github.com/boltgolt/howdy/blob/beta/howdy/src/cli/add.py#L171) but the RGB frame for getting the actual encoding of the face (https://github.com/boltgolt/howdy/blob/beta/howdy/src/cli/add.py#L201) which feels quite sketchy since it would defeat the whole thing with the IR camera...

Could someone please explain this to me? What am I missing?

[boltgolt]

The "RGB" frame is not really in color of course, it's just the grayscale version with exactly equal R, G and B channels. This is why it works with the model trained on color images.

As for the exact why, i think that has been lost to time over the last 6 years