(obsolete - please ignore) use the MMPose Beta online demo to drive T2iKeypose

[AugmentedRealityCat]

All the information below is now obsolete: I found a one-click solution ! There is a huggingface online live demo that works perfectly - in fact you can even use it as a benchmark as it's based on the official code.

https://huggingface.co/spaces/Adapter/T2I-Adapter

---

(THIS IS OBSOLETE INFORMATION - PLEASE DISREGARD)

I found a way to extract character poses from an image in a format that is compatible with the T2iKeypose model. It is different from OpenPose, and the kind of image we want to produce looks like this:

According to the information I found over here, the model was trained using the HRnet algorithm on the COCO dataset.

That's how I found MMPose, which is still in beta phase, has a free online demo that allows you to do that. Here is a screenshot of the interface:

You have to select the following options
top down method | HRNet | COCO

There is a button to download the result, or you can right-click and save the picture as - but in both cases you'll get the colored pose skeleton AND the source image blended together as one picture.

This is NOT what we want, we want just the coloured bones, nothing else.

That's where we have to hack the image using our favourite image editor. In there, you will load both your source image and the result you downloaded (source image + keypose bones) and you will use a process called difference keying to find which pixels are different when you compare both pictures. In photoshop, you could do that by having the second image applied with the "difference" blending mode as a layer above the first image, and then selecting all pixels that are not white to create a selection area that correspond with the keypose colored bones, which you can then copy and paste on a new layer, over black. Here is what you get:

One more step before we go: there is a blue rectangle around our bones. We don't need that - in fact we need to get rid of it.
Its color is RGB(0,0,255) - which is pure blue, hex code 0000ff - so just select everything of that color (with color range for example) and make them black or transparent so we don't see them anymore.

Voila, you have your keypose reference for T2i, as per the example below. You can also crop it to a size that makes sense for SD. Here it is in 1280x1024

But now, if you compare with the example at the top, it's still different: first, we have extra bones for hands, and second we have little white lozenges acting as face trackers.
Solution: chop their hands and face ! Use the same principle as before for the face by selecting the white color - nothing else should be white in your image by now. As for the hands, it's simpler to simply paint over them with black color or to erase them.

Finally ! Here is the image we will feed to the T2i model.

Procedure in Stable Diffusion WebUi

First we set the preprocessor to none. There is no preprocessor for this model yet - that's why I was looking for an alternative like this one here !
Second, we set the model to T2i (...) keypose (...) SD14v1
The name can vary a bit, hence the (...) and it should not be a problem as long as you have keypose in there.
You can crank the weight a little bit and reduce the end (T) value a bit as well. Taste and season accordingly.
Here is what I got with the standard 1.5 model and the simple prompt: 3 jumping wookies

Laugh all you want - this is one of the best images I managed to get out of it. Human faces were absolutely horrible - I tried to prompt clowns to make those faces more acceptable, and it was just worse. Negative prompting seems to send the whole thing into a mess. I've put a screenshot of the interface below so you can reuse the parameters if you want (I don't recommend it).

So, after all those steps, was it worth it ? NO.
Just NO !
Way too much work.
This will need a proper pre-processor to be useful, or some other custom bones structure you can edit quickly and easily in a 3d app (more on that soon !).