How to run the demo locally and correctly?

[ifsheldon]

Hi! I've read README and tried to run the demo on my server, but I think there're a lot of code that is out of sync or missing. And the guides are incomplete.

Here are my steps:

1. I installed the dependencies according to INSTALL.md
2. I ran the app.py in this repo
3. A lot of models are missing:
   • I need GLEE models so I downloaded them on huggingface demo directory
   • Misc models are missing, then I found the ones mentioned in TRAIN.md and downloaded them
4. There are bugs in app.py on lines like (outputs,_) = GLEEmodel(...), which should be ((outputs, _), _, _) = GLEEmodel(...)
5. Then I ran the app.py in this repo again, but the results are just random like below.

Did I do anything wrong? Should I just clone the huggingface repo instead?

[knightdby]

+1

[heroinlin]

+1

[heroinlin]

@ifsheldon @knightdby
check out to huggingface repo, and it work well!

[cteant]

The problem is that in app.py when loading the glee model, the key does not match and the model is not correctly loaded. Use the following code in L88 in app.py

GLEEmodel_r50.load_state_dict({k.replace('glee.', ''): v for k, v in checkpoints_r50.items()}, strict=False)

Also in L98 in app.py

GLEEmodel_swin.load_state_dict({k.replace('glee.', ''): v for k, v in checkpoints_swin.items()}, strict=False)

[2anchao]

The problem is that in app.py when loading the glee model, the key does not match and the model is not correctly loaded. Use the following code in L88 in app.py

GLEEmodel_r50.load_state_dict({k.replace('glee.', ''): v for k, v in checkpoints_r50.items()}, strict=False)

Also in L98 in app.py

GLEEmodel_swin.load_state_dict({k.replace('glee.', ''): v for k, v in checkpoints_swin.items()}, strict=False)

nice

[victoriazinkovich]

When testing app.py with the model GLEE_Lite_joint.pth, I also encountered an error at line 127. The issue was with the image transposition. To fix this, change the following line:

ori_image = torch.as_tensor(np.ascontiguousarray(copyed_img.transpose(2, 0, 1)))

to:

ori_image = torch.as_tensor(np.ascontiguousarray(np.transpose(copyed_img, (2, 0, 1))))

This resolves the TypeError: Image.transpose() takes 2 positional arguments but 4 were given error.