The reconstruction images have a lot of noise

[powermano]

The following image is the result of epoch=69.

The reconstruction of unmasked part have lots of noise, is this a normal phenomenon？

And I have found a strange problem. If i replace the first conv in the stem with MinkowskiDepthwiseConvolution, the training loss will quickly drop to 0. I can't understand why this is happening.

[Mogul5306]

How to reconstuct the image. Could you provide the code?

[powermano]

How to reconstuct the image. Could you provide the code?

same as MAE. https://github.com/facebookresearch/mae/blob/main/demo/mae_visualize.ipynb

[Mogul5306]

I encountered the same question. Have you solved this problem? The reconstruction images have a lot of noise.

[powermano]

I encountered the same question. Have you solved this problem? The reconstruction images have a lot of noise.

I did not encounter the problem, maybe you can check you visualization codes.

[powermano]

I using another self-supervised learning for CNN, SparK(https://github.com/keyu-tian/SparK), and the reconstruction of unmasked part have lots of noise too.

Maybe the reason is due to loss fuction, as the reconstruction loss is only applied to the part of the mask, and the part without the mask is not constrained

[supertx]

Hi @powermano,

I saw your preview and noticed you're working on facial reconstruction. It seems you've made some progress. I've been working on a similar project but haven't achieved the desired results. Could you share any insights or experiences🙏🏻?

Thanks!