Attention map implementation

[samanehAb]

Hi,

I couldn't find the implementation of the attention layer inside the network models. In the SAGAN paper it is mentioned that they have added the self-attention mechanism at different stages and compared them with each other. Would you please let me know where you have considered that?

Bests,
Samaneh

[TLMichael]

This is the self-attention module.

[samanehAb]

Hello,

Thanks for your comment. I am wondering why you only have it in generator_test and discriminator_test modules. Why two different architectures are defined for the generator and discriminator?

Best Regards,
Samaneh

[TLMichael]

Sorry, I am not the coder. But I think one is SN-GAN, the other is SA-GAN.

[samanehAb]

Thanks for your response. That makes sense. Because in the SAGAN paper both during training and test time, there is attention layer in both the discriminator and the generator.

[TLMichael]

It's my pleasure~