CA module

[Zhangyuhaoo]

Sorry to bother you. Hello, I have read your paper and found it very impressive. I have a small question: can I use the CA module you proposed to replace the FFN layer in ViT? Again, I apologize for my interruption and look forward to your reply and suggestions. Thank you!

[Lupin1998]

Hello, @Zhangyuhaoo. Sorry for the late reply. The proposed CA module in MogaNet enhances the MixFFN, which inserts a DWConv3x3 in FFN (proposed in PVTv2), which can be regarded as the GRN block proposed in ConvNeXtv2. You could replace the FFN block with CA+MixFNN in ViTs to enhance the overall performance, i.e., adding the CA module and the DWConv3x3 in the FFN block. Hoping this message is not too late for you to be helpful, and feel free to ask me if there are more questions.