Spectralnormalization GAN seems to be seriously broken

[Mut1nyJD]

Description

The SN_GAN example in the gluon/examples directory does not seem to work. I am wondering if the spectral normalization code in it is correct at all.

1.) The example does not use HybridBlocks which shouldn't be too hard to do I would think
2.) Some of the default values do not match what they say in the description (latent_z defaults to 10 not to 100 as by description)
3.) The results I am getting are terrible even after training for over 80 epochs the latent space seems to collapse to a single class or a specific input image which is weired because spectral normalization should actually cause the opposite to happen according to the research papers cited

(See linked output example for what I am seeing)
Output example after 86 epochs

Environment info (Required)

MXNet 1.3 Anaconda Python 3.4.1 Windows 10 nVidia Geforce1080ti

[zachgk]

Thank you for submitting the issue! I'm labeling it so the MXNet community members can help resolve it. @mxnet-label-bot add [Python, Gluon, Example]

@stu1130 Can you take a look?

[seujung]

The problem is related to the update u matrix.
I found that parameter is not updated using setattr method.
When changing code like this, the paramter is updated successfully.

        self.params.setattr('u', _u)

         with autograd.pause():
             self.u.set_data(_u)

[stu1130]

@Mut1nyJD @zachgk will take a look at it.
@seujung Thanks for your solution!

[seujung]

PR #13868

[stu1130]

@Mut1nyJD

1. Yes it's possible to create HybridBlocks here. Possible solution would be separate the SNConv2D into Conv2D and Spectral Normalization hybrid block. I created a issue here Make Spectral Normalization as HybridBlocks #13879 Sorry I got another important work to do now, call for contribution for this one.
2. Updated the doc
3. I think @seujung update Spectral Normalization Code #13868 fix this, Thanks again!