Added Weight Normalization

[rajuthegr8]

No description provided.

[rajuthegr8]

Added feature for #942 @MikeInnes @dhairyagandhi96

[CarloLucibello]

It would nice to have a utility function to apply this to any module as pytorch does:
https://github.com/pytorch/pytorch/blob/master/torch/nn/utils/weight_norm.py
https://pytorch.org/docs/stable/nn.html#weight-norm

[bhvieira]

It would nice to have a utility function to apply this to any module as pytorch does:
https://github.com/pytorch/pytorch/blob/master/torch/nn/utils/weight_norm.py
https://pytorch.org/docs/stable/nn.html#weight-norm

I've been thinking of ways of doing that.
I think we could define an enclosure (like Chain) whose parameters are defined in the Weight Normalization fashion in regards to the parameters of its constituents.

Or we could do something with custom adjoints too, that's done explicitly in the paper.

---

Don't know what to do with the bias terms, however.

[bhvieira]

I thought of a better way that mimics PyTorch, will write a PR soon.

[bhvieira]

BTW in the paper the norm should be taken by output dimension, so g is a vector and v is a matrix.
Your current implementation equals to PyTorch with dim=None.

[CarloLucibello]

we need something more general, along the lines of #1005. Closing this now, feel free to open a new PR in case you want to tackle the problem again