Script MultiheadAttention (#1524)

Summary:
Pull Request resolved: facebookresearch/fairseq#1524

Make fairseq MultiheadAttention scriptable. Looking for feedbacks.

1. Add types
2. Move incremental state management logic from util functions to initializers. TorchScript in general doesn't support global dict. As a result modules with multihead attention in it would assign itself fairseq_instance_id in the initializer.
3. There might be opportunities to make assertions and annotations cleaner.

Differential Revision: D18772594

fbshipit-source-id: 81b830b16fbaa9c6fc34dee0672054f146060ea4

pytorch_translate/average_attention.py

@@ -9,10 +9,10 @@
 import torch
 import torch.nn.functional as F
 from fairseq import utils
-from torch import nn
+from fairseq.modules import BaseFairseqModule
 
 
-class AttentionAbstract(nn.Module):
+class AttentionAbstract(BaseFairseqModule):
     """Abstract class for attention modules
     """