You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The concept of unbounded/bouned memory is fristed proposed by NTM . They can be used as a kind of external memory for complicated sequence tasks.
In attention mechanism, the encoder outputs can be regarded as an unbounded memory for the decoder. It means at each time step of decoder, the data in unbounded memory always remain constant, and the decoder read it. BUT, because an unbounded memory is involved in decoder's forward computation, it will also backpropagate the gradients it receives.
The unbounded/bounded memory is important for attention, but I do not find any example to show how to implement it?
The text was updated successfully, but these errors were encountered:
lcy-seso
changed the title
How to implement unbounded/bounded memory for while_op
How to implement the unbounded/bounded memory for while_op
Dec 25, 2017
The concept of unbounded/bouned memory is fristed proposed by NTM . They can be used as a kind of external memory for complicated sequence tasks.
In attention mechanism, the encoder outputs can be regarded as an unbounded memory for the decoder. It means at each time step of decoder, the data in unbounded memory always remain constant, and the decoder read it. BUT, because an unbounded memory is involved in decoder's forward computation, it will also backpropagate the gradients it receives.
The unbounded/bounded memory is important for attention, but I do not find any example to show how to implement it?
The text was updated successfully, but these errors were encountered: