Releases: Rishit-dagli/Nystromformer
Releases · Rishit-dagli/Nystromformer
Nystromformer v0.1.0
This is the initial release of Nystromformer and implements Nystromformer as a subclassed TensorFlow model.
- MoorePenrosePseudoinverse: Implement the Moore Penrose Inverse as a Keras layer
- NystromAttention: Implment Nystrom Attention as a Keras layer
- Nystromformer: Put together a complete Nystromformer model stacking Nystrom Attention.
- PreNorm: Normalize the activations of the previous layer for each given example in a batch independently and apply some function to it, implemented as a TensorFlow Keras Layer.
- FeedForward: Create a FeedForward neural net with two Dense layers and GELU activation, implemented as a TensorFlow Keras Layer.
Full Changelog: https://github.com/Rishit-dagli/Nystromformer/commits/v0.1.0