-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
92c1d97
commit 1c96c58
Showing
2 changed files
with
17 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,18 @@ | ||
# language_modelling | ||
# Language Modelling | ||
Repository containing the results of the Natural Language Understanding course project about Language Modelling. | ||
|
||
## Project description | ||
The proposed task of Language Modelling (LM) for the NLU course required to: | ||
|
||
1. implement a Language Model using one of the RNN architectures (eg. Vanilla, LSTM, GRU); | ||
2. train it and evaluate its performance on the word-level Penn Treebank (PTB) dataset; | ||
3. reach a baseline value of 140 PP using a Vanilla RNN, or 90.7 PP using an LSTM. | ||
|
||
As a starting point, I decided to implement a very basic model made of: | ||
- a neural embedding layer; | ||
- an LSTM, to capture context information; | ||
- a fully connected layer, for the final word prediction; and obtained 137 PP. | ||
|
||
To improve such results, I have considered the techniques described by Merity et. al, reaching 81.43 PP. | ||
|
||
 |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.