Skip to content

Tweak IMDB TorchText interpretation tutorial #1516

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions tutorials/IMDB_TorchText_Interpret.ipynb
Original file line number Diff line number Diff line change
@@ -13,11 +13,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook loads pretrained CNN model for sentiment analysis on IMDB dataset. It makes predictions on test samples and interprets those predictions using integrated gradients method.\n",
"This notebook loads a pretrained CNN model for sentiment analysis on an IMDB dataset. It makes predictions on test samples and interprets those predictions using the Integrated Gradients method.\n",
"\n",
"The model was trained using an open source sentiment analysis tutorials described in: https://github.com/bentrevett/pytorch-sentiment-analysis/blob/master/4%20-%20Convolutional%20Sentiment%20Analysis.ipynb with the following changes:\n",
"\n",
"- TEXT: set lower=True at initialization and call build_vocab() on the entire training data including validation to avoid mismatched indices\n",
"- TEXT: set lower=True at initialization and call build_vocab() on the entire training data including validation, to avoid mismatched indices\n",
"- model: save the entire model instead of just model.state_dict()\n",
"\n",
"**Note:** Before running this tutorial, please install the spacy package, and its NLP modules for English language."
@@ -81,7 +81,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The dataset used for training this model can be found in: https://ai.stanford.edu/~amaas/data/sentiment/\n",
"The dataset used for training this model can be found in: https://ai.stanford.edu/~amaas/data/sentiment/.\n",
"\n",
"Redefining the model in order to be able to load it.\n"
]
@@ -113,31 +113,31 @@
" \n",
" def forward(self, text):\n",
" \n",
" #text = [sent len, batch size]\n",
" # text = [sent len, batch size]\n",
" \n",
" #text = text.permute(1, 0)\n",
" # text = text.permute(1, 0)\n",
" \n",
" #text = [batch size, sent len]\n",
" # text = [batch size, sent len]\n",
" \n",
" embedded = self.embedding(text)\n",
"\n",
" #embedded = [batch size, sent len, emb dim]\n",
" # embedded = [batch size, sent len, emb dim]\n",
" \n",
" embedded = embedded.unsqueeze(1)\n",
" \n",
" #embedded = [batch size, 1, sent len, emb dim]\n",
" # embedded = [batch size, 1, sent len, emb dim]\n",
" \n",
" conved = [F.relu(conv(embedded)).squeeze(3) for conv in self.convs]\n",
" \n",
" #conved_n = [batch size, n_filters, sent len - filter_sizes[n] + 1]\n",
" # conved_n = [batch size, n_filters, sent len - filter_sizes[n] + 1]\n",
" \n",
" pooled = [F.max_pool1d(conv, conv.shape[2]).squeeze(2) for conv in conved]\n",
" \n",
" #pooled_n = [batch size, n_filters]\n",
" # pooled_n = [batch size, n_filters]\n",
" \n",
" cat = self.dropout(torch.cat(pooled, dim = 1))\n",
"\n",
" #cat = [batch size, n_filters * len(filter_sizes)]\n",
" # cat = [batch size, n_filters * len(filter_sizes)]\n",
" \n",
" return self.fc(cat)"
]
@@ -148,7 +148,7 @@
"source": [
"Loads pretrained model and sets the model to eval mode.\n",
"\n",
"The model can be downloaded here: https://github.com/pytorch/captum/blob/master/tutorials/models/imdb-model-cnn-large.pt"
"The model can be downloaded here: https://github.com/pytorch/captum/blob/master/tutorials/models/imdb-model-cnn-large.pt."
]
},
{
@@ -166,7 +166,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Forward function that supports sigmoid"
"Forward function that supports sigmoid."
]
},
{
@@ -290,7 +290,7 @@
"metadata": {},
"source": [
"Let's create an instance of `LayerIntegratedGradients` using forward function of our model and the embedding layer.\n",
"This instance of layer integrated gradients will be used to interpret movie rating review.\n",
"This instance of Layer Integrated Gradients will be used to interpret movie rating review.\n",
"\n",
"Layer Integrated Gradients will allow us to assign an attribution score to each word/token embedding tensor in the movie review text. We will ultimately sum the attribution scores across all embedding dimensions for each word/token in order to attain a word/token level attribution score.\n",
"\n",
@@ -319,7 +319,7 @@
"metadata": {},
"outputs": [],
"source": [
"# accumalate couple samples in this array for visualization purposes\n",
"# accumulate couple samples in this array for visualization purposes\n",
"vis_data_records_ig = []\n",
"\n",
"def interpret_sentence(model, sentence, min_len = 7, label = 0):\n",