NLP/3_nltk/nltk_built-in_dataset.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to load and analysis corpus shakespeare"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 如何使用nltk的shakespeare数据集\n",
    "* ref: https://www.nltk.org/book/ch02.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. download dataset/corpus"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package shakespeare to /root/nltk_data...\n",
      "[nltk_data]   Unzipping corpora/shakespeare.zip.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "nltk.download('shakespeare')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. check the text names at this corpus"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['a_and_c.xml',\n",
       " 'dream.xml',\n",
       " 'hamlet.xml',\n",
       " 'j_caesar.xml',\n",
       " 'macbeth.xml',\n",
       " 'merchant.xml',\n",
       " 'othello.xml',\n",
       " 'r_and_j.xml']"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "nltk.corpus.shakespeare.fileids()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. access hamlet words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "hamlet = nltk.corpus.shakespeare.words('hamlet.xml')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "40379"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(hamlet)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['The',\n",
       " 'Tragedy',\n",
       " 'of',\n",
       " 'Hamlet',\n",
       " ',',\n",
       " 'Prince',\n",
       " 'of',\n",
       " 'Denmark',\n",
       " 'Dramatis',\n",
       " 'Personae',\n",
       " 'CLAUDIUS',\n",
       " ',',\n",
       " 'king',\n",
       " 'of',\n",
       " 'Denmark',\n",
       " '.',\n",
       " 'HAMLET',\n",
       " ',',\n",
       " 'son',\n",
       " 'to',\n",
       " 'the',\n",
       " 'late',\n",
       " ',',\n",
       " 'and',\n",
       " 'nephew',\n",
       " 'to',\n",
       " 'the',\n",
       " 'present',\n",
       " 'king',\n",
       " '.',\n",
       " 'POLONIUS',\n",
       " ',',\n",
       " 'lord',\n",
       " 'chamberlain',\n",
       " '.',\n",
       " 'HORATIO',\n",
       " ',',\n",
       " 'friend',\n",
       " 'to',\n",
       " 'Hamlet',\n",
       " '.',\n",
       " 'LAERTES',\n",
       " ',',\n",
       " 'son',\n",
       " 'to',\n",
       " 'Polonius',\n",
       " '.',\n",
       " 'LUCIANUS',\n",
       " ',',\n",
       " 'nephew',\n",
       " 'to',\n",
       " 'the',\n",
       " 'king',\n",
       " '.',\n",
       " 'VOLTIMAND',\n",
       " 'CORNELIUS',\n",
       " 'ROSENCRANTZ',\n",
       " 'GUILDENSTERN',\n",
       " 'OSRIC',\n",
       " 'courtiers',\n",
       " '.',\n",
       " 'A',\n",
       " 'Gentleman',\n",
       " 'A',\n",
       " 'Priest',\n",
       " '.',\n",
       " 'MARCELLUS',\n",
       " 'BERNARDO',\n",
       " 'officers',\n",
       " '.',\n",
       " 'FRANCISCO',\n",
       " ',',\n",
       " 'a',\n",
       " 'soldier',\n",
       " '.',\n",
       " 'REYNALDO',\n",
       " ',',\n",
       " 'servant',\n",
       " 'to',\n",
       " 'Polonius',\n",
       " '.',\n",
       " 'Players',\n",
       " '.',\n",
       " 'Two',\n",
       " 'Clowns',\n",
       " ',',\n",
       " 'grave',\n",
       " '-',\n",
       " 'diggers',\n",
       " '.',\n",
       " 'FORTINBRAS',\n",
       " ',',\n",
       " 'prince',\n",
       " 'of',\n",
       " 'Norway',\n",
       " '.',\n",
       " 'A',\n",
       " 'Captain',\n",
       " '.',\n",
       " 'English']"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hamlet[:100]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. search by word"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Displaying 25 of 472 matches:\n",
      "The Tragedy of Hamlet , Prince of Denmark Dramatis Person\n",
      "rsonae CLAUDIUS , king of Denmark . HAMLET , son to the late , and nephew to t\n",
      "d chamberlain . HORATIO , friend to Hamlet . LAERTES , son to Polonius . LUCIA\n",
      " , queen of Denmark , and mother to Hamlet . OPHELIA , daughter to Polonius . \n",
      "s , and other Attendants . Ghost of Hamlet ' s Father . SCENE Denmark . HAMLET\n",
      "Hamlet ' s Father . SCENE Denmark . HAMLET ACT I SCENE I . Elsinore . A platfo\n",
      "o the combat ; in which our valiant Hamlet -- For so this side of our known wo\n",
      "he article design ' d , His fell to Hamlet . Now , sir , young Fortinbras , Of\n",
      " we have seen to - night Unto young Hamlet ; for , upon my life , This spirit \n",
      "er KING CLAUDIUS , QUEEN GERTRUDE , HAMLET , POLONIUS , LAERTES , VOLTIMAND , \n",
      "endants KING CLAUDIUS Though yet of Hamlet our dear brother ' s death The memo\n",
      "t at thy will ! But now , my cousin Hamlet , and my son ,-- HAMLET Aside KING \n",
      ", my cousin Hamlet , and my son ,-- HAMLET Aside KING CLAUDIUS How is it that \n",
      "that the clouds still hang on you ? HAMLET Not so , my lord ; I am too much i \n",
      "h i ' the sun . QUEEN GERTRUDE Good Hamlet , cast thy nighted colour off , And\n",
      "assing through nature to eternity . HAMLET Ay , madam , it is common . QUEEN G\n",
      " seems it so particular with thee ? HAMLET Seems , madam ! nay it is ; I know \n",
      "et and commendable in your nature , Hamlet , To give these mourning duties to \n",
      "t not thy mother lose her prayers , Hamlet : I pray thee , stay with us ; go n\n",
      "ay with us ; go not to Wittenberg . HAMLET I shall in all my best obey you , m\n",
      " This gentle and unforced accord of Hamlet Sits smiling to my heart : in grace\n",
      "hunder . Come away . Exeunt all but HAMLET HAMLET O , that this too too solid \n",
      ". Come away . Exeunt all but HAMLET HAMLET O , that this too too solid flesh w\n",
      "RDO HORATIO Hail to your lordship ! HAMLET I am glad to see you well : Horatio\n",
      "lord , and your poor servant ever . HAMLET Sir , my good friend ; I ' ll chang\n"
     ]
    }
   ],
   "source": [
    "# words to text\n",
    "text = nltk.Text(hamlet)\n",
    "# search\n",
    "text.concordance(\"hamlet\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}