深度学习用于文本和序列.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用于处理序列的两种基本的深度学习算法\n",
    "* **循环神经网络**\n",
    "* **一维卷积神经网络**  \n",
    "\n",
    "这些算法的应用包括：  \n",
    "* 文档分类和时间序列分类，比如识别文章的主题或书的作者\n",
    "* 时间序列对比，比如预测两个文档或两支股票行情的相关程度\n",
    "* 序列到序列的学习，比如将英语翻译成法语\n",
    "* 情感分析，比如将推文或电影评论的情感划分为正面或负面\n",
    "* 时间序列预测，比如根据某地最近的天气数据来预测未来天气"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 处理文本数据\n",
    "深度学习模型不会接收原始文本作为输入，它只能处理张量值。**文本向量化**是指将文本转换为数值张量的过程。它有多种实现方法  \n",
    "* 将文本分割为单词，并将每个单词转换为一个向量\n",
    "* 将文本分割为字符，并将每个字符转换为一个向量\n",
    "* 提取单词或字符的n-gram，并将每个n-gram转换为一个向量。n-gram是多个连续单词或字符的集合（n-gram之间可以重叠）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 单词级的one-hot编码示例"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'The': 1, 'cat': 2, 'sat': 3, 'on': 4, 'the': 5, 'mat.': 6, 'dog': 7, 'ate': 8, 'my': 9, 'homework.': 10}\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "samples = ['The cat sat on the mat.', 'The dog ate my homework.']\n",
    "\n",
    "token_index = {}\n",
    "for sample in samples:\n",
    "    for word in sample.split():\n",
    "        if word not in token_index:\n",
    "            token_index[word] = len(token_index) + 1\n",
    "print(token_index)\n",
    "max_length = 10\n",
    "results = np.zeros(shape=(len(samples), max_length, max(token_index.values()) + 1))\n",
    "\n",
    "for i, sample in enumerate(samples):\n",
    "    for j, word in list(enumerate(sample.split()))[:max_length]:\n",
    "        index = token_index.get(word)\n",
    "        results[i, j, index] = 1."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 字符级的one-hot编码示例"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~ \t\n",
      "\r",
      "\u000b",
      "\f",
      "\n",
      "{'0': 1, '1': 2, '2': 3, '3': 4, '4': 5, '5': 6, '6': 7, '7': 8, '8': 9, '9': 10, 'a': 11, 'b': 12, 'c': 13, 'd': 14, 'e': 15, 'f': 16, 'g': 17, 'h': 18, 'i': 19, 'j': 20, 'k': 21, 'l': 22, 'm': 23, 'n': 24, 'o': 25, 'p': 26, 'q': 27, 'r': 28, 's': 29, 't': 30, 'u': 31, 'v': 32, 'w': 33, 'x': 34, 'y': 35, 'z': 36, 'A': 37, 'B': 38, 'C': 39, 'D': 40, 'E': 41, 'F': 42, 'G': 43, 'H': 44, 'I': 45, 'J': 46, 'K': 47, 'L': 48, 'M': 49, 'N': 50, 'O': 51, 'P': 52, 'Q': 53, 'R': 54, 'S': 55, 'T': 56, 'U': 57, 'V': 58, 'W': 59, 'X': 60, 'Y': 61, 'Z': 62, '!': 63, '\"': 64, '#': 65, '$': 66, '%': 67, '&': 68, \"'\": 69, '(': 70, ')': 71, '*': 72, '+': 73, ',': 74, '-': 75, '.': 76, '/': 77, ':': 78, ';': 79, '<': 80, '=': 81, '>': 82, '?': 83, '@': 84, '[': 85, '\\\\': 86, ']': 87, '^': 88, '_': 89, '`': 90, '{': 91, '|': 92, '}': 93, '~': 94, ' ': 95, '\\t': 96, '\\n': 97, '\\r': 98, '\\x0b': 99, '\\x0c': 100}\n",
      "[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 0. 0. 0.]\n"
     ]
    }
   ],
   "source": [
    "import string\n",
    "\n",
    "samples = ['The cat sat on the mat.', 'The dog ate my homework.']\n",
    "characters = string.printable\n",
    "print(characters)\n",
    "\n",
    "token_index = dict(zip(characters, range(1, len(characters) + 1)))\n",
    "print(token_index)\n",
    "\n",
    "max_length = 50\n",
    "results = np.zeros((len(samples), max_length, max(token_index.values()) + 1))\n",
    "for i, sample in enumerate(samples):\n",
    "    for j, character in enumerate(sample):\n",
    "        index = token_index.get(character)\n",
    "        results[i, j, index] = 1.\n",
    "print(results[0][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### keras实现单词级one-hot编码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1, 2, 3, 4, 1, 5], [1, 6, 7, 8, 9]]\n",
      "{'the': 1, 'cat': 2, 'sat': 3, 'on': 4, 'mat': 5, 'dog': 6, 'ate': 7, 'my': 8, 'homework': 9}\n",
      "9\n"
     ]
    }
   ],
   "source": [
    "from keras.preprocessing.text import Tokenizer\n",
    "\n",
    "samples = ['The cat sat on the mat.', 'The dog ate my homework.']\n",
    "\n",
    "tokenizer = Tokenizer(num_words=1000)\n",
    "tokenizer.fit_on_texts(samples)\n",
    "\n",
    "sequences = tokenizer.texts_to_sequences(samples)\n",
    "print(sequences)\n",
    "\n",
    "one_hot_results = tokenizer.texts_to_matrix(samples, mode='binary')\n",
    "word_index = tokenizer.word_index\n",
    "print(word_index)\n",
    "print(len(word_index))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TODO\n",
    "有些地方看的一头雾水，先停了"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}