example-notebook/Tutorial_07 How to save the model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 如何使用 tf.train.Saver() 来保存模型\n",
    "之前一直出错，主要是因为坑爹的编码问题。所以要注意文件的路径绝对不不要出现什么中文呀。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model saved in file: ../ckpt/test-model.ckpt-1\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "config = tf.ConfigProto()\n",
    "config.gpu_options.allow_growth = True\n",
    "sess = tf.Session(config=config)\n",
    "\n",
    "# Create some variables.\n",
    "v1 = tf.Variable([1.0, 2.3], name=\"v1\")\n",
    "v2 = tf.Variable(55.5, name=\"v2\")\n",
    "\n",
    "# Add an op to initialize the variables.\n",
    "init_op = tf.global_variables_initializer()\n",
    "\n",
    "# Add ops to save and restore all the variables.\n",
    "saver = tf.train.Saver()\n",
    "\n",
    "ckpt_path = '../ckpt/test-model.ckpt'\n",
    "# Later, launch the model, initialize the variables, do some work, save the\n",
    "# variables to disk.\n",
    "sess.run(init_op)\n",
    "save_path = saver.save(sess, ckpt_path, global_step=1)\n",
    "print(\"Model saved in file: %s\" % save_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意，在上面保存完了模型之后。**Restart kernel** 之后才能使用下面的模型导入。否则会因为两次命名 \"v1\" 而导致名字错误。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ../ckpt/test-model.ckpt-1\n",
      "Model restored.\n",
      "[1.  2.3]\n",
      "55.5\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "config = tf.ConfigProto()\n",
    "config.gpu_options.allow_growth = True\n",
    "sess = tf.Session(config=config)\n",
    "\n",
    "# Create some variables.\n",
    "v1 = tf.Variable([11.0, 16.3], name=\"v1\")\n",
    "v2 = tf.Variable(33.5, name=\"v2\")\n",
    "\n",
    "# Add ops to save and restore all the variables.\n",
    "saver = tf.train.Saver()\n",
    "\n",
    "# Later, launch the model, use the saver to restore variables from disk, and\n",
    "# do some work with the model.\n",
    "# Restore variables from disk.\n",
    "ckpt_path = '../ckpt/test-model.ckpt'\n",
    "saver.restore(sess, ckpt_path + '-'+ str(1))\n",
    "print(\"Model restored.\")\n",
    "\n",
    "print(sess.run(v1))\n",
    "print(sess.run(v2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "导入模型之前，必须重新再定义一遍变量。\n",
    "\n",
    "但是并不需要全部变量都重新进行定义，只定义我们需要的变量就行了。\n",
    "\n",
    "也就是说，**你所定义的变量一定要在 checkpoint 中存在；但不是所有在checkpoint中的变量，你都要重新定义。**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ../ckpt/test-model.ckpt-1\n",
      "Model restored.\n",
      "[1.  2.3]\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "config = tf.ConfigProto()\n",
    "config.gpu_options.allow_growth = True\n",
    "sess = tf.Session(config=config)\n",
    "\n",
    "# Create some variables.\n",
    "v1 = tf.Variable([11.0, 16.3], name=\"v1\")\n",
    "\n",
    "# Add ops to save and restore all the variables.\n",
    "saver = tf.train.Saver()\n",
    "\n",
    "# Later, launch the model, use the saver to restore variables from disk, and\n",
    "# do some work with the model.\n",
    "# Restore variables from disk.\n",
    "ckpt_path = '../ckpt/test-model.ckpt'\n",
    "saver.restore(sess, ckpt_path + '-'+ str(1))\n",
    "print(\"Model restored.\")\n",
    "\n",
    "print(sess.run(v1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 关于模型保存的一点心得\n",
    "\n",
    "```\n",
    "saver = tf.train.Saver(max_to_keep=3)\n",
    "```\n",
    "在定义 saver 的时候一般会定义最多保存模型的数量，一般来说，我都不会保存太多模型，模型多了浪费空间，很多时候其实保存最好的模型就够了。方法就是每次迭代到一定步数就在验证集上计算一次 accuracy 或者 f1 值，如果本次结果比上次好才保存新的模型，否则没必要保存。\n",
    "\n",
    "如果你想用不同 epoch 保存下来的模型进行融合的话，3到5 个模型已经足够了，假设这各融合的模型成为 M，而最好的一个单模型称为 m_best, 这样融合的话对于M 确实可以比 m_best 更好。但是如果拿这个模型和其他结构的模型再做融合的话，M 的效果并没有 m_best 好，因为M 相当于做了平均操作，减少了该模型的“特性”。"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}