5. 损失函数.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "NN的优化目标：loss最小\n",
    "\n",
    "loss函数：\n",
    "    - mse(Mean Square Error)\n",
    "    - 自定义\n",
    "    - ce(Cross Entropy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1 - MES"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$MES(y, y\\_) = \\frac{\\sum_{i=1}^n(y - y\\_)^2}{n}$$\n",
    "\n",
    "```python\n",
    "loss = tf.reduce_mean(tf.square(y-y_))\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "预测酸奶的日销量，x1和x2是影响日销量的因素。建模前，应该预先采集的数据有：每日x1、x2和销量y。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "rds = np.random.RandomState(23455)\n",
    "X = rds.rand(32, 2)\n",
    "Y = [[x0 + x1 + rds.rand()/10.0 - 0.05] for (x0, x1) in X]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w =  [[-0.80974597]\n",
      " [ 1.4852903 ]]\n",
      "w =  [[0.08942621]\n",
      " [1.673328  ]]\n",
      "w =  [[0.42332518]\n",
      " [1.4907393 ]]\n",
      "w =  [[0.61732584]\n",
      " [1.3329403 ]]\n",
      "w =  [[0.7443861]\n",
      " [1.2228197]]\n",
      "w =  [[0.8294814]\n",
      " [1.1482829]]\n",
      "w =  [[0.88669145]\n",
      " [1.0980824 ]]\n",
      "w =  [[0.92517716]\n",
      " [1.0643018 ]]\n",
      "w =  [[0.95107025]\n",
      " [1.0415728 ]]\n",
      "w =  [[0.9684917]\n",
      " [1.0262802]]\n"
     ]
    }
   ],
   "source": [
    "BATCH_SIZE = 8\n",
    "\n",
    "x = tf.placeholder(tf.float32, shape=(None, 2))\n",
    "y = tf.placeholder(tf.float32, shape=(None, 1))\n",
    "\n",
    "w = tf.Variable(tf.random_normal([2, 1], stddev=1, seed=1))\n",
    "y_ = tf.matmul(x, w)\n",
    "\n",
    "loss = tf.reduce_mean(tf.square(y - y_))\n",
    "train = tf.train.GradientDescentOptimizer(0.001).minimize(loss)\n",
    "\n",
    "init = tf.global_variables_initializer()\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    sess.run(init)\n",
    "    STEP = 20000\n",
    "    for i in range(STEP):\n",
    "        start = (i * BATCH_SIZE) % 32\n",
    "        end = start + BATCH_SIZE\n",
    "        sess.run(train, feed_dict = {x:X[start:end], y:Y[start:end]})\n",
    "        if i % 2000 == 0:\n",
    "            print(\"w = \", sess.run(w, feed_dict = {x: X[start:end], y:Y[start:end]}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2 - 自定义的损失函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果预测商品的日销量，预测多了($y\\_ > y$)，损失成本(cost)；预测少了($y\\_ < y$)，损失利润(profit)。如果成本不等于利润，则mes无法利益最大化。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "f(y, y\\_) = \n",
    "\\begin{cases}\n",
    "\\text{cost}*(y\\_ - y)\\text{,  if }y < y\\_ \\text{，预测多了} \\\\\n",
    "\\text{profit}*(y - y\\_)\\text{,  if }y > y\\_ \\text{，预测少了} \\\\\n",
    "\\end{cases}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "loss = tf.reduce_sum(tf.where(tf.greater(y_, y), cost*(y_ - y), profit*(y - y_)))\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.1 - 当成本高"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w =  [[-0.80594873]\n",
      " [ 1.4873729 ]]\n",
      "w =  [[0.9602475]\n",
      " [0.9742084]]\n",
      "w =  [[0.95985126]\n",
      " [0.9805035 ]]\n",
      "w =  [[0.9641995]\n",
      " [0.9676626]]\n",
      "w =  [[0.964802 ]\n",
      " [0.9759015]]\n",
      "w =  [[0.95873684]\n",
      " [0.9744887 ]]\n",
      "w =  [[0.96004957]\n",
      " [0.9690877 ]]\n",
      "w =  [[0.96499455]\n",
      " [0.98077846]]\n",
      "w =  [[0.95892936]\n",
      " [0.97936565]]\n",
      "w =  [[0.9642764 ]\n",
      " [0.96846855]]\n"
     ]
    }
   ],
   "source": [
    "BATCH_SIZE = 8\n",
    "COST = 9\n",
    "PROFIT = 1\n",
    "\n",
    "x = tf.placeholder(tf.float32, shape=(None, 2))\n",
    "y = tf.placeholder(tf.float32, shape=(None, 1))\n",
    "\n",
    "w = tf.Variable(tf.random_normal([2, 1], stddev=1, seed=1))\n",
    "y_ = tf.matmul(x, w)\n",
    "\n",
    "loss = tf.reduce_sum(tf.where(tf.greater(y_, y), COST*(y_ - y), PROFIT*(y - y_)))\n",
    "train = tf.train.GradientDescentOptimizer(0.001).minimize(loss)\n",
    "\n",
    "init = tf.global_variables_initializer()\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    sess.run(init)\n",
    "    STEP = 20000\n",
    "    for i in range(STEP):\n",
    "        start = (i * BATCH_SIZE) % 32\n",
    "        end = start + BATCH_SIZE\n",
    "        sess.run(train, feed_dict = {x:X[start:end], y:Y[start:end]})\n",
    "        if i % 2000 == 0:\n",
    "            print(\"w = \", sess.run(w, feed_dict = {x: X[start:end], y:Y[start:end]}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.2 - 当利润高"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w =  [[-0.762993 ]\n",
      " [ 1.5095658]]\n",
      "w =  [[1.0179386]\n",
      " [1.0412899]]\n",
      "w =  [[1.021926 ]\n",
      " [1.0465802]]\n",
      "w =  [[1.0185053]\n",
      " [1.0389466]]\n",
      "w =  [[1.0224928]\n",
      " [1.0442369]]\n",
      "w =  [[1.0199078]\n",
      " [1.0392858]]\n",
      "w =  [[1.0238953]\n",
      " [1.044576 ]]\n",
      "w =  [[1.0191677]\n",
      " [1.0463983]]\n",
      "w =  [[1.0252978]\n",
      " [1.0449152]]\n",
      "w =  [[1.0205702]\n",
      " [1.0467374]]\n"
     ]
    }
   ],
   "source": [
    "BATCH_SIZE = 8\n",
    "COST = 1\n",
    "PROFIT = 9\n",
    "\n",
    "x = tf.placeholder(tf.float32, shape=(None, 2))\n",
    "y = tf.placeholder(tf.float32, shape=(None, 1))\n",
    "\n",
    "w = tf.Variable(tf.random_normal([2, 1], stddev=1, seed=1))\n",
    "y_ = tf.matmul(x, w)\n",
    "\n",
    "loss = tf.reduce_sum(tf.where(tf.greater(y_, y), COST*(y_ - y), PROFIT*(y - y_)))\n",
    "train = tf.train.GradientDescentOptimizer(0.001).minimize(loss)\n",
    "\n",
    "init = tf.global_variables_initializer()\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    sess.run(init)\n",
    "    STEP = 20000\n",
    "    for i in range(STEP):\n",
    "        start = (i * BATCH_SIZE) % 32\n",
    "        end = start + BATCH_SIZE\n",
    "        sess.run(train, feed_dict = {x:X[start:end], y:Y[start:end]})\n",
    "        if i % 2000 == 0:\n",
    "            print(\"w = \", sess.run(w, feed_dict = {x: X[start:end], y:Y[start:end]}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3 - 交叉熵（CE）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "表征两个概率分布之间的距离：\n",
    "$$H(y\\_, y) = -\\sum ylogy\\_$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "loss = -tf.reduce_mean(y * tf.log(tf.clip_by_value(y_, 1e-12, 1.0)))\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在实际的操作中，为了使得前向传播的计算结果满足概率设定，引入了softmax函数。"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}