Sample Code/Day_36_SampleCode.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 今天的範例-常見偵測異常值的方法\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Z9YHBcEGYdBV"
   },
   "source": [
    "### 範例資料\n",
    "鐵達尼號 train 資料集\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1558,
     "status": "ok",
     "timestamp": 1578021044012,
     "user": {
      "displayName": "Mora chen",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AAuE7mB40f7sDArbZ5_DYq02nNcnLD0Ryaf7AhsASSQeLQ=s64",
      "userId": "03171203089166907199"
     },
     "user_tz": -480
    },
    "id": "yz28_IgkYdBW",
    "outputId": "a12f486c-18b3-4fb5-d06c-f162aebd9444"
   },
   "outputs": [],
   "source": [
    "# library\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn  as sns\n",
    "from scipy import stats\n",
    "import math\n",
    "import statistics\n",
    "#讓圖可以在 jupyter notebook顯示\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train = pd.read_csv(\"Titanic_train.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 探索性分析:\n",
    "*  觀察-是否有異常值  \n",
    "我們來觀察一下 Age 變數吧!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Q1:觀察票價是否有異常值的現象?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "29.69911764705882\n",
      "14.526497332334044\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x192720e53c8>"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAPRklEQVR4nO3df+xddX3H8edLChphKMVCKuCKGwHR\njV91iEy3WV2Yc5ZN2EBY6sLC/nAON40r849pzDJNFiPZhrERtf6a/HKjaRaRFVh0WSpFAS0VYcqg\nWqDVicMsauW9P86pfm2/5Xvb773f+7nf+3wkN/eec+8933fb77uv+znn3M9JVSFJUmueNu4CJEma\njQElSWqSASVJapIBJUlqkgElSWqSASVJapIBtUCSvD3J1iT3JLkrydlD2u5rk6wd0raeGMI2np7k\n2iQPJNmcZMX8K9O0mKI+eXmSLybZneSCYdS1GC0ZdwHTIMk5wGuAM6vqB0meAxx2AO9fUlW7Z3uu\nqjYAG4ZT6VBcBvxPVf1ikouA9wB/MOaaNAGmrE8eAt4AvHXMdTTNEdTCWA7sqqofAFTVrqr6FkCS\nB/tGJMnKJLf3j9+RZF2SzwIf7UcjL9yzwSS3JzkryRuS/EOSZ/Xbelr//DOTPJzk0CS/kOQzSe5M\n8rkkp/SvOTHJfya5I8m7hvRnXQ2s7x/fAKxKkiFtW4vb1PRJVT1YVfcATw5je4uVAbUwPguckORr\nSa5O8msDvu8sYHVVvR74FPD7AEmWA8+tqjv3vLCqHgfuBvZs+3eAm6vqR8A64E1VdRbdJ7ar+9dc\nBby/ql4MPLK/IvpmvWuW2ytneflxwMN9TbuBx4GjB/zzarpNU59oAO7iWwBV9USSs4CXAb8BXJtk\nbVV9ZI63bqiq/+sfXwfcAvw1XQNeP8vrr6XbnXYbcBFwdZIjgJcC188YyDy9vz8XeF3/+GN0u+Nm\nq/9lc9Q502yjJefT0pymrE80AANqgVTVj4HbgduTfBlYA3wE2M1PR7LP2Ott35/x/m8m+XaSX6Zr\nrj+Z5cdsAP42yVK6T5W3AocD362q0/dX2ly1J/kc8HOzPPXWqvq3vdZtB04AtidZAjwL+M5cP0OC\nqeoTDcBdfAsgyclJTpqx6nTgv/vHD9I1Cfz0U9r+fAp4G/Csqvry3k9W1RPAF+h2SWysqh9X1feA\nbyS5sK8lSU7r3/IfdJ8gAS7Z3w+tqpdV1emz3GZrug10/6kAXADcWs5IrAFMWZ9oAAbUwjgCWJ/k\n3iT3AKcC7+ifeydwVf/p68dzbOcGuka57ilecy1waX+/xyXAZUnuBrbSncgAcAXwxiR30I10huEa\n4OgkDwB/AQzl1F5NhanpkyQvTrIduBD4QJKtw9juYhM/3EqSWuQISpLUJANKktQkA0qS1CQDSpLU\npAUNqPPOO6/ovk/gzds03A6KfeJtCm+zWtCA2rVr10L+OGki2SdSx118kqQmGVCSpCYZUJKkJhlQ\nkqQmGVCSpCYZUJKkJnk9qBH55OaHBnrd689+3ogrkaTJ5AhKktQkA0qS1CQDSpLUJANKktQkA0qS\n1CQDSpLUJANKktQkA0qS1CQDSpLUJANKktQkA0qS1CQDSpLUJANKktQkA0qS1CQDSpLUJANKktSk\ngQIqybOT3JDkq0m2JTknydIktyS5v78/atTFSpKmx6AjqKuAz1TVKcBpwDZgLbCpqk4CNvXLkiQN\nxZwBleRI4OXANQBV9cOq+i6wGljfv2w9cP6oipQkTZ9BRlDPB3YCH07ypSQfTHI4cGxV7QDo74+Z\n7c1JLk+yJcmWnTt3Dq1waTGxT6R9DRJQS4AzgfdX1RnA9zmA3XlVta6qVlbVymXLlh1kmdLiZp9I\n+1oywGu2A9uranO/fANdQD2aZHlV7UiyHHhsVEW25JObHxp3CZI0FeYcQVXVI8DDSU7uV60C7gU2\nAGv6dWuAm0ZSoSRpKg0yggJ4E/CJJIcBXwf+iC7crktyGfAQcOFoSpQkTaOBAqqq7gJWzvLUquGW\nI0lSx5kkJElNMqAkSU0yoCRJTTKgJElNMqAkSU0yoCRJTTKgJElNMqAkSU0yoCRJTTKgJElNGnQu\nvkXPWcolqS2OoCRJTTKgJElNMqAkSU3yGNSYDXLs6/VnP28BKpGktjiCkiQ1yYCSJDXJgJIkNcmA\nkiQ1yYCSJDXJgJIkNcmAkiQ1yYCSJDXJgJIkNcmAkiQ1yYCSJDXJgJIkNcmAkiQ1ydnMJ8CgV/t1\n1nNJi4kjKElSkxxBSRqpQfcAHAj3FkyHgUdQSQ5J8qUkG/vlpUluSXJ/f3/U6MqUJE2bAxlBXQFs\nA47sl9cCm6rq3UnW9st/OeT6JGkfBzIqc7Q1uQYaQSU5Hvht4IMzVq8G1veP1wPnD7c0SdI0G3QX\n3/uAtwFPzlh3bFXtAOjvj5ntjUkuT7IlyZadO3fOq1hpsbJPpH3NGVBJXgM8VlV3HswPqKp1VbWy\nqlYuW7bsYDYhLXr2ibSvQY5BnQu8NsmrgWcARyb5OPBokuVVtSPJcuCxURYqSZouc46gqurKqjq+\nqlYAFwG3VtWlwAZgTf+yNcBNI6tSkjR15vNF3XcDr0pyP/CqflmSpKE4oC/qVtXtwO39428Dq4Zf\nkiRJTnUkSWqUASVJapIBJUlqkgElSWqSASVJapKX21hEvLChpMXEEZQkqUkGlCSpSQaUJKlJHoOS\ndMBGcRl3aW+OoCRJTTKgJElNMqAkSU3yGJQ0gQ7kGJDfe9OkcgQlSWqSASVJapIBJUlqkgElSWqS\nASVJapJn8Ula1DzjcXI5gpIkNcmAkiQ1yYCSJDXJgJIkNcmAkiQ1yYCSJDXJgJIkNcmAkiQ1yYCS\nJDXJmSS0X4N+A99v30saBUdQkqQmOYKaQgcyN5kkjcucI6gkJyS5Lcm2JFuTXNGvX5rkliT39/dH\njb5cSdK0GGQX327gLVX1AuAlwBuTnAqsBTZV1UnApn5ZkqShmDOgqmpHVX2xf/y/wDbgOGA1sL5/\n2Xrg/FEVKUmaPgd0kkSSFcAZwGbg2KraAV2IAcfs5z2XJ9mSZMvOnTvnV620SNkn0r4GDqgkRwA3\nAm+uqu8N+r6qWldVK6tq5bJlyw6mRmnRs0+kfQ0UUEkOpQunT1TVp/vVjyZZ3j+/HHhsNCVKkqbR\nIGfxBbgG2FZV753x1AZgTf94DXDT8MuTJE2rQb4HdS7wh8CXk9zVr/sr4N3AdUkuAx4CLhxNiZKk\naTRnQFXV54Hs5+lVwy1HkqSOM0lIUs/5J9viXHySpCY5gpKkA3Qg81k62jp4jqAkSU0yoCRJTTKg\nJElNMqAkSU0yoCRJTWryLL5hXvHVM2gkaTI5gpIkNcmAkiQ1yYCSJDWpyWNQkhbeMI/9SsPgCEqS\n1CRHUJo3z7qU9s95+w6eIyhJUpMMKElSkwwoSVKTDChJUpMMKElSkwwoSVKTFv1p5n75cLIM+u/l\n6biDswcmh7//P8sRlCSpSYt+BKXp5idSaXI5gpIkNcmAkiQ1yYCSJDXJY1CaSJ6Zpmk2LRPQOoKS\nJDXJgJIkNcmAkiQ1yYCSJDVpXgGV5Lwk9yV5IMnaYRUlSdJBn8WX5BDgH4FXAduBO5JsqKp7h1Wc\nJKk9C3UW4XxGUL8CPFBVX6+qHwKfAlbPY3uSJP3EfL4HdRzw8Izl7cDZe78oyeXA5f3iE0nue4pt\nPgfYNY+aFpK1jsZYar3k4N42V62fqarzBtmQfdKERVnrQf5uD80lg9U6a6/MJ6Ayy7raZ0XVOmDd\nQBtMtlTVynnUtGCsdTSmtVb7ZPysdTTmU+t8dvFtB06YsXw88K15bE+SpJ+YT0DdAZyU5MQkhwEX\nARuGU5Ykadod9C6+qtqd5E+Bm4FDgA9V1dZ51jPQLo5GWOtoWGu7P/dgWOtoTEWtqdrnsJEkSWPn\nTBKSpCYZUJKkJjURUC1PmZTkhCS3JdmWZGuSK/r1S5PckuT+/v6ocde6R5JDknwpycZ+uclakzw7\nyQ1Jvtr//Z7TcK1/3v/7fyXJPyV5xjhqbbVX7JPRmeY+GXtAzZgy6beAU4GLk5w63qp+xm7gLVX1\nAuAlwBv7+tYCm6rqJGBTv9yKK4BtM5ZbrfUqui/onQKcRldzc7UmOQ74M2BlVb2I7qSgi1jgWhvv\nFftkdKa3T6pqrDfgHODmGctXAleOu66nqPcmuvkH7wOW9+uWA/eNu7a+luP7X4JXABv7dc3VChwJ\nfIP+RJ0Z61usdc+sKUvpznzdCPzmQtc6Sb1inwytzqnuk7GPoJh9yqTjxlTLU0qyAjgD2AwcW1U7\nAPr7Y8ZX2c94H/A24MkZ61qs9fnATuDD/W6WDyY5nAZrrapvAn8HPATsAB6vqs+y8LVORK/YJ0M1\n1X3SQkANNGXSuCU5ArgReHNVfW/c9cwmyWuAx6rqznHXMoAlwJnA+6vqDOD7NLCbYjb9PvPVwInA\nc4HDk1w6jlJmWddUr9gnQzfVfdJCQDU/ZVKSQ+ma7hNV9el+9aNJlvfPLwceG1d9M5wLvDbJg3Sz\ny78iycdps9btwPaq2twv30DXiC3W+krgG1W1s6p+BHwaeCkLX2vTvWKfjMRU90kLAdX0lElJAlwD\nbKuq9854agOwpn+8hm6f+1hV1ZVVdXxVraD7e7y1qi6lzVofAR5OcnK/ahVwLw3WSrfL4iVJntn/\nPqyiO1C90LU22yv2yWhMfZ+M+8Baf+Ds1cDXgP8C3j7uevaq7VfpdqPcA9zV314NHE13kPX+/n7p\nuGvdq+5f56cHf5usFTgd2NL/3f4LcFTDtb4T+CrwFeBjwNPHUWurvWKfjLTGqe0TpzqSJDWphV18\nkiTtw4CSJDXJgJIkNcmAkiQ1yYCSJDXJgFoEkvxukkpyyrhrkVpmr0wWA2pxuBj4PN2XDiXtn70y\nQQyoCdfPfXYucBl90yV5WpKr++uybEzyr0ku6J87K8m/J7kzyc17piCRFjt7ZfIYUJPvfLprxXwN\n+E6SM4HfA1YAvwT8Md1lGvbMlfb3wAVVdRbwIeBvxlG0NAb2yoRZMu4CNG8X0106ALqJLy8GDgWu\nr6ongUeS3NY/fzLwIuCWbqosDqGbFl+aBvbKhDGgJliSo+kuuPaiJEXXRAX88/7eAmytqnMWqESp\nCfbKZHIX32S7APhoVf18Va2oqhPorr65C3hdv3/9WLoJMaG7suWyJD/ZjZHkheMoXFpg9soEMqAm\n28Xs+wnwRrqLhW2nm1H4A3RXNn28qn5I16jvSXI33YzTL124cqWxsVcmkLOZL1JJjqiqJ/pdG18A\nzq3u2jKSZrBX2uUxqMVrY5JnA4cB77LhpP2yVxrlCEqS1CSPQUmSmmRASZKaZEBJkppkQEmSmmRA\nSZKa9P/v+s4vni4LugAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(df_train['Age'].mean()) #age平均\n",
    "print(df_train['Age'].std()) #age標準差\n",
    "# 連續型用分布圖行來看\n",
    "g = sns.FacetGrid(df_train, col='Survived')\n",
    "g.map(sns.distplot, 'Age', kde=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 方法1\n",
    "* 進行一個描述性的統計（規定範圍），從而檢視哪些值是不合理的（範圍以外的值）。  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "count    714.000000\n",
      "mean      29.699118\n",
      "std       14.526497\n",
      "min        0.420000\n",
      "25%       20.125000\n",
      "50%       28.000000\n",
      "75%       38.000000\n",
      "max       80.000000\n",
      "Name: Age, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "#先透過 計算統計值， 分別呈現數量，年齡平均，標準差，最大值和最小值。\n",
    "print(df_train['Age'].describe()) #age平均\n",
    "# 透過這樣的方式，年齡最小 0.4 歲和 80 歲都在合理範圍內，所以第一關 pass。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 方法 2\n",
    "* 進行3倍標準差原則的計算，從而檢視哪些值是可疑的異常值。  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Z-score method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[630 851]\n",
      "用第二種方法的找出的 outlier 有哪些?\n",
      "630    80.0\n",
      "851    74.0\n",
      "Name: Age, dtype: float64\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\suipi\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:9: RuntimeWarning: invalid value encountered in greater\n",
      "  if __name__ == '__main__':\n"
     ]
    }
   ],
   "source": [
    "#http://colingorrie.github.io/outlier-detection.html\n",
    "import numpy as np\n",
    "\n",
    "# 創建一個函數，計算在這個資料中， ys:資料，times : 幾倍標準差，找出在這樣條件下的異常值。\n",
    "def outliers_z_score(ys,times):\n",
    "    mean_y = np.mean(ys)\n",
    "    stdev_y = np.std(ys)\n",
    "    z_scores = [(y - mean_y) / stdev_y for y in ys]\n",
    "    return np.where(np.abs(z_scores) > times)\n",
    "\n",
    "out_index=outliers_z_score(df_train['Age'],3)\n",
    "print(out_index[0])\n",
    "print(\"用第二種方法的找出的 outlier 有哪些?\")\n",
    "print(df_train.loc[out_index[0],'Age'])\n",
    "#  所以用這個方法，找出的 outlier 就是少數族群，年齡較大的，一位是 80 和 74。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 異常值的判別方法3-盒鬚圖判別法(IQR method)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "# 透過一個函數來判斷是否為異常值\n",
    "def outliers_iqr(ys,times):\n",
    "    #注意 使用 np.percentile 要去除 nan 要不然計算出來會錯誤，所以我們採用下者的程式 np.nanpercentile\n",
    "    #quartile_1, quartile_3 = np.percentile(ys, [25, 75])\n",
    "    quartile_1, quartile_3 = np.nanpercentile(ys, [25, 75])\n",
    "    iqr = quartile_3 - quartile_1\n",
    "    lower_bound = quartile_1 - (iqr * times)\n",
    "    upper_bound = quartile_3 + (iqr * times)\n",
    "    return np.where((ys > upper_bound) | (ys < lower_bound))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(array([ 33,  54,  96, 116, 280, 456, 493, 630, 672, 745, 851], dtype=int64),)\n",
      "用第三種方法的找出的 outlier 有哪些?(1.5 倍IQR)\n",
      "33     66.0\n",
      "54     65.0\n",
      "96     71.0\n",
      "116    70.5\n",
      "280    65.0\n",
      "456    65.0\n",
      "493    71.0\n",
      "630    80.0\n",
      "672    70.0\n",
      "745    70.0\n",
      "851    74.0\n",
      "Name: Age, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "out_index2=outliers_iqr(df_train['Age'],1.5)\n",
    "print(out_index2)\n",
    "print(\"用第三種方法的找出的 outlier 有哪些?(1.5 倍IQR)\")\n",
    "print(df_train.loc[out_index2[0],'Age'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(array([630, 851], dtype=int64),)\n",
      "用第三種方法的找出的 outlier 有哪些?(2 倍IQR)\n",
      "630    80.0\n",
      "851    74.0\n",
      "Name: Age, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "out_index3=outliers_iqr(df_train['Age'],2)\n",
    "print(out_index3)\n",
    "print(\"用第三種方法的找出的 outlier 有哪些?(2 倍IQR)\")\n",
    "print(df_train.loc[out_index3[0],'Age'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 畫盒鬚圖\n",
    "方法：利用matplotlib包中axes物件的boxplot()方法。，\n",
    "透過 whis 來設定 IQR 的倍數"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAQoElEQVR4nO3df6yeZ13H8feHbjBgwFp2OsugFLRi\nSQ1DThCFYEqZjii0iRmw8KOBahMjA9TEFKoBEqszwZ+NMVY6OAtQGDBcRYI0xyJpgsgZDN0opIjb\nGNT2sJXwc7jNr388d+HQnu7c55znOWfX+n4lJ9dzX8/9PNf3/NHPrl3nvu8rVYUkqT0PW+4CJEkL\nY4BLUqMMcElqlAEuSY0ywCWpUQa4JDXKAJfmIcm7kvzRctchgQGuhiS5Lcn3k3wnyckk/5TkSSMY\n5xNJ7unG+UaSG5KsWcD3VJKfGnZ90ikGuFrz4qq6EFgDHAf2jGic13Xj/DRwEfAXIxpHWjADXE2q\nqnuADwJPP9WX5HFJrksyneT2JH+Q5GFJViW5M8mLu/MuTPLlJK/uMc7dwIeAjbO9n+Q3u++6O8mB\nJE/o+j/ZnfL5bib/skX+ytIZDHA1KcmjgJcB/zajew/wOOCpwC8BrwZe04Xwa4G/T7KawWz65qq6\nrsc4FwO/DnxulvdeAPwJ8FIG/0dwO/A+gKp6fnfaM6rqwqp6/0J+T+mBnLfcBUjz9A9J7gMuBE4A\nvwKQZAWDQH9mVX0b+HaSPwNeBeyrqo8n+QAwCTwe+Nk5xvnrJG8Hvgt8AvjdWc55BXBtVX22q+FN\nwMkk66rqtsX9mtLcnIGrNVur6iLgEcDrgH9N8hPAxcDDGcyCT7kduHTG8V4GSyHvrKq75hjn9VV1\nUVVdWlWvqKrpWc55wszxquo7wF2njSmNjAGuJlXV/VV1A3A/8DzgG8C9wJNnnLYW+Br8cIb+d8B1\nwG8N6eqQr88cL8mjGczuvzaE75bmZICrSRnYAqwEjlTV/cD1wO4kj0nyZAbLHu/uPvLmrn0t8Hbg\nui7UF+O9wGuSXJbkEcAfA5+esXxynMF6vDQSBrha849JvgN8C9gNbKuqW7v3rmawZv0V4DCDgL02\nybMYhPmru6D/U6CAnYsppKomgT9kcJXKMeAngZfPOOWtwESSbyZ56WLGkmYTN3SQpDY5A5ekRhng\nktQoA1ySGmWAS1KjlvROzIsvvrjWrVu3lENKUvNuuummb1TV2On9Sxrg69atY2pqaimHlKTmJbl9\ntn6XUCSpUQa4JDXKAJekRhngktQoA1ySGtUrwJP8TpJbk9ySZH+SC7ptqg4mOdq1K0ddrDRs+/fv\nZ+PGjaxYsYKNGzeyf//+5S5J6m3OAE9yKfB6YLyqNgIrGDxxbScwWVXrGexysqgnu0lLbf/+/eza\ntYs9e/Zwzz33sGfPHnbt2mWIqxl9l1DOAx6Z5DzgUQweZL8FmOjenwC2Dr88aXR2797Nvn372LRp\nE+effz6bNm1i37597N69e7lLk3rp9TjZJG9g8Ozl7wMfr6pXJPlmt7XVqXNOVtUZyyhJdgA7ANau\nXfus22+f9Xp0acmtWLGCe+65h/PPP/+Hfffeey8XXHAB999//zJWJv24JDdV1fjp/X2WUFYymG0/\nhcEegI9O8sq+A1fV3qoar6rxsbEz7gSVls2GDRs4fPjwj/UdPnyYDRs2LFNF0vz0WUJ5IfDfVTVd\nVfcCNwC/CBxPsgaga0+Mrkxp+Hbt2sX27ds5dOgQ9957L4cOHWL79u3s2rVruUuTeunzLJQ7gOck\neRSDJZTNwBSDrau2Add07Y2jKlIahauuugqAq6++miNHjrBhwwZ27979w37pwa7vGvjbgJcB9wGf\nA34DuJDBJrJrGYT8lVV19wN9z/j4ePkwK0man7Otgfd6GmFVvQV4y2ndP2AwG5ckLQPvxJSkRhng\nktQoA1ySGmWAS1KjDHBJapQBLkmNMsB1TvNxsmrZku5KLz2YnHqc7L59+3je857H4cOH2b59O4B3\nY6oJve7EHBbvxNSDycaNG9mzZw+bNm36Yd+hQ4e4+uqrueWWW5axMunHne1OTANc5ywfJ6tWLPhx\nstJDlY+TVesMcJ2zfJysWucfMXXO8nGyap1r4JL0IOcauCQ9xBjgktSoPpsaPy3JzTN+vpXkjUlW\nJTmY5GjXnrEjvSRpdOYM8Kr6UlVdVlWXAc8Cvgd8GNgJTFbVemCyO5YeFJIsyY+0nOa7hLIZ+K+q\nuh3YAkx0/RPA1mEWJi1GVc3rZyGfWcoLAKTZzDfAXw6cetrPJVV1DKBrV8/2gSQ7kkwlmZqenl54\npZKkH9M7wJM8HHgJ8IH5DFBVe6tqvKrGx8bG5lufJOks5jMDfxHw2ao63h0fT7IGoGtPDLs4SdLZ\nzSfAr+JHyycAB4Bt3ettwI3DKkqSNLdeAZ7kUcDlwA0zuq8BLk9ytHvvmuGXJ0k6m17PQqmq7wGP\nP63vLgZXpUiSloF3YkpSowxwSWqUAS5JjTLAJalRBrgkNcoAl6RGGeCS1CgDXJIaZYBLUqMMcElq\nlAEuSY0ywCWpUQa4JDXKAJekRhngktQoA1ySGtV3R56LknwwyReTHEnyC0lWJTmY5GjXrhx1sZKk\nH+k7A/8r4GNV9TPAM4AjwE5gsqrWA5PdsSRpicwZ4EkeCzwf2AdQVf9bVd8EtgAT3WkTwNZRFSlJ\nOlOfGfhTgWngnUk+l+QdSR4NXFJVxwC6dvVsH06yI8lUkqnp6emhFS5J57o+AX4e8HPA31bVM4Hv\nMo/lkqraW1XjVTU+Nja2wDIlSafrE+B3AndW1ae74w8yCPTjSdYAdO2J0ZQoSZrNnAFeVf8DfDXJ\n07quzcAXgAPAtq5vG3DjSCqUJM3qvJ7nXQ28J8nDga8Ar2EQ/tcn2Q7cAVw5mhIlSbPpFeBVdTMw\nPstbm4dbjiSpL+/ElKRGGeCS1CgDXJIaZYBLUqMMcElqlAEuSY0ywCWpUQa4JDXKAJekRhngktQo\nA1ySGmWAS1KjDHBJapQBLkmNMsAlqVEGuCQ1qteGDkluA74N3A/cV1XjSVYB7wfWAbcBL62qk6Mp\nU5J0uvnMwDdV1WVVdWpnnp3AZFWtByaZx071kqTFW8wSyhZgons9AWxdfDmSpL76BngBH09yU5Id\nXd8lVXUMoGtXz/bBJDuSTCWZmp6eXnzFkiSg/670z62qrydZDRxM8sW+A1TVXmAvwPj4eC2gRknS\nLHrNwKvq6117Avgw8GzgeJI1AF17YlRFSpLONGeAJ3l0kseceg38MnALcADY1p22DbhxVEVKks7U\nZwnlEuDDSU6d/96q+liSzwDXJ9kO3AFcOboyJUmnmzPAq+orwDNm6b8L2DyKoiRJc/NOTElqlAEu\nSY0ywCWpUQa4JDXKAJekRhngktQoA1ySGmWAS1KjDHBJapQBLkmNMsAlqVEGuCQ1ygCXpEYZ4JLU\nKANckhrVO8CTrEjyuSQf6Y5XJTmY5GjXrhxdmZKk081nBv4G4MiM453AZFWtBya7Y0nSEukV4Eme\nCPwq8I4Z3VuAie71BLB1uKVJkh5I3xn4XwK/D/zfjL5LquoYQNeuHnJtkqQH0GdX+l8DTlTVTQsZ\nIMmOJFNJpqanpxfyFZKkWfSZgT8XeEmS24D3AS9I8m7geJI1AF17YrYPV9XeqhqvqvGxsbEhlS1J\nmjPAq+pNVfXEqloHvBz4l6p6JXAA2Nadtg24cWRVSpLOsJjrwK8BLk9yFLi8O5YkLZHz5nNyVX0C\n+ET3+i5g8/BLkiT1Ma8Al5bDqlWrOHny5MjHSTLS71+5ciV33333SMfQucUA14PeyZMnqarlLmPR\nRv0fCJ17fBaKJDXKAJekRhngktQoA1ySGmWAS1KjDHBJapQBLkmNMsAlqVEGuCQ1ygCXpEYZ4JLU\nKANckhplgEtSowxwSWpUn02NL0jy70k+n+TWJG/r+lclOZjkaNeuHH25kqRT+szAfwC8oKqeAVwG\nXJHkOcBOYLKq1gOT3bEkaYn02dS4quo73eH53U8BW4CJrn8C2DqSCiVJs+q1Bp5kRZKbgRPAwar6\nNHBJVR0D6NrVZ/nsjiRTSaamp6eHVbcknfN6BXhV3V9VlwFPBJ6dZGPfAapqb1WNV9X42NjYQuuU\nJJ1mXlehVNU3GexKfwVwPMkagK49MfTqJEln1ecqlLEkF3WvHwm8EPgicADY1p22DbhxVEVKks7U\nZ1f6NcBEkhUMAv/6qvpIkk8B1yfZDtwBXDnCOiVJp5kzwKvqP4BnztJ/F7B5FEVJM9VbHgtvfdxy\nl7Fo9ZbHLncJeojpMwOXllXe9i2qarnLWLQk1FuXuwo9lHgrvSQ1ygCXpEYZ4JLUKANckhplgEtS\nowxwSWqUAS5JjTLAJalRBrgkNcoAl6RGGeCS1CgDXJIaZYBLUqMMcElqlAEuSY3qs6Xak5IcSnIk\nya1J3tD1r0pyMMnRrl05+nIlSaf0mYHfB/xeVW0AngP8dpKnAzuByapaD0x2x5KkJTJngFfVsar6\nbPf628AR4FJgCzDRnTYBbB1VkZKkM81rS7Uk6xjsj/lp4JKqOgaDkE+y+iyf2QHsAFi7du1iatU5\nLMlyl7BoK1e6yqjh6h3gSS4EPgS8saq+1fcfVFXtBfYCjI+Pt7+xoZbcUuyHmeQhse+mzi29rkJJ\ncj6D8H5PVd3QdR9PsqZ7fw1wYjQlSpJm0+cqlAD7gCNV9ecz3joAbOtebwNuHH55kqSz6bOE8lzg\nVcB/Jrm563szcA1wfZLtwB3AlaMpUZI0mzkDvKoOA2db8N483HIkSX15J6YkNcoAl6RGGeCS1CgD\nXJIaZYBLUqMMcElqlAEuSY0ywCWpUQa4JDXKAJekRhngktQoA1ySGmWAS1KjDHBJapQBLkmNMsAl\nqVF9tlS7NsmJJLfM6FuV5GCSo13rdtuStMT6zMDfBVxxWt9OYLKq1gOT3bEkaQnNGeBV9Ung7tO6\ntwAT3esJYOuQ65IkzWGha+CXVNUxgK5dfbYTk+xIMpVkanp6eoHDSZJON/I/YlbV3qoar6rxsbGx\nUQ8nSeeMhQb48SRrALr2xPBKkiT1sdAAPwBs615vA24cTjmSpL76XEa4H/gU8LQkdybZDlwDXJ7k\nKHB5dyxJWkLnzXVCVV11lrc2D7kWSdI8eCemJDXKAJekRhngktQoA1ySGmWAS1KjDHBJapQBLkmN\nMsAlqVEGuCQ1ygCXpEYZ4JLUKANckhplgEtSowxwSWqUAS5JjTLAJalRiwrwJFck+VKSLyfZOayi\nJElzW3CAJ1kB/A3wIuDpwFVJnj6swiRJD2zOLdUewLOBL1fVVwCSvA/YAnxhGIVJi5FkST5TVfP+\njDQsiwnwS4Gvzji+E/j5009KsgPYAbB27dpFDCf1Z7DqXLCYNfDZpitn/Kupqr1VNV5V42NjY4sY\nTpI002IC/E7gSTOOnwh8fXHlSJL6WkyAfwZYn+QpSR4OvBw4MJyyJElzWfAaeFXdl+R1wD8DK4Br\nq+rWoVUmSXpAi/kjJlX1UeCjQ6pFkjQP3okpSY0ywCWpUQa4JDUqS3nDQ5Jp4PYlG1Dq72LgG8td\nhHQWT66qM26kWdIAlx6skkxV1fhy1yHNh0soktQoA1ySGmWASwN7l7sAab5cA5ekRjkDl6RGGeCS\n1CgDXOe0JNcmOZHkluWuRZovA1znuncBVyx3EdJCGOA6p1XVJ4G7l7sOaSEMcElqlAEuSY0ywCWp\nUQa4JDXKANc5Lcl+4FPA05LcmWT7ctck9eWt9JLUKGfgktQoA1ySGmWAS1KjDHBJapQBLkmNMsAl\nqVEGuCQ16v8BJGmmIX5imIAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 使用np.isnana(data)，找出在 age中的遺失值，然後逐位反轉，讓遺失值為 0,則可以透過索引的方式，濾掉遺失值。\n",
    "plt.boxplot(df_train['Age'][~np.isnan(df_train['Age'])],whis=2)    #作圖\n",
    "plt.title('Box Plot')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 但在用任何方法後，都需要做判斷，這是 outlier 嗎?\n",
    "* 需有業務面的資訊可以輔助你回答這問題，   \n",
    "在鐵達尼案例中，年紀的似乎沒有 outlier，\n",
    "但有時候 outlier 判斷無法從單一變數檢視出來，\n",
    "假如 80.0 的這個人，但是在姓名中，發現她可能是個小孩，那 80 歲的年紀，就有可能是 outlier。"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "Day33.YOLO 細節理解 - 網路輸出的後處理_HW.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}