scikit-learn-基本功能.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xjjJlcaRduED"
   },
   "source": [
    "# scikit-learn-基本功能\n",
    "\n",
    "## 教學目標\n",
    "\n",
    "這份教學的目標是介紹基本的 `scikit-learn` 功能，並學習撰寫機器學習的程式碼。\n",
    "\n",
    "## 適用對象\n",
    "\n",
    "適用於有程式基礎，且擁有 python、`numpy`、`pandas` 與 `matplotlib` 基礎的學生。\n",
    "\n",
    "若沒有先學過 python，請參考 [python-入門語法](./python-入門語法.ipynb) 教學。\n",
    "\n",
    "若沒有先學過 `numpy`，請參考 [numpy-基本功能](./numpy-基本功能.ipynb) 教學。\n",
    "\n",
    "若沒有先學過 `pandas`，請參考 [pandas-基本功能](./pandas-基本功能.ipynb) 教學。\n",
    "\n",
    "若沒有先學過 `matplotlib`，請參考 [matplotlib-資料視覺化](./matplotlib-資料視覺化.ipynb) 教學。\n",
    "\n",
    "## 執行時間\n",
    "\n",
    "本教學全部執行時間約為 2.0011203289031982 秒。\n",
    "\n",
    "|測試環境|名稱|\n",
    "|-|-|\n",
    "|主機板|X570 AORUS ELITE|\n",
    "|處理器|AMD Ryzen 7 3700X 8-Core Processor|\n",
    "|記憶體|Kingston KHX3200C16D4/16GX|\n",
    "|硬碟|Seagate ST1000DM003-1ER1|\n",
    "|顯示卡|GeForce RTX 2080|\n",
    "|作業系統|Ubuntu 18.04 LTS|"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b4T2fs87duEJ"
   },
   "source": [
    "## 大綱\n",
    "\n",
    "- [簡介](#簡介)\n",
    "- [安裝](#安裝)\n",
    "- [機器學習流程](#機器學習流程)\n",
    "- [資料前處理](#資料前處理)\n",
    "- [機器學習模型](#機器學習模型)\n",
    "- [模型驗證](#模型驗證)\n",
    "- [評估方法](#評估方法)\n",
    "- [練習](#練習)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "z20dVKbzduEK"
   },
   "source": [
    "## 簡介\n",
    "\n",
    "`scikit-learn` 是以 python 為介面的**機器學習函式庫**，提供種類豐富的**機器學習模型**與**分析工具**，能夠套用在各式各樣的資料集上。\n",
    "\n",
    "根據 [scikit-learn 官方網站](https://scikit-learn.org/stable/)（v0.22）：\n",
    "\n",
    "> Simple and efficient tools for predictive data analysis\n",
    "> \n",
    "> 簡單且有效率的預測型資料分析工具\n",
    "> \n",
    "> Accessible to everybody, and reusable in various contexts\n",
    ">\n",
    "> 容易取得，能夠在不同的環境下重複利用執行結果\n",
    ">\n",
    "> Built on NumPy, SciPy, and matplotlib\n",
    ">\n",
    "> 基於 `numpy`、`scipy` 與 `matplotlib` 等工具進行開發\n",
    ">\n",
    "> Open source, commercially usable - BSD license\n",
    ">\n",
    "> 以 BSD 授權條款開源且可以進行商業使用"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FIFYSXMIduEL"
   },
   "source": [
    "## 安裝\n",
    "\n",
    "透過 `pip` 安裝 `scikit-learn`：\n",
    "\n",
    "```sh\n",
    "pip install scikit-learn\n",
    "```\n",
    "\n",
    "❌ 請注意不是 ```\n",
    "pip install sklearn\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "96l-_vcBduEL"
   },
   "outputs": [],
   "source": [
    "# 匯入 scikit-learn 模組\n",
    "# 在 python 中的介面名稱為 sklearn\n",
    "\n",
    "import sklearn\n",
    "\n",
    "# 處理資料時會用到的工具\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "# 儲存模型的工具\n",
    "import pickle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "F3JtYX5QduEN"
   },
   "source": [
    "## 機器學習流程\n",
    "\n",
    "機器學習基本上分成以下流程：\n",
    "\n",
    "1. **訓練**模型\n",
    "2. **驗證**與**測試**模型\n",
    "3. 轉換為**應用**程式\n",
    "\n",
    "### 訓練模型（Training）\n",
    "\n",
    "1. 給予訓練資料\n",
    "2. 資料前處理\n",
    "3. 選擇機器學習模型\n",
    "4. 設定模型參數\n",
    "5. 使用 `model.fit()` 進行模型訓練\n",
    "6. 確認模型是否訓練成功\n",
    "7. 保存訓練後的模型\n",
    "\n",
    "### 驗證與測試（Validation & Test）\n",
    "\n",
    "1. 給予驗證/測試資料\n",
    "2. 資料前處理\n",
    "3. 載入訓練過的模型\n",
    "4. 使用 `model.predict()` 進行預測\n",
    "5. 評估預測結果\n",
    "\n",
    "### 應用（Deploy）\n",
    "\n",
    "1. 串接資料流（包含前處理）\n",
    "2. 決定是否動態載入模型\n",
    "3. 預測輸入資料\n",
    "4. 轉換預測結果成為可解讀的資訊"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "DHaXmskLduEN"
   },
   "outputs": [],
   "source": [
    "# 訓練模型\n",
    "\n",
    "# 1. 給予訓練資料\n",
    "# 此範例中我們要學習的目標是 y = 10x - 12\n",
    "train_x = np.random.rand(10, 1) * 10\n",
    "train_y = 10 * train_x - 12\n",
    "\n",
    "# 2. 資料前處理\n",
    "# 此範例中不需要進行前處理\n",
    "\n",
    "# 3. 選擇機器學習模型\n",
    "# 此範例中我們選擇 Linear Regression 模型\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "# 4. 設定模型參數\n",
    "# 此範例中我們參考 Linear Regression 預設值\n",
    "# 將 fit_intercept 設為 True\n",
    "\n",
    "model = LinearRegression(\n",
    "    fit_intercept=True, # 是否考慮到截距來最佳化模型\n",
    ")\n",
    "\n",
    "# 5. 使用 model.fit 進行模型訓練\n",
    "# 模型輸入為 train_x，預測目標為 train_y\n",
    "model.fit(train_x, train_y)\n",
    "\n",
    "# 6. 確認模型是否訓練成功\n",
    "# 此範例中使用 model.score 與視覺化的方式觀察\n",
    "print(model.score(train_x, train_y))\n",
    "plt.scatter(\n",
    "    train_x,\n",
    "    train_y,\n",
    "    marker='1'\n",
    ")\n",
    "plt.scatter(\n",
    "    train_x,\n",
    "    model.predict(train_x),\n",
    "    marker='2',\n",
    "    color='red'\n",
    ")\n",
    "plt.show()\n",
    "\n",
    "# 7. 保存訓練後的模型\n",
    "# 使用 pickle 將模型儲存於 ./model.pickle\n",
    "# wb: write in binary mode\n",
    "pickle.dump(model, open('./model.pickle', 'wb'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "hCSaSfVNduEO"
   },
   "outputs": [],
   "source": [
    "# 驗證與測試\n",
    "\n",
    "# 1. 給予驗證/測試資料\n",
    "# 此範例中我們要測試模型是否學到 y = 10x - 12\n",
    "test_x = np.random.rand(5, 1) * 10\n",
    "test_y = 10 * test_x - 12\n",
    "\n",
    "# 2. 資料前處理\n",
    "# 此範例中不需要進行前處理\n",
    "\n",
    "# 3. 載入訓練過的模型\n",
    "# 使用 pickle 將儲存於 ./data/model.pickle 的模型讀入\n",
    "# rb: read in binary mode\n",
    "model = pickle.load(open('./model.pickle', 'rb'))\n",
    "\n",
    "# 4. 使用 `model.predict()` 進行預測\n",
    "# 模型輸入為 test_x，將預測結果保留在 pred_y\n",
    "pred_y = model.predict(test_x)\n",
    "\n",
    "# 5. 評估預測結果\n",
    "# 此範例中使用 model.score 與視覺化的方式觀察\n",
    "print(model.score(test_x, test_y))\n",
    "plt.scatter(\n",
    "    test_x,\n",
    "    test_y,\n",
    "    marker='1'\n",
    ")\n",
    "plt.scatter(\n",
    "    test_x,\n",
    "    pred_y,\n",
    "    marker='2',\n",
    "    color='red'\n",
    ")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "h7bfbQQoduEP"
   },
   "outputs": [],
   "source": [
    "# 應用\n",
    "\n",
    "# 1. 串接資料流（包含前處理）\n",
    "# 此範例中將流程改為可以只處理一筆資料\n",
    "def stream_data(model, x):\n",
    "    y = model.predict([[x]])\n",
    "    return y[0, 0]\n",
    "\n",
    "# 2. 決定是否動態載入模型\n",
    "# 此範例中不使用動態載入模型，所有程式共享一份記憶體\n",
    "model = pickle.load(open('./model.pickle', 'rb'))\n",
    "\n",
    "# 3. 預測輸入資料\n",
    "# 使用事先載入記憶體中的模型 model 預測單一輸入 x\n",
    "# 將預測結果保留在 output_y\n",
    "input_x = 1\n",
    "output_y = stream_data(model, input_x)\n",
    "\n",
    "# 4. 轉換預測結果成為可解讀的資訊X\n",
    "# 單純將輸入與預測結果結合後輸出\n",
    "print('input {} will get {}'.format(input_x, output_y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PnQ4PfXzduEQ"
   },
   "source": [
    "## 資料前處理\n",
    "\n",
    "讀取資料集後並對資料進行前處理，包含：\n",
    "\n",
    "- 處理**不存在**或是**遺失**的值\n",
    "- 去除**離群值（Outlier）**\n",
    "- **取出**需要分析的資料欄位\n",
    "- 針對**不同資料型態**進行特別處理，通常分為兩種型態：\n",
    "    - **數值**型態\n",
    "    - **類別**型態"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8PL_IssbduEQ"
   },
   "source": [
    "### 資料集\n",
    "\n",
    "本教學使用 [Kaggle Titanic](https://www.kaggle.com/c/titanic/overview) 所提供的資料，根據鐵達尼號乘客資料**預測生還者**。\n",
    "\n",
    "點選[資料集分頁](https://www.kaggle.com/c/titanic/data)後，點擊 `Download All` 下載所有資料並解壓縮，或是只下載 `train.csv`, `test.csv`。\n",
    "\n",
    "將下載後的資料放入 `course_material/data` 資料夾中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "TCClqKbEduER"
   },
   "outputs": [],
   "source": [
    "# 資料集\n",
    "# 讀取訓練資料\n",
    "train_df = pd.read_csv('./data/train.csv')\n",
    "# 預覽訓練資料前 5 筆\n",
    "train_df.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "PnrYZymuduER"
   },
   "outputs": [],
   "source": [
    "# 輸出訓練資料型態與大小\n",
    "train_df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "mWIgXH0JduER"
   },
   "outputs": [],
   "source": [
    "# 輸出訓練資料統計資訊\n",
    "train_df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "SkVV2zRXduES"
   },
   "outputs": [],
   "source": [
    "# 取出訓練資料需要分析的資料欄位\n",
    "train_x = train_df[['Sex', 'Age']] \n",
    "# 取出訓練資料的答案\n",
    "train_y = train_df['Survived']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QmzU-P3nduES"
   },
   "source": [
    "### 填補缺失值\n",
    "\n",
    "使用 `sklearn.impute.SimpleImputer` 進行**填補缺失**的動作，策略包含：\n",
    "\n",
    "|策略|意義|\n",
    "|-|-|\n",
    "|`strategy=mean`|使用平均數填補缺失值|\n",
    "|`strategy=median`|使用中位數填補缺失值|\n",
    "|`strategy=most_frequent`|使用眾數填補缺失值|\n",
    "|`strategy=costant`|使用常數填補缺失值，必須使用 `fill_value=constant` 設定填補常數|\n",
    "\n",
    "範例程式碼：\n",
    "\n",
    "```python\n",
    "from sklearn.impute import SimpleImputer # 匯入填補缺失值的工具\n",
    "\n",
    "imputer = SimpleImputer(strategy='mean') # 創造 imputer 並設定填補策略\n",
    "imputer.fit(values)                      # 根據資料學習需要填補的值\n",
    "values = imputer.transform(values)       # 填補缺失值\n",
    "```\n",
    "\n",
    "更多填補缺失值的工具請參考 [`sklearn.impute`](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.impute)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "mq7ZrJ2iduES"
   },
   "outputs": [],
   "source": [
    "# 查看資料缺失情形\n",
    "train_df.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "MQlNjmuoduES"
   },
   "outputs": [],
   "source": [
    "# 數值型態資料前處理\n",
    "\n",
    "# 匯入填補缺失值的工具\n",
    "from sklearn.impute import SimpleImputer\n",
    "\n",
    "# 創造 imputer 並設定填補策略\n",
    "imputer = SimpleImputer(strategy='mean')\n",
    "age = train_x['Age'].to_numpy().reshape(-1, 1)\n",
    "# 根據資料學習需要填補的值\n",
    "imputer.fit(age)                       \n",
    "# 填補缺失值        \n",
    "train_x['Age'] = imputer.transform(age)\n",
    "\n",
    "# 顯示填補後的資料\n",
    "train_x.tail(5)                               "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eWDiEhknduET"
   },
   "source": [
    "### 數值資料（Numerical Data）\n",
    "\n",
    "若資料型態為**數值**，且包含**範圍**很大（例如最小值與最大值差超過 $10^2$），則可以考慮以下作法：\n",
    "\n",
    "#### 標準化（Normalization）\n",
    "\n",
    "![Normal Distribution](https://upload.wikimedia.org/wikipedia/commons/thumb/7/74/Normal_Distribution_PDF.svg/680px-Normal_Distribution_PDF.svg.png)\n",
    "\n",
    "將數值縮小到**方便計算**的範圍，例如 $[0, 1]$ 之間；常見作法為 $\\frac{x - \\mu}{\\sigma}$\n",
    "\n",
    "- $x$ 為每一筆資料\n",
    "- $\\mu$ 為所有資料的平均值\n",
    "- $\\sigma$ 為所有資料的標準差\n",
    "\n",
    "範例程式碼 -- 使用 [StandardScaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html)：\n",
    "\n",
    "```python\n",
    "from sklearn.preprocessing import StandardScaler # 匯入標準化的工具\n",
    "\n",
    "normalizer = StandardScaler()                    # 創造標準化工具\n",
    "normalizer.fit(values)                           # 根據資料學習平均值與標準差\n",
    "values = normalizer.transform(values)            # 進行標準化\n",
    "```\n",
    "\n",
    "其他標準化的作法可參考[維基百科 Normalization](https://en.wikipedia.org/wiki/Normalization_(statistics))。\n",
    "\n",
    "#### 轉換數值空間（Domain Transformation）\n",
    "\n",
    "![Domain Transformation](https://upload.wikimedia.org/wikipedia/commons/thumb/0/00/Population_vs_area.svg/650px-Population_vs_area.svg.png)\n",
    "\n",
    "將數值轉換到**不同的空間**；常見作法為取 $\\log(x)$。\n",
    "\n",
    "- $x$ 為每一筆資料\n",
    "- 數值必須為正數\n",
    "\n",
    "範例程式碼 -- 使用 [FunctionTransformer](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.FunctionTransformer.html)：\n",
    "\n",
    "```python\n",
    "import numpy as np\n",
    "from sklearn.preprocessing import FunctionTransformer # 匯入轉換數值的工具\n",
    "\n",
    "transformer = FunctionTransformer(func=np.log)             # 創造轉換數值工具\n",
    "values = transformer.transform(values)                # 進行轉換數值\n",
    "```\n",
    "\n",
    "其他轉換數值的作法可參考[維基百科 Data Transformation](https://en.wikipedia.org/wiki/Data_transformation_(statistics))。\n",
    "\n",
    "#### 去除離群值（Remove Outlier）\n",
    "\n",
    "![Quartile](https://upload.wikimedia.org/wikipedia/commons/thumb/1/1a/Boxplot_vs_PDF.svg/440px-Boxplot_vs_PDF.svg.png)\n",
    "\n",
    "**丟棄不合理**的資料；常見作法為只保留 $x \\in [Q_1 - 1.5(Q_3 - Q_1), Q_3 + 1.5(Q_3 - Q_1)]$\n",
    "\n",
    "- $x$ 為每一筆資料\n",
    "- $Q_1$ 代表第一四分位距\n",
    "- $Q_3$ 代表第一四分位距\n",
    "\n",
    "範例程式碼：\n",
    "\n",
    "```python\n",
    "import numpy as np\n",
    "\n",
    "q1, q3 = np.quantile(values, [0.25, 0.75])          # 計算第一與第三四分位數\n",
    "low = q1 - 1.5 * (q3 - q1)                          # 計算下界\n",
    "high = q3 + 1.5 * (q3 - q1)                         # 計算上界\n",
    "values = values[(values >= low) & (values <= high)] # 濾除離群值\n",
    "```\n",
    "\n",
    "其他計算離群值的作法可參考[維基百科 Outlier](https://en.wikipedia.org/wiki/Outlier)。\n",
    "\n",
    "更多數值型態資料前處理的工具請參考 [`sklearn.preprocessing`](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.preprocessing)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "a0jWbuLFduET"
   },
   "source": [
    "### 類別資料（Categorical Data）\n",
    "\n",
    "若資料型態為**類別**，則可以考慮以下作法：\n",
    "\n",
    "#### Label Encoding\n",
    "\n",
    "給予每個類別 $c$ 一個**對應的數字** $n_c$，之後都**以相同的數字代表該類別**：\n",
    "而數字 $n_c$ 必須符合以下規則：\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "& n_c \\in \\{1, \\dots, c\\} \\\\\n",
    "& n_{c_1} \\neq n_{c_2} \\text{ if } c_1 \\neq c_2\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "- 優點：使用數字**方便計算**\n",
    "- 缺點：數字本身包含**距離關係**\n",
    "\n",
    "範例程式碼：\n",
    "\n",
    "```python\n",
    "from sklearn.preprocessing import LabelEncoder # 匯入 Label Encoder\n",
    "\n",
    "le = LabelEncoder()                            # 創造 Label Encoder\n",
    "le.fit(values)                                 # 給予每個類別一個數值\n",
    "values = le.transform(values)                  # 轉換所有類別成為數值\n",
    "```\n",
    "\n",
    "#### One-hot Vector\n",
    "\n",
    "給予每個類別 $c$ 一個**對應的向量** $v_c$，之後都**以相同的向量代表該類別**。\n",
    "而向量 $v_c$ 必須符合以下規則：\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "& v_c =\n",
    "\\begin{pmatrix}\n",
    "0 \\\\\n",
    "\\vdots \\\\\n",
    "0 \\\\\n",
    "1 \\\\\n",
    "0 \\\\\n",
    "\\vdots \\\\\n",
    "0\n",
    "\\end{pmatrix}\n",
    "\\in \\mathbb{R}^c \\\\\n",
    "& v_c[i] =\n",
    "\\begin{cases}\n",
    "0 & \\text{if } i \\neq c \\\\\n",
    "1 & \\text{if } i = c\n",
    "\\end{cases} \\\\\n",
    "& v_{c_1} \\neq v_{c_2} \\text{ if } c_1 \\neq c_2\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "- 優點：類別與類別之間彼此**沒有關係**\n",
    "- 缺點：需要**較多的記憶體**空間紀錄類別資訊\n",
    "\n",
    "範例程式碼：\n",
    "\n",
    "```python\n",
    "from sklearn.preprocessing import OneHotEncoder # 匯入 One-hot Encoder\n",
    "\n",
    "ohe = OneHotEncoder()                           # 創造 One-hot Encoder\n",
    "ohe.fit(values)                                 # 給予每個類別一個 One-hot Vector\n",
    "values = ohe.transform(values)                  # 轉換所有類別成為 One-hot Vector\n",
    "```\n",
    "\n",
    "更多類別型態資料前處理的工具請參考 [`sklearn.preprocessing`](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.preprocessing)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "eI-EZ0uaduEU"
   },
   "outputs": [],
   "source": [
    "# 類別型態資料前處理\n",
    "\n",
    "# 匯入 Label Encoder\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "# 創造 Label Encoder\n",
    "le = LabelEncoder()\n",
    "# 給予每個類別一個數值\n",
    "le.fit(train_x['Sex'])\n",
    "# 轉換所有類別成為數值\n",
    "train_x['Sex'] = le.transform(train_x['Sex'])  \n",
    "\n",
    "# 顯示轉換後的資料\n",
    "train_x.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ds9u_5WVduEU"
   },
   "outputs": [],
   "source": [
    "# 類別型態資料轉換為 one-hot 型式\n",
    "\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "\n",
    "reshaped_data = np.expand_dims(train_x['Sex'], axis=1)\n",
    "print(f\"擴充維度後的資料維度屬性為: {reshaped_data.shape}\")\n",
    "\n",
    "onehot_enc = OneHotEncoder()\n",
    "transformed = onehot_enc.fit_transform(reshaped_data).toarray()\n",
    "\n",
    "print(transformed)\n",
    "print(f\"Samples 數量為: {len(transformed)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xSBOxTkYduEU"
   },
   "source": [
    "## 機器學習模型\n",
    "\n",
    "在選擇機器學習模型時需要考慮以下條件：\n",
    "\n",
    "- 任務**類型**\n",
    "- 是否擁有**標記資料**\n",
    "- 擁有標記資料的**數量**\n",
    "\n",
    "### 任務類型\n",
    "\n",
    "機器學習任務類型主要分為兩種：\n",
    "\n",
    "#### 數值預測\n",
    "\n",
    "![Linear Regression](https://upload.wikimedia.org/wikipedia/commons/b/be/Normdist_regression.png)\n",
    "\n",
    "若給予一筆資料 $x$，希望能夠讓機器學習模型自動輸出對應的**數值** $y$，則我們稱任務「用 $x$ 預測**數值** $y$」為**迴歸分析（Regression）**。\n",
    "\n",
    "令訓練資料集 $D = \\{(x_1, y_1), \\dots, (x_n, y_n)\\}$。\n",
    "若迴歸模型為 $f$，參數為 $\\theta$，則目標是根據訓練資料 $D$ 找出表現最好的參數 $\\theta_*$ 並且滿足以下需求：\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "\\theta_* &= \\arg\\min_{\\theta} \\mathcal{L} \\big( f(x_1; \\theta), y_1 \\big) + \\mathcal{L} \\big( f(x_2; \\theta), y_2 \\big) + \\dots + \\mathcal{L} \\big( f(x_n; \\theta), y_n \\big) \\\\\n",
    "&= \\arg\\min_{\\theta} \\sum_{i = 1}^n \\mathcal{L} \\big( f(x_i; \\theta), y_i \\big) \\\\\n",
    "&= \\arg\\min_{\\theta} \\frac{1}{n} \\sum_{i = 1}^n \\mathcal{L} \\big( f(x_i; \\theta), y_i \\big) \\\\\n",
    "&= \\arg\\min_{\\theta} \\frac{1}{n} \\sum_{i = 1}^n \\big( f(x_i; \\theta) - y_i \\big)^2\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "- 模型**輸出**可以是**任意數字**\n",
    "    - 模型 $f$ 在樣本 $x_i$ 得到的輸出為 $f(x_i) \\in \\mathbb{R}$\n",
    "    - 樣本 $x_i$ 真正對應到的答案為 $y_i \\in \\mathbb{R}$\n",
    "- 目標函數 $\\mathcal{L}$ 稱為**最小平方差（Mean Square Error，MSE）**\n",
    "    - 訓練資料中的每一個樣本都會用於訓練，所以使用 $\\sum_{i = 1}^n$ 將所有樣本誤差結合\n",
    "    - 不論誤差是正數還是負數都是誤差，但正負誤差值直接相加會互相抵消，所以需要取平方\n",
    "    - 誤差值會透過 $\\sum_{i = 1}^n$ 累加，導致樣本數愈多誤差愈大，所以使用 $\\frac{1}{n}$ 抵消樣本數的影響\n",
    "    - 使用不同的模型參數 $\\theta$ 會影響目標函數 $\\mathcal{L}$，所以使用 $\\arg\\min_{\\theta}$ 找出 $\\min_{\\theta} \\frac{1}{n} \\sum_{i = 1}^n \\mathcal{L} \\big( f(x_i; \\theta), y_i \\big)$（即誤差最少）的參數 $\\theta_*$\n",
    "- 目標函數 $\\mathcal{L}$ 可以替換成其他作法\n",
    "    - L2 norm：$\\frac{1}{n} \\sum_{i = 1}^n \\sqrt{\\left(f(x_i; \\theta) - y_i\\right)^2}$\n",
    "    - 絕對值：$\\frac{1}{n} \\sum_{i = 1}^n \\left|f(x_i; \\theta) - y_i\\right|$\n",
    "- 更多資訊可以參考以下連結\n",
    "    - [迴歸分析](https://zh.wikipedia.org/zh-tw/%E8%BF%B4%E6%AD%B8%E5%88%86%E6%9E%90)\n",
    "    - [最小平方法](https://zh.wikipedia.org/zh-tw/%E6%9C%80%E5%B0%8F%E4%BA%8C%E4%B9%98%E6%B3%95)\n",
    "\n",
    "#### 類別預測\n",
    "\n",
    "![Classification](https://upload.wikimedia.org/wikipedia/commons/thumb/f/fe/Kernel_Machine.svg/640px-Kernel_Machine.svg.png?1586847480231)\n",
    "\n",
    "若給予一筆資料 $x$，希望能夠讓機器學習模型自動輸出對應的**類別** $y$，則我們稱任務「用 $x$ 預測**類別** $y$」為**分類問題（Classification）**。\n",
    "\n",
    "令訓練資料集 $D = \\{(x_1, y_1), \\dots, (x_n, y_n)\\}$，總共有 $C$ 個類別（即 $y_i \\in \\{1, \\dots, C\\}$）。\n",
    "若分類模型為 $f$，參數為 $\\theta$，則目標是根據訓練資料 $D$ 找出表現最好的參數 $\\theta_*$ 並且滿足以下需求：\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "\\theta_* &= \\arg\\min_{\\theta} \\mathcal{L} \\big( f(x_1; \\theta), y_1 \\big) + \\mathcal{L} \\big( f(x_2; \\theta), y_2 \\big) + \\dots + \\mathcal{L} \\big( f(x_n; \\theta), y_n \\big) \\\\\n",
    "&= \\arg\\min_{\\theta} \\sum_{i = 1}^n \\mathcal{L} \\big( f(x_i; \\theta), y_i \\big) \\\\\n",
    "&= \\arg\\min_{\\theta} \\frac{1}{n} \\sum_{i = 1}^n \\mathcal{L} \\big( f(x_i; \\theta), y_i \\big) \\\\\n",
    "&= \\arg\\min_{\\theta} \\frac{1}{n} \\sum_{i = 1}^n D_{KL} \\Big( \\mathbb{1}_{y_i} \\Big|\\Big| f(x_i;\\theta) \\Big) \\\\\n",
    "&= \\arg\\min_{\\theta} \\frac{1}{n} \\sum_{i = 1}^n \\sum_{c = 1}^C \\bigg[ - \\big( \\mathbf{1}(y_i = c) \\big) \\big( \\log f(x_i;\\theta) \\big)_c \\bigg] \\\\\n",
    "&= \\arg\\min_{\\theta} \\frac{1}{n} \\sum_{i = 1}^n \\big( - \\log f(x_i;\\theta) \\big)_{y_i} \\\\\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "- 模型**輸出**為**類別**或**機率值**\n",
    "    - 模型 $f$ 在樣本 $x_i$ 得到的輸出為 $f(x_i) \\in \\mathbb{R}^C$\n",
    "    - 樣本 $x_i$ 真正對應到的答案為 $y_i \\in \\mathbb{R}$\n",
    "- 目標函數 $\\mathcal{L}$ 稱為**交叉熵（Cross Entropy）**\n",
    "    - 訓練資料中的每一個樣本都會用於訓練，所以使用 $\\sum_{i = 1}^n$ 將所有樣本誤差結合\n",
    "    - 誤差值會透過 $\\sum_{i = 1}^n$ 累加，導致樣本數愈多誤差愈大，所以使用 $\\frac{1}{n}$ 抵消樣本數的影響\n",
    "    - $D_{KL}$ 稱為相對熵，用來比較兩個機率分佈 $D_{KL}(P||Q)$ 的相似度，相似度愈高相對熵愈低\n",
    "    - $\\mathbb{1}_{y_i} \\in \\mathbb{R}^C$ 為 one-hot vector，位置 $y_i$ 的數值為 $1$，其他為 $0$\n",
    "    - $\\mathbf{1}(y_i = c)$ 為條件式，如果 $y_i = c$ 則輸出 $1$，否則輸出 $0$\n",
    "    - 使用不同的模型參數 $\\theta$ 會影響目標函數 $\\mathcal{L}$，所以使用 $\\arg\\min_{\\theta}$ 找出 $\\min_{\\theta} \\frac{1}{n} \\sum_{i = 1}^n \\mathcal{L} \\big( f(x_i; \\theta), y_i \\big)$（即誤差最少）的參數 $\\theta_*$\n",
    "- 更多資訊可以參考以下連結\n",
    "    - [Statistical Classification](https://en.wikipedia.org/wiki/Statistical_classification)\n",
    "    - [相對熵](https://zh.wikipedia.org/zh-tw/%E7%9B%B8%E5%AF%B9%E7%86%B5)\n",
    "    - [交叉熵](https://zh.wikipedia.org/zh-tw/%E4%BA%A4%E5%8F%89%E7%86%B5)\n",
    "\n",
    "### 標記資料\n",
    "\n",
    "根據是否擁有標記資料與標記資料數量可以將訓練方法分成以下種類：\n",
    "\n",
    "|訓練方法|有無標記|常見用途|範例模型|\n",
    "|-|-|-|-|\n",
    "|監督式學習|有|訓練預測模型|`sklearn.tree.DecisionTreeClassifier`|\n",
    "|非監督式學習|無|分析資料分佈、抽取資料特徵|`sklearn.cluster.KMeans`|\n",
    "|半監督式學習|部份|監督+半監督、自動標記資料|`sklearn.semi_supervised.LabelPropagation`|\n",
    "\n",
    "### 超參數（Hyperparameters）\n",
    "\n",
    "機器學習模型擁有的參數就稱為**超參數**，調整超參數可以**影響訓練結果**，為機器學習實驗中的**控制變因**。\n",
    "\n",
    "以下給予 `sklearn.tree.DecisionTreeClassifier` 的部份超參數以及用途：\n",
    "\n",
    "|參數|意義|型態或範圍|\n",
    "|-|-|-|\n",
    "|`criterion`|最佳化模型的方法|`'gini'` 或 `'entropy'`|\n",
    "|`max_depth`|決策樹最大深度|`int`|\n",
    "|`max_leaf_nodes`|決策樹最多葉子個數|`int`|\n",
    "\n",
    "完整參數列表請參考 [`sklearn.tree.DecisionTreeClassifier`](https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html#sklearn.tree.DecisionTreeClassifier)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "k1XCBKKPduEW"
   },
   "outputs": [],
   "source": [
    "# 任務類型 & 標記資料\n",
    "# 本範例使用 titanic 資料集，所以為類別預測 + 監督式學習\n",
    "\n",
    "# 匯入決策樹模型\n",
    "from sklearn.tree import DecisionTreeClassifier \n",
    "# 匯入準確度計算工具\n",
    "from sklearn.metrics import accuracy_score      \n",
    "\n",
    "# 創造決策樹模型\n",
    "model = DecisionTreeClassifier()                \n",
    "# 訓練決策樹模型\n",
    "model.fit(train_x, train_y)                     \n",
    "\n",
    "# 確認模型是否訓練成功\n",
    "pred_y = model.predict(train_x)                 \n",
    "# 計算準確度\n",
    "acc = accuracy_score(train_y, pred_y)           \n",
    "\n",
    "# 輸出準確度\n",
    "print('accuracy: {}'.format(acc))               "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "W-0MZFDiduEX"
   },
   "outputs": [],
   "source": [
    "# 超參數\n",
    "\n",
    "# 匯入決策樹模型\n",
    "from sklearn.tree import DecisionTreeClassifier \n",
    "# 匯入準確度計算工具\n",
    "from sklearn.metrics import accuracy_score      \n",
    "\n",
    "# 創造決策樹模型\n",
    "# 設定最佳化方法為 Gini Index\n",
    "# 設定最大深度為 2\n",
    "# 設定最多葉子個數為 4\n",
    "model = DecisionTreeClassifier(                 \n",
    "    criterion='gini',                           \n",
    "    max_depth=2,                                \n",
    "    max_leaf_nodes=2 ** 2\n",
    ")                      \n",
    "# 訓練決策樹模型\n",
    "model.fit(train_x, train_y)                    \n",
    "\n",
    "# 確認模型是否訓練成功\n",
    "pred_y = model.predict(train_x)                \n",
    "# 計算準確度 \n",
    "acc = accuracy_score(train_y, pred_y)          \n",
    "\n",
    "# 輸出準確度\n",
    "print('accuracy: {}'.format(acc))              "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "wgoAPEk4duEY"
   },
   "outputs": [],
   "source": [
    "# 匯入決策樹繪圖工具\n",
    "from sklearn.tree import plot_tree       \n",
    "\n",
    "# 創造繪圖環境\n",
    "fig, ax = plt.subplots(figsize=(10, 10)) \n",
    "\n",
    "# 繪製訓練後的模型\n",
    "plot_tree(model, ax=ax)                  \n",
    "\n",
    "# 繪製圖表\n",
    "plt.show()                               "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s5MTVcg9duEY"
   },
   "source": [
    "## 模型驗證\n",
    "\n",
    "![Overfitting & Underfitting](https://upload.wikimedia.org/wikipedia/commons/thumb/1/1f/Overfitting_svg.svg/600px-Overfitting_svg.svg.png)\n",
    "\n",
    "機器學習模型在訓練資料上進行學習後可以得到三種結果：\n",
    "\n",
    "|訓練資料|測試資料|結果|\n",
    "|-|-|-|\n",
    "|表現不好||無法適應（Underfitting）|\n",
    "|表現良好|表現良好|任務完成|\n",
    "|表現良好|表現不好|過度適應（Overfitting）|\n",
    "\n",
    "### 無法適應（Underfitting）\n",
    "\n",
    "解決方法包含：\n",
    "\n",
    "- 更換模型\n",
    "- 更換演算法\n",
    "- 增加參數\n",
    "\n",
    "### 過度適應（Overfitting）\n",
    "\n",
    "解決方法包含：\n",
    "\n",
    "- 減少參數\n",
    "- 使用更多的訓練資料\n",
    "- 使用驗證資料評估模型\n",
    "\n",
    "### 驗證資料（Validation Set）\n",
    "\n",
    "使用 `sklearn.model_selection.train_test_split` 將訓練資料集分成兩個部份：\n",
    "\n",
    "- 訓練資料集（Training Set）\n",
    "    - 模型直接訓練的資料\n",
    "- 驗證資料集（Validation Set）\n",
    "    - 用來驗證模型的資料\n",
    "    - 驗證完畢後確認沒有問題，可以與訓練資料集**合併**後再次訓練\n",
    "\n",
    "```python\n",
    "from sklearn.model_selection import train_test_split # 匯入分離訓練資料與驗證資料的工具\n",
    "\n",
    "(train_x_split, valid_x_split,\n",
    " train_y_split, valid_y_split) = train_test_split(   # 分離訓練資料與驗證資料\n",
    "    train_x,                                         # 原始訓練資料\n",
    "    train_y,                                         # 原始訓練資料標記\n",
    "    test_size=0.33                                   # 分離比例\n",
    ")\n",
    "```\n",
    "\n",
    "更多的驗證方法請參考 [`sklearn.model_selection`](https://scikit-learn.org/stable/modules/cross_validation.html#cross-validation)\n",
    "\n",
    "### 交叉驗證（Cross Validation）\n",
    "\n",
    "![K-fold Cross Validation](https://scikit-learn.org/stable/_images/grid_search_cross_validation.png)\n",
    "\n",
    "使用不同的資料訓練的結果可能不同，為了消除驗證資料的**隨機性**，通常會將驗證資料用來訓練，並把訓練資料用來驗證，此過程稱為**交叉驗證**。\n",
    "可是驗證資料比起訓練資料通常會**較少**，導致模型無法適應，所以可以使用 $K$ 次交叉驗證（K-fold Cross Validation）**消除隨機性**，並將 $K$ 次的實驗結果取**平均**作為最終實驗結果。\n",
    "\n",
    "`sklearn` 提供了 `sklearn.model_selection.KFold` 進行交叉驗證：\n",
    "\n",
    "```python\n",
    "from sklearn.model_selection import KFold          # 匯入 K 次交叉驗證工具\n",
    "\n",
    "kf = KFold(n_splits=3)                             # 設定 K 值\n",
    "kf.get_n_splits(train_x)                           # 查看資料被分成幾組\n",
    "\n",
    "for train_index, valid_index in kf.split(train_x): # 每個迴圈都會產生不同部份的資料\n",
    "    train_x_split = train_x[train_index]           # 產生訓練資料\n",
    "    train_y_split = train_y[train_index]           # 產生訓練資料標籤\n",
    "    valid_x_split = train_x[valid_index]           # 產生驗證資料\n",
    "    valid_y_split = train_y[valid_index]           # 產生驗證資料標籤\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "VXCotvOtduEY"
   },
   "outputs": [],
   "source": [
    "# 驗證資料\n",
    "\n",
    "# 匯入分離訓練資料與驗證資料的工具\n",
    "from sklearn.model_selection import train_test_split \n",
    "# 匯入決策樹模型\n",
    "from sklearn.tree import DecisionTreeClassifier      \n",
    "# 匯入準確度計算工具\n",
    "from sklearn.metrics import accuracy_score           \n",
    "\n",
    "# 分離訓練資料與驗證資料\n",
    "# train_x：原始訓練資料\n",
    "# train_y：原始訓練資料標記\n",
    "# test_size：分離比例\n",
    "# random_state：控制隨機亂數\n",
    "(train_x_split, valid_x_split,\n",
    " train_y_split, valid_y_split) = train_test_split(   \n",
    "    train_x,                                         \n",
    "    train_y,                                         \n",
    "    test_size=0.33,                                  \n",
    "    random_state=1012                                \n",
    ")\n",
    "\n",
    "# 創造決策樹模型\n",
    "model = DecisionTreeClassifier(random_state=1012)    \n",
    "# 訓練決策樹模型\n",
    "model.fit(train_x_split, train_y_split)              \n",
    "\n",
    "# 確認模型是否訓練成功\n",
    "train_pred_y = model.predict(train_x_split)          \n",
    "# 計算訓練資料準確度\n",
    "train_acc = accuracy_score(\n",
    "    train_y_split,            \n",
    "    train_pred_y\n",
    ")\n",
    "# 驗證模型是否訓練成功\n",
    "valid_pred_y = model.predict(valid_x_split)          \n",
    "# 計算驗證資料準確度\n",
    "valid_acc = accuracy_score(\n",
    "    valid_y_split,            \n",
    "    valid_pred_y\n",
    ")\n",
    "\n",
    "# 輸出訓練準確度\n",
    "print('train accuracy: {}'.format(train_acc))\n",
    "# 輸出驗證準確度\n",
    "print('valid accuracy: {}'.format(valid_acc))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "7q9FqWsMduEY"
   },
   "outputs": [],
   "source": [
    "# 交叉驗證\n",
    "\n",
    "# 匯入 K 次交叉驗證工具\n",
    "from sklearn.model_selection import KFold             \n",
    "# 匯入決策樹模型\n",
    "from sklearn.tree import DecisionTreeClassifier       \n",
    "# 匯入準確度計算工具\n",
    "from sklearn.metrics import accuracy_score            \n",
    "\n",
    "# 設定 K 值\n",
    "kf = KFold(\n",
    "    n_splits=3,                                \n",
    "    random_state=1012,\n",
    "    shuffle=True\n",
    ")\n",
    "# 查看資料被分成幾組\n",
    "kf.get_n_splits(train_x)                            \n",
    "\n",
    "# 儲存每次訓練模型的準確度\n",
    "train_acc_list = []                                   \n",
    "# 儲存每次驗證模型的準確度\n",
    "valid_acc_list = []                                   \n",
    "\n",
    "# 每個迴圈都會產生不同部份的資料\n",
    "for train_index, valid_index in kf.split(train_x):    \n",
    "    # 產生訓練資料\n",
    "    train_x_split = train_x.iloc[train_index]         \n",
    "    # 產生訓練資料標籤\n",
    "    train_y_split = train_y.iloc[train_index]         \n",
    "    # 產生驗證資料\n",
    "    valid_x_split = train_x.iloc[valid_index]         \n",
    "    # 產生驗證資料標籤\n",
    "    valid_y_split = train_y.iloc[valid_index]         \n",
    "    \n",
    "    # 創造決策樹模型\n",
    "    model = DecisionTreeClassifier(random_state=1012) \n",
    "    # 訓練決策樹模型\n",
    "    model.fit(train_x_split, train_y_split)           \n",
    "    \n",
    "    # 確認模型是否訓練成功\n",
    "    train_pred_y = model.predict(train_x_split)       \n",
    "    # 計算訓練資料準確度\n",
    "    train_acc = accuracy_score(\n",
    "        train_y_split,         \n",
    "        train_pred_y\n",
    "    )\n",
    "    # 驗證模型是否訓練成功\n",
    "    valid_pred_y = model.predict(valid_x_split)       \n",
    "    # 計算驗證資料準確度\n",
    "    valid_acc = accuracy_score(\n",
    "        valid_y_split,         \n",
    "        valid_pred_y\n",
    "    )\n",
    "    \n",
    "    train_acc_list.append(train_acc)\n",
    "    valid_acc_list.append(valid_acc)\n",
    "\n",
    "\n",
    "# 輸出平均訓練準確度、最低訓練準確度、最高訓練準確度\n",
    "print((\n",
    "    'average train accuracy: {}\\n' +\n",
    "    'min train accuracy: {}\\n' +\n",
    "    'max train accuracy: {}').format(\n",
    "    np.mean(train_acc_list),                          \n",
    "    np.min(train_acc_list),                           \n",
    "    np.max(train_acc_list)                            \n",
    "))\n",
    "print()\n",
    "\n",
    "# 輸出平均驗證準確度、最低驗證準確度、最高驗證準確度\n",
    "print((\n",
    "    'average valid accuracy: {}\\n' +\n",
    "    'min valid accuracy: {}\\n' +\n",
    "    'max valid accuracy: {}').format(\n",
    "    np.mean(valid_acc_list),                          \n",
    "    np.min(valid_acc_list),                           \n",
    "    np.max(valid_acc_list)                            \n",
    "))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ysafI8hOduEZ"
   },
   "source": [
    "## 評估方法\n",
    "\n",
    "根據任務類型可以選擇不同的評估方法\n",
    "\n",
    "### 數值預測\n",
    "\n",
    "數值預測的評估方法通常都是計算**預測數值**與**真實答案**的**差距**：\n",
    "\n",
    "|評估方法|意義|公式|\n",
    "|-|-|-|\n",
    "|`sklearn.metrics.mean_squared_error`|計算均方誤差（Mean Squared Error）|$$\\frac{1}{n}\\sum_{i=1}^n\\big(f(x_i;\\theta)-y_i\\big)^2$$|\n",
    "|`sklearn.metrics.mean_squared_log_error`|將誤差經過 $\\log$ 轉換後再計算均方誤差|$$\\frac{1}{n}\\sum_{i=1}^n\\Big(\\log\\big(1+f(x_i;\\theta)\\big)-\\log\\big(1+y_i\\big)\\Big)^2$$|\n",
    "|`sklearn.metrics.mean_absolute_error`|計算平均絕對誤差（Mean Absolute Error）|$$\\frac{1}{n}\\sum_{i=1}^n\\big\\vert f(x_i;\\theta)-y_i\\big\\vert$$|\n",
    "\n",
    "更多數值預測的評估方法請參考 [`sklearn.model_evaluation`](https://scikit-learn.org/stable/modules/model_evaluation.html#regression-metrics)\n",
    "\n",
    "### 類別預測\n",
    "\n",
    "理解類別預測的評估方法需要理解**混淆矩陣（Confusion Matrix）**：\n",
    "\n",
    "![Confusion Matrix](https://upload.wikimedia.org/wikipedia/commons/thumb/e/e7/Sensitivity_and_specificity.svg/700px-Sensitivity_and_specificity.svg.png)\n",
    "\n",
    "|與標記答案是否相同\\模型預測|模型預測答案為正（Positive，P）|模型預測答案為負（Negative，N）|\n",
    "|-|-|-|\n",
    "|與標記答案相同（True，T）|模型預測答案為正且與標記答案相同（True Positive，TP）|模型預測答案為負且與標記答案相同（True Negative，TN）|\n",
    "|與標記答案不相同（False，F）|模型預測答案為正但與標記答案不相同（False Positive，FP）|模型預測答案為負但與標記答案不相同（False Negative，FN）|\n",
    "\n",
    "而類別預測的評估方法通常都是計算**預測類別**與**真實答案是否相同**：\n",
    "\n",
    "|評估方法|意義|公式|\n",
    "|-|-|-|\n",
    "|`sklearn.metrics.accuracy_score`|計算準確度（Accuracy），即預測結果與答案完全相同的比例|$$\\frac{\\text{TP}+\\text{TN}}{\\text{TP}+\\text{TN}+\\text{FP}+\\text{FN}}$$|\n",
    "|`sklearn.metrics.precision_score`|計算精密度（Precision），即預測答案為正的資料中正確的比例|$$\\frac{\\text{TP}}{\\text{TP}+\\text{FP}}$$|\n",
    "|`sklearn.metrics.recall_score`|計算召回率（Recall），即答案為正的資料中預測正確的比例|$$\\frac{\\text{TP}}{\\text{TP}+\\text{FN}}$$|\n",
    "|`sklearn.metrics.f1_score`|計算 F1 score，同時考慮 Precision 與 Recall 的分數|$$\\frac{2 \\times \\text{Precision} \\times \\text{Recall}}{\\text{Precision}+\\text{Recall}}$$|\n",
    "\n",
    "更多類別預測的評估方法請參考 [`sklearn.model_evaluation`](https://scikit-learn.org/stable/modules/model_evaluation.html#classification-metrics)\n",
    "\n",
    "### 視覺化\n",
    "\n",
    "可以將模型訓練結果以視覺化的方式呈現，以下為部份範例：\n",
    "\n",
    "|模型|方法|\n",
    "|-|-|\n",
    "|`sklearn.linear_model.LinearRegression`|將訓練資料 $x, y$ 與其預測的結果 $x, y'$ 以折線圖繪製|\n",
    "|`sklearn.tree.DecisionTreeClassifier`|透過 `sklearn.tree.plot_tree` 畫出決策樹|"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GM1CNpBj2X58"
   },
   "source": [
    "舉例:\n",
    "\n",
    "假設有1000人接受新冠肺炎確診檢查，實際上有2人確診，如果醫療檢驗設備檢查出3人為陽性，則其準確率為 999/1000=99.9%，這結果大家應該都很滿意，但是，如果有一天這台設備故障了，同樣檢查1000人，無人為陽性，這時準確率為 998/1000=99.8%，這結果大家應該很傻眼吧。\n",
    "\n",
    "使用精確率（Precision）計算看看：\n",
    "\n",
    "\n",
    "\n",
    "1.   未故障： 2/3\n",
    "2.   故障時：0/0\n",
    "    \n",
    "    應該就可以發現設備有問題了。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "bPDYeOLhduEZ"
   },
   "outputs": [],
   "source": [
    "# 類別預測 & 視覺化\n",
    "\n",
    "# 匯入決策樹模型\n",
    "from sklearn.tree import DecisionTreeClassifier \n",
    "# 匯入決策樹繪圖工具\n",
    "from sklearn.tree import plot_tree              \n",
    "# 匯入準確度計算工具\n",
    "from sklearn.metrics import accuracy_score      \n",
    "\n",
    "# 創造決策樹模型\n",
    "# 設定最佳化方法為 Gini Index\n",
    "# 設定最大深度為 3\n",
    "# 設定最多葉子個數為 8\n",
    "model = DecisionTreeClassifier(                 \n",
    "    criterion='gini',                           \n",
    "    max_depth=3,                                \n",
    "    max_leaf_nodes=2 ** 3\n",
    ")                      \n",
    "# 訓練決策樹模型\n",
    "model.fit(train_x, train_y)                     \n",
    "\n",
    "# 確認模型是否訓練成功\n",
    "pred_y = model.predict(train_x)    \n",
    "            \n",
    "# 計算準確度\n",
    "acc = accuracy_score(train_y, pred_y)          \n",
    "\n",
    "# 輸出準確度\n",
    "print('accuracy: {}'.format(acc))               \n",
    "\n",
    "# 創造繪圖環境\n",
    "fig, ax = plt.subplots(figsize=(15, 15))        \n",
    "\n",
    "# 繪製訓練後的模型\n",
    "plot_tree(model, ax=ax)                         \n",
    "\n",
    "# 繪製圖表\n",
    "plt.show()                                      "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BbMvCvZEduEZ"
   },
   "source": [
    "## 練習\n",
    "\n",
    "本練習使用 [Kaggle Titanic](https://www.kaggle.com/c/titanic/overview) 所提供的資料，根據鐵達尼號乘客資料**預測生還者**。\n",
    "\n",
    "點選[資料集分頁](https://www.kaggle.com/c/titanic/data)後，點擊 `Download All` 下載所有資料並解壓縮，或是只下載 `train.csv`, `test.csv`。\n",
    "\n",
    "將下載後的資料放入 `course_material/data` 資料夾中。\n",
    "\n",
    "### 練習 1：改善決策樹分類模型\n",
    "\n",
    "請改進 `sklearn.tree.DecisionTreeClassifier` 模型分類效果，可以嘗試：\n",
    "\n",
    "- 增加更多的輸入特徵\n",
    "- 使用不同的前處理方法\n",
    "- 調整超參數\n",
    "\n",
    "### 練習 2：使用不同的模型\n",
    "\n",
    "請使用不同的模型打敗使用 `sklearn.tree.DecisionTreeClassifier` 模型分類效果，可以嘗試課堂所提過的方法：\n",
    "\n",
    "|模型|名稱|\n",
    "|-|-|\n",
    "|[`sklearn.naive_bayes`](https://scikit-learn.org/stable/modules/classes.html?highlight=metric#module-sklearn.naive_bayes)|樸素貝氏分類器（Naive Bayes Classifier）|\n",
    "|[`sklearn.svm`](https://scikit-learn.org/stable/modules/classes.html?highlight=metric#module-sklearn.svm)|支援向量機（Support Vector Machines）|\n",
    "|[`sklearn.neighbors`](https://scikit-learn.org/stable/modules/classes.html?highlight=metric#module-sklearn.neighbors)|近鄰演算法（Nearest Neighbors）|\n",
    "|[`sklearn.ensemble`](https://scikit-learn.org/stable/modules/classes.html?highlight=metric#module-sklearn.ensemble)|集合學習（Ensemble）|"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "EBn9SP0cduEZ"
   },
   "outputs": [],
   "source": [
    "# 練習 1 解答\n",
    "\n",
    "# 匯入填補缺失值的工具\n",
    "from sklearn.impute import SimpleImputer          \n",
    "# 匯入 Label Encoder\n",
    "from sklearn.preprocessing import LabelEncoder    \n",
    "# 匯入決策樹模型\n",
    "from sklearn.tree import DecisionTreeClassifier   \n",
    "# 匯入準確度計算工具\n",
    "from sklearn.metrics import accuracy_score        \n",
    "\n",
    "train_df = pd.read_csv('./data/train.csv')\n",
    "\n",
    "# 取出訓練資料需要分析的資料欄位\n",
    "train_x = train_df[['Sex', 'Age', 'Fare']]        \n",
    "# 取出訓練資料的答案\n",
    "train_y = train_df['Survived']                    \n",
    "\n",
    "# 數值型態資料前處理\n",
    "# 創造 imputer 並設定填補策略\n",
    "imputer = SimpleImputer(strategy='median')        \n",
    "age = train_x['Age'].to_numpy().reshape(-1, 1)\n",
    "# 根據資料學習需要填補的值\n",
    "imputer.fit(age)                                  \n",
    "# 填補缺失值\n",
    "train_x['Age'] = imputer.transform(age)           \n",
    "\n",
    "# 類別型態資料前處理\n",
    "# 創造 Label Encoder\n",
    "le = LabelEncoder()                               \n",
    "# 給予每個類別一個數值\n",
    "le.fit(train_x['Sex'])                            \n",
    "# 轉換所有類別成為數值\n",
    "train_x['Sex'] = le.transform(train_x['Sex'])     \n",
    "\n",
    "# 創造決策樹模型\n",
    "model = DecisionTreeClassifier(random_state=1012) \n",
    "# 訓練決策樹模型\n",
    "model.fit(train_x, train_y)                       \n",
    "\n",
    "# 確認模型是否訓練成功\n",
    "pred_y = model.predict(train_x)                   \n",
    "# 計算準確度\n",
    "acc = accuracy_score(train_y, pred_y)             \n",
    "\n",
    "# 輸出準確度\n",
    "print('accuracy: {}'.format(acc))                 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "aBtSe-mBduEZ"
   },
   "outputs": [],
   "source": [
    "# 練習 2 解答\n",
    "\n",
    "# 匯入填補缺失值的工具\n",
    "from sklearn.impute import SimpleImputer          \n",
    "# 匯入 Label Encoder\n",
    "from sklearn.preprocessing import LabelEncoder    \n",
    "# 匯入支援向量機模型\n",
    "from sklearn.svm import SVC                       \n",
    "# 匯入準確度計算工具\n",
    "from sklearn.metrics import accuracy_score        \n",
    "\n",
    "train_df = pd.read_csv('./data/train.csv')\n",
    "\n",
    "# 取出訓練資料需要分析的資料欄位\n",
    "train_x = train_df[['Sex', 'Age', 'Fare']]        \n",
    "# 取出訓練資料的答案\n",
    "train_y = train_df['Survived']                    \n",
    "\n",
    "# 數值型態資料前處理\n",
    "# 創造 imputer 並設定填補策略\n",
    "imputer = SimpleImputer(strategy='median')        \n",
    "age = train_x['Age'].to_numpy().reshape(-1, 1)\n",
    "# 根據資料學習需要填補的值\n",
    "imputer.fit(age)                                  \n",
    "# 填補缺失值\n",
    "train_x['Age'] = imputer.transform(age)           \n",
    "\n",
    "# 類別型態資料前處理\n",
    "# 創造 Label Encoder\n",
    "le = LabelEncoder()                               \n",
    "# 給予每個類別一個數值\n",
    "le.fit(train_x['Sex'])                            \n",
    "# 轉換所有類別成為數值\n",
    "train_x['Sex'] = le.transform(train_x['Sex'])     \n",
    "\n",
    "# 創造支援向量機模型\n",
    "model = SVC(random_state=1012)                    \n",
    "# 訓練支援向量機模型\n",
    "model.fit(train_x, train_y)                       \n",
    "\n",
    "# 確認模型是否訓練成功\n",
    "pred_y = model.predict(train_x)                   \n",
    "# 計算準確度\n",
    "acc = accuracy_score(train_y, pred_y)             \n",
    "\n",
    "# 輸出準確度\n",
    "print('accuracy: {}'.format(acc))                 "
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "scikit-learn-基本功能.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  },
  "vscode": {
   "interpreter": {
    "hash": "c722923b96b9c31396b2182f935fa631109324cb0f0f8144167b2ddca282865c"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}