diff --git a/notebooks/03_sgd_from_scratch.ipynb b/notebooks/03_sgd_from_scratch.ipynb index ad66cb4..0916a0f 100644 --- a/notebooks/03_sgd_from_scratch.ipynb +++ b/notebooks/03_sgd_from_scratch.ipynb @@ -1,4097 +1,4101 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "D7ExutUSGoY-" - }, - "source": [ - "# บทที่ 3 - Stochastic Gradient Descent ตั้งแต่เริ่มต้น" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ai-builders/curriculum/blob/main/notebooks/03_sgd_from_scratch.ipynb)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "jUUXviwGayJP" - }, - "source": [ - "ในบทเรียนนี้ เราจะทำการสร้างวิธีที่โมเดลของเราเรียนรู้ในบทเรียนที่แล้วๆมา เรียกว่า stochastic gradient descent ขึ้นมาเองตั้งแต่ต้นโดยใช้เพียงแค่ Pytorch สำหรับ linear algebra และการทำ partial derivatives เท่านั้น ด้วยตัวอย่างการจำแนกรูปภาพตัวเลข 3 และ 7 ออกจากกัน\n", - "\n", - "บทเรียนแปล-สรุปมาจาก [04_mnist_basics.ipynb](https://github.com/fastai/fastbook/blob/master/04_mnist_basics.ipynb) ของ [fastai](https://course.fast.ai/) ผู้ที่สนใจสามารถไปติดตามบทเรียนต้นทางได้ที่ [course.fast.ai](https://course.fast.ai/)" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "6IlUoU8YGlQ7", - "outputId": "2aae9add-1fe0-40fe-ed88-c7b83114ce47" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\u001b[K |████████████████████████████████| 720 kB 26.4 MB/s \n", - "\u001b[K |████████████████████████████████| 48 kB 5.5 MB/s \n", - "\u001b[K |████████████████████████████████| 1.2 MB 46.0 MB/s \n", - "\u001b[K |████████████████████████████████| 189 kB 63.3 MB/s \n", - "\u001b[K |████████████████████████████████| 56 kB 5.0 MB/s \n", - "\u001b[K |████████████████████████████████| 51 kB 310 kB/s \n", - "\u001b[K |████████████████████████████████| 558 kB 63.9 MB/s \n", - "\u001b[K |████████████████████████████████| 130 kB 55.7 MB/s \n", - "\u001b[?25h" - ] - } - ], - "source": [ - "#ติดตั้ง fastai\n", - "!pip install -q fastbook\n", - "import fastbook" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "id": "-2VyXV7kpoju" - }, - "outputs": [], - "source": [ - "from fastai.vision.all import *\n", - "from fastbook import *\n", - "import torch\n", - "\n", - "matplotlib.rc('image', cmap='Greys')" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "gBWvqPiOpoju" - }, - "source": [ - "# เทรนโมเดลจำแนกรูปเลข 3 และเลข 7 จาก [ชุดข้อมูล MNIST](http://yann.lecun.com/exdb/mnist/)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "_5WWrXnD6Vpj" - }, - "source": [ - "## โหลดข้อมูลรูปเลข 3 และเลข 7" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 37 - }, - "id": "wkh3fHu5pojw", - "outputId": "d204e01b-6cb2-4ad4-f8de-119b5bcf34b4" - }, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "\n", - "
\n", - " \n", - " 100.14% [3219456/3214948 00:00<00:00]\n", - "
\n", - " " - ], - "text/plain": [ - "" + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "D7ExutUSGoY-" + }, + "source": [ + "# บทที่ 3 - Stochastic Gradient Descent ตั้งแต่เริ่มต้น" ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "path = untar_data(URLs.MNIST_SAMPLE)" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "Z6OjxHEPpojw", - "outputId": "d76dc5d8-0237-45e4-9ece-b94ee12f6bbc" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(#3) [Path('/root/.fastai/data/mnist_sample/valid'),Path('/root/.fastai/data/mnist_sample/train'),Path('/root/.fastai/data/mnist_sample/labels.csv')]" + "cell_type": "markdown", + "metadata": { + "id": "GtlT-MCeHftk" + }, + "source": [ + "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ai-builders/curriculum/blob/main/notebooks/03_sgd_from_scratch.ipynb)" ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#แบ่งเป็น train, validation, test \n", - "path.ls()" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "mMmCdDscpojw", - "outputId": "68396480-226d-41b6-e0d0-81977e28ba38" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(#2) [Path('/root/.fastai/data/mnist_sample/train/3'),Path('/root/.fastai/data/mnist_sample/train/7')]" + "cell_type": "markdown", + "metadata": { + "id": "jUUXviwGayJP" + }, + "source": [ + "ในบทเรียนนี้ เราจะทำการสร้างวิธีที่โมเดลของเราเรียนรู้ในบทเรียนที่แล้วๆมา เรียกว่า stochastic gradient descent ขึ้นมาเองตั้งแต่ต้นโดยใช้เพียงแค่ Pytorch สำหรับ linear algebra และการทำ partial derivatives เท่านั้น ด้วยตัวอย่างการจำแนกรูปภาพตัวเลข 3 และ 7 ออกจากกัน\n", + "\n", + "บทเรียนแปล-สรุปมาจาก [04_mnist_basics.ipynb](https://github.com/fastai/fastbook/blob/master/04_mnist_basics.ipynb) ของ [fastai](https://course.fast.ai/) ผู้ที่สนใจสามารถไปติดตามบทเรียนต้นทางได้ที่ [course.fast.ai](https://course.fast.ai/)" ] - }, - "execution_count": 7, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#ในแต่ละ set จะมีเลข 3 และ 7\n", - "(path/'train').ls()" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "EilsFTaWpojx", - "outputId": "6da64090-f81d-46b1-ae05-86a5b3fd4187" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(#6131) [Path('/root/.fastai/data/mnist_sample/train/3/10.png'),Path('/root/.fastai/data/mnist_sample/train/3/10000.png'),Path('/root/.fastai/data/mnist_sample/train/3/10011.png'),Path('/root/.fastai/data/mnist_sample/train/3/10031.png'),Path('/root/.fastai/data/mnist_sample/train/3/10034.png'),Path('/root/.fastai/data/mnist_sample/train/3/10042.png'),Path('/root/.fastai/data/mnist_sample/train/3/10052.png'),Path('/root/.fastai/data/mnist_sample/train/3/1007.png'),Path('/root/.fastai/data/mnist_sample/train/3/10074.png'),Path('/root/.fastai/data/mnist_sample/train/3/10091.png')...]" + "cell_type": "code", + "execution_count": 1, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "6IlUoU8YGlQ7", + "outputId": "96ea0400-f259-4d81-dcbf-4a7665998010" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\u001b[K |████████████████████████████████| 720 kB 12.8 MB/s \n", + "\u001b[K |████████████████████████████████| 188 kB 49.3 MB/s \n", + "\u001b[K |████████████████████████████████| 1.2 MB 45.6 MB/s \n", + "\u001b[K |████████████████████████████████| 60 kB 5.6 MB/s \n", + "\u001b[?25h" + ] + } + ], + "source": [ + "#ติดตั้ง fastai\n", + "!pip install -q fastbook\n", + "import fastbook" ] - }, - "execution_count": 8, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#ในละ folder 3 และ 7 จะเป็นไฟล์รูป\n", - "threes = (path/'train'/'3').ls().sorted()\n", - "sevens = (path/'train'/'7').ls().sorted()\n", - "threes" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "2nKh8v3OBG5G", - "outputId": "bf8f9a5e-c62c-48ec-a199-83ac22bac45e" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(6131, 6265)" + "cell_type": "code", + "execution_count": 2, + "metadata": { + "id": "-2VyXV7kpoju" + }, + "outputs": [], + "source": [ + "from fastai.vision.all import *\n", + "from fastbook import *\n", + "import torch\n", + "\n", + "matplotlib.rc('image', cmap='Greys')\n", + "\n", + "#fix plot_function as new pytorch requires steps argument for torch.linspace\n", + "def plot_function(f, tx=None, ty=None, title=None, min=-2, max=2, figsize=(6,4)):\n", + " x = torch.linspace(min, max, steps=100)\n", + " fig,ax = plt.subplots(figsize=figsize)\n", + " ax.plot(x,f(x))" ] - }, - "execution_count": 9, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#จำนวนรูปในแต่ละ class\n", - "len(threes), len(sevens)" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 45 - }, - "id": "jJEUNNQKpojx", - "outputId": "c84f0303-b2e4-47e5-c7bd-a823c28f4b71" - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAAA3UlEQVR4nGNgGNxA1XnOxX//vxlgkVIqe//379+///4+EkCX4lv//e+/v38vnf33968wumTd379/rzUZ8lf9e1XChC4Zf39RAAMDA+eFf2txuIg18htuObW/f9/y45CM/Pt3iwYOuaxvv7Zz4pBjmPvvoxwSF9XNrxhZ/XFpZOCf9+tjMk5Zhsl/d+AyloHhI4MsTo22T//uF8cu5Xbo/79/8eiiXGVlyqyOk3/9/fe7nRld0urv3/uH/v79+/d1GKZ5In///v339+/zDdiClbVu9sF/X2cr4XQoVQEA4o1d+YAEyFQAAAAASUVORK5CYII=\n", - "text/plain": [ - "" - ] - }, - "execution_count": 10, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "im3_path = threes[6000]\n", - "im3 = Image.open(im3_path)\n", - "im3" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "GBYX4Mf0BTb6", - "outputId": "02cc967e-9295-4da6-806b-8b3e58ca3ae3" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(28, 28)" + "cell_type": "markdown", + "metadata": { + "id": "gBWvqPiOpoju" + }, + "source": [ + "# เทรนโมเดลจำแนกรูปเลข 3 และเลข 7 จาก [ชุดข้อมูล MNIST](http://yann.lecun.com/exdb/mnist/)" ] - }, - "execution_count": 11, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#รูปขนาด 28 x 28 pixels \n", - "array(im3).shape" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "RZphr7qzpojx", - "outputId": "257cba1a-9290-4a12-98f8-f3d9d5d9a2dc" - }, - "outputs": [ - { - "data": { - "text/plain": [ - "array([[0, 0, 0, 0, 0, 0],\n", - " [0, 0, 0, 0, 0, 0],\n", - " [0, 0, 0, 0, 0, 0],\n", - " [0, 0, 0, 0, 0, 0],\n", - " [0, 0, 0, 0, 0, 0],\n", - " [0, 0, 0, 0, 0, 0]], dtype=uint8)" - ] - }, - "execution_count": 12, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#เปลี่ยนจาก numpy array\n", - "array(im3)[4:10,4:10]" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "o52Dig-Vpojx", - "outputId": "c2aae0e8-5e86-4eeb-88a8-180112bf9675" - }, - "outputs": [ - { - "data": { - "text/plain": [ - "tensor([[0, 0, 0, 0, 0, 0],\n", - " [0, 0, 0, 0, 0, 0],\n", - " [0, 0, 0, 0, 0, 0],\n", - " [0, 0, 0, 0, 0, 0],\n", - " [0, 0, 0, 0, 0, 0],\n", - " [0, 0, 0, 0, 0, 0]], dtype=torch.uint8)" - ] - }, - "execution_count": 13, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#เป็น torch tensor\n", - "tensor(im3)[4:10,4:10]" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 394 - }, - "id": "0yZCaBRNpojy", - "outputId": "f4d506e0-bf3b-473f-fe3d-bf17277cd1b4" - }, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
 01234567891011121314151617
0000000000000000000
10000000000376715620925425524648
20000000034118239253253253253254253226
300000014175247253254253253210205254253253
40000001262532532532141304915122254234116
5000000952231628000092082541730
60000000000058924625417300
700000000005382532532371500
800000000008925325318040000
900000000010624625018390000
100000000001572542413000000
\n" - ], - "text/plain": [ - "" - ] - }, - "execution_count": 14, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "im3_t = tensor(im3)\n", - "df = pd.DataFrame(im3_t[4:15,4:22])\n", - "df.style.set_properties(**{'font-size':'6pt'}).background_gradient('Greys')" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "jMdNhmiXpojy" - }, - "source": [ - "## วิธีที่ง่ายที่สุด: ดูว่ามี pixel เหมือนกันแค่ไหน" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "MqYq_GbFT7XA" - }, - "source": [ - "สมมุติเราไม่ทำ ML อะไรเลย แต่เราพยามสร้างกฎแบบ Rule-based Systems โดยบอกว่า \"รูปที่มี pixels ใกล้เคียงกับ pixels เฉลี่ยของเลข 3 และ 7 มากกว่า ให้ถือว่าเป็นเลขนั้น\"" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "JDsXZWi8pojy", - "outputId": "deb4fb4e-a3dd-44ce-d5c3-93272004588b" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(6131, 6265)" + "cell_type": "markdown", + "metadata": { + "id": "_5WWrXnD6Vpj" + }, + "source": [ + "## โหลดข้อมูลรูปเลข 3 และเลข 7" ] - }, - "execution_count": 15, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "seven_tensors = [tensor(Image.open(o)) for o in sevens]\n", - "three_tensors = [tensor(Image.open(o)) for o in threes]\n", - "len(three_tensors),len(seven_tensors)" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 85 - }, - "id": "9h_vpbbvpojy", - "outputId": "cadff4bb-f297-4f52-e3ec-17508f92f686" - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAJHElEQVR4nO2bXXMSZxuAL1jYXYQsiCYxIWIIjImJ0TaVTu2H41FnnGmPPOtMf0NP+i/6H9oDx+OOOtMjW6cdGz/Sk1ZqxkRDhCTQEAjfsLDsvgeWbbMmxgrEzDtcR5ln+bi5eJ5n7/t+iM0wDPr8g/1tB3DY6Aux0BdioS/EQl+IBcc+1/+fb0G23Qb7M8RCX4iFvhALfSEW+kIs9IVY6Aux0BdiYb/ErGMMw6DVatFsNqlUKmiaRrPZpNls0mg0Xnq8LMtIkoSu6+i6jtPpxOFw4HQ6EQQBWZZxOHoXds+FaJpGvV4nHo/zww8/kMlkSKVSxONxnjx5wr/7MTabjffee4+pqSmq1SqqqjIyMsKxY8eIRCIMDw8zOzuLz+frWbxdF6LrujkDisUilUqFbDbLn3/+yeLiIoVCgfX1dTKZDJVKBVEUEUWRer1OrVZjZWUFh8NhCqlUKmxublIulzl+/DgTExM9FWLbp2P2n2sZVVXZ3t5mdXWV77//nlQqxaNHj8jlcqTTaQzDwDAMZFnG7XYzODhIIBDgyZMnPH/+HLvdjt1uN2eOzWbDZrPh8/nwer1cv36daDT6hh93B7vWMh3PEE3TqNVq1Go1UqkUlUqFVCrFysoKT58+pVQqIQgCExMTRKNRnE4nkiQhiiIulwtFUfB6vczOzpLJZFheXubZs2eUy2VqtZr5Po1GA1VV0XW905BfScdCVFUlFovx22+/8c0331CpVGg2m7RaLRqNBoFAgGg0ysWLF7l69Soejwe3220+3263Y7PZ0HUdwzC4ceMG165dIxaLkUwmOw3vP9OxEMMwaDQaVKtVSqUS1WoVXdex2+2IokggEGBubo5z587h8/lwOp04nU7z+e0l8e+lJAgCNtvOGe33+wkGg8iy3GnIr6QrQqrVKrVaDVVV0TQNAFEUOXr0KHNzc3zxxRf4fD4URXnpg7ZpjzudTux2+0vXpqenmZmZQVGUTkN+JR0LcTqdhEIhRFEkn8/TbDaBF0Lcbjdzc3N4vV5EUdxTBrzYizRNI5/Pk8/nzRzFZrMhCALDw8OEw2FcLlenIb+SjoVIksTp06cJh8N88MEH5nh7KQiCgCiK+75Oo9GgVCqxtrZGMpmkWq0CmM+PRCJEo9Ed+08v6FhI+1sXBGHH3tC+Zp3+e7G+vs7PP//M77//TqlUQtM0BEEgHA4zPj7OzMwMJ06ceOk9uk1XErP2bHidmbAXd+7c4auvvkLTNHRdx+FwIIoiFy9eJBqN8u6773LixIluhPtKep66WzEMA13XqdfrlMtlM2FbWFgwN2S73U4kEiESifDhhx8SjUZ7vpm2OXAhuq6jaRrZbJbHjx/z448/cvPmTbLZrHm7djgcXLhwgY8//phPP/2UkydPHlh8B1LtGoZBsVhkdXXVrGWSySSrq6ssLS2Ry+Wo1+vAi3zD7/dz+vRpzp8/z8DAQK9D3MGBlf/JZJLvvvuO5eVlHjx4gKqqO1LzNkNDQ0xPT3PhwgUmJyd7fpu1ciBLRtd1CoUCjx49Yn19HVVVzXzFSiaTIRaLcevWLeLxOMeOHcPj8TA2NobX62VwcLCnkg5syWxtbfHgwQM0TTM31t3IZDJkMhnW19dxu90oioLH4+HKlSucP3+eS5cuIcvyK5O8Tuh6+f/SC/y9ZDY2Nrh9+zblcplCoUCxWCSXy5mPi8fjLC0tmT0UURRxOp1mv2RqaorR0VE++ugjzpw5w9mzZ/F6vQiC8Nq5joVdjfZciJVarWbKWFtbM8d/+uknfvnlF1ZXV0mn07s+t91Ri0QifP3110xOTiJJEoIgvEkovemH/FecTieKoiDL8o7O19DQEJcuXWJtbY10Ok0mkyGXy3H//n3i8TjwYrYlEgmq1SrPnz9ncHCQ48ePv6mQXTlwIQ6HA4fDgcvlwuv1muPDw8NMT09TrVbNW/TKygrZbNYUArC5uUkulyMejxMKhTh69Gh34+vqq3VAuxBsd9VlWSYYDBKPx/nrr79IJBJsb28DL2bKxsYG8XicU6dOdTWOQ3Mu0y4EJUkye63BYJDZ2VmmpqZ2pO6GYZgzR1XVrsZxaITsRbtwtI55vd6eVL+HXgjAbndCt9uN3+/v6oYKh2gPsVIsFtne3mZhYYGHDx/uyFlsNhunTp0iEol01HLYjUMppF0MJpNJEokEiURiR2YrCAJDQ0P4/f6uH2seOiHVapVKpcK9e/e4c+cOf/zxh3lEATA5Ocn4+DiBQABZlt80S92TQyOk/YHr9TrZbJZYLMb8/DypVMq8ZrfbCQaDRCIRvF4vDoej6zXNoRFSKBRIp9Pcvn2bu3fv8vjxY5LJpNknGRgYwO12c/XqVS5fvszo6Oiu5zed8laFtCthwzDI5/M8ffqUhw8fcuPGDbO3Ci820SNHjjA4OMi5c+cIhUI9kQFvUYiqqqiqysbGBktLS9y9e5f5+XkSiQTNZtNcJi6XC1mW+fLLL/nkk08Ih8M9kwFdFvLvb9yaULXH2383Gg0KhQLxeJz5+XkWFha4d++e+fj2me/AwACKonD27FneeecdPB5Pz2RAF4Vomka1WqVcLrO2toaiKIyMjJiH3pVKha2tLUqlEpubmywvL7O4uGjWKfl8HvgnM41EIoTDYa5cuUI0GiUUCqEoSk9/PQRdFNJqtSiVSmxtbRGLxRgdHUWSJPMXRLlcjuXlZba2tsxl0u6tqqq645RPFEXGx8cJh8NEo1FmZmbMhlGv6ZqQfD7Pt99+SzKZ5P79++Zhd6vVotVqmecwqqqaM6ZSqZgb5/DwMGNjY7z//vtmk/nkyZMoioIkSV3PN/aia0IajQaJRIJnz56xuLj4Wj9ssdlsSJKEJEmMjY0RiUQ4c+YM0WiUiYkJ/H5/t8J7bbomxOPxcPnyZTweD7/++uu+QmRZ5siRI3z++ed89tlnhEIhRkZGcLlcB7Y8dqNrQhwOB8FgkHQ6TSAQMI8l98LlcuHz+ZicnGR2dpahoaEdHbS3RdeazK1WyzxvKRaL+7/x3w0ht9ttdsm6XcrvF8KugwfddT9E9P+j6nXoC7HQF2KhL8TCfrfd3lVRh5T+DLHQF2KhL8RCX4iFvhALfSEW/gcMlBno19ugeQAAAABJRU5ErkJggg==\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "show_image(three_tensors[1]);" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "D5B7mZNspojy", - "outputId": "bc93149a-adf0-42f7-fe24-bd1dc286b5e9" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(torch.Size([6131, 28, 28]), torch.Size([6265, 28, 28]))" + "cell_type": "code", + "execution_count": 3, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 37 + }, + "id": "wkh3fHu5pojw", + "outputId": "c0fe5213-0052-462a-b8ee-84882e0c73cd" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "\n", + "\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "\n", + "
\n", + " \n", + " 100.14% [3219456/3214948 00:00<00:00]\n", + "
\n", + " " + ] + }, + "metadata": {} + } + ], + "source": [ + "path = untar_data(URLs.MNIST_SAMPLE)" ] - }, - "execution_count": 17, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#เรามัดเลข 3 และ 7 ทั้งหมดรวมกันตามคลาสแล้วหารมันด้วย 255 เพื่อให้ได้ค่าระหว่าง 0 และ 1\n", - "stacked_sevens = torch.stack(seven_tensors).float()/255\n", - "stacked_threes = torch.stack(three_tensors).float()/255\n", - "stacked_threes.shape, stacked_sevens.shape" - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "K4mmFyngpojz", - "outputId": "4b1ba479-b693-4f5a-bcd3-260a3283ed07" - }, - "outputs": [ { - "data": { - "text/plain": [ - "3" + "cell_type": "code", + "execution_count": 4, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Z6OjxHEPpojw", + "outputId": "b1c65a6c-315f-4233-fe92-215e817ecdb3" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(#3) [Path('/root/.fastai/data/mnist_sample/valid'),Path('/root/.fastai/data/mnist_sample/labels.csv'),Path('/root/.fastai/data/mnist_sample/train')]" + ] + }, + "metadata": {}, + "execution_count": 4 + } + ], + "source": [ + "#แบ่งเป็น train, validation, test \n", + "path.ls()" ] - }, - "execution_count": 18, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "len(stacked_threes.shape)" - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "Jck_enHBpojz", - "outputId": "1bc5de1f-50f5-45cd-ed75-65f44ac5c8fe" - }, - "outputs": [ { - "data": { - "text/plain": [ - "3" + "cell_type": "code", + "execution_count": 5, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "mMmCdDscpojw", + "outputId": "fc98608c-0131-4be1-d56e-922e924a03af" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(#2) [Path('/root/.fastai/data/mnist_sample/train/7'),Path('/root/.fastai/data/mnist_sample/train/3')]" + ] + }, + "metadata": {}, + "execution_count": 5 + } + ], + "source": [ + "#ในแต่ละ set จะมีเลข 3 และ 7\n", + "(path/'train').ls()" ] - }, - "execution_count": 19, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "stacked_threes.ndim" - ] - }, - { - "cell_type": "code", - "execution_count": 20, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 103 - }, - "id": "cxcx0MIlg-fI", - "outputId": "0cf1655d-5ce7-4044-b1f5-b36dd0a93b3b" - }, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 20, - "metadata": {}, - "output_type": "execute_result" - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAIEElEQVR4nO2bS08TbRuArx6ntJQONBQoYIFgEPCQYBQ0Rl2YuDFGoy504Q/xJ7j1B7hwR+LGjboQSZRo8IQBLIdWC5aDI1Baep525luYzgulKF++TjHv1yvpZmae9u41z9z3c2gNqqpS5R+MBx3A30ZVSBFVIUVUhRRRFVKE+Q/n/80lyFDqYLWHFFEVUkRVSBFVIUVUhRRRFVLEn8pu2VEUhXw+Tz6fR5ZlMpkMqVQKs9mMxWLBbDZjMpm06wVBwGKxYDCUrJJlp+JCstksW1tbrK6uMjs7y/v373nz5g0+nw+v16u9Cpw5c4bm5maMRmNFpFRMiCzLpNNp1tfX+fbtG9++fWN+fp5AIEAoFCKbzZJMJolEIqysrABgNBrp7u6mvr4eq9WK2ax/uBUTEo1GmZiY4MWLFzx69IhUKkUymSSXy5HP51laWuL9+/cYjUaMxl+pzWAwIIoiDoeDlpYWamtrdY+zYkIsFgv19fXYbDaSySTpdJp0Oq2dz+fzJdvNzc3x7t07hoaGMJlMWp7RDVVVf/cqG7Isq4lEQh0eHlZbWlpUp9Op8muu9NuXy+VS29vb1QcPHqh+v1+Nx+PlCqnkd65Y2TUajZjNZlpaWjh9+jRtbW2YTCYtUZrNZux2+667n0ql2Nzc5MePH0iShCzL+sap67tv/yCjEavVSldXF3fv3uXixYsIgqAJqK2tpampCYfDsaNdNpslHo8TDAYZHx8nHo/rG6eu716C2tpaenp6aG9vx+VyIQgCACaTCUEQdoxBtuPxeDhy5Ag1NTW6xldxIU6nk76+PoaGhvD5fLhcLgCsVit1dXVYLJZdbQwGA8ePH+fs2bO6V5qKD8wKOcPtdjM0NIQoilq1kSRpR+U5CCoupIDX6+XWrVuMjIwQi8UIhUKEQqE9ry8M+fXmwCZ3NTU1tLe3c/78eW7cuMG5c+dwu90lc4SqqgQCAfx+P6lUSte4DqyHOBwOHA4HTU1NDAwM4Ha7CYVCLC4u7vrSqqoyPj7O1tYWXq8XURR1i+vAhKTTaeLxOIlEgmg0SiAQYHV1lUQisetag8GA2+2mpaUFq9Wqa1wHJmRrawu/38/y8jLhcJhPnz7x/ft31D32mj0eDz09PbqX3YoJyeVyyLLM2toac3NzzM/PMzMzQywWIxqNMjc3t6cMgKamJrq7u7HZbLrGWVEh8XicsbEx7t+/jyRJ/PjxA0VRUBTlt20NBgM+n4/Ozk5tIKcXFasy+XyeWCyGJEmsra2RSCRQFOW3vaKAqqrMz8/z5cuXf0+VyeVybGxsIEkSP3/+RJblP/aM7czNzWGz2Whra9NGt3pQMSGCINDR0cGFCxcIh8P4/X4+fvy4r0cGIJlMEovFdB+cVUyIzWbDZrPR39/PlStXEASBycnJffUUVVVJJBLE43FyuZyucVa87IqiyODgIKIo0tDQQDabJZvNaudVVUVRFMbHx5menkaW5YoM2QtUXIjdbsdut+N0OmlsbCSTyZDJZLTzBSHpdJpgMKhtWVSKAxuY1dTU4PP5Sk7aFEXh4sWLKIrC69evCQaDrK6u4nA42NjYIJ1OY7FY9lw7+V84MCGCIOw5plBVlRMnThCLxVhYWCAYDLK5ucni4iKxWIxMJoPJZNJFyF+3lVl4ZKampnj8+DEzMzPAP2uyem9YHVgP2YuCkEAgwOjoqHa8sAXxfydkdXWVYDDI/Py8dsxgMDAwMMCxY8doa2vDZrPp8riADkIK+xvb7+J/c0clSWJsbIzv37/vOO71eunv70cUxZLrruWibEIURdGG569evaKhoYGOjg5EUcTtdv+xfS6XI5fLMT09zdOnT3f1kK6uLk6ePIndbi9XyCUpW1JVVRVZlvn58yfPnj1jdHSUUChEJBLZcxK3fccsl8uRyWRYXFzk8+fPrK+vA79kmEwmmpqaaGxs1LV3QBl7yNbWFs+fP2dycpKXL1/idrtZWlri1KlTXL9+HavVitVqxWKxIAgC6XSaVCqlbXr7/X4+fPjAq1evtJkwwODgIL29vfT19emaOwqUTUgqlWJiYoJAIEA4HCYSiZBOp3E6nUiSpK2hOhwOzGYz2WyWSCRCNBolEonw9u1bRkZGCIVC2nzFaDTS2dnJsWPHaGho0MqunpRNiNPp5OrVq3z69InFxUXW1tZYWFjgyZMnzM7OarmksDYaDof5+vUr8XicaDTK8vIy6+vr2r5MQ0MDoihy6dIlLl++TH19/Y69YL0omxCz2UxrayvxeJy2tjby+TwrKyuEw2FCoRC1tbV4PB6am5s5dOgQX79+xe/3k8lkdkzu4FfecLlc+Hw+Dh8+jMfjqYgMAMMfVqz2/dNuRVHIZrMsLS3x8OFDVFXFarXi9/sZHh7WNrutVqv2G5FkMrkr4dpsNgRB4N69e9y8eROPx4PdbsdgMJRbSMk3K1sPMRqN2Gw26urqaG1tRRAEvF4vsixjt9u1JJnJZDQRpWaxdrsdURQ5ceIEXV1d5Qpv35R9YCaKIrdv3wZ+df3CD+YKQjY3N5EkCb/fz9TUlNauUH3u3LnDtWvXOHr0aLlD2xdlF2KxWBBFUZuTNDc3MzAwoPUGSZJwuVwoioIkSVq7worakSNH6O3txel0lju0fVG2HFKy8bYBV+FzCo9KJpPZsdNfyBGiKFJTU1OJElsyh+gq5C+n+n+Z/VAVUkRVSBFVIUVUhRRRFVLEnwZmlfmTyl9EtYcUURVSRFVIEVUhRVSFFFEVUsR/AP0FXN1zCRLUAAAAAElFTkSuQmCC\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "#เลข 3 ที่ index 0\n", - "show_image(stacked_threes[0,:,:])" - ] - }, - { - "cell_type": "code", - "execution_count": 21, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 85 - }, - "id": "2Es3hPcXpojz", - "outputId": "14820e4c-7b0d-42b4-cb01-fe33175307c2" - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAJtUlEQVR4nO1b2XLiWhJM7QsChDG22x3h//+qfnKzWVhoX5HmoaNqDufK9jRge2aCiiCEAS0nVUtWlqz0fY+r/dvU776A/za7AiLZFRDJroBIdgVEMv2D7/+fS5Ay9OHVQyS7AiLZFRDJroBI9lFS/RQ7tV1QlME8eFH7dEDkxdPf4udDAMmLVxQFfd8Pfn5JuyggQ4vs+/7o1XUdfy6+l01RFCiKAlX9E9WqqvJn9JJ/fwm7CCDy4ruu423XdTgcDjgcDqiqCm3boq5rtG2LpmlwOBzQti3vr6oqVFWFaZrQNA2WZUHXdViWBU3ToOs6NE3j3xFQZOcCcxYgQ15AIHRdh7Zt0bYtqqpCXdcoigJlWaIoCtR1jTzPUdc1yrLk4+i6DlVVMRqNYFnW0dY0Tdi2DcMwYBgGNE1jEGRgTrWTARHBkD2haRpUVYU8z1EUBcIwRJIk2Gw2CMMQu90OaZoiiiKUZYk8z3E4HNB1HUzThGEY8DwPo9EIi8UCs9kMj4+PmM1mmM/ncF0X4/EYlmUdeY4YYqeCc7aHDAFSliWqqkKSJMjzHLvdDmEYYrPZII5jBEGALMsQRRGyLEOWZRw6dOdnsxk8z0PXdSiKAoqioKoq6LqOw+EAXdfR9z17CYXPUOL9dEDkXCHmiKqqEMcx0jTFdrtFGIZ4fn7Gfr/HZrNBmqYIggBJkiCKIlRVhbIs0bYtDocDn8NxHFiWhYeHB9ze3iIMQ9zc3CDLMtze3qJtW0wmEyiKAsdxjpLvOXZ2UhXzBy1K3LZty+GlaRpM08RoNIKiKNA0jQFpmgZt27Knkaf0fY+6rlHXNYdhlmX8N53DNE2+ji/3EAJiKHfQhdZ1zVVEVVVYlsVxb9s270P7U/WhvNM0DZqmga7rnJgp76iqivl8Dk3TMB6Poes6uq7jkKEbcAowJ4eMaHRiimNd19kTyHNs24Zt27xQApRAITAJkKIokOc5NE07SpbyfqKHXsIuRszoonVdh+M4nOw8z4Pv++wB8j60QAJgv98jjmOuTMRZdP3PpYqe+R4Q31Jl6MQiGADQdR2Tp6ZpOESImYpGn2dZBl3XUVUViqJgPkILo5xD5zEM44ikib87x04CRD45ubVpmnyRXdfBcZyjaiTfTSJvTdPAsiyYpvmPUAFwlBNM02SCRpxlCLxT7SwPES9AVVVehBgKsnuLpZq25BVhGGK/32O/3yNNU+R5zhVLVVUYhnHEXonWEykb6nG+HBDxLhIQcqKTgaBS3Pc9V4+Xlxes12usViu8vLwgiiLs93tesKZpsG0bk8kEvu9jNBrBdV0GhRL6uXYyIDIQiqKg67pBUESeIvKJJEmYwa7Xa2y3WwRBgN1uhyzLkOc5JpMJl2rXdeF5HsbjMRzHOQpRsRv+FkBEUAgEIlJvtfvU4NHdXy6XWK1W7BUESBiGHIau60LTNDiOg8lkwpTecRw4jnPkHd+WQwgA8b28JRAOhwN3tHEcI4oiDo3lcsmhst1usdlsUBQFqqri5EkVi7jNUHW5VIU5GZD/BBSRxRZFgSRJsN1usVqt8OvXL6zXazw/P+P3799YLpeI4xhJkvAxPc+D53kA/lQxSqhi6y+LRnQt59jFRGbxQihcyDuKouCmbrPZYLPZIAgCLJdLBEGANE1R1zULRCLPAHBUiaiBpLZAZqvnMtazc8iQZirS67qukWUZ4jjGer1mQJbLJZbLJZIkQZIkXIYVRTnyAjpWVVUsFbiue9QMUh/zrSHznhFIIg+h5EpyoO/7WCwWGI1GmEwmDKSmaZxEKUwo7KIoQhAE3NRR9yxrr8A3UnfRZJFZJmZEvy3Lgud5uLu7Y3WNTNRLAcAwDHRdhzzPoaoqXl9foWkabm5uoOs6bNvm4367h7w3SlAUhXOB67po2xY/fvxghlkUBbIs49AiI/BkQbrve65UqqoiiiJomgbXdTnviJ5C1/C3dramKr8XL0am2/P5HI7jwHVdFn1kSk+9DVH3OI4ZOGK1qqpiv9/DsixUVcVeJHriqXaWHjK0FUsxjRNc14VhGDBNE3VdYz6fs2fIgJCC9vr6ijiOmXiRStY0DVet0WiEsixhmib3O8SWvyyHvFVVxO+o+pD7ip0pdbhDs5yu61igtm2bQ4tAAoC6rqFpGqv1ojInMmU69t8Cc5aHyG39EDAU3zRzeav5I05BJZd6HgLGMAxOvqJsQFLkECf58hwytDBxS6CQejaUa2QPIbNtm0uvyErl38tgnGt/BYjsGVQd3tM236PW4gLp+LRYyhUkWFOYDc13L8FQyU7OISIAlBxliVAeWL8Finx8keWSQCQn7KF9ZftS1V2Me3FoTQuStVZRURMBot9TcozjmGn+arXCbrfjkSclTmK7NBCn7ve9pwM+DRBZ6xABaZrmKBcQ42zbli9cFKMVRWFQReEoTVNW37MsQ1EURyFD9F7WU7+NqYqA0BCpbVu+8Kqq+Ht5VkPAkFG1oMZts9mwRvL6+ookSVAUBatjJBT5vo/pdMojTzr2uU3eSUlVBoW8g2a0RVFwDiAKL48OyGhARSradrvFbrdDEAQcKsRGiehRBSJuQ99dwlP+ChBZFBJPTqSqLEsEQcC0m7xIfIaDRo9Ex8uyRJqmSNOU5YA8z1GWJfMQz/MwnU5xf3+Pu7s73N3dYTKZwPM82LY9mEc+HRARGPmkYnIkgTiKIvYcAk0UpEV5kYbYSZLw4xF93zMpcxwHnudhMplgMpnAcRxmwEO66qn214CIYFBiI+2TdFAAR3khDEPUdY04jo9yDok8IuOkhOn7PsbjMR4fH+H7Pp6ennB/f4+npydMp1PMZjMGhfb58pAZAoUSpvwiLxCbsd1ux5WE8g6Vb3J3Yqeu68L3ffi+j/l8jsVigdvb26MwuVQiPRkQOinxCHERNL60bRsAMJ1Ooes69vs9DMNAkiQwDIPlRNI66O7SvGU+n2M8HuP+/h6z2Qw/f/7EYrHAzc0NC88iGEPc5ssAkcERgSHdAwCr5VmWQVGOH4Ui8IjIkUfNZjOMRiPMZjP4vs9PDj08PGA8HnPeoFmMqKxdcgyhfNADDH451NOQuEP6Z9M0XCnSNOVHrUg9pypDi6PRpLilZ0rkofZQiT0BjMEdzgJkiLVSmaXRgchPaEu5gzQTEotN02SSRS+x2x16NvUMr7gcIPzlAFED/tn9fjQ7kZO03JO81aOcGSKDO19ktiv/LbblQ9uPjvfW9q3zXtLO8pCP7BIaxScu/vIe8uEZP/FOfpZd/4FIsisgkn0UMv97Pn+mXT1Esisgkl0BkewKiGRXQCS7AiLZvwBtCZqwAvXF1QAAAABJRU5ErkJggg==\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "#เลข 3 โดยเฉลี่ย\n", - "mean3 = stacked_threes.mean(0)\n", - "show_image(mean3);" - ] - }, - { - "cell_type": "code", - "execution_count": 22, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 85 - }, - "id": "ambkeHzzpojz", - "outputId": "2ef89ced-bead-4371-ff36-d472f5fa0121" - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAI6klEQVR4nO1baVPiTBc9ZF/IhoOWOjUf5v//KgelNEZICCErvB+eutemjaNC8N04VanG7H1y99uOdrsdzniF8u9+gf80nAmRcCZEwpkQCWdCJGgfHP9fdkGjvp1nCZFwJkTCmRAJZ0IknAmRcCZEwpkQCWdCJJwJkfBRpHoQPlNjkc8ZjXoDxzf47HmHYhBCxMnR74/Gz0IkYDQa8d/y2Hf+ITiKEHmS2+2Wx91uxxv9TcfFY+L1BHGy9FtRFIxGo96RjsvXH4KDCBEnIk6aJt51HbbbLbquQ9d1aNuWR9pP58n3kUET1zQNqqrCMAyoqgrTNKGqKjRNg6IoexvhEGK+TIj44kQCEdA0DZqmQVmWaJoGm80GdV1jvV6jrmvkeY6qqrDZbNA0Deq6Rtu2TFSfJKmqCkVR4LouTNPExcUFXNdFFEVwHAe+78M0TViWxQSNRiOoqordbvdlUr5EiCwZ9KVJAoqiQNM0WK/XKMsSWZahKAokSYLNZoMsy1CWJRPVNA1fS6SKKgaACRmPxzBNE9PpFJ7n4devX/B9H5qmYbfbMRHb7RaKohxExpcI6VMPmgxNcLVaoSgKxHGMNE0xn8+RZRniOEae51gsFlitVlgulywpoiqJ6kQbffUgCOB5Hn7//o0wDJGmKS4vLwEAQRBA0/6ZCqnYyQnpI0ckhlSFJCHLMqRpymOe50iSBOv1GqvVCm3boq7rvfuJIHLatkVVVRiNRmiaBlEUQVEU5HkO13VZ0kjCjsWXVUYkous6NE2DqqpQliXyPEeapojjGMvlEnEcI8sy3N/fY7VaIUkS1HWNsiyZAEVR2DCS1xCfQXamrmvoug7TNFHXNS4vL2EYBvI8h23bLK3vGeeTEPI3kIskEdd1HbquwzAM+L4PVVUBgL+6fA55ETKyy+US6/UaWZZhvV7zMwDskSkSSb9Fd/1VfIqQjxgXyaAJmqYJ27aZhPF4jDAMoaoqu03HcfhcXdehqip7qiRJkGUZ7u7u8PT0xAabDCdBdrnHkPFpQshIyQQoisISsd1uYZomuq5DEARQFAV1XcO2bei6ziqg6zosy4Jt23BdF5ZlwbIslpDNZoOyLHl/nucoioLVgQik0bIsjk2+jRCRiD5CDMMAALiuC1VV0XUdTNOEoiioqgphGLKtoNjB8zy4rssTo3sSIZ7nYTabIc9zrNdrVFWFtm1hWRYcx4Ft27Btm4kjCRNV5tu8jKizAKDrOn89ALBtm41j13WoqgqapsE0Tbiuy5LhOA50XedYYrfb7UWmwD/qRt5IVVU4jgPXdTEejxEEAUuIaJiPwacJER+kKAoHQPTypNuqqrL6ULS42+1YVSzLYsmwLIuJJZUiIk3TZEKqquKYxHEcOI4Dz/M4WqUoVbQjJydEJIaCHjHxAv6RFABMhghR98muEJF0Pbnbuq6RZRkHcuRlDMPAeDyG7/sIw5CjV13X3+Qxh+JglQFeJYV0V9d1Jqxt2z2dFr0P6TtNgK6h2KYsS6RpisVigcViwUEYqVwQBIii6A0hx7rcLxMiehvRsJLEkOEkkogQ2i8TQRCDvDzP8fz8jMfHRzw/PyNNU9R1zaG77/sIggCu68K27T3SAey938mTO3oQPVj0OuRxSBrETJU2MVUX70PGl/Kh+XyOp6cnzGYzpGmKpmmgaRo8z4PneQjDkCWGpGMoHBypytICvNoSsh80igUdoL+EUBQF0jTF/f097u7uEMcx4jhG0zRQVRVhGGIymfBIrvZYFZFxVOjeV84Tv5ZMmLyfJKNtW2w2G86QHx4eMJ/PkaYpezFys7TJrnYoUg42qrItod8A9ryGDLmMQMlekiT48+cPZrMZ5vM5FosFmqZhb3J1dYXLy0tMp1OMx2OOP0TJG4KUoyVEtCVkYGkTiZOLS2RI67pm6Xh4eEAcx3h4eECe5+i6DoZhIIoihGGIi4sLNqhiZErvMgSOtiF9Ve/tdvsm/wH2SaGSY1EUeHl5wWw2w2w2w8vLC6uK53m4vb3Fz58/cX19jZubGwRBwEmhHIx9u9v9G+RIVm5NyG6RCktUR0mSBEmSYLFYoCxLqKoK27bx48cPXFxcYDKZYDKZwHEcmKb5xn70kfBtucxnH0xSIhdtKHCjuuv9/T3HHGVZQtM0RFGEIAhwfX2N29tb3NzcIIoi2LbNkXBfyn+sCg1iQ+S/33sZUWWoClYUBZbLJfI8R5Zl7GZ938d0OkUURZhMJvB9H7Zt73mX98g4BkdLSB8phPdUheqvaZri6ekJSZJwi4LynNvbW0ynU5YQMqaidPTZjW/Ldv8GedLv7SPVIemgEiGRQV5FzGajKILneRyIDW1EZQza7O7zLMCrVxGN6HK5xOPjI6vLdruF4zgIggC2bePq6gpXV1fchxErY3LkKz7/WJx8OURfy4J6Mnmec08HANdIKGchcvq8CtAfFB6LQSVEVg+5b0M9mcVigefnZywWC2w2G4xGI1aJyWSCKIrY1ZLdoJrr0KG6jJOtD/lbD2ez2aAoCpRlia7rOF/RdZ2Lz5TeExGniEr7cJL1IWJoTipSFAVWqxWyLMPLywvyPOe2AnkWUUJEdZELQKfE4CrTJx3Ua6mqipO5uq65qKxpGtsPalGIlfTvIgMYgJC+tSLUZyXpoJ7ver1mQ0pVNbHOats2wjCE7/uwLOvDmOMUGNyG9NkOMqo00nIHsTBMTScav8uIyhhsSdV7RpQWxtBGrQbTNN8siKEikOhqRYP6X6EyIvoW1IgrgyiUp8YUqYSqqiwdhmFwi2KIPstXMWhy99451AR3XZcnKa8CIBsitiffa1mI49A4SRwC7Lc7adLU6gReWw+apnGb0zAM3khy/lbzOAUpow++8KdWnvTZEHHJFdkSMqxt2+6pEEkRkUNumEj5qDJ2IDG9Fw1eMaNqmdinESdMC+xEQoDXxXWiIZXvcYp0/808hpAQPrknYhX391XPel+qJ4EbqiImPqZ355CE8EXvlADeO953ft/EB5aKgwj5v8P530MknAmRcCZEwpkQCWdCJJwJkfAv6ObhbeIGuNEAAAAASUVORK5CYII=\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "#เลข 7 โดยเฉลี่ย\n", - "mean7 = stacked_sevens.mean(0)\n", - "show_image(mean7);" - ] - }, - { - "cell_type": "code", - "execution_count": 23, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 85 - }, - "id": "aKESuqYHpojz", - "outputId": "fc68b38d-ba24-4c02-9b92-154e5e4908f9" - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAJUElEQVR4nO2by08bVxuHn/HM2OP7BbsmYJNQKCQUCGkuyratmi7aRbfZt1L/ga66Sfbtv9FVF11E6aKLKG2RqBIlaUpSih0IoSF2YkrAl7l5xl1830w/GxJKGBP0yY+EkOacmfPOb95zed9zLLRaLXr8g+91G3DY6AnSQU+QDnqCdNATpANpl/L/5ylI2Oliz0M66AnSQU+QDnqCdNATpIOeIB3sNu16hm3bWJaFaZo0Gg0Mw0BV1RfW9/l8+Hw+UqkUiqIgiiI+X/e/34EJYhgG1WqVxcVFrl69ysLCAtevX39h/VgsRiwW48svv+Tdd98lGo0SCAS6bmfXBFFVlSdPniAIAqIoUq/XefbsGXfu3KFQKLC4uMj6+vpL769Wq9y6dYu+vj6GhoZIJBJdF0bYJR/yyivV3377jc8//xxJkkin0zx//pwnT57w/Plz1tfXsSwLy7JebJggIAgCwWCQQCDAxx9/zNtvv83FixfJ5/OvalZbEztd9NxDTNNEVVXK5TKlUgmAjY0N6vU6m5ubaJqGaZr/sUjY0SaXVquFqqrous6jR48IBAJsbm6SzWaRJKkrY4rngjQaDW7evMns7Cy1Wo1Go4GqqjieuNcMXavVwrIs7ty5w9LSEh9++CHZbJZYLNaVruOZIM7s8fjxY2ZnZykUCqiqSrPZxLZtt14qlSKfz5PJZHjjjTe2PadYLLKyssLW1lbbLNRsNjEMA13XMU1zz8L+WzwTRFVVbty4wezsLF999RWapu04RkxPT/PJJ59w9uxZJicnt5VfuXKFb7/9llu3brGysuJet23b7Y6GYRx+QVqtFoZhYJomhmFsE0OSJBRFIZfLMTk5STabbXN5VVWp1WqUy2XW1tbQNK3tflmWURSFvr4+otEooih6ZXq7nV4+zBFkp68XCoXIZrNMTExw+vRpAoEAfr/fLS+VSszPz3P79m3u3r2LYRht9weDQZLJJLlcjlQqteuA/Kp4JojP5yORSBCJRJAkyZ0yE4kEw8PDDA4OMjo6yvnz5/H7/QiCgGmaaJpGtVplfn6e69evs7y8jGma7rgjiiKSJHHy5EnGx8dJJpNdEwM8FEQURdLpNOl0GkmSCAaDZLNZTp06xYULFzhx4gQTExPIskwgEHAHyKWlJX755ReuXbvGlStX0HWdZrPpPtfv96MoCu+99x4XLlzYcSD2Es8EkSSJTCbD6dOn+fTTTxFFkUQiwdDQEDMzM6RSKWRZdvu+rutuN5mbm6NQKKDruusZ8XiccDjM5OQkg4ODnDlzhv7+fmRZ9srkHfF8pWoYBrVaDZ/PhyiKrkd0uvnq6io//fQT33//Pd988822cWd8fJzR0VE++ugjzp07x8jICPF4fK/mvIyDWamKooiiKAiC4IqyU5+vVqvcu3ePUqnUJkYmkyGdTvPBBx/wzjvvMD09TX9//4EEdtAlQUKh0K71KpUKc3NzLC0ttV0fHh7m5MmTXLx4kZmZGSRJ6toUuxOvNUHUarW2dRVd1914Z6fybnPoMma6rlOv19F1HcuyDlwQ8dKlSy8rf2nhflBVlXq9jm3bPHr0yPWGVqtFo9EAoFarYVkWzWbT/e+MSx5weaeLr00QQRBIJpM8ffqU+fl5bNum2WyiqiqVSoWVlRV+/fVXNzgUBIFWq0UwGPRq6j18goTDYSKRCOFwGFEUKZfLbeOGruuoqsrjx49ZXl5maWmJ/v5+r3KsOwpyYDnVToLBIMFgkJmZGaLRKLZtc/fuXQAsy6JWq7nBHkAymSQejzM0NEQkEkGWZSTJe/Nfm4c4+Hw+IpEImUyGY8eOceTIESKRCK1Wa9tMY5ompmny4MEDMpkMsiy3rX73yOHyEAfHUxRF4ejRo8zNzREOhwHcQdeyLDRNQ9M0vvvuOxRFYWBgAEmSCIVCbVHzfnntHuIgCAKKopDJZDhx4gTHjx9ncnKSTCaDoihomuZm0GzbJhaLUalUiEQi2LaNoih77UKH00Mc/H4/fr+fcDhMLpfj6NGjnDp1ilgsht/v56+//mJ9fR3btmm1Wty4cYM///yTaDTK1NQUyWQSRVH2bUfXtiH2i5NsevbsGZVKhZ9//plCocAPP/xAsVgkFosRDAaZmpoin89z6dKlvW5PHExw5xWOxwQCAY4cOYKiKAwNDbGwsECxWGRra4utrS2q1SqpVIovvvjCk3YPrSAOTrSczWaRZZlMJtPV9g69IM5WaDgcRhCEfxVJ74dDL4iz5bm2tsbq6qq7G9gtPBWkc4B+1WTw/+7ymaZJs9mkXC5z//59NjY2trXh5ZamZ4Lous7Tp09pNpvouk40GiWdTrsGO67vBHGw87Zms9nENE1WV1dZXV1lcXGRUqlEsVjk4cOHLC8vu3VFUeTMmTOMjIy4i7n94okgzpcslUrutkI2myUSibhBmLO8dl74RckfVVVRVZU//viDmzdv8vvvv7OwsEC5XGZra8ut5/P5kGWZfD7PyMiIZynGfQtiGAabm5sUCgW+/vprd082FArR19dHOBwmnU6TzWY5fvw4Dx484N69e+5SvBNHsFKpRKVSoVqt0mg02uoKgsDZs2d58803+eyzz3jrrbdIJBL7fRXAA0Fs26Zer7O2tsaPP/5IrVZD13W3PB6Pk8vlGB8fZ2Njg8XFRa5du0aj0aBer++pLafbybLMsWPHGB0dZWRkhP7+/v2+xj9t7Helats2hmHw8OFDLl++TLFY5Pbt2207b4qiEAqFiEaj1Go1NjY2sCyr7VTANsP+e2DG+QuFQgQCAd5//32Gh4c5f/48uVyOsbGxVx0/urNS9fl8KIpCLBYjn89jmiYLCwvu0tuyLOr1unuk6qUWCgKSJCFJEn6/3824OymCcDjM2NgY09PTbjfxOifiWSxTr9e5f/8+a2trzM3Nsby8zNWrVzFNs60LvYhIJEI0GuXcuXOMjY0xMTHB8PAw8XicUCjkzlTOGTO/34/P59vPSaLuxjKSJDEwMEAwGETTNCRJYnBwEFVV0TRt1+x5KpUilUoxOjrKzMwMU1NTDAwMEA6HD2yTCjz0EGd94RxsMU2TarXqlu2GMzUrikIgEHBThM551S6wo4cc2vD/AOj9Xubf0BOkg54gHfQE6aAnSAc9QTrYbWHWveN+h5Seh3TQE6SDniAd9ATpoCdIBz1BOvgb74E4FYpj4GsAAAAASUVORK5CYII=\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "#เลข 3 อันที่ 125\n", - "a_3 = stacked_threes[125]\n", - "show_image(a_3);" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "k4Rm7EZjU-Gb" - }, - "source": [ - "เราจะเห็นได้ว่าหากเราคำนวน mean absolute error และ mean squared error ของแต่ละ pixel ระหว่าง \"เลข 3 อันที่ 125\" กับ \"เลข 3 โดยเฉลี่ย\" และ \"เลข 7 โดยเฉลี่ย\" เราจะเห็นกว่าค่าของ \"เลข 3 อันที่ 125\" กับ \"เลข 3 โดยเฉลี่ย\" มีค่าน้อยกว่า และในกรณีนี้ระบบ (ที่ไม่ใช่ ML) ของเราจะทำนายถูกว่า \"เลข 3 อันที่ 125\" คือเลข 3" - ] - }, - { - "cell_type": "code", - "execution_count": 24, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "JbfdH2zWpoj0", - "outputId": "68513061-cf01-4337-adba-950be0372b4b" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(tensor(0.1259), tensor(0.2290))" + "cell_type": "code", + "execution_count": 6, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "EilsFTaWpojx", + "outputId": "97aecb3b-fce5-4f7c-9b62-e6d43253e616" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(#6131) [Path('/root/.fastai/data/mnist_sample/train/3/10.png'),Path('/root/.fastai/data/mnist_sample/train/3/10000.png'),Path('/root/.fastai/data/mnist_sample/train/3/10011.png'),Path('/root/.fastai/data/mnist_sample/train/3/10031.png'),Path('/root/.fastai/data/mnist_sample/train/3/10034.png'),Path('/root/.fastai/data/mnist_sample/train/3/10042.png'),Path('/root/.fastai/data/mnist_sample/train/3/10052.png'),Path('/root/.fastai/data/mnist_sample/train/3/1007.png'),Path('/root/.fastai/data/mnist_sample/train/3/10074.png'),Path('/root/.fastai/data/mnist_sample/train/3/10091.png')...]" + ] + }, + "metadata": {}, + "execution_count": 6 + } + ], + "source": [ + "#ในละ folder 3 และ 7 จะเป็นไฟล์รูป\n", + "threes = (path/'train'/'3').ls().sorted()\n", + "sevens = (path/'train'/'7').ls().sorted()\n", + "threes" ] - }, - "execution_count": 24, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#ความห่างระหว่า \"เลข 3 อันที่ 125\" กับ \"เลข 3 โดยเฉลี่ย\"\n", - "dist_3_abs = (a_3 - mean3).abs().mean()\n", - "dist_3_sqr = ((a_3 - mean3)**2).mean().sqrt()\n", - "dist_3_abs,dist_3_sqr" - ] - }, - { - "cell_type": "code", - "execution_count": 25, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "s5xmB8f1poj0", - "outputId": "d9967f38-5193-4528-b890-5de8e16a42a8" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(tensor(0.1836), tensor(0.3390))" + "cell_type": "code", + "execution_count": 7, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "2nKh8v3OBG5G", + "outputId": "9442d7c1-6125-4d88-c81a-e7e87d8b49ce" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(6131, 6265)" + ] + }, + "metadata": {}, + "execution_count": 7 + } + ], + "source": [ + "#จำนวนรูปในแต่ละ class\n", + "len(threes), len(sevens)" ] - }, - "execution_count": 25, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#ความห่างระหว่า \"เลข 3 อันที่ 125\" กับ \"เลข 7 โดยเฉลี่ย\"\n", - "dist_7_abs = (a_3 - mean7).abs().mean()\n", - "dist_7_sqr = ((a_3 - mean7)**2).mean().sqrt()\n", - "dist_7_abs,dist_7_sqr" - ] - }, - { - "cell_type": "code", - "execution_count": 26, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "lDlxHuO8Vazc", - "outputId": "a20160f4-2575-47c5-a8dc-bb5169feb261" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(tensor(0.1836), tensor(0.3390))" + "cell_type": "code", + "execution_count": 8, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 45 + }, + "id": "jJEUNNQKpojx", + "outputId": "16277235-20fb-47c8-d1f4-bb6bbb0483a9" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAAA3UlEQVR4nGNgGNxA1XnOxX//vxlgkVIqe//379+///4+EkCX4lv//e+/v38vnf33968wumTd379/rzUZ8lf9e1XChC4Zf39RAAMDA+eFf2txuIg18htuObW/f9/y45CM/Pt3iwYOuaxvv7Zz4pBjmPvvoxwSF9XNrxhZ/XFpZOCf9+tjMk5Zhsl/d+AyloHhI4MsTo22T//uF8cu5Xbo/79/8eiiXGVlyqyOk3/9/fe7nRld0urv3/uH/v79+/d1GKZ5In///v339+/zDdiClbVu9sF/X2cr4XQoVQEA4o1d+YAEyFQAAAAASUVORK5CYII=\n" + }, + "metadata": {}, + "execution_count": 8 + } + ], + "source": [ + "im3_path = threes[6000]\n", + "im3 = Image.open(im3_path)\n", + "im3" ] - }, - "execution_count": 26, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#ใช้ function ของ pytorch คิดก็ได้\n", - "F.l1_loss(a_3.float(), mean7), F.mse_loss(a_3, mean7).sqrt()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "4Lx6GAdSVgD2" - }, - "source": [ - "เรื่องน่าคิดถึงความแตกต่างระหว่าง mean squared error และ mean absolute error" - ] - }, - { - "cell_type": "code", - "execution_count": 27, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "IcAtO7IwDzUp", - "outputId": "cb2fc2a2-cf78-4ada-895d-29ab5a4e3cbb" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(1.1666666269302368, 1.1902379989624023)" + "cell_type": "code", + "execution_count": 9, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "GBYX4Mf0BTb6", + "outputId": "b77ba285-8176-455e-bd86-6d16b75f9acd" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(28, 28)" + ] + }, + "metadata": {}, + "execution_count": 9 + } + ], + "source": [ + "#รูปขนาด 28 x 28 pixels \n", + "array(im3).shape" ] - }, - "execution_count": 27, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "a = torch.tensor([1, 2, 3]).float()\n", - "b = torch.tensor([2.,3.,4.5])\n", - "c = torch.tensor([2.,3.,40.])\n", - "\n", - "#mse และ mae ไม่ต่างกันเท่าไหร่สำหรับ a และ b\n", - "(a-b).abs().mean().item(), \\\n", - "((a-b)**2).mean().sqrt().item()," - ] - }, - { - "cell_type": "code", - "execution_count": 28, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "y5zmXGk2poj0", - "outputId": "96a206ec-10da-4054-8de4-13e538559074" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(13.0, 21.3775577545166)" + "cell_type": "code", + "execution_count": 10, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "RZphr7qzpojx", + "outputId": "ba2bf4b9-1cdd-4583-8734-abf34a064f96" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "array([[0, 0, 0, 0, 0, 0],\n", + " [0, 0, 0, 0, 0, 0],\n", + " [0, 0, 0, 0, 0, 0],\n", + " [0, 0, 0, 0, 0, 0],\n", + " [0, 0, 0, 0, 0, 0],\n", + " [0, 0, 0, 0, 0, 0]], dtype=uint8)" + ] + }, + "metadata": {}, + "execution_count": 10 + } + ], + "source": [ + "#เปลี่ยนจาก numpy array\n", + "array(im3)[4:10,4:10]" ] - }, - "execution_count": 28, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#mse และ mae ต่างกันเกือบเท่าตัวสำหรับ a และ c\n", - "(a-c).abs().mean().item(),\\\n", - "((a-c)**2).mean().sqrt().item()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "ErTcHuGkpoj4" - }, - "source": [ - "## Stochastic Gradient Descent (SGD)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "XTAhc8bJWi6y" - }, - "source": [ - "หากยังจำได้จากบทที่ 1 เราเรียนรู้ว่าเราจะคำนวณ `Gradients` จาก `Loss` แล้วให้ Optimizer ทำหน้าที่ update `Weights` ในบทเรียนนี้เราจะมาเรียนรู้ขั้นตอนเหล่านี้กัน" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "V9fzeVckWhP9" - }, - "source": [ - "" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "OISepsl-poj5" - }, - "source": [ - "### คำนวณ Gradients เพื่อทำ Backpropagation" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "_9WrHtiu0ScN" - }, - "source": [ - "หากคุณยังไม่เคยเรียนเกี่ยวกับ [partial derivative](https://en.wikipedia.org/wiki/Partial_derivative) และ [chain rule](https://www.khanacademy.org/math/ap-calculus-ab/ab-differentiation-2-new/ab-3-1a/a/chain-rule-review) ในชั้นเรียนมัธยมปลาย คุณอาจจะไม่จำเป็นต้องเข้าใจเนื้อหาส่วนนี้ทั้งหมดก็ได้ ใจความสำคัญคือเราสามารถปรับแต่ง `Weights` ได้ด้วย `Gradients` ที่ถูกคำนวณมาจาก `Loss` เพื่อให้ได้ `Loss` ที่น้อยที่สุดเท่าที่จะทำได้ใน iteration ต่อๆไป" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "tP7xMjbi6er4" - }, - "source": [ - "ตัวอย่างการทำ Backpropagation อย่างง่ายจาก [cs231n](https://cs231n.github.io/optimization-2/#backprop)\n", - "\n", - "Independent variables:\n", - "(เราอาจจะคิดว่า `x, y, z` เป็น `Inputs` หรือ `Weights` สำหรับโมเดลก็ได้)\n", - "\\begin{align}\n", - "x & = -2 \\\\\n", - "y & = 5 \\\\\n", - "z & = -4 \\\\\n", - "\\end{align}\n", - "\n", - "Dependent variables: \n", - "(`q` และ `f` คือฟังชั่นอะไรบางอย่าง เช่น `Loss Function` ของโมเดลก็ได้)\n", - "\n", - "ถ้าแทนค่า `x, y, z` เข้าไปในฟังชั่น `q, f` จะได้\n", - "\n", - "\\begin{align}\n", - "q & = x+y = -2+5 = 3\\\\\n", - "f & = q*z = 3*-4 = -12\n", - "\\end{align}\n", - "\n", - "การทำแบบนี้เปรียบเสมือนการเปลี่ยน `Inputs` เป็น `Predictions` ด้วย `Weights` เรียกว่า `Forward Pass`\n", - "\n", - "หลังจากนั้น เราสามารถคำนวณหา `Gradients` ซึ่งโดยทั่วไปแล้วหมายถึงค่าอัตราการเปลี่ยนแปลงของฟังชั่นท้ายสุด (ในที่นี้คือ `f`) เทียบกับตัวแปรแรกสุด (ในที่นี้คือ `x, y, z`) ได้แก่ $\\frac{df}{dx}$, $\\frac{df}{dy}$, $\\frac{df}{dz}$ เราสามารถหาสิ่งนี้ด้วยการหา [partial derivative](https://en.wikipedia.org/wiki/Partial_derivative) และ [chain rule](https://www.khanacademy.org/math/ap-calculus-ab/ab-differentiation-2-new/ab-3-1a/a/chain-rule-review) เรียกว่า `Backward Pass`\n", - "\n", - "\\begin{align}\n", - "\\frac{df}{dq} & = z = -4\\\\\n", - "\\frac{dq}{dx} & = 1\\\\\n", - "\\frac{df}{dx} & = \\frac{df}{dq} * \\frac{dq}{dx}\\\\\n", - "& = -4*1\\\\\n", - "& = -4\\\\\n", - "\\end{align}\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": 29, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "VXdDIlJl6jYC", - "outputId": "326dcf7c-8210-49c7-918f-e4067ee240ab" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(-4.0, -4.0)" + "cell_type": "code", + "execution_count": 11, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "o52Dig-Vpojx", + "outputId": "9295e887-7371-4494-fca8-829823e96a64" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "tensor([[0, 0, 0, 0, 0, 0],\n", + " [0, 0, 0, 0, 0, 0],\n", + " [0, 0, 0, 0, 0, 0],\n", + " [0, 0, 0, 0, 0, 0],\n", + " [0, 0, 0, 0, 0, 0],\n", + " [0, 0, 0, 0, 0, 0]], dtype=torch.uint8)" + ] + }, + "metadata": {}, + "execution_count": 11 + } + ], + "source": [ + "#เป็น torch tensor\n", + "tensor(im3)[4:10,4:10]" ] - }, - "execution_count": 29, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# set some inputs\n", - "x = -2; y = 5; z = -4\n", - "\n", - "# perform the forward pass\n", - "q = x + y # q becomes 3\n", - "f = q * z # f becomes -12\n", - "\n", - "# perform the backward pass (backpropagation) in reverse order:\n", - "# first backprop through f = q * z\n", - "dfdz = q # df/dz = q, so gradient on z becomes 3\n", - "dfdq = z # df/dq = z, so gradient on q becomes -4\n", - "# now backprop through q = x + y\n", - "dfdx = 1.0 * dfdq # dq/dx = 1. And the multiplication here is the chain rule!\n", - "dfdy = 1.0 * dfdq # dq/dy = 1\n", - "\n", - "dfdx, dfdy" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "kt9JIcVZ3CuU" - }, - "source": [ - "Pytorch สามารถทำ `Backward Pass` ให้เราโดยอัตโนมัติด้วยฟังชั่น Autograd โดยที่เราไม่ต้องคิด partial derivative เอง ผลข้างเคียงอีกอย่างคือเราสามารถใช้ Pytorch ช่วยทำการบ้านวิชาแคลคูลัสเวลาเราหา derivative ที่ยากเกินไปไม่ออกได้อีกด้วย" - ] - }, - { - "cell_type": "code", - "execution_count": 30, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "SWg90ZHApoj6", - "outputId": "96fde65e-54bc-43c4-a8e1-e4217f5441c1" - }, - "outputs": [ { - "data": { - "text/plain": [ - "tensor([ 3., 4., 10.], requires_grad=True)" + "cell_type": "code", + "execution_count": 12, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 394 + }, + "id": "0yZCaBRNpojy", + "outputId": "9e3f04a1-6be4-48db-ac47-3ee450e0c536" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
 01234567891011121314151617
0000000000000000000
10000000000376715620925425524648
20000000034118239253253253253254253226
300000014175247253254253253210205254253253
40000001262532532532141304915122254234116
5000000952231628000092082541730
60000000000058924625417300
700000000005382532532371500
800000000008925325318040000
900000000010624625018390000
100000000001572542413000000
\n" + ] + }, + "metadata": {}, + "execution_count": 12 + } + ], + "source": [ + "im3_t = tensor(im3)\n", + "df = pd.DataFrame(im3_t[4:15,4:22])\n", + "df.style.set_properties(**{'font-size':'6pt'}).background_gradient('Greys')" ] - }, - "execution_count": 30, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "xt = torch.tensor([3.,4.,10.]).requires_grad_()\n", - "xt" - ] - }, - { - "cell_type": "code", - "execution_count": 31, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "WpM1Q5nApoj6", - "outputId": "f969c1d9-e911-4bf7-9622-2bf843b63be3" - }, - "outputs": [ { - "data": { - "text/plain": [ - "tensor(125., grad_fn=)" + "cell_type": "markdown", + "metadata": { + "id": "jMdNhmiXpojy" + }, + "source": [ + "## วิธีที่ง่ายที่สุด: ดูว่ามี pixel เหมือนกันแค่ไหน" ] - }, - "execution_count": 31, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#ฟังชั่น f(x) = (x1^2 + x2^2 +...+xn^2)\n", - "def f(x): return (x**2).sum()\n", - "\n", - "yt = f(xt) #ใส่ 3, 4, 10 เข้าไปได้ 3^2+4^2+10^2 = 125\n", - "yt" - ] - }, - { - "cell_type": "code", - "execution_count": 32, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "POJB2ewmpoj6", - "outputId": "de1feba8-9f02-4654-8622-11a1b4b30b51" - }, - "outputs": [ { - "data": { - "text/plain": [ - "tensor([ 6., 8., 20.])" + "cell_type": "markdown", + "metadata": { + "id": "MqYq_GbFT7XA" + }, + "source": [ + "สมมุติเราไม่ทำ ML อะไรเลย แต่เราพยามสร้างกฎแบบ Rule-based Systems โดยบอกว่า \"รูปที่มี pixels ใกล้เคียงกับ pixels เฉลี่ยของเลข 3 และ 7 มากกว่า ให้ถือว่าเป็นเลขนั้น\"" ] - }, - "execution_count": 32, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#ถ้าคิดด้วยมือ df(x)/dx = 2x \n", - "#สำหรับ x1=3, x2=4, x3=10 ก็จะเป็น \n", - "#df(x1)/dx1 = 6, df(x2)/dx2 = 8, df(x3)/dx3 = 20\n", - "\n", - "#ใช้ autograd หา df(x1)/dx1, df(x2)/dx2, df(x3)/dx3\n", - "yt.backward()\n", - "xt.grad" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "mHHqwvz2poj_" - }, - "source": [ - "## สร้าง Loss Function สำหรับจำแนกรูปเลข 3 และเลข 7" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "U0EQ6WD4zEf0" - }, - "source": [ - "#### สร้าง X และ y" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "t2aoyhac4ZrI" - }, - "source": [ - "จัดการ `Inputs` คือรูปตัวเลข 28x28 pixels และ `Labels` คือ `1 ถ้าเป็นเลข 3` และ `0 ถ้าเป็นเลข 7`" - ] - }, - { - "cell_type": "code", - "execution_count": 33, - "metadata": { - "id": "-4LTBmPcpoj_" - }, - "outputs": [], - "source": [ - "train_x = torch.cat([stacked_threes, stacked_sevens]).view(-1, 28*28)" - ] - }, - { - "cell_type": "code", - "execution_count": 34, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "Q5bU0JpipokA", - "outputId": "fa8b6406-e3a7-4721-fef1-218c6e38fbf7" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(torch.Size([12396, 784]), torch.Size([12396, 1]))" + "cell_type": "code", + "execution_count": 13, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "JDsXZWi8pojy", + "outputId": "34c39b1d-84e9-43ed-8364-e30f8b4969a3" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(6131, 6265)" + ] + }, + "metadata": {}, + "execution_count": 13 + } + ], + "source": [ + "seven_tensors = [tensor(Image.open(o)) for o in sevens]\n", + "three_tensors = [tensor(Image.open(o)) for o in threes]\n", + "len(three_tensors),len(seven_tensors)" ] - }, - "execution_count": 34, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "train_y = tensor([1]*len(threes) + [0]*len(sevens)).unsqueeze(1)\n", - "train_x.shape,train_y.shape" - ] - }, - { - "cell_type": "code", - "execution_count": 35, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "rkavLIe1IcmD", - "outputId": "10446b01-f0b3-470d-cdcd-fb1752c76a8e" - }, - "outputs": [ { - "data": { - "text/plain": [ - "[(1, 4), (2, 5), (3, 6)]" + "cell_type": "code", + "execution_count": 14, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 85 + }, + "id": "9h_vpbbvpojy", + "outputId": "3873c893-9aad-43cb-be95-7ba2aa239f63" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAJHElEQVR4nO2bXXMSZxuAL1jYXYQsiCYxIWIIjImJ0TaVTu2H41FnnGmPPOtMf0NP+i/6H9oDx+OOOtMjW6cdGz/Sk1ZqxkRDhCTQEAjfsLDsvgeWbbMmxgrEzDtcR5ln+bi5eJ5n7/t+iM0wDPr8g/1tB3DY6Aux0BdioS/EQl+IBcc+1/+fb0G23Qb7M8RCX4iFvhALfSEW+kIs9IVY6Aux0BdiYb/ErGMMw6DVatFsNqlUKmiaRrPZpNls0mg0Xnq8LMtIkoSu6+i6jtPpxOFw4HQ6EQQBWZZxOHoXds+FaJpGvV4nHo/zww8/kMlkSKVSxONxnjx5wr/7MTabjffee4+pqSmq1SqqqjIyMsKxY8eIRCIMDw8zOzuLz+frWbxdF6LrujkDisUilUqFbDbLn3/+yeLiIoVCgfX1dTKZDJVKBVEUEUWRer1OrVZjZWUFh8NhCqlUKmxublIulzl+/DgTExM9FWLbp2P2n2sZVVXZ3t5mdXWV77//nlQqxaNHj8jlcqTTaQzDwDAMZFnG7XYzODhIIBDgyZMnPH/+HLvdjt1uN2eOzWbDZrPh8/nwer1cv36daDT6hh93B7vWMh3PEE3TqNVq1Go1UqkUlUqFVCrFysoKT58+pVQqIQgCExMTRKNRnE4nkiQhiiIulwtFUfB6vczOzpLJZFheXubZs2eUy2VqtZr5Po1GA1VV0XW905BfScdCVFUlFovx22+/8c0331CpVGg2m7RaLRqNBoFAgGg0ysWLF7l69Soejwe3220+3263Y7PZ0HUdwzC4ceMG165dIxaLkUwmOw3vP9OxEMMwaDQaVKtVSqUS1WoVXdex2+2IokggEGBubo5z587h8/lwOp04nU7z+e0l8e+lJAgCNtvOGe33+wkGg8iy3GnIr6QrQqrVKrVaDVVV0TQNAFEUOXr0KHNzc3zxxRf4fD4URXnpg7ZpjzudTux2+0vXpqenmZmZQVGUTkN+JR0LcTqdhEIhRFEkn8/TbDaBF0Lcbjdzc3N4vV5EUdxTBrzYizRNI5/Pk8/nzRzFZrMhCALDw8OEw2FcLlenIb+SjoVIksTp06cJh8N88MEH5nh7KQiCgCiK+75Oo9GgVCqxtrZGMpmkWq0CmM+PRCJEo9Ed+08v6FhI+1sXBGHH3tC+Zp3+e7G+vs7PP//M77//TqlUQtM0BEEgHA4zPj7OzMwMJ06ceOk9uk1XErP2bHidmbAXd+7c4auvvkLTNHRdx+FwIIoiFy9eJBqN8u6773LixIluhPtKep66WzEMA13XqdfrlMtlM2FbWFgwN2S73U4kEiESifDhhx8SjUZ7vpm2OXAhuq6jaRrZbJbHjx/z448/cvPmTbLZrHm7djgcXLhwgY8//phPP/2UkydPHlh8B1LtGoZBsVhkdXXVrGWSySSrq6ssLS2Ry+Wo1+vAi3zD7/dz+vRpzp8/z8DAQK9D3MGBlf/JZJLvvvuO5eVlHjx4gKqqO1LzNkNDQ0xPT3PhwgUmJyd7fpu1ciBLRtd1CoUCjx49Yn19HVVVzXzFSiaTIRaLcevWLeLxOMeOHcPj8TA2NobX62VwcLCnkg5syWxtbfHgwQM0TTM31t3IZDJkMhnW19dxu90oioLH4+HKlSucP3+eS5cuIcvyK5O8Tuh6+f/SC/y9ZDY2Nrh9+zblcplCoUCxWCSXy5mPi8fjLC0tmT0UURRxOp1mv2RqaorR0VE++ugjzpw5w9mzZ/F6vQiC8Nq5joVdjfZciJVarWbKWFtbM8d/+uknfvnlF1ZXV0mn07s+t91Ri0QifP3110xOTiJJEoIgvEkovemH/FecTieKoiDL8o7O19DQEJcuXWJtbY10Ok0mkyGXy3H//n3i8TjwYrYlEgmq1SrPnz9ncHCQ48ePv6mQXTlwIQ6HA4fDgcvlwuv1muPDw8NMT09TrVbNW/TKygrZbNYUArC5uUkulyMejxMKhTh69Gh34+vqq3VAuxBsd9VlWSYYDBKPx/nrr79IJBJsb28DL2bKxsYG8XicU6dOdTWOQ3Mu0y4EJUkye63BYJDZ2VmmpqZ2pO6GYZgzR1XVrsZxaITsRbtwtI55vd6eVL+HXgjAbndCt9uN3+/v6oYKh2gPsVIsFtne3mZhYYGHDx/uyFlsNhunTp0iEol01HLYjUMppF0MJpNJEokEiURiR2YrCAJDQ0P4/f6uH2seOiHVapVKpcK9e/e4c+cOf/zxh3lEATA5Ocn4+DiBQABZlt80S92TQyOk/YHr9TrZbJZYLMb8/DypVMq8ZrfbCQaDRCIRvF4vDoej6zXNoRFSKBRIp9Pcvn2bu3fv8vjxY5LJpNknGRgYwO12c/XqVS5fvszo6Oiu5zed8laFtCthwzDI5/M8ffqUhw8fcuPGDbO3Ci820SNHjjA4OMi5c+cIhUI9kQFvUYiqqqiqysbGBktLS9y9e5f5+XkSiQTNZtNcJi6XC1mW+fLLL/nkk08Ih8M9kwFdFvLvb9yaULXH2383Gg0KhQLxeJz5+XkWFha4d++e+fj2me/AwACKonD27FneeecdPB5Pz2RAF4Vomka1WqVcLrO2toaiKIyMjJiH3pVKha2tLUqlEpubmywvL7O4uGjWKfl8HvgnM41EIoTDYa5cuUI0GiUUCqEoSk9/PQRdFNJqtSiVSmxtbRGLxRgdHUWSJPMXRLlcjuXlZba2tsxl0u6tqqq645RPFEXGx8cJh8NEo1FmZmbMhlGv6ZqQfD7Pt99+SzKZ5P79++Zhd6vVotVqmecwqqqaM6ZSqZgb5/DwMGNjY7z//vtmk/nkyZMoioIkSV3PN/aia0IajQaJRIJnz56xuLj4Wj9ssdlsSJKEJEmMjY0RiUQ4c+YM0WiUiYkJ/H5/t8J7bbomxOPxcPnyZTweD7/++uu+QmRZ5siRI3z++ed89tlnhEIhRkZGcLlcB7Y8dqNrQhwOB8FgkHQ6TSAQMI8l98LlcuHz+ZicnGR2dpahoaEdHbS3RdeazK1WyzxvKRaL+7/x3w0ht9ttdsm6XcrvF8KugwfddT9E9P+j6nXoC7HQF2KhL8TCfrfd3lVRh5T+DLHQF2KhL8RCX4iFvhALfSEW/gcMlBno19ugeQAAAABJRU5ErkJggg==\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "show_image(three_tensors[1]);" ] - }, - "execution_count": 35, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#เราสามารถนำ iterables สองอันมาต่อกันแบบนี้ได้ด้วย zip\n", - "a = [1,2,3]\n", - "b = [4,5,6]\n", - "list(zip(a,b))" - ] - }, - { - "cell_type": "code", - "execution_count": 36, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "08BIemoApokA", - "outputId": "1ed03c6d-72af-4ff9-cb58-11251e7790e6" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(torch.Size([784]), torch.Size([1]))" + "cell_type": "code", + "execution_count": 15, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "D5B7mZNspojy", + "outputId": "0d8770e5-0ccc-4af4-8781-f871c04e7b14" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(torch.Size([6131, 28, 28]), torch.Size([6265, 28, 28]))" + ] + }, + "metadata": {}, + "execution_count": 15 + } + ], + "source": [ + "#เรามัดเลข 3 และ 7 ทั้งหมดรวมกันตามคลาสแล้วหารมันด้วย 255 เพื่อให้ได้ค่าระหว่าง 0 และ 1\n", + "stacked_sevens = torch.stack(seven_tensors).float()/255\n", + "stacked_threes = torch.stack(three_tensors).float()/255\n", + "stacked_threes.shape, stacked_sevens.shape" ] - }, - "execution_count": 36, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "dset = list(zip(train_x,train_y))\n", - "example = dset[0]\n", - "\n", - "#คู่ Inputs　และ Labels\n", - "example[0].shape, example[1].shape" - ] - }, - { - "cell_type": "code", - "execution_count": 37, - "metadata": { - "id": "Y6ImID5ApokA" - }, - "outputs": [], - "source": [ - "#สร้าง validation set ในแบบเดียวกัน\n", - "valid_3_tens = torch.stack([tensor(Image.open(o)) \n", - " for o in (path/'valid'/'3').ls()])\n", - "valid_3_tens = valid_3_tens.float()/255\n", - "valid_7_tens = torch.stack([tensor(Image.open(o)) \n", - " for o in (path/'valid'/'7').ls()])\n", - "valid_7_tens = valid_7_tens.float()/255\n", - "\n", - "valid_x = torch.cat([valid_3_tens, valid_7_tens]).view(-1, 28*28)\n", - "valid_y = tensor([1]*len(valid_3_tens) + [0]*len(valid_7_tens)).unsqueeze(1)\n", - "valid_dset = list(zip(valid_x,valid_y))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "RR1v1Z2tzHmo" - }, - "source": [ - "#### Initiate `Weights`" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "7slIAUaOI0_d" - }, - "source": [ - "สมมุติว่าเราจะใช้ architecture สุดเรียบง่าย แค่คูณค่า pixels ของรูป `Inputs` ด้วย `W` และบวกด้วย `b`\n", - "\n", - "$$prediction = \\Sigma(xW^T) + b$$\n", - "\n", - "เราสามารถเริ่มตั้ง `Weights` เป็นการ random จาก standard normal distribution" - ] - }, - { - "cell_type": "code", - "execution_count": 62, - "metadata": { - "id": "WQ9vpYHIpokA" - }, - "outputs": [], - "source": [ - "def init_params(size, std=1.0): return (torch.randn(size)*std).requires_grad_()" - ] - }, - { - "cell_type": "code", - "execution_count": 88, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "itXv1TZ9pokA", - "outputId": "50f3e5d0-626e-4633-a395-92a02991d36f" - }, - "outputs": [ { - "data": { - "text/plain": [ - "torch.Size([784, 1])" + "cell_type": "code", + "execution_count": 16, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "K4mmFyngpojz", + "outputId": "03cb8688-0ce1-44a6-c239-25654368389a" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "3" + ] + }, + "metadata": {}, + "execution_count": 16 + } + ], + "source": [ + "len(stacked_threes.shape)" ] - }, - "execution_count": 88, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#Inputs มี dimension (batch_size, 28*28)\n", - "#เพราะงั้นถ้าเราจะคูณรายตัว (element-wise multiplication) ด้วย W^T, W ต้องมี dimension (28*28,1)\n", - "weights = init_params((28*28,1))\n", - "weights.shape" - ] - }, - { - "cell_type": "code", - "execution_count": 103, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "7Ef5fK0KpokA", - "outputId": "767be632-37cc-4583-b733-0d8c382d6ffb" - }, - "outputs": [ { - "data": { - "text/plain": [ - "tensor([0.6971], requires_grad=True)" + "cell_type": "code", + "execution_count": 17, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Jck_enHBpojz", + "outputId": "7c732263-92aa-40d7-caec-2c98b91026aa" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "3" + ] + }, + "metadata": {}, + "execution_count": 17 + } + ], + "source": [ + "stacked_threes.ndim" ] - }, - "execution_count": 103, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#bias มีแค่ (1) dimension แล้วจะถูก broadcast ไปทุก dimension ของ batch size เอง\n", - "# (1) -> (batch_size,1)\n", - "bias = init_params(1)\n", - "bias" - ] - }, - { - "cell_type": "code", - "execution_count": 104, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "pnMvUz7kFoIU", - "outputId": "903243e1-5149-4c37-c02c-554d8a5fe665" - }, - "outputs": [ - { - "data": { - "text/plain": [ - "(torch.Size([784, 1]),\n", - " torch.Size([1]),\n", - " torch.Size([12396, 784]),\n", - " torch.Size([784]))" - ] - }, - "execution_count": 104, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "weights.shape, bias.shape, train_x.shape, train_x[7].shape" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "KXK2LAZqzLYk" - }, - "source": [ - "#### Forward Pass" - ] - }, - { - "cell_type": "code", - "execution_count": 105, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "S9md33qTEu6n", - "outputId": "4b257356-e622-40c9-ea2c-aac1966e4fcb" - }, - "outputs": [ { - "data": { - "text/plain": [ - "tensor([[0.5587]], grad_fn=)" + "cell_type": "code", + "execution_count": 18, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 103 + }, + "id": "cxcx0MIlg-fI", + "outputId": "b7af8b59-816c-4937-fe79-d9b1514d5a3d" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "" + ] + }, + "metadata": {}, + "execution_count": 18 + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAIEElEQVR4nO2bS08TbRuArx6ntJQONBQoYIFgEPCQYBQ0Rl2YuDFGoy504Q/xJ7j1B7hwR+LGjboQSZRo8IQBLIdWC5aDI1Baep525luYzgulKF++TjHv1yvpZmae9u41z9z3c2gNqqpS5R+MBx3A30ZVSBFVIUVUhRRRFVKE+Q/n/80lyFDqYLWHFFEVUkRVSBFVIUVUhRRRFVLEn8pu2VEUhXw+Tz6fR5ZlMpkMqVQKs9mMxWLBbDZjMpm06wVBwGKxYDCUrJJlp+JCstksW1tbrK6uMjs7y/v373nz5g0+nw+v16u9Cpw5c4bm5maMRmNFpFRMiCzLpNNp1tfX+fbtG9++fWN+fp5AIEAoFCKbzZJMJolEIqysrABgNBrp7u6mvr4eq9WK2ax/uBUTEo1GmZiY4MWLFzx69IhUKkUymSSXy5HP51laWuL9+/cYjUaMxl+pzWAwIIoiDoeDlpYWamtrdY+zYkIsFgv19fXYbDaSySTpdJp0Oq2dz+fzJdvNzc3x7t07hoaGMJlMWp7RDVVVf/cqG7Isq4lEQh0eHlZbWlpUp9Op8muu9NuXy+VS29vb1QcPHqh+v1+Nx+PlCqnkd65Y2TUajZjNZlpaWjh9+jRtbW2YTCYtUZrNZux2+667n0ql2Nzc5MePH0iShCzL+sap67tv/yCjEavVSldXF3fv3uXixYsIgqAJqK2tpampCYfDsaNdNpslHo8TDAYZHx8nHo/rG6eu716C2tpaenp6aG9vx+VyIQgCACaTCUEQdoxBtuPxeDhy5Ag1NTW6xldxIU6nk76+PoaGhvD5fLhcLgCsVit1dXVYLJZdbQwGA8ePH+fs2bO6V5qKD8wKOcPtdjM0NIQoilq1kSRpR+U5CCoupIDX6+XWrVuMjIwQi8UIhUKEQqE9ry8M+fXmwCZ3NTU1tLe3c/78eW7cuMG5c+dwu90lc4SqqgQCAfx+P6lUSte4DqyHOBwOHA4HTU1NDAwM4Ha7CYVCLC4u7vrSqqoyPj7O1tYWXq8XURR1i+vAhKTTaeLxOIlEgmg0SiAQYHV1lUQisetag8GA2+2mpaUFq9Wqa1wHJmRrawu/38/y8jLhcJhPnz7x/ft31D32mj0eDz09PbqX3YoJyeVyyLLM2toac3NzzM/PMzMzQywWIxqNMjc3t6cMgKamJrq7u7HZbLrGWVEh8XicsbEx7t+/jyRJ/PjxA0VRUBTlt20NBgM+n4/Ozk5tIKcXFasy+XyeWCyGJEmsra2RSCRQFOW3vaKAqqrMz8/z5cuXf0+VyeVybGxsIEkSP3/+RJblP/aM7czNzWGz2Whra9NGt3pQMSGCINDR0cGFCxcIh8P4/X4+fvy4r0cGIJlMEovFdB+cVUyIzWbDZrPR39/PlStXEASBycnJffUUVVVJJBLE43FyuZyucVa87IqiyODgIKIo0tDQQDabJZvNaudVVUVRFMbHx5menkaW5YoM2QtUXIjdbsdut+N0OmlsbCSTyZDJZLTzBSHpdJpgMKhtWVSKAxuY1dTU4PP5Sk7aFEXh4sWLKIrC69evCQaDrK6u4nA42NjYIJ1OY7FY9lw7+V84MCGCIOw5plBVlRMnThCLxVhYWCAYDLK5ucni4iKxWIxMJoPJZNJFyF+3lVl4ZKampnj8+DEzMzPAP2uyem9YHVgP2YuCkEAgwOjoqHa8sAXxfydkdXWVYDDI/Py8dsxgMDAwMMCxY8doa2vDZrPp8riADkIK+xvb7+J/c0clSWJsbIzv37/vOO71eunv70cUxZLrruWibEIURdGG569evaKhoYGOjg5EUcTtdv+xfS6XI5fLMT09zdOnT3f1kK6uLk6ePIndbi9XyCUpW1JVVRVZlvn58yfPnj1jdHSUUChEJBLZcxK3fccsl8uRyWRYXFzk8+fPrK+vA79kmEwmmpqaaGxs1LV3QBl7yNbWFs+fP2dycpKXL1/idrtZWlri1KlTXL9+HavVitVqxWKxIAgC6XSaVCqlbXr7/X4+fPjAq1evtJkwwODgIL29vfT19emaOwqUTUgqlWJiYoJAIEA4HCYSiZBOp3E6nUiSpK2hOhwOzGYz2WyWSCRCNBolEonw9u1bRkZGCIVC2nzFaDTS2dnJsWPHaGho0MqunpRNiNPp5OrVq3z69InFxUXW1tZYWFjgyZMnzM7OarmksDYaDof5+vUr8XicaDTK8vIy6+vr2r5MQ0MDoihy6dIlLl++TH19/Y69YL0omxCz2UxrayvxeJy2tjby+TwrKyuEw2FCoRC1tbV4PB6am5s5dOgQX79+xe/3k8lkdkzu4FfecLlc+Hw+Dh8+jMfjqYgMAMMfVqz2/dNuRVHIZrMsLS3x8OFDVFXFarXi9/sZHh7WNrutVqv2G5FkMrkr4dpsNgRB4N69e9y8eROPx4PdbsdgMJRbSMk3K1sPMRqN2Gw26urqaG1tRRAEvF4vsixjt9u1JJnJZDQRpWaxdrsdURQ5ceIEXV1d5Qpv35R9YCaKIrdv3wZ+df3CD+YKQjY3N5EkCb/fz9TUlNauUH3u3LnDtWvXOHr0aLlD2xdlF2KxWBBFUZuTNDc3MzAwoPUGSZJwuVwoioIkSVq7worakSNH6O3txel0lju0fVG2HFKy8bYBV+FzCo9KJpPZsdNfyBGiKFJTU1OJElsyh+gq5C+n+n+Z/VAVUkRVSBFVIUVUhRRRFVLEnwZmlfmTyl9EtYcUURVSRFVIEVUhRVSFFFEVUsR/AP0FXN1zCRLUAAAAAElFTkSuQmCC\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "#เลข 3 ที่ index 0\n", + "show_image(stacked_threes[0,:,:])" ] - }, - "execution_count": 105, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#คำนวณ forward pass สำหรับตัวอย่าง 7\n", - "(train_x[7]*weights.T).sum(1)[:,None] + bias" - ] - }, - { - "cell_type": "code", - "execution_count": 106, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "22qYPEQDEwjC", - "outputId": "a4b12d0c-c2fa-435b-ae26-611940166c7e" - }, - "outputs": [ - { - "data": { - "text/plain": [ - "tensor([[ 7.6066],\n", - " [22.2802],\n", - " [ 9.7072],\n", - " ...,\n", - " [ 8.1622],\n", - " [ 9.2091],\n", - " [-0.7656]], grad_fn=)" - ] - }, - "execution_count": 106, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#คำนวณ forward pass สำหรับทุกตัวอย่าง\n", - "(train_x*weights.T).sum(1)[:,None] + bias" - ] - }, - { - "cell_type": "code", - "execution_count": 107, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "I65On42BpokB", - "outputId": "3c2f1528-6aae-4999-ee30-c086090ea535" - }, - "outputs": [ - { - "data": { - "text/plain": [ - "tensor([[ 7.6066],\n", - " [22.2802],\n", - " [ 9.7072],\n", - " ...,\n", - " [ 8.1622],\n", - " [ 9.2091],\n", - " [-0.7656]], grad_fn=)" - ] - }, - "execution_count": 107, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#เขียนเป็นฟังชั่น; @ คือ matrix multiplication ใน pytorch\n", - "def linear1(xb): return xb@weights + bias\n", - "preds = linear1(train_x)\n", - "preds" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "XJKeTmrPzOr3" - }, - "source": [ - "#### Metric ใช้ Accuracy" - ] - }, - { - "cell_type": "code", - "execution_count": 111, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "SLnbggPopokB", - "outputId": "e41a9fc7-78f5-4d9e-f46d-b9e08a62eaf9" - }, - "outputs": [ - { - "data": { - "text/plain": [ - "tensor([[ True],\n", - " [ True],\n", - " [ True],\n", - " ...,\n", - " [False],\n", - " [False],\n", - " [ True]])" - ] - }, - "execution_count": 111, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#ถ้าสมมุติว่า Predictions >0 ให้ทายเป็น 1 (เลข 3)\n", - "corrects = (preds>0.0).float() == train_y\n", - "corrects" - ] - }, - { - "cell_type": "code", - "execution_count": 112, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "5DVZlAVzJqbV", - "outputId": "60c481ad-1b1b-40a8-e74c-0fff4962eda5" - }, - "outputs": [ { - "data": { - "text/plain": [ - "Counter({0.0: 647, 1.0: 11749})" + "cell_type": "code", + "execution_count": 19, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 85 + }, + "id": "2Es3hPcXpojz", + "outputId": "492d2c79-b892-48a9-f20d-8c4eba02024e" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAJtUlEQVR4nO1b2XLiWhJM7QsChDG22x3h//+qfnKzWVhoX5HmoaNqDufK9jRge2aCiiCEAS0nVUtWlqz0fY+r/dvU776A/za7AiLZFRDJroBIdgVEMv2D7/+fS5Ay9OHVQyS7AiLZFRDJroBI9lFS/RQ7tV1QlME8eFH7dEDkxdPf4udDAMmLVxQFfd8Pfn5JuyggQ4vs+/7o1XUdfy6+l01RFCiKAlX9E9WqqvJn9JJ/fwm7CCDy4ruu423XdTgcDjgcDqiqCm3boq5rtG2LpmlwOBzQti3vr6oqVFWFaZrQNA2WZUHXdViWBU3ToOs6NE3j3xFQZOcCcxYgQ15AIHRdh7Zt0bYtqqpCXdcoigJlWaIoCtR1jTzPUdc1yrLk4+i6DlVVMRqNYFnW0dY0Tdi2DcMwYBgGNE1jEGRgTrWTARHBkD2haRpUVYU8z1EUBcIwRJIk2Gw2CMMQu90OaZoiiiKUZYk8z3E4HNB1HUzThGEY8DwPo9EIi8UCs9kMj4+PmM1mmM/ncF0X4/EYlmUdeY4YYqeCc7aHDAFSliWqqkKSJMjzHLvdDmEYYrPZII5jBEGALMsQRRGyLEOWZRw6dOdnsxk8z0PXdSiKAoqioKoq6LqOw+EAXdfR9z17CYXPUOL9dEDkXCHmiKqqEMcx0jTFdrtFGIZ4fn7Gfr/HZrNBmqYIggBJkiCKIlRVhbIs0bYtDocDn8NxHFiWhYeHB9ze3iIMQ9zc3CDLMtze3qJtW0wmEyiKAsdxjpLvOXZ2UhXzBy1K3LZty+GlaRpM08RoNIKiKNA0jQFpmgZt27Knkaf0fY+6rlHXNYdhlmX8N53DNE2+ji/3EAJiKHfQhdZ1zVVEVVVYlsVxb9s270P7U/WhvNM0DZqmga7rnJgp76iqivl8Dk3TMB6Poes6uq7jkKEbcAowJ4eMaHRiimNd19kTyHNs24Zt27xQApRAITAJkKIokOc5NE07SpbyfqKHXsIuRszoonVdh+M4nOw8z4Pv++wB8j60QAJgv98jjmOuTMRZdP3PpYqe+R4Q31Jl6MQiGADQdR2Tp6ZpOESImYpGn2dZBl3XUVUViqJgPkILo5xD5zEM44ikib87x04CRD45ubVpmnyRXdfBcZyjaiTfTSJvTdPAsiyYpvmPUAFwlBNM02SCRpxlCLxT7SwPES9AVVVehBgKsnuLpZq25BVhGGK/32O/3yNNU+R5zhVLVVUYhnHEXonWEykb6nG+HBDxLhIQcqKTgaBS3Pc9V4+Xlxes12usViu8vLwgiiLs93tesKZpsG0bk8kEvu9jNBrBdV0GhRL6uXYyIDIQiqKg67pBUESeIvKJJEmYwa7Xa2y3WwRBgN1uhyzLkOc5JpMJl2rXdeF5HsbjMRzHOQpRsRv+FkBEUAgEIlJvtfvU4NHdXy6XWK1W7BUESBiGHIau60LTNDiOg8lkwpTecRw4jnPkHd+WQwgA8b28JRAOhwN3tHEcI4oiDo3lcsmhst1usdlsUBQFqqri5EkVi7jNUHW5VIU5GZD/BBSRxRZFgSRJsN1usVqt8OvXL6zXazw/P+P3799YLpeI4xhJkvAxPc+D53kA/lQxSqhi6y+LRnQt59jFRGbxQihcyDuKouCmbrPZYLPZIAgCLJdLBEGANE1R1zULRCLPAHBUiaiBpLZAZqvnMtazc8iQZirS67qukWUZ4jjGer1mQJbLJZbLJZIkQZIkXIYVRTnyAjpWVVUsFbiue9QMUh/zrSHznhFIIg+h5EpyoO/7WCwWGI1GmEwmDKSmaZxEKUwo7KIoQhAE3NRR9yxrr8A3UnfRZJFZJmZEvy3Lgud5uLu7Y3WNTNRLAcAwDHRdhzzPoaoqXl9foWkabm5uoOs6bNvm4367h7w3SlAUhXOB67po2xY/fvxghlkUBbIs49AiI/BkQbrve65UqqoiiiJomgbXdTnviJ5C1/C3dramKr8XL0am2/P5HI7jwHVdFn1kSk+9DVH3OI4ZOGK1qqpiv9/DsixUVcVeJHriqXaWHjK0FUsxjRNc14VhGDBNE3VdYz6fs2fIgJCC9vr6ijiOmXiRStY0DVet0WiEsixhmib3O8SWvyyHvFVVxO+o+pD7ip0pdbhDs5yu61igtm2bQ4tAAoC6rqFpGqv1ojInMmU69t8Cc5aHyG39EDAU3zRzeav5I05BJZd6HgLGMAxOvqJsQFLkECf58hwytDBxS6CQejaUa2QPIbNtm0uvyErl38tgnGt/BYjsGVQd3tM236PW4gLp+LRYyhUkWFOYDc13L8FQyU7OISIAlBxliVAeWL8Finx8keWSQCQn7KF9ZftS1V2Me3FoTQuStVZRURMBot9TcozjmGn+arXCbrfjkSclTmK7NBCn7ve9pwM+DRBZ6xABaZrmKBcQ42zbli9cFKMVRWFQReEoTVNW37MsQ1EURyFD9F7WU7+NqYqA0BCpbVu+8Kqq+Ht5VkPAkFG1oMZts9mwRvL6+ookSVAUBatjJBT5vo/pdMojTzr2uU3eSUlVBoW8g2a0RVFwDiAKL48OyGhARSradrvFbrdDEAQcKsRGiehRBSJuQ99dwlP+ChBZFBJPTqSqLEsEQcC0m7xIfIaDRo9Ex8uyRJqmSNOU5YA8z1GWJfMQz/MwnU5xf3+Pu7s73N3dYTKZwPM82LY9mEc+HRARGPmkYnIkgTiKIvYcAk0UpEV5kYbYSZLw4xF93zMpcxwHnudhMplgMpnAcRxmwEO66qn214CIYFBiI+2TdFAAR3khDEPUdY04jo9yDok8IuOkhOn7PsbjMR4fH+H7Pp6ennB/f4+npydMp1PMZjMGhfb58pAZAoUSpvwiLxCbsd1ux5WE8g6Vb3J3Yqeu68L3ffi+j/l8jsVigdvb26MwuVQiPRkQOinxCHERNL60bRsAMJ1Ooes69vs9DMNAkiQwDIPlRNI66O7SvGU+n2M8HuP+/h6z2Qw/f/7EYrHAzc0NC88iGEPc5ssAkcERgSHdAwCr5VmWQVGOH4Ui8IjIkUfNZjOMRiPMZjP4vs9PDj08PGA8HnPeoFmMqKxdcgyhfNADDH451NOQuEP6Z9M0XCnSNOVHrUg9pypDi6PRpLilZ0rkofZQiT0BjMEdzgJkiLVSmaXRgchPaEu5gzQTEotN02SSRS+x2x16NvUMr7gcIPzlAFED/tn9fjQ7kZO03JO81aOcGSKDO19ktiv/LbblQ9uPjvfW9q3zXtLO8pCP7BIaxScu/vIe8uEZP/FOfpZd/4FIsisgkn0UMv97Pn+mXT1Esisgkl0BkewKiGRXQCS7AiLZvwBtCZqwAvXF1QAAAABJRU5ErkJggg==\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "#เลข 3 โดยเฉลี่ย\n", + "mean3 = stacked_threes.mean(0)\n", + "show_image(mean3);" ] - }, - "execution_count": 112, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from collections import Counter\n", - "Counter((preds>0.0).float().numpy()[:,0])" - ] - }, - { - "cell_type": "code", - "execution_count": 115, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "CiSVOFonpokB", - "outputId": "8a2b8106-f5f1-4b7f-dab9-4dae99c5bf88" - }, - "outputs": [ { - "data": { - "text/plain": [ - "0.5121006965637207" + "cell_type": "code", + "execution_count": 20, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 85 + }, + "id": "ambkeHzzpojz", + "outputId": "e21cad11-daa9-459a-ebd7-a5a7169567c4" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAI6klEQVR4nO1baVPiTBc9ZF/IhoOWOjUf5v//KgelNEZICCErvB+eutemjaNC8N04VanG7H1y99uOdrsdzniF8u9+gf80nAmRcCZEwpkQCWdCJGgfHP9fdkGjvp1nCZFwJkTCmRAJZ0IknAmRcCZEwpkQCWdCJJwJkfBRpHoQPlNjkc8ZjXoDxzf47HmHYhBCxMnR74/Gz0IkYDQa8d/y2Hf+ITiKEHmS2+2Wx91uxxv9TcfFY+L1BHGy9FtRFIxGo96RjsvXH4KDCBEnIk6aJt51HbbbLbquQ9d1aNuWR9pP58n3kUET1zQNqqrCMAyoqgrTNKGqKjRNg6IoexvhEGK+TIj44kQCEdA0DZqmQVmWaJoGm80GdV1jvV6jrmvkeY6qqrDZbNA0Deq6Rtu2TFSfJKmqCkVR4LouTNPExcUFXNdFFEVwHAe+78M0TViWxQSNRiOoqordbvdlUr5EiCwZ9KVJAoqiQNM0WK/XKMsSWZahKAokSYLNZoMsy1CWJRPVNA1fS6SKKgaACRmPxzBNE9PpFJ7n4devX/B9H5qmYbfbMRHb7RaKohxExpcI6VMPmgxNcLVaoSgKxHGMNE0xn8+RZRniOEae51gsFlitVlgulywpoiqJ6kQbffUgCOB5Hn7//o0wDJGmKS4vLwEAQRBA0/6ZCqnYyQnpI0ckhlSFJCHLMqRpymOe50iSBOv1GqvVCm3boq7rvfuJIHLatkVVVRiNRmiaBlEUQVEU5HkO13VZ0kjCjsWXVUYkous6NE2DqqpQliXyPEeapojjGMvlEnEcI8sy3N/fY7VaIUkS1HWNsiyZAEVR2DCS1xCfQXamrmvoug7TNFHXNS4vL2EYBvI8h23bLK3vGeeTEPI3kIskEdd1HbquwzAM+L4PVVUBgL+6fA55ETKyy+US6/UaWZZhvV7zMwDskSkSSb9Fd/1VfIqQjxgXyaAJmqYJ27aZhPF4jDAMoaoqu03HcfhcXdehqip7qiRJkGUZ7u7u8PT0xAabDCdBdrnHkPFpQshIyQQoisISsd1uYZomuq5DEARQFAV1XcO2bei6ziqg6zosy4Jt23BdF5ZlwbIslpDNZoOyLHl/nucoioLVgQik0bIsjk2+jRCRiD5CDMMAALiuC1VV0XUdTNOEoiioqgphGLKtoNjB8zy4rssTo3sSIZ7nYTabIc9zrNdrVFWFtm1hWRYcx4Ft27Btm4kjCRNV5tu8jKizAKDrOn89ALBtm41j13WoqgqapsE0Tbiuy5LhOA50XedYYrfb7UWmwD/qRt5IVVU4jgPXdTEejxEEAUuIaJiPwacJER+kKAoHQPTypNuqqrL6ULS42+1YVSzLYsmwLIuJJZUiIk3TZEKqquKYxHEcOI4Dz/M4WqUoVbQjJydEJIaCHjHxAv6RFABMhghR98muEJF0Pbnbuq6RZRkHcuRlDMPAeDyG7/sIw5CjV13X3+Qxh+JglQFeJYV0V9d1Jqxt2z2dFr0P6TtNgK6h2KYsS6RpisVigcViwUEYqVwQBIii6A0hx7rcLxMiehvRsJLEkOEkkogQ2i8TQRCDvDzP8fz8jMfHRzw/PyNNU9R1zaG77/sIggCu68K27T3SAey938mTO3oQPVj0OuRxSBrETJU2MVUX70PGl/Kh+XyOp6cnzGYzpGmKpmmgaRo8z4PneQjDkCWGpGMoHBypytICvNoSsh80igUdoL+EUBQF0jTF/f097u7uEMcx4jhG0zRQVRVhGGIymfBIrvZYFZFxVOjeV84Tv5ZMmLyfJKNtW2w2G86QHx4eMJ/PkaYpezFys7TJrnYoUg42qrItod8A9ryGDLmMQMlekiT48+cPZrMZ5vM5FosFmqZhb3J1dYXLy0tMp1OMx2OOP0TJG4KUoyVEtCVkYGkTiZOLS2RI67pm6Xh4eEAcx3h4eECe5+i6DoZhIIoihGGIi4sLNqhiZErvMgSOtiF9Ve/tdvsm/wH2SaGSY1EUeHl5wWw2w2w2w8vLC6uK53m4vb3Fz58/cX19jZubGwRBwEmhHIx9u9v9G+RIVm5NyG6RCktUR0mSBEmSYLFYoCxLqKoK27bx48cPXFxcYDKZYDKZwHEcmKb5xn70kfBtucxnH0xSIhdtKHCjuuv9/T3HHGVZQtM0RFGEIAhwfX2N29tb3NzcIIoi2LbNkXBfyn+sCg1iQ+S/33sZUWWoClYUBZbLJfI8R5Zl7GZ938d0OkUURZhMJvB9H7Zt73mX98g4BkdLSB8phPdUheqvaZri6ekJSZJwi4LynNvbW0ynU5YQMqaidPTZjW/Ldv8GedLv7SPVIemgEiGRQV5FzGajKILneRyIDW1EZQza7O7zLMCrVxGN6HK5xOPjI6vLdruF4zgIggC2bePq6gpXV1fchxErY3LkKz7/WJx8OURfy4J6Mnmec08HANdIKGchcvq8CtAfFB6LQSVEVg+5b0M9mcVigefnZywWC2w2G4xGI1aJyWSCKIrY1ZLdoJrr0KG6jJOtD/lbD2ez2aAoCpRlia7rOF/RdZ2Lz5TeExGniEr7cJL1IWJoTipSFAVWqxWyLMPLywvyPOe2AnkWUUJEdZELQKfE4CrTJx3Ua6mqipO5uq65qKxpGtsPalGIlfTvIgMYgJC+tSLUZyXpoJ7ver1mQ0pVNbHOats2wjCE7/uwLOvDmOMUGNyG9NkOMqo00nIHsTBMTScav8uIyhhsSdV7RpQWxtBGrQbTNN8siKEikOhqRYP6X6EyIvoW1IgrgyiUp8YUqYSqqiwdhmFwi2KIPstXMWhy99451AR3XZcnKa8CIBsitiffa1mI49A4SRwC7Lc7adLU6gReWw+apnGb0zAM3khy/lbzOAUpow++8KdWnvTZEHHJFdkSMqxt2+6pEEkRkUNumEj5qDJ2IDG9Fw1eMaNqmdinESdMC+xEQoDXxXWiIZXvcYp0/808hpAQPrknYhX391XPel+qJ4EbqiImPqZ355CE8EXvlADeO953ft/EB5aKgwj5v8P530MknAmRcCZEwpkQCWdCJJwJkfAv6ObhbeIGuNEAAAAASUVORK5CYII=\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "#เลข 7 โดยเฉลี่ย\n", + "mean7 = stacked_sevens.mean(0)\n", + "show_image(mean7);" ] - }, - "execution_count": 115, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#จะเห็นได้ว่ามันถูกเครื่องๆ (เพราะเราเดาสุ่มด้วย weights ที่สุ่มมา)\n", - "corrects.float().mean().item()" - ] - }, - { - "cell_type": "code", - "execution_count": 116, - "metadata": { - "id": "RwkizTBqpokC" - }, - "outputs": [], - "source": [ - "#ถ้าเราเปลี่ยน weights เล็กน้อย\n", - "with torch.no_grad():\n", - " weights *= 5.0001\n", - " bias+=10" - ] - }, - { - "cell_type": "code", - "execution_count": 118, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "d5Kv_vFrpokC", - "outputId": "11814646-8f8b-44ed-eb89-9a6deed931ef" - }, - "outputs": [ { - "data": { - "text/plain": [ - "0.5012907385826111" + "cell_type": "code", + "execution_count": 21, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 85 + }, + "id": "aKESuqYHpojz", + "outputId": "2ef7a17e-7e6e-4844-b49a-c094483c5679" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAJUElEQVR4nO2by08bVxuHn/HM2OP7BbsmYJNQKCQUCGkuyratmi7aRbfZt1L/ga66Sfbtv9FVF11E6aKLKG2RqBIlaUpSih0IoSF2YkrAl7l5xl1830w/GxJKGBP0yY+EkOacmfPOb95zed9zLLRaLXr8g+91G3DY6AnSQU+QDnqCdNATpANpl/L/5ylI2Oliz0M66AnSQU+QDnqCdNATpIOeIB3sNu16hm3bWJaFaZo0Gg0Mw0BV1RfW9/l8+Hw+UqkUiqIgiiI+X/e/34EJYhgG1WqVxcVFrl69ysLCAtevX39h/VgsRiwW48svv+Tdd98lGo0SCAS6bmfXBFFVlSdPniAIAqIoUq/XefbsGXfu3KFQKLC4uMj6+vpL769Wq9y6dYu+vj6GhoZIJBJdF0bYJR/yyivV3377jc8//xxJkkin0zx//pwnT57w/Plz1tfXsSwLy7JebJggIAgCwWCQQCDAxx9/zNtvv83FixfJ5/OvalZbEztd9NxDTNNEVVXK5TKlUgmAjY0N6vU6m5ubaJqGaZr/sUjY0SaXVquFqqrous6jR48IBAJsbm6SzWaRJKkrY4rngjQaDW7evMns7Cy1Wo1Go4GqqjieuNcMXavVwrIs7ty5w9LSEh9++CHZbJZYLNaVruOZIM7s8fjxY2ZnZykUCqiqSrPZxLZtt14qlSKfz5PJZHjjjTe2PadYLLKyssLW1lbbLNRsNjEMA13XMU1zz8L+WzwTRFVVbty4wezsLF999RWapu04RkxPT/PJJ59w9uxZJicnt5VfuXKFb7/9llu3brGysuJet23b7Y6GYRx+QVqtFoZhYJomhmFsE0OSJBRFIZfLMTk5STabbXN5VVWp1WqUy2XW1tbQNK3tflmWURSFvr4+otEooih6ZXq7nV4+zBFkp68XCoXIZrNMTExw+vRpAoEAfr/fLS+VSszPz3P79m3u3r2LYRht9weDQZLJJLlcjlQqteuA/Kp4JojP5yORSBCJRJAkyZ0yE4kEw8PDDA4OMjo6yvnz5/H7/QiCgGmaaJpGtVplfn6e69evs7y8jGma7rgjiiKSJHHy5EnGx8dJJpNdEwM8FEQURdLpNOl0GkmSCAaDZLNZTp06xYULFzhx4gQTExPIskwgEHAHyKWlJX755ReuXbvGlStX0HWdZrPpPtfv96MoCu+99x4XLlzYcSD2Es8EkSSJTCbD6dOn+fTTTxFFkUQiwdDQEDMzM6RSKWRZdvu+rutuN5mbm6NQKKDruusZ8XiccDjM5OQkg4ODnDlzhv7+fmRZ9srkHfF8pWoYBrVaDZ/PhyiKrkd0uvnq6io//fQT33//Pd988822cWd8fJzR0VE++ugjzp07x8jICPF4fK/mvIyDWamKooiiKAiC4IqyU5+vVqvcu3ePUqnUJkYmkyGdTvPBBx/wzjvvMD09TX9//4EEdtAlQUKh0K71KpUKc3NzLC0ttV0fHh7m5MmTXLx4kZmZGSRJ6toUuxOvNUHUarW2dRVd1914Z6fybnPoMma6rlOv19F1HcuyDlwQ8dKlSy8rf2nhflBVlXq9jm3bPHr0yPWGVqtFo9EAoFarYVkWzWbT/e+MSx5weaeLr00QQRBIJpM8ffqU+fl5bNum2WyiqiqVSoWVlRV+/fVXNzgUBIFWq0UwGPRq6j18goTDYSKRCOFwGFEUKZfLbeOGruuoqsrjx49ZXl5maWmJ/v5+r3KsOwpyYDnVToLBIMFgkJmZGaLRKLZtc/fuXQAsy6JWq7nBHkAymSQejzM0NEQkEkGWZSTJe/Nfm4c4+Hw+IpEImUyGY8eOceTIESKRCK1Wa9tMY5ompmny4MEDMpkMsiy3rX73yOHyEAfHUxRF4ejRo8zNzREOhwHcQdeyLDRNQ9M0vvvuOxRFYWBgAEmSCIVCbVHzfnntHuIgCAKKopDJZDhx4gTHjx9ncnKSTCaDoihomuZm0GzbJhaLUalUiEQi2LaNoih77UKH00Mc/H4/fr+fcDhMLpfj6NGjnDp1ilgsht/v56+//mJ9fR3btmm1Wty4cYM///yTaDTK1NQUyWQSRVH2bUfXtiH2i5NsevbsGZVKhZ9//plCocAPP/xAsVgkFosRDAaZmpoin89z6dKlvW5PHExw5xWOxwQCAY4cOYKiKAwNDbGwsECxWGRra4utrS2q1SqpVIovvvjCk3YPrSAOTrSczWaRZZlMJtPV9g69IM5WaDgcRhCEfxVJ74dDL4iz5bm2tsbq6qq7G9gtPBWkc4B+1WTw/+7ymaZJs9mkXC5z//59NjY2trXh5ZamZ4Lous7Tp09pNpvouk40GiWdTrsGO67vBHGw87Zms9nENE1WV1dZXV1lcXGRUqlEsVjk4cOHLC8vu3VFUeTMmTOMjIy4i7n94okgzpcslUrutkI2myUSibhBmLO8dl74RckfVVVRVZU//viDmzdv8vvvv7OwsEC5XGZra8ut5/P5kGWZfD7PyMiIZynGfQtiGAabm5sUCgW+/vprd082FArR19dHOBwmnU6TzWY5fvw4Dx484N69e+5SvBNHsFKpRKVSoVqt0mg02uoKgsDZs2d58803+eyzz3jrrbdIJBL7fRXAA0Fs26Zer7O2tsaPP/5IrVZD13W3PB6Pk8vlGB8fZ2Njg8XFRa5du0aj0aBer++pLafbybLMsWPHGB0dZWRkhP7+/v2+xj9t7Helats2hmHw8OFDLl++TLFY5Pbt2207b4qiEAqFiEaj1Go1NjY2sCyr7VTANsP+e2DG+QuFQgQCAd5//32Gh4c5f/48uVyOsbGxVx0/urNS9fl8KIpCLBYjn89jmiYLCwvu0tuyLOr1unuk6qUWCgKSJCFJEn6/3824OymCcDjM2NgY09PTbjfxOifiWSxTr9e5f/8+a2trzM3Nsby8zNWrVzFNs60LvYhIJEI0GuXcuXOMjY0xMTHB8PAw8XicUCjkzlTOGTO/34/P59vPSaLuxjKSJDEwMEAwGETTNCRJYnBwEFVV0TRt1+x5KpUilUoxOjrKzMwMU1NTDAwMEA6HD2yTCjz0EGd94RxsMU2TarXqlu2GMzUrikIgEHBThM551S6wo4cc2vD/AOj9Xubf0BOkg54gHfQE6aAnSAc9QTrYbWHWveN+h5Seh3TQE6SDniAd9ATpoCdIBz1BOvgb74E4FYpj4GsAAAAASUVORK5CYII=\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "#เลข 3 อันที่ 125\n", + "a_3 = stacked_threes[125]\n", + "show_image(a_3);" ] - }, - "execution_count": 118, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#ความแม่นยำก็เปลี่ยนนิดนึง\n", - "preds = linear1(train_x)\n", - "((preds>0.0).float() == train_y).float().mean().item()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "PTOZYvLRy-Vz" - }, - "source": [ - "#### Loss Function อย่างง่าย" - ] - }, - { - "cell_type": "code", - "execution_count": 120, - "metadata": { - "id": "w7wVT8TjpokC" - }, - "outputs": [], - "source": [ - "trgts = tensor([1,0,1])\n", - "prds = tensor([0.9, 0.4, 0.2])" - ] - }, - { - "cell_type": "code", - "execution_count": 121, - "metadata": { - "id": "9HgV5HQppokC" - }, - "outputs": [], - "source": [ - "#loss function ที่จะเป็นจะเป็น Predictions ของ class ที่ผิด\n", - "#เพราะงั้นยิ่งทาย class ที่ผิดแบบมั่นใจมาก Loss ก็จะยิ่งสูง\n", - "def mnist_loss(predictions, targets):\n", - " return torch.where(targets==1, 1-predictions, predictions).mean()" - ] - }, - { - "cell_type": "code", - "execution_count": 122, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "5xGnIzeHpokC", - "outputId": "f6f63235-c18f-4fb3-ce36-ecb7a1b332d0" - }, - "outputs": [ { - "data": { - "text/plain": [ - "tensor([0.1000, 0.4000, 0.8000])" + "cell_type": "markdown", + "metadata": { + "id": "k4Rm7EZjU-Gb" + }, + "source": [ + "เราจะเห็นได้ว่าหากเราคำนวน mean absolute error และ mean squared error ของแต่ละ pixel ระหว่าง \"เลข 3 อันที่ 125\" กับ \"เลข 3 โดยเฉลี่ย\" และ \"เลข 7 โดยเฉลี่ย\" เราจะเห็นกว่าค่าของ \"เลข 3 อันที่ 125\" กับ \"เลข 3 โดยเฉลี่ย\" มีค่าน้อยกว่า และในกรณีนี้ระบบ (ที่ไม่ใช่ ML) ของเราจะทำนายถูกว่า \"เลข 3 อันที่ 125\" คือเลข 3" ] - }, - "execution_count": 122, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#if target == 1, loss is how far it is from 1\n", - "#if target == 0, loss is how far it is from 0\n", - "torch.where(trgts==1, 1-prds, prds)" - ] - }, - { - "cell_type": "code", - "execution_count": 123, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "dhQH45dYpokC", - "outputId": "6eb43351-b5a6-4092-b7ac-ccc2ad266ce3" - }, - "outputs": [ { - "data": { - "text/plain": [ - "tensor(0.4333)" + "cell_type": "code", + "execution_count": 22, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "JbfdH2zWpoj0", + "outputId": "a2d7d370-8bae-45ab-fa02-e10e6ad40d2d" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(tensor(0.1259), tensor(0.2290))" + ] + }, + "metadata": {}, + "execution_count": 22 + } + ], + "source": [ + "#ความห่างระหว่า \"เลข 3 อันที่ 125\" กับ \"เลข 3 โดยเฉลี่ย\"\n", + "dist_3_abs = (a_3 - mean3).abs().mean()\n", + "dist_3_sqr = ((a_3 - mean3)**2).mean().sqrt()\n", + "dist_3_abs,dist_3_sqr" ] - }, - "execution_count": 123, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "mnist_loss(prds,trgts)" - ] - }, - { - "cell_type": "code", - "execution_count": 125, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "uiUjD7aJpokD", - "outputId": "63c11348-6649-4688-f672-032837dbd862" - }, - "outputs": [ { - "data": { - "text/plain": [ - "tensor(0.2333)" + "cell_type": "code", + "execution_count": 23, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "s5xmB8f1poj0", + "outputId": "fd56a041-cae9-4195-be48-44491f38be2e" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(tensor(0.1836), tensor(0.3390))" + ] + }, + "metadata": {}, + "execution_count": 23 + } + ], + "source": [ + "#ความห่างระหว่า \"เลข 3 อันที่ 125\" กับ \"เลข 7 โดยเฉลี่ย\"\n", + "dist_7_abs = (a_3 - mean7).abs().mean()\n", + "dist_7_sqr = ((a_3 - mean7)**2).mean().sqrt()\n", + "dist_7_abs,dist_7_sqr" ] - }, - "execution_count": 125, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#ถ้าทายแม่นขึ้น loss ก็ลดลง\n", - "mnist_loss(tensor([0.9, 0.4, 0.8]), trgts)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "dqsrmegupokD" - }, - "source": [ - "### Activation Function - Sigmoid" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "-0DXOc8w0YeU" - }, - "source": [ - "ปัญหาอย่างนึงของเราตอนนี้คือโมเดลเราให้ค่า `Predictions` เป็นจำนวนจริง แต่ `Loss Function` ของเราต้องการค่าระหว่าง 0 ถึง 1 เราจึงใช้ `Activation Function` ที่ชื่อ Sigmoid เพื่อเปลี่ยนให้ตัวเลขอยู่ระหว่าง 0 ถึง 1\n", - "\n", - "`Activation Function` นั้นมีมากมายหลายประเภท ที่นิยมใช้ เช่น [ReLU](https://en.wikipedia.org/wiki/Rectifier_(neural_networks))(x) - ฟังชั่นที่บอกว่าถ้าน้อยกว่า 0 ให้เป็น นอกนั้นเป็น x, [tanh](https://mathworld.wolfram.com/HyperbolicTangent.html)(x) - ฟังชั่นที่เปลี่ยนค่าจำนวนจริงให้อยู่ระหว่าง -1 และ 1 เป็นต้น อ่านเพิ่มเกี่ยวกับ [Activation Functions](https://en.wikipedia.org/wiki/Activation_function)" - ] - }, - { - "cell_type": "code", - "execution_count": 126, - "metadata": { - "id": "sQIUTk5ZpokD" - }, - "outputs": [], - "source": [ - "def sigmoid(x): return 1/(1+torch.exp(-x))" - ] - }, - { - "cell_type": "code", - "execution_count": 127, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 340 - }, - "id": "TY9tdedYpokD", - "outputId": "966a16d1-0eb5-42b7-d056-08c80a35909a" - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/usr/local/lib/python3.7/dist-packages/fastbook/__init__.py:74: UserWarning: Not providing a value for linspace's steps is deprecated and will throw a runtime error in a future release. This warning will appear only once per process. (Triggered internally at ../aten/src/ATen/native/RangeFactories.cpp:23.)\n", - " x = torch.linspace(min,max)\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEMCAYAAAA/Jfb8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU9b3G8c8XCCQkJGwh7DvIpoBEEBStVety61ZstSrutaLWrfXW6tVatbW112urtS63KIriWnGjaqtWxaXKGiDs+04Cgex7vvePCb0xJmaAJGdm8rxfr3npnPnN8BhnHk5+58zvmLsjIiKxpVXQAUREpPGp3EVEYpDKXUQkBqncRURikMpdRCQGqdxFRGKQyl1ijpndZWZrg86xn5nNMLP3GhhzqZlVNFcmiX0qd4kqZpZgZveY2RozKzazHDObZ2bX1xj238DRQWWsww3A94MOIS1Lm6ADiBygR4ETCBVmBpAMjAX67h/g7gVAQSDp6uDuuUFnkJZHe+4Sbc4Gfu/ur7n7BnfPcPcZ7n73/gF1TcuY2Y1mttXMiszsXTObamZuZr2rH7/UzCrM7AQzW1r9W8GHZtbTzI4zs0VmVmhm75lZr1qvfYmZLTezsuo/414za1Pj8a9My5hZq+rfPrLMrMDMXgQ6NdUPTFomlbtEmx3AqWbWOdwnmNn3CE3V/B4YDTwP/K6Ooa2AXwJXAscAvYAXgbuBadXbegP/U+O1/wN4EpgJjAJ+Clxb/Tr1+QlwM3ALcCSwoIHxIgfO3XXTLWpuhAp2E1AJLAGeILQ3bzXG3AWsrXH/U2Bmrdf5LeBA7+r7l1bfH1NjzC3V28bV2HYTsLvG/bnAS7Ve+wagGGhbfX8G8F6Nx7cCv671nFeAiqB/vrrFzk177hJV3P1TYBAwGXgaSCNUjG+YmdXztBHAv2pt+7yulweW1ri/s/qfS2pt62JmravvjwQ+rvU6HwHx1Tm/wsySCf1G8Fmthz6pJ7vIQVG5S9Rx9wp3/8zdH3D3swjtdX8XOO6bnhbGS1e5e2Xt57h7eR2vU99fJCIRQeUusWBF9T+71fP4cmBirW2NdapkJl//S+V4QtMy62oPdvc8YBswqdZDxzRSHhFAp0JKlDGzjwgdEJ0PZAODgd8A+4B/1vO0B4AXzexL4G1CxXpx9WOHekGD+4A3zexW4FVgDKE5/wfcvewb8txjZisJTRedCZx0iDlEvkJ77hJt3gYuBP4GrAKeAtYAx7j77rqe4O6vAv8J3EpoTv1C4FfVD5ccShh3/xtwOXAJsAx4EPhzjdevyx+Bh6rHLib0W8Xd3zBe5ICZu67EJC2Pmd0JXO/uXYPOItIUNC0jMc/M4gidf/43oJDQN1xvAR4JMpdIU9Keu8S86m+LvgWMAzoAG4BnCH3TVYt1SUxSuYuIxCAdUBURiUERMefetWtX79+/f9AxRESiyoIFC3a7e2pdj0VEuffv35/58+cHHUNEJKqY2ab6HtO0jIhIDAqr3M3sOjObb2alZjajgbE3mdlOM8szsyfNrF2jJBURkbCFu+e+HbiX0LrV9TKzUwh9C/BEoB8wkG/+pp6IiDSBsMrd3V9199eAPQ0MvQSY7u6Z7r4XuIfQin0iItKMGnvOfSSh61rulwGkmVmXRv5zRETkGzR2uScBNS8GvP/fO9QeaGZXVc/jz8/Ozm7kGCIiLVtjl3sBoavR77f/3/NrD3T3J9w93d3TU1PrPE1TREQOUmOf555J6ALEL1XfHw3scveG5upFRGKau5NTWMbOvBKy8krJyi9hV14pY/t2ZPKQxt/BDavcqxdeagO0BlqbWTyhi/nWXnTpGWCGmT1H6Ayb/yJ0cWARkZhWVlHFtn3FbN1bxNa9xWzbW8z2fcVs21fMjtwSduaVUFZR9bXnTfvWoODKnVBJ/7LG/YuAX5nZk4QuYTbC3Te7+ztmdj+hK+IkAH+t9TwRkahVXlnF5pwi1mcXsmF3ARt2F7FxdyGbc4rYkVtMVY11GFu3Mronx9OzYzxj+nSkR8d4uieHbt2S4+nWoR2pHdoRH9e6/j/wEETEqpDp6emu5QdEJFJUVjkbdhewcmc+q3cVsGZXPmuyCti0p5Dyyv/vzE7t4+jfNZF+ndvTt0sifTu3p0+nBHp3bk9ah3a0ad20iwCY2QJ3T6/rsYhYW0ZEJCgl5ZWs2pnP0m25ZG7PJXN7Hqt25lNaPYXSyqBfl0QGd0vi5BFpDE5NYmBqIgO7JpHSPi7g9PVTuYtIi+HubM4pYsGmvSzavI+MrftYsSPv33vjKQlxjOyZzNSj+zG8RzLDenRgUGpSk02dNCWVu4jErKoqZ8XOPL5Yn8OXG3KYv2kvuwtKAUhs25ojenfkyskDOaJXCqN6pdC7UwJmFnDqxqFyF5GYsnF3IZ+s3c2na3fz2bo95BaXA9C7UwKTh3RlXL9OpPfvxJBuHWjdKjaKvC4qdxGJaiXllXy+bg8frsriw9XZbNpTBEDPlHi+MyKNiYO6MGFgF3p1TAg4afNSuYtI1NlXVMY/lu/ivRW7+Hj1borLK4mPa8WkQV254tgBTB6SSv8u7WNmiuVgqNxFJCrsLSzjncyd/G3pDj5ft4eKKqdHSjznjuvNicO7cfTALlF54LOpqNxFJGIVl1Xy9+U7eWPxdj5anU1FldOvS3t+dNxAThvVncN7pbTovfNvonIXkYji7izcvJdXFmzlrYwd5JdW0D05nsuPHcCZo3sysmeyCj0MKncRiQi5ReX8deFWZn25mbVZBSTEteb0w3swZVwvjh7QhVYxfGZLU1C5i0iglm/PY8ZnG3h98XZKK6oY3acj9085gtOP6EFSO1XUwdJPTkSaXVWV896KXUz/ZANfbMghPq4V3zuyNxcd3ZeRPVOCjhcTVO4i0mxKKyp5bdE2Hv94PeuzC+nVMYHbTh/Geel9I3qdlmikcheRJldSXskLX27msY/WszOvhJE9k3noh2M5fVT3Jl85saVSuYtIkykpr+S5Lzbz2EfryM4vZfyAzvz++0dw7OCuOuOliancRaTRVVRW8cqCrfzx/TXsyC1h0qAuPPzDsRw9sEvQ0VoMlbuINBp3570VWdz39grWZxcypk9HHvj+aCYN7hp0tBZH5S4ijWLZtlzueWs5X2zIYWBqIk9MHcfJI9I0/RIQlbuIHJKcwjJ+/+4qXpi3mU7t23LPWSM5f3xf4nSgNFAqdxE5KFVVzqwvN/P7d1dRUFrBZZMGcOPJQ0iO1ymNkUDlLiIHbOXOPH7x6lIWbd7HxIFd+NVZIxma1iHoWFKDyl1EwlZSXslD76/hiY/Xk5wQx4PnjebsMb00rx6BVO4iEpZFm/dyyytLWJtVwLnjenP76cPplNg26FhSD5W7iHyj0opKHvzHGp74eB1pyfE8ffl4jh+aGnQsaYDKXUTqtXpXPje8sJgVO/I4L70Pt393uA6YRgmVu4h8jbvz9Gcbue/tlSS1a8NfLk7npBFpQceSA6ByF5Gv2FdUxs9eXsJ7K3ZxwmGp3H/uaFI7tAs6lhwglbuI/NuCTXu5/vlFZOWXcMd3R3D5Mf11JkyUUrmLCO7OU59u5Dd/W0GPjvG8cvUkRvfpGHQsOQQqd5EWrqisgl+8upTXF2/npOFpPPCD0aQk6KBptFO5i7Rgm/cUcdXM+azalc8tpxzGtOMH6ULUMSKslX3MrLOZzTazQjPbZGYX1DOunZk9Zma7zCzHzN40s16NG1lEGsPn6/Zw1iOfsCO3hBmXjefaEwar2GNIuMu2PQKUAWnAhcCjZjayjnE3ABOBI4CewF7g4UbIKSKNaNYXm5k6/Qu6JLXj9WuP0ZeSYlCD5W5micAU4A53L3D3T4A3gKl1DB8AvOvuu9y9BHgRqOsvAREJQGWVc89by7lt9lKOHdKVV6+ZRP+uiUHHkiYQzpz7UKDC3VfX2JYBHF/H2OnAH82sJ7CP0F7+24ecUkQOWXFZJTe+uIh3M3dx6aT+3PHdEbTWNEzMCqfck4C8WttygbrW91wDbAG2AZXAUuC6ul7UzK4CrgLo27dvmHFF5GDsKSjliqfnk7F1H3d+dwSXHzsg6EjSxMKZcy8AkmttSwby6xj7CNAO6AIkAq9Sz567uz/h7ununp6aqvk+kaayJaeIcx/7nJU783jsonEq9hYinHJfDbQxsyE1to0GMusYOwaY4e457l5K6GDqeDPT1XFFArBiRx5THv2MnMIynrtyAqeM7B50JGkmDZa7uxcS2gO/28wSzewY4CxgZh3D5wEXm1mKmcUB1wDb3X13Y4YWkYbN25jDDx7/nFZmvHz1RMb16xx0JGlG4Z4KeQ2QAGQBzwPT3D3TzCabWUGNcT8DSgjNvWcDpwPnNGJeEQnD3DXZTJ3+BalJ7Xhl2kRdAq8FCusbqu6eA5xdx/a5hA647r+/h9AZMiISkL9n7uS6WYsYmJrIzCsmaEXHFkrLD4jEkDcztnPji4sZ1SuFpy87io7tdRm8lkrlLhIjXl+8jZteXEx6v85MvzSdDrpiUoumcheJAa8t2sbNLy3mqP6defLSo0hsp492S6d3gEiU21/s4weEir19W32sReUuEtXmLNnBzS8tZsKALjx56VEktG0ddCSJEOGeCikiEebvmTu54YVFHNm3E9MvTVexy1eo3EWi0Eers7lu1iJG9krhqcs0FSNfp3IXiTLzN+bw45nzGdQtiWcuG6+zYqROKneRKLJ8ex6XzZhHz5QEZl4xnpT2Knapm8pdJEps2F3IxU9+SVK7Nsy8cgJdk/TNU6mfyl0kCmTllTB1+hdUuTPzign06pgQdCSJcCp3kQiXV1LOJU/NI6ewjBmXHcXgbkkNP0laPJW7SAQrrajk6pkLWLMrn8cuGscRvTsGHUmihM6fEolQVVXOz15ewmfr9vDgeaM5bqiuWCbh0567SIT63bsreTNjO7eeNoxzxvYOOo5EGZW7SAR69l+bePyj9Vx0dF9+fNzAoONIFFK5i0SYD1bu4s7Xl/HtYd2464yRmFnQkSQKqdxFIsjy7XlcN2sRI3om8/APx9KmtT6icnD0zhGJEFl5JVz59DxSEuKYfonWZJdDo3ePSAQoLqvkR8/MZ19xOS9fPZG05PigI0mUU7mLBCx0ymMGS7bl8vhF4xjZMyXoSBIDNC0jErCHPljDnKU7uPXUYXxnZPeg40iMULmLBOjtpTv4w3trmHJkb67SKY/SiFTuIgHJ3J7LzS9lMLZvR359ziid8iiNSuUuEoDdBaVc9cwCOraP4/Gp44iP0yXypHHpgKpIMyuvrOLa5xayu6CUV66eRLcOOjNGGp/KXaSZ/XrOCr7YkMOD543m8N46M0aahqZlRJrRy/O3MOOzjVxx7AAtBiZNSuUu0kyWbN3H7a8tY9KgLvzitGFBx5EYp3IXaQZ7Ckq5euYCUpPa8acLjtSaMdLkNOcu0sQqKqu4/oVF7C4s469XT6JzYtugI0kLENbug5l1NrPZZlZoZpvM7IJvGHukmX1sZgVmtsvMbmi8uCLR57//vppP1+7h3rNH6QCqNJtw99wfAcqANGAMMMfMMtw9s+YgM+sKvAPcBLwCtAV01EharHeW7eSxj9ZxwYS+/CC9T9BxpAVpcM/dzBKBKcAd7l7g7p8AbwBT6xh+M/Cuuz/n7qXunu/uKxo3skh0WJ9dwM9ezmB0n4788owRQceRFiacaZmhQIW7r66xLQMYWcfYo4EcM/vMzLLM7E0z69sYQUWiSVFZBdOeXUhca+PPFx5Juzb6Bqo0r3DKPQnIq7UtF+hQx9jewCXADUBfYAPwfF0vamZXmdl8M5ufnZ0dfmKRCOfu3D57Gauz8vnD+WPp1TEh6EjSAoVT7gVAcq1tyUB+HWOLgdnuPs/dS4BfAZPM7GtHkdz9CXdPd/f01NTUA80tErGe+2Izsxdt48YTh3L8UL23JRjhlPtqoI2ZDamxbTSQWcfYJYDXuO91jBGJWUu35nL3m8s5bmgqP/n24KDjSAvWYLm7eyHwKnC3mSWa2THAWcDMOoY/BZxjZmPMLA64A/jE3XMbM7RIJMotKueaWQvoktSWP5w3hlattISvBCfcr8ldAyQAWYTm0Ke5e6aZTTazgv2D3P0D4DZgTvXYwUC958SLxAp352evZLBjXwl/uuBIfVFJAhfWee7ungOcXcf2uYQOuNbc9ijwaKOkE4kSf5m7gX8s38Ud3x3BuH6dgo4jorVlRA7Vgk17+d07Kzl1ZHcuP6Z/0HFEAJW7yCHZW1jGT2YtpEfHeH537hG6VJ5EDC0cJnKQqqqcn76cwe6CMv46bRIpCXFBRxL5N+25ixyk/527ng9WZnH7fwzXgmAScVTuIgdhwaYc7n93Facf3p2LJ/YLOo7I16jcRQ5QaJ59Eb06JvDbKZpnl8ikOXeRA+Du/KzGPHtyvObZJTJpz13kAPxl7gbeX5nFbacP0zy7RDSVu0iYFm0Onc9+ysg0LpnUP+g4It9I5S4Shtyicq6btYjuKfHcf+5ozbNLxNOcu0gD3J2f/3UJu/JKePnqiTqfXaKC9txFGvDM55t4J3MnPz91GGP7at0YiQ4qd5FvsGxbLr+es4JvD+vGlZMHBB1HJGwqd5F65JeUc92shXRJassD39c8u0QXzbmL1MHduW32MrbsLeaFq46mk9ZnlyijPXeROrw4bwtvZmzn5pOHclT/zkHHETlgKneRWlbtzOeXb2Ry7OCuTDt+UNBxRA6Kyl2khqKyCq6dtZAO8XE8qOugShTTnLtIDXe+nsm67AKevWICqR3aBR1H5KBpz12k2l8XbOWVBVv5ybeHcMzgrkHHETkkKncRYG1WPv/12jLGD+jMDScOCTqOyCFTuUuLV1xWybXPLSKhbWseOn8srTXPLjFAc+7S4t31RiarduXz9OXj6Z4SH3QckUahPXdp0V5btI0X52/hmm8N4vihqUHHEWk0KndpsdZmFXDb7KUc1b8TN588NOg4Io1K5S4tUmiefSHxca156IdjadNaHwWJLZpzlxZp/zz7jMuOokdKQtBxRBqddlekxXl14VZenL+Fa08YxLcO6xZ0HJEmoXKXFmXNrnxunx06n/2mkzTPLrFL5S4tRmFpBdOeW0hiu9b8SfPsEuM05y4tgrtz++ylrK9eN6Zbss5nl9gW1q6LmXU2s9lmVmhmm8zsggbGtzWzFWa2tXFiihyaWV9u5rXF27nppKFM0rox0gKEu+f+CFAGpAFjgDlmluHumfWMvwXIBjocekSRQ7Nk6z5+9cZyjh+ayrUnDA46jkizaHDP3cwSgSnAHe5e4O6fAG8AU+sZPwC4CLivMYOKHIx9RWVMe3YhqR3aaX12aVHCmZYZClS4++oa2zKAkfWMfxi4DSg+xGwih6SqyrnxxcVk55fy5wuPpLOugyotSDjlngTk1dqWSx1TLmZ2DtDa3Wc39KJmdpWZzTez+dnZ2WGFFTkQD3+wlg9XZXPnGSMY3adj0HFEmlU45V4AJNfalgzk19xQPX1zP3B9OH+wuz/h7ununp6aqgWbpHF9uCqLP7y/mnPG9uLCCX2DjiPS7MI5oLoaaGNmQ9x9TfW20UDtg6lDgP7AXDMDaAukmNlO4Gh339goiUUasHlPETe8sJjD0jrwm3MOp/r9KNKiNFju7l5oZq8Cd5vZlYTOljkLmFRr6DKgT437k4A/AUcSOnNGpMkVl1Vy9bMLcHcenzqOhLatg44kEohwv6J3DZAAZAHPA9PcPdPMJptZAYC7V7j7zv03IAeoqr5f2STpRWpwd25/bSkrdubxx/PH0q9LYtCRRAIT1nnu7p4DnF3H9rmEDrjW9ZwPgd6HEk7kQDzz+SZeXbiNG08awgnDtCCYtGxaXENiwufr9nD3W8s5aXga139bF7gWUblL1Nu2r5hrZy2kf5f2PHjeaH1RSQSVu0S5kvJKfjxzPuUVVTxxcTod4uOCjiQSEbQqpEQtd+eWV5aQuT2Pv1yczqDUOg//iLRI2nOXqPXnD9fxZsZ2bjnlME4cnhZ0HJGIonKXqPT3zJ38/t1VnDWmJ9OOHxR0HJGIo3KXqLNyZx43vbiY0b1T+N2UI/QNVJE6qNwlqmTnl3LFjPkkxbfh8anpxMfpG6giddEBVYkaJeWVXDVzPjmFZbx89US6p+hSeSL1UblLVNh/Zsyizft47KJxjOqVEnQkkYimaRmJCg/+YzVvZmzn56cO49RR3YOOIxLxVO4S8V6at4WHPljLeel9uPr4gUHHEYkKKneJaHPXZHPb7KUcNzSVe88ZpTNjRMKkcpeItWJHHtOeXcjgbkk8csFY4lrr7SoSLn1aJCJt3VvEpU99SVK7Njx12VFaM0bkAKncJeLsLSzj4ie/pLiskmeuGE+PlISgI4lEHZ0KKRGluKySy5+ex9a9xTx7xQSGpnUIOpJIVNKeu0SMsooqrnluARlb9vHQ+WMZP6Bz0JFEopb23CUiVFY5P305g3+uyua+7x2uc9lFDpH23CVw7s6dry/jzYzt3HraMH44vm/QkUSinspdAuXu3P/uKp77YjNXHz+Iq7V8r0ijULlLoB56fy2PfriOCyb05eenHhZ0HJGYoXKXwDz+0ToefG81547rzb1n6dunIo1J5S6BeOrTDdz39krOGN2T3005glatVOwijUlny0ize/KTDdz91nJOHdmd//nBaFqr2EUancpdmtVf5q7n3jkrOHVkdx7WejEiTUafLGk2+4v9tFEqdpGmpj13aXLuzsMfrOV//rGa/zi8B384f4yKXaSJqdylSbk7v31nJY9/tJ4pR/bmd1MOp42KXaTJqdylyVRWOb98YxnP/mszU4/ux6/OHKmzYkSaicpdmkRpRSU3v5jBnKU7+PHxA7n11GE6j12kGYX1+7GZdTaz2WZWaGabzOyCesbdYmbLzCzfzDaY2S2NG1eiQUFpBZfPmMecpTu4/fTh/OK04Sp2kWYW7p77I0AZkAaMAeaYWYa7Z9YaZ8DFwBJgEPB3M9vi7i80VmCJbFl5JVz+9DxW7Mjnge+PZsq43kFHEmmRGtxzN7NEYApwh7sXuPsnwBvA1Npj3f1+d1/o7hXuvgp4HTimsUNLZFq9K59z/vwZ67ML+cvF6Sp2kQCFMy0zFKhw99U1tmUAI7/pSRb6PXwyUHvvXmLQp2t3M+XPn1FWWcVLP57ICcO6BR1JpEULp9yTgLxa23KBhq5/dlf16z9V14NmdpWZzTez+dnZ2WHEkEj13BebuOTJL+nRMZ7Xrj2GUb1Sgo4k0uKFM+deACTX2pYM5Nf3BDO7jtDc+2R3L61rjLs/ATwBkJ6e7mGllYhSUVnFPW8t5+nPN3H80FQevmAsyfFxQccSEcIr99VAGzMb4u5rqreNpp7pFjO7HLgVOM7dtzZOTIk0OYVlXP/8Ij5Zu5sfTR7AracN1wJgIhGkwXJ390IzexW428yuJHS2zFnApNpjzexC4DfACe6+vrHDSmRYujWXq59dQHZBKfefewQ/SO8TdCQRqSXc74FfAyQAWcDzwDR3zzSzyWZWUGPcvUAXYJ6ZFVTfHmvcyBKkl+ZvYcpjnwHwytUTVewiESqs89zdPQc4u47tcwkdcN1/f0DjRZNIUlRWwZ2vZ/LKgq0cO7grD/1wLJ0T2wYdS0TqoeUHpEGrduZz7ayFrMsu4PoTh3DDiUM0vy4S4VTuUi9359kvNvPrOctJahfHs1dM4JjBXYOOJSJhULlLnbLzS/n5X5fwwcosjhuayn9//wi6dYgPOpaIhEnlLl/zzrKd3D57KfmlFdx1xggunthfS/WKRBmVu/xbTmEZv3wjkzcztjOyZzLPnzeGoWkNfRFZRCKRyl1wd+Ys3cFdb2SSW1zOzScPZdq3BulSeCJRTOXewm3JKeLO15fxz1XZHN4rhZlXTGB4j9qrTYhItFG5t1ClFZVM/2QDD7+/FjO447sjuGRiP13fVCRGqNxboA9XZfGrN5ezYXchJ49I464zR9KrY0LQsUSkEancW5A1u/K57+2VfLAyi4FdE3n68vEcPzQ16Fgi0gRU7i1Adn4pf3hvNS/M20L7tq35xWnDuOyYAbRtoykYkVilco9huUXlPDF3HU9+spHyyiqmHt2P608cojVhRFoAlXsMyisp5+lPN/K/c9eTV1LBmaN7ctPJQxnQNTHoaCLSTFTuMWRfURlPfbqRJz/dQH5JBScN78bNJx/GiJ46tVGkpVG5x4Bt+4qZPncDL8zbTFFZJaeMTOMn3x6ia5mKtGAq9yi2eMs+nvp0A28t2YEBZ4zuyVXHDdSXkERE5R5tSsoreWfZTmZ8tpHFW/aR1K4Nl0zszxWTB+hcdRH5N5V7lFifXcDzX27mlQVb2VtUzoCuidx1xgjOTe9DUjv9bxSRr1IrRLC8knLmLNnBKwu2smDTXtq0Mk4ekcaFE/oxaVAXLcMrIvVSuUeYkvJKPlyVzRsZ23h/RRalFVUM7pbEracN43tje9EtWRfMEJGGqdwjQEl5JR+vzubtZTt5b8Uu8ksq6JrUlvOP6sPZY3sxpk9HzLSXLiLhU7kHJKewjH+uzOK9Fbv4eHU2hWWVpCTEccrI7pw5uieTBnXRCo0ictBU7s2ksspZti2XD1dl89HqLBZv2UeVQ1pyO84c04vTRnVn4qAuukCGiDQKlXsTcXfWZRfyr/V7+HTtbj5bt4fc4nLM4IheKVz37SGcNLwbo3qm6MCoiDQ6lXsjKa+sYsWOPOZv3Mv8TTl8uSGH3QVlAPRMiec7I9I4dkhXjh3clS5J7QJOKyKxTuV+ENydrXuLWbotl8Vb9rF4yz6Wbs2luLwSCJX55CGpTBjQmQkDu9C/S3sdEBWRZqVyb0BZRRXrsgtYuTOPFTvyWb49j2Xbc9lXVA5A29atGNEzmfOO6kN6/04c2bcTPfVNUREJmMq9Wkl5JRt2F7Iuu4C1WQWsySpg9c58NuwupKLKAWjbphVD05I4bVR3RvVKYVTPFIb3SNZFL0Qk4rSocs8tLmfr3iK25BSxaU8Rm3OK2LinkI27i9ieW4yHOhwz6NOpPUPTkjhpRBrDundgeI9kBnZN1OmJIhIVYqbcC0sryAMVYosAAAVeSURBVMovZUduMbvyStiZW8r2fcXsyC1m274Stu4tIr+k4ivP6dg+jv5dEhk/oDP9uyQyMDWRwd2SGNA1kfi41gH9l4iIHLqoLvd/rszi7reWk5VXQmFZ5dceT0mIo2fHBHqmxDO+fyd6d2pPr04J9O3cnj6d25OSEBdAahGRphdWuZtZZ2A68B1gN/ALd59VxzgDfgtcWb3pL8Ct7vsnPBpXx/ZxjOiRzLcOS6Vbh3i6dWhHj5R4ulff2reN6r+7REQOWrjt9whQBqQBY4A5Zpbh7pm1xl0FnA2MBhz4B7ABeKxx4n7V2L6deOTCTk3x0iIiUa3Bo4NmlghMAe5w9wJ3/wR4A5hax/BLgAfcfau7bwMeAC5txLwiIhKGcE79GApUuPvqGtsygJF1jB1Z/VhD40REpAmFU+5JQF6tbblAh3rG5tYal2R1fD3TzK4ys/lmNj87OzvcvCIiEoZwyr0AqH3F5WQgP4yxyUBBXQdU3f0Jd0939/TU1NRw84qISBjCKffVQBszG1Jj22ig9sFUqreNDmOciIg0oQbL3d0LgVeBu80s0cyOAc4CZtYx/BngZjPrZWY9gZ8CMxoxr4iIhCHc79JfAyQAWcDzwDR3zzSzyWZWUGPc48CbwFJgGTCnepuIiDSjsM5zd/ccQuev194+l9BB1P33HfjP6puIiATEmujLowcWwiwb2HSQT+9K6FuzkSZSc0HkZlOuA6NcByYWc/Vz9zrPSImIcj8UZjbf3dODzlFbpOaCyM2mXAdGuQ5MS8ul9WtFRGKQyl1EJAbFQrk/EXSAekRqLojcbMp1YJTrwLSoXFE/5y4iIl8XC3vuIiJSi8pdRCQGqdxFRGJQzJW7mQ0xsxIzezboLABm9qyZ7TCzPDNbbWZXNvysJs/Uzsymm9kmM8s3s8VmdlrQuQDM7LrqpaBLzWxGwFk6m9lsMyus/lldEGSe6kwR8/OpKcLfUxH3GaypqTor5sqd0CUB5wUdoob7gP7ungycCdxrZuMCztQG2AIcD6QA/wW8ZGb9A8y033bgXuDJoIPw1ctLXgg8amZBX3wmkn4+NUXyeyoSP4M1NUlnxVS5m9n5wD7g/aCz7Ofume5euv9u9W1QgJFw90J3v8vdN7p7lbu/Rehat4G/4d39VXd/DdgTZI4DvLxks4mUn09tEf6eirjP4H5N2VkxU+5mlgzcDdwcdJbazOzPZlYErAR2AH8LONJXmFkaocspau39/3cgl5eUWiLtPRWJn8Gm7qyYKXfgHmC6u28NOkht7n4NocsSTia0Nn7pNz+j+ZhZHPAc8LS7rww6TwQ5kMtLSg2R+J6K0M9gk3ZWVJS7mX1oZl7P7RMzGwOcBDwYSblqjnX3yupf7XsD0yIhl5m1InTRlTLguqbMdCC5IsSBXF5SqjX3e+pANOdnsCHN0VlhreceNHf/1jc9bmY3Av2BzdXX4k4CWpvZCHc/Mqhc9WhDE8/3hZOr+qLl0wkdLDzd3cubMlO4uSLIvy8v6e5rqrfpspHfIIj31EFq8s9gGL5FE3dWVOy5h+EJQv+zxlTfHiN0FahTggxlZt3M7HwzSzKz1mZ2CvBDIuOA76PAcOAMdy8OOsx+ZtbGzOKB1oTe7PFm1uw7IQd4eclmEyk/n3pE3Hsqgj+DTd9Z7h5zN+Au4NkIyJEKfEToaHgeocsP/igCcvUjdMZACaHph/23CyMg2138/xkN+293BZSlM/AaUAhsBi7Qzye63lOR+hms5/9ro3aWFg4TEYlBsTItIyIiNajcRURikMpdRCQGqdxFRGKQyl1EJAap3EVEYpDKXUQkBqncRURi0P8BDYVAdlrTNlwAAAAASUVORK5CYII=\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "plot_function(torch.sigmoid, title='Sigmoid', min=-4, max=4)" - ] - }, - { - "cell_type": "code", - "execution_count": 129, - "metadata": { - "id": "htbxxyxlpokD" - }, - "outputs": [], - "source": [ - "def mnist_loss(predictions, targets):\n", - " predictions = predictions.sigmoid() #ใส่ sigmoid ไป; pytorch tensor มี built-in function ให้แล้ว\n", - " return torch.where(targets==1, 1-predictions, predictions).mean()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "UUxkL3_2pokD" - }, - "source": [ - "### SGD and Mini-Batches" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "rxCnTrNRInAz" - }, - "source": [ - "หลายครั้งข้อมูลทั้งชุดของเราใหญ่เกินไปที่จะใส่เข้าไปใน memory ของเครื่อง เราจึงต้องใส่เข้าไปทีละ batch ด้วย `DataLoader`" - ] - }, - { - "cell_type": "code", - "execution_count": 132, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "2-twSiAspokD", - "outputId": "f22c96c8-2188-4b31-c97a-2eccb24f46c7" - }, - "outputs": [ - { - "data": { - "text/plain": [ - "[tensor([ 8, 9, 5, 11, 4]),\n", - " tensor([ 7, 13, 14, 2, 3]),\n", - " tensor([ 6, 12, 10, 1, 0])]" - ] - }, - "execution_count": 132, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#ทดลองใช้ Dataloader สุ่มตัวเลขจาก 0-14; batch ละ 5 ตัวอย่าง; ให้สุ่มด้วย\n", - "coll = range(15)\n", - "dl = DataLoader(coll, batch_size=5, shuffle=True)\n", - "list(dl)" - ] - }, - { - "cell_type": "code", - "execution_count": 133, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "7W43iIU_pokE", - "outputId": "8cedff83-1aa4-458d-b8e2-0372eb1520d5" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(#26) [(0, 'a'),(1, 'b'),(2, 'c'),(3, 'd'),(4, 'e'),(5, 'f'),(6, 'g'),(7, 'h'),(8, 'i'),(9, 'j')...]" + "cell_type": "code", + "execution_count": 24, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "lDlxHuO8Vazc", + "outputId": "11eda802-037f-4abb-8308-f5045f90d468" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(tensor(0.1836), tensor(0.3390))" + ] + }, + "metadata": {}, + "execution_count": 24 + } + ], + "source": [ + "#ใช้ function ของ pytorch คิดก็ได้\n", + "F.l1_loss(a_3.float(), mean7), F.mse_loss(a_3, mean7).sqrt()" ] - }, - "execution_count": 133, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#สมมุติเรามี DataSet　แบบนี้\n", - "ds = L(enumerate(string.ascii_lowercase))\n", - "ds" - ] - }, - { - "cell_type": "code", - "execution_count": 134, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "GOccHc_rpokE", - "outputId": "f9664233-48a4-4f7d-f5ef-5be94f04a9de" - }, - "outputs": [ - { - "data": { - "text/plain": [ - "[(tensor([19, 14, 0, 24, 20, 12]), ('t', 'o', 'a', 'y', 'u', 'm')),\n", - " (tensor([23, 8, 9, 3, 16, 6]), ('x', 'i', 'j', 'd', 'q', 'g')),\n", - " (tensor([ 4, 7, 1, 13, 2, 22]), ('e', 'h', 'b', 'n', 'c', 'w')),\n", - " (tensor([ 5, 17, 18, 10, 11, 15]), ('f', 'r', 's', 'k', 'l', 'p')),\n", - " (tensor([25, 21]), ('z', 'v'))]" - ] - }, - "execution_count": 134, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#ก็ใส่เข้าไปใน dataloader ได้เช่นกัน\n", - "dl = DataLoader(ds, batch_size=6, shuffle=True)\n", - "list(dl)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "tvWlUYU8pokE" - }, - "source": [ - "## Putting It All Together" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "fjUk_EOS0deu" - }, - "source": [ - "#### Initialize weights" - ] - }, - { - "cell_type": "code", - "execution_count": 186, - "metadata": { - "id": "lK9dlPP4pokE" - }, - "outputs": [], - "source": [ - "weights = init_params((28*28,1))\n", - "bias = init_params(1)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "L4ejZTDi0fds" - }, - "source": [ - "#### Initialize dataloader" - ] - }, - { - "cell_type": "code", - "execution_count": 169, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "XesYQeEUpokE", - "outputId": "fc6aab3d-26af-4611-ea8d-bcb308310b41" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(torch.Size([256, 784]), torch.Size([256, 1]))" + "cell_type": "markdown", + "metadata": { + "id": "4Lx6GAdSVgD2" + }, + "source": [ + "เรื่องน่าคิดถึงความแตกต่างระหว่าง mean squared error และ mean absolute error" ] - }, - "execution_count": 169, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#สร้าง dataloader สำหรับ train\n", - "dl = DataLoader(dset, batch_size=256, shuffle=True)\n", - "xb,yb = first(dl)\n", - "xb.shape,yb.shape" - ] - }, - { - "cell_type": "code", - "execution_count": 170, - "metadata": { - "id": "hlw0qizVpokE" - }, - "outputs": [], - "source": [ - "#เราจะไม่ shuffle validation set\n", - "valid_dl = DataLoader(valid_dset, batch_size=256, shuffle=False)" - ] - }, - { - "cell_type": "code", - "execution_count": 171, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "opSgyb70pokF", - "outputId": "eac98990-8410-46e5-8356-68c03c412118" - }, - "outputs": [ { - "data": { - "text/plain": [ - "torch.Size([4, 784])" + "cell_type": "code", + "execution_count": 25, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "IcAtO7IwDzUp", + "outputId": "a4f00685-f56e-4a9b-ca7b-051ba45ff758" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(1.1666666269302368, 1.1902379989624023)" + ] + }, + "metadata": {}, + "execution_count": 25 + } + ], + "source": [ + "a = torch.tensor([1, 2, 3]).float()\n", + "b = torch.tensor([2.,3.,4.5])\n", + "c = torch.tensor([2.,3.,40.])\n", + "\n", + "#mse และ mae ไม่ต่างกันเท่าไหร่สำหรับ a และ b\n", + "(a-b).abs().mean().item(), \\\n", + "((a-b)**2).mean().sqrt().item()," ] - }, - "execution_count": 171, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#(batch size, 28*28)\n", - "batch = train_x[:4]\n", - "batch.shape" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "d_OZXa2l0iwq" - }, - "source": [ - "#### Forward pass" - ] - }, - { - "cell_type": "code", - "execution_count": 196, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "IQ5eVzE5pokF", - "outputId": "91143405-c419-41be-ffa8-1f485be9b9ef" - }, - "outputs": [ { - "data": { - "text/plain": [ - "torch.Size([256, 1])" + "cell_type": "code", + "execution_count": 26, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "y5zmXGk2poj0", + "outputId": "5457536b-f88b-4077-f57f-8b3aa5021cdf" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(13.0, 21.3775577545166)" + ] + }, + "metadata": {}, + "execution_count": 26 + } + ], + "source": [ + "#mse และ mae ต่างกันเกือบเท่าตัวสำหรับ a และ c\n", + "(a-c).abs().mean().item(),\\\n", + "((a-c)**2).mean().sqrt().item()" ] - }, - "execution_count": 196, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "def linear1(xb): return xb@weights + bias\n", - "preds = linear1(xb)\n", - "preds.shape" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "P0dawUHM0kpf" - }, - "source": [ - "#### Calculate loss" - ] - }, - { - "cell_type": "code", - "execution_count": 197, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "_zK9HJE7pokF", - "outputId": "84527bdf-28f4-4a64-86a5-17ad6c3c08ec" - }, - "outputs": [ { - "data": { - "text/plain": [ - "tensor(0.2793, grad_fn=)" + "cell_type": "markdown", + "metadata": { + "id": "ErTcHuGkpoj4" + }, + "source": [ + "## Stochastic Gradient Descent (SGD)" ] - }, - "execution_count": 197, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "def mnist_loss(predictions, targets):\n", - " predictions = predictions.sigmoid()\n", - " return torch.where(targets==1, 1-predictions, predictions).mean()\n", - "loss = mnist_loss(preds, xb)\n", - "loss" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "tMIHxJcw0mW5" - }, - "source": [ - "#### Backward pass - get gradients and update weights" - ] - }, - { - "cell_type": "code", - "execution_count": 198, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "lMdSQrorpokF", - "outputId": "30ef9ce3-f371-434e-8598-f81970ce1cbf" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(torch.Size([784, 1]), tensor(0.0098), tensor([0.0854]))" + "cell_type": "markdown", + "metadata": { + "id": "XTAhc8bJWi6y" + }, + "source": [ + "หากยังจำได้จากบทที่ 1 เราเรียนรู้ว่าเราจะคำนวณ `Gradients` จาก `Loss` แล้วให้ Optimizer ทำหน้าที่ update `Weights` ในบทเรียนนี้เราจะมาเรียนรู้ขั้นตอนเหล่านี้กัน" ] - }, - "execution_count": 198, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "loss.backward()\n", - "weights.grad.shape,weights.grad.mean(), bias.grad" - ] - }, - { - "cell_type": "code", - "execution_count": 199, - "metadata": { - "id": "K4x5HLp4pokF" - }, - "outputs": [], - "source": [ - "#คำนวณ gradient ด้วย .backward()\n", - "def calc_grad(xb, yb, model):\n", - " preds = model(xb)\n", - " loss = mnist_loss(preds, yb)\n", - " loss.backward()" - ] - }, - { - "cell_type": "code", - "execution_count": 217, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "CZzqYNtgpokF", - "outputId": "433c868f-f1d4-449f-a127-7187ec620785" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(tensor(-3.8863e-05), tensor([-0.0003]))" + "cell_type": "markdown", + "metadata": { + "id": "V9fzeVckWhP9" + }, + "source": [ + "" ] - }, - "execution_count": 217, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "calc_grad(batch, train_y[:4], linear1)\n", - "weights.grad.mean(),bias.grad" - ] - }, - { - "cell_type": "code", - "execution_count": 218, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "R5yY7Az-pokF", - "outputId": "ffe44369-ddd7-42fe-f61d-37ed1101133f" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(tensor(-4.2396e-05), tensor([-0.0003]))" + "cell_type": "markdown", + "metadata": { + "id": "OISepsl-poj5" + }, + "source": [ + "### คำนวณ Gradients เพื่อทำ Backpropagation" ] - }, - "execution_count": 218, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "calc_grad(batch, train_y[:4], linear1)\n", - "weights.grad.mean(),bias.grad" - ] - }, - { - "cell_type": "code", - "execution_count": 219, - "metadata": { - "id": "R-Bzu26wpokG" - }, - "outputs": [], - "source": [ - "#เราจำเป็นต้อง reset gradients เพื่อไม่ให้มันบวกเพิ่มขึ้นเรือ่ยๆ\n", - "weights.grad.zero_();\n", - "bias.grad.zero_();" - ] - }, - { - "cell_type": "code", - "execution_count": 232, - "metadata": { - "id": "gKkqYbaqpokG" - }, - "outputs": [], - "source": [ - "#เทรน 1 epoch\n", - "def train_epoch(model, lr, params):\n", - " #โยนทุก batch ให้โมเดลที่ละ batch จนหมด\n", - " for xb,yb in dl:\n", - " #คำนวน loss และ gradients\n", - " calc_grad(xb, yb, model)\n", - " #เปลี่ยน weights ด้วย gradient * learning rate (lr)\n", - " for p in params:\n", - " p.data -= p.grad*lr\n", - " p.grad.zero_()" - ] - }, - { - "cell_type": "code", - "execution_count": 233, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "2XWh1J1ppokG", - "outputId": "5d18d3ee-0fa7-4394-f366-86a8f68351a2" - }, - "outputs": [ { - "data": { - "text/plain": [ - "tensor(0.3438)" + "cell_type": "markdown", + "metadata": { + "id": "_9WrHtiu0ScN" + }, + "source": [ + "หากคุณยังไม่เคยเรียนเกี่ยวกับ [partial derivative](https://en.wikipedia.org/wiki/Partial_derivative) และ [chain rule](https://www.khanacademy.org/math/ap-calculus-ab/ab-differentiation-2-new/ab-3-1a/a/chain-rule-review) ในชั้นเรียนมัธยมปลาย คุณอาจจะไม่จำเป็นต้องเข้าใจเนื้อหาส่วนนี้ทั้งหมดก็ได้ ใจความสำคัญคือเราสามารถปรับแต่ง `Weights` ได้ด้วย `Gradients` ที่ถูกคำนวณมาจาก `Loss` เพื่อให้ได้ `Loss` ที่น้อยที่สุดเท่าที่จะทำได้ใน iteration ต่อๆไป" ] - }, - "execution_count": 233, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#คิด accuracy\n", - "((preds>0.0).float() == yb).float().mean()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "4ZfrHoey0w2z" - }, - "source": [ - "### Calculate metric (accuracy in this case)" - ] - }, - { - "cell_type": "code", - "execution_count": 234, - "metadata": { - "id": "5QPI55L4pokG" - }, - "outputs": [], - "source": [ - "def batch_accuracy(xb, yb):\n", - " preds = xb.sigmoid()\n", - " correct = (preds>0.5) == yb\n", - " return correct.float().mean()" - ] - }, - { - "cell_type": "code", - "execution_count": 235, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "8UVCx6RApokG", - "outputId": "29839e53-a855-4967-b6f1-e4edc8d51571" - }, - "outputs": [ { - "data": { - "text/plain": [ - "tensor(0.)" + "cell_type": "markdown", + "metadata": { + "id": "tP7xMjbi6er4" + }, + "source": [ + "ตัวอย่างการทำ Backpropagation อย่างง่ายจาก [cs231n](https://cs231n.github.io/optimization-2/#backprop)\n", + "\n", + "Independent variables:\n", + "(เราอาจจะคิดว่า `x, y, z` เป็น `Inputs` หรือ `Weights` สำหรับโมเดลก็ได้)\n", + "\\begin{align}\n", + "x & = -2 \\\\\n", + "y & = 5 \\\\\n", + "z & = -4 \\\\\n", + "\\end{align}\n", + "\n", + "Dependent variables: \n", + "(`q` และ `f` คือฟังชั่นอะไรบางอย่าง เช่น `Loss Function` ของโมเดลก็ได้)\n", + "\n", + "ถ้าแทนค่า `x, y, z` เข้าไปในฟังชั่น `q, f` จะได้\n", + "\n", + "\\begin{align}\n", + "q & = x+y = -2+5 = 3\\\\\n", + "f & = q*z = 3*-4 = -12\n", + "\\end{align}\n", + "\n", + "การทำแบบนี้เปรียบเสมือนการเปลี่ยน `Inputs` เป็น `Predictions` ด้วย `Weights` เรียกว่า `Forward Pass`\n", + "\n", + "หลังจากนั้น เราสามารถคำนวณหา `Gradients` ซึ่งโดยทั่วไปแล้วหมายถึงค่าอัตราการเปลี่ยนแปลงของฟังชั่นท้ายสุด (ในที่นี้คือ `f`) เทียบกับตัวแปรแรกสุด (ในที่นี้คือ `x, y, z`) ได้แก่ $\\frac{df}{dx}$, $\\frac{df}{dy}$, $\\frac{df}{dz}$ เราสามารถหาสิ่งนี้ด้วยการหา [partial derivative](https://en.wikipedia.org/wiki/Partial_derivative) และ [chain rule](https://www.khanacademy.org/math/ap-calculus-ab/ab-differentiation-2-new/ab-3-1a/a/chain-rule-review) เรียกว่า `Backward Pass`\n", + "\n", + "\\begin{align}\n", + "\\frac{df}{dq} & = z = -4\\\\\n", + "\\frac{dq}{dx} & = 1\\\\\n", + "\\frac{df}{dx} & = \\frac{df}{dq} * \\frac{dq}{dx}\\\\\n", + "& = -4*1\\\\\n", + "& = -4\\\\\n", + "\\end{align}\n", + "\n" ] - }, - "execution_count": 235, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "batch_accuracy(linear1(batch), train_y[:4])" - ] - }, - { - "cell_type": "code", - "execution_count": 236, - "metadata": { - "id": "GbJQiOgIpokG" - }, - "outputs": [], - "source": [ - "#ทำการ validate ด้วย validation set\n", - "def validate_epoch(model):\n", - " accs = [batch_accuracy(model(xb), yb) for xb,yb in valid_dl]\n", - " return round(torch.stack(accs).mean().item(), 4)" - ] - }, - { - "cell_type": "code", - "execution_count": 237, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "txrLkdJQpokH", - "outputId": "83acd8ad-c9dd-4852-d105-fe35af5c3f3b" - }, - "outputs": [ { - "data": { - "text/plain": [ - "0.3447" + "cell_type": "code", + "execution_count": 27, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "VXdDIlJl6jYC", + "outputId": "9770d555-8e05-4b73-bf31-48d1d6b782c7" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(-4.0, -4.0)" + ] + }, + "metadata": {}, + "execution_count": 27 + } + ], + "source": [ + "# set some inputs\n", + "x = -2; y = 5; z = -4\n", + "\n", + "# perform the forward pass\n", + "q = x + y # q becomes 3\n", + "f = q * z # f becomes -12\n", + "\n", + "# perform the backward pass (backpropagation) in reverse order:\n", + "# first backprop through f = q * z\n", + "dfdz = q # df/dz = q, so gradient on z becomes 3\n", + "dfdq = z # df/dq = z, so gradient on q becomes -4\n", + "# now backprop through q = x + y\n", + "dfdx = 1.0 * dfdq # dq/dx = 1. And the multiplication here is the chain rule!\n", + "dfdy = 1.0 * dfdq # dq/dy = 1\n", + "\n", + "dfdx, dfdy" ] - }, - "execution_count": 237, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "validate_epoch(linear1)" - ] - }, - { - "cell_type": "code", - "execution_count": 238, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "HRIapIkfpokH", - "outputId": "f26dd6f4-c57f-481a-dcc1-b8199c3803ad" - }, - "outputs": [ { - "data": { - "text/plain": [ - "0.714" + "cell_type": "markdown", + "metadata": { + "id": "kt9JIcVZ3CuU" + }, + "source": [ + "Pytorch สามารถทำ `Backward Pass` ให้เราโดยอัตโนมัติด้วยฟังชั่น Autograd โดยที่เราไม่ต้องคิด partial derivative เอง ผลข้างเคียงอีกอย่างคือเราสามารถใช้ Pytorch ช่วยทำการบ้านวิชาแคลคูลัสเวลาเราหา derivative ที่ยากเกินไปไม่ออกได้อีกด้วย" ] - }, - "execution_count": 238, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#เทรนไป 1 epoch; accuracy เพิ่มเกือบเท่าตัว!\n", - "lr = 1.\n", - "\n", - "params = weights, bias\n", - "train_epoch(linear1, lr, params)\n", - "validate_epoch(linear1)" - ] - }, - { - "cell_type": "code", - "execution_count": 239, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "hvBUTwULpokH", - "outputId": "20f9b9df-8efb-4ead-baee-9e644f908d79" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "0.9068 0.9328 0.947 0.9569 0.9603 0.9618 0.9613 0.9627 0.9642 0.9677 0.9686 0.9696 0.9701 0.9706 0.972 0.972 0.973 0.973 0.9735 0.974 " - ] - } - ], - "source": [ - "#เทรนไป 20 epoch; ทายถูกเกือบหมด\n", - "for i in range(20):\n", - " train_epoch(linear1, lr, params)\n", - " print(validate_epoch(linear1), end=' ')" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "uVt65caepokH" - }, - "source": [ - "### Optimizer as a class" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "gyvY33oqMdZk" - }, - "source": [ - "หลายครั้งเรารวบรวมหน้าที่การ update weights มาเป็น class ชื่อ optimizer" - ] - }, - { - "cell_type": "code", - "execution_count": 240, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "ETZ0sdlXpokH", - "outputId": "5be434b1-64b6-4080-de5b-51216745e07b" - }, - "outputs": [ { - "data": { - "text/plain": [ - "Linear(in_features=784, out_features=1, bias=True)" + "cell_type": "code", + "execution_count": 28, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "SWg90ZHApoj6", + "outputId": "d7fbd5b9-1170-424e-e049-26bb3ca5a1a1" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "tensor([ 3., 4., 10.], requires_grad=True)" + ] + }, + "metadata": {}, + "execution_count": 28 + } + ], + "source": [ + "xt = torch.tensor([3.,4.,10.]).requires_grad_()\n", + "xt" ] - }, - "execution_count": 240, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "#nn.Linear ก็คือฟังชั่นที่ x*W.T + b เราเขียนเมื่อกี้เลย\n", - "linear_model = nn.Linear(28*28,1, bias=True)\n", - "linear_model" - ] - }, - { - "cell_type": "code", - "execution_count": 242, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "Op_9VxfipokH", - "outputId": "6096f6bb-ffd3-4f4b-8c6b-46412d73cd1a" - }, - "outputs": [ { - "data": { - "text/plain": [ - "(torch.Size([1, 784]), torch.Size([1]))" + "cell_type": "code", + "execution_count": 29, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "WpM1Q5nApoj6", + "outputId": "826205b5-d40f-4b04-986d-7c0beb6a0265" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "tensor(125., grad_fn=)" + ] + }, + "metadata": {}, + "execution_count": 29 + } + ], + "source": [ + "#ฟังชั่น f(x) = (x1^2 + x2^2 +...+xn^2)\n", + "def f(x): return (x**2).sum()\n", + "\n", + "yt = f(xt) #ใส่ 3, 4, 10 เข้าไปได้ 3^2+4^2+10^2 = 125\n", + "yt" ] - }, - "execution_count": 242, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "w,b = linear_model.parameters()\n", - "w.shape,b.shape" - ] - }, - { - "cell_type": "code", - "execution_count": 243, - "metadata": { - "id": "HDPzEqO-pokI" - }, - "outputs": [], - "source": [ - "#optimizer　แบบง่ายที่สุด\n", - "class BasicOptim:\n", - " def __init__(self,params,lr): \n", - " self.params,self.lr = list(params),lr\n", - "\n", - " #step ทำการ update weights\n", - " def step(self, *args, **kwargs):\n", - " for p in self.params: p.data -= p.grad.data * self.lr\n", - "\n", - " #zero grad เพื่อทำการรีเซต gradients\n", - " def zero_grad(self, *args, **kwargs):\n", - " for p in self.params: p.grad = None" - ] - }, - { - "cell_type": "code", - "execution_count": 244, - "metadata": { - "id": "UIAhC389pokI" - }, - "outputs": [], - "source": [ - "opt = BasicOptim(linear_model.parameters(), lr)" - ] - }, - { - "cell_type": "code", - "execution_count": 245, - "metadata": { - "id": "zNokDJCupokI" - }, - "outputs": [], - "source": [ - "def train_epoch(model):\n", - " for xb,yb in dl:\n", - " calc_grad(xb, yb, model)\n", - " #เปลี่ยนโค้ดเป็น optimizer แทน\n", - " opt.step()\n", - " opt.zero_grad()" - ] - }, - { - "cell_type": "code", - "execution_count": 246, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "pUr7KQecpokI", - "outputId": "0515d9ee-847a-497e-8f13-20fe853d9548" - }, - "outputs": [ { - "data": { - "text/plain": [ - "0.3604" + "cell_type": "code", + "execution_count": 30, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "POJB2ewmpoj6", + "outputId": "83cd31b2-2071-4a9e-af97-f3b301b2c807" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "tensor([ 6., 8., 20.])" + ] + }, + "metadata": {}, + "execution_count": 30 + } + ], + "source": [ + "#ถ้าคิดด้วยมือ df(x)/dx = 2x \n", + "#สำหรับ x1=3, x2=4, x3=10 ก็จะเป็น \n", + "#df(x1)/dx1 = 6, df(x2)/dx2 = 8, df(x3)/dx3 = 20\n", + "\n", + "#ใช้ autograd หา df(x1)/dx1, df(x2)/dx2, df(x3)/dx3\n", + "yt.backward()\n", + "xt.grad" ] - }, - "execution_count": 246, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "validate_epoch(linear_model)" - ] - }, - { - "cell_type": "code", - "execution_count": 247, - "metadata": { - "id": "wQptVRJfpokI" - }, - "outputs": [], - "source": [ - "def train_model(model, epochs):\n", - " for i in range(epochs):\n", - " train_epoch(model)\n", - " print(validate_epoch(model), end=' ')" - ] - }, - { - "cell_type": "code", - "execution_count": 248, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "kRbgjPURpokI", - "outputId": "e95b776d-8a7c-46e5-b56e-b596a5237fce" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "0.9702 0.9745 0.9765 0.9785 0.9794 0.9789 0.9789 0.9799 0.9794 0.9799 0.9809 0.9809 0.9814 0.9819 0.9819 0.9819 0.9824 0.9819 0.9829 0.9829 " - ] - } - ], - "source": [ - "#ได้ผลดีเหมือนเดิม\n", - "train_model(linear_model, 20)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "rBjvWLYwNII8" - }, - "source": [ - "## ทำงานง่ายขึ้นด้วย PyTorch" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "60qaqoMbNMA4" - }, - "source": [ - "Pytorch ได้ทำการเปลี่ยนฟังชั่นและ class ต่างๆที่เราใช้งานเป็นฟังชั่นพื้นฐานของ package ให้เราเรียกใช้ได้" - ] - }, - { - "cell_type": "code", - "execution_count": 249, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "DY1kWv7lpokI", - "outputId": "1da51068-8204-48d0-b3cb-10136c208ec5" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "0.9696 0.9741 0.976 0.977 0.977 0.978 0.9775 0.9789 0.9794 0.9799 0.9804 0.9794 0.9794 0.9804 0.9804 0.9799 0.9799 0.9814 0.9809 0.9804 " - ] - } - ], - "source": [ - "linear_model = nn.Linear(28*28,1,bias=False) #เหมือนกับ init_weights และการคูณ matrix ที่เราเพิ่งทำไป\n", - "opt = SGD(linear_model.parameters(), lr) #เหมือนกับ BasicOptim ที่เราเพิ่งทำไป\n", - "\n", - "#เทรนโมเดลได้ผลดีเหมือนเดิม\n", - "train_model(linear_model, 20)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "6ctjZxC3Nxi8" - }, - "source": [ - "## ใช้ fastai ให้สะดวกยิ่งขึ้น" - ] - }, - { - "cell_type": "code", - "execution_count": 250, - "metadata": { - "id": "jtzVcFkxpokJ" - }, - "outputs": [], - "source": [ - "dls = DataLoaders(dl, valid_dl)\n", - "#รวมทุกอย่างเข้าด้วยกันด้วย class Learner\n", - "learn = Learner(dls, nn.Linear(28*28,1), opt_func=SGD,\n", - " loss_func=mnist_loss, metrics=batch_accuracy)" - ] - }, - { - "cell_type": "code", - "execution_count": 251, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 676 - }, - "id": "0WcORvumpokJ", - "outputId": "e0988105-59a7-4c16-9d8a-466003017eb5" - }, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n" + }, + { + "cell_type": "markdown", + "metadata": { + "id": "mHHqwvz2poj_" + }, + "source": [ + "## สร้าง Loss Function สำหรับจำแนกรูปเลข 3 และเลข 7" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "U0EQ6WD4zEf0" + }, + "source": [ + "#### สร้าง X และ y" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "t2aoyhac4ZrI" + }, + "source": [ + "จัดการ `Inputs` คือรูปตัวเลข 28x28 pixels และ `Labels` คือ `1 ถ้าเป็นเลข 3` และ `0 ถ้าเป็นเลข 7`" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": { + "id": "-4LTBmPcpoj_" + }, + "outputs": [], + "source": [ + "train_x = torch.cat([stacked_threes, stacked_sevens]).view(-1, 28*28)" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Q5bU0JpipokA", + "outputId": "798e931e-2e7b-41bf-e94c-231da4a25c83" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(torch.Size([12396, 784]), torch.Size([12396, 1]))" + ] + }, + "metadata": {}, + "execution_count": 32 + } ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
epochtrain_lossvalid_lossbatch_accuracytime
00.0572380.0413640.97055900:00
10.0403980.0342190.97546600:00
20.0318900.0315480.97595700:00
30.0277810.0294280.97742900:00
40.0254070.0281660.97792000:00
50.0235450.0267280.97939200:00
60.0222740.0261640.97841000:00
70.0212850.0253030.97939200:00
80.0204780.0247790.97890100:00
90.0197470.0245610.98037300:00
100.0197390.0242870.98037300:00
110.0191990.0233610.97988200:00
120.0186020.0228020.98086400:00
130.0180490.0226690.98086400:00
140.0177650.0224540.98184500:00
150.0174720.0220460.98135400:00
160.0177030.0223970.98135400:00
170.0173600.0217730.98233600:00
180.0169900.0215650.98233600:00
190.0166550.0213360.98184500:00
" + "source": [ + "train_y = tensor([1]*len(threes) + [0]*len(sevens)).unsqueeze(1)\n", + "train_x.shape,train_y.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "rkavLIe1IcmD", + "outputId": "c7cabbb3-e47f-4403-b5d7-b09f38036698" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "[(1, 4), (2, 5), (3, 6)]" + ] + }, + "metadata": {}, + "execution_count": 33 + } ], - "text/plain": [ - "" + "source": [ + "#เราสามารถนำ iterables สองอันมาต่อกันแบบนี้ได้ด้วย zip\n", + "a = [1,2,3]\n", + "b = [4,5,6]\n", + "list(zip(a,b))" ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "learn.fit(20, lr=lr)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "H_UoKZ24pokJ" - }, - "source": [ - "## สร้าง Architecture ที่ Deep ขึ้น" - ] - }, - { - "cell_type": "code", - "execution_count": 252, - "metadata": { - "id": "4s1EGNszpokK" - }, - "outputs": [], - "source": [ - "#แทนที่จะทำ matrix multipilcation + bias ครั้งเดียว เราใส่ ReLU activation ไปเพิ่ม\n", - "def simple_net(xb): \n", - " res = xb@w1 + b1\n", - " res = res.max(tensor(0.0)) #นี่คือ relu\n", - " res = res@w2 + b2 #แล้วทำ matrix multiplication + bias อีกรอบ\n", - " return res" - ] - }, - { - "cell_type": "code", - "execution_count": 253, - "metadata": { - "id": "bKDW0LWlpokK" - }, - "outputs": [], - "source": [ - "w1 = init_params((28*28,30))\n", - "b1 = init_params(30)\n", - "w2 = init_params((30,1))\n", - "b2 = init_params(1)" - ] - }, - { - "cell_type": "code", - "execution_count": 254, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 268 - }, - "id": "Fza6OpHHpokK", - "outputId": "55ede7e1-b109-4bd7-ba79-bdb62f172b02" - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD7CAYAAABt0P8jAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU5dn/8c8lW9jXgGvYBBGQNe5r1RaXqrSolcXlqRUF0aqt1VatuNStrXUpD5anWlQQFwTXutQqj4pbQ9gMQlBkEQRC2JIAAZLr98dMfs84JuQEZsnMfN+v17yYuc99zlznMHPlzD1nrtvcHRERyRz7JTsAERFJLCV+EZEMo8QvIpJhlPhFRDKMEr+ISIZpmOwAgujQoYN36dIl2WGIiKSUOXPmbHD37Oj2lEj8Xbp0IS8vL9lhiIikFDNbUV27hnpERDKMEr+ISIZR4hcRyTBK/CIiGUaJX0Qkw9Sa+M2siZk9bmYrzKzEzOaZ2Zl76H+9ma01s61m9oSZNYlY1sXM3jOzbWa22MxOj9WOiIhIMEHO+BsCq4CTgdbArcDzZtYluqOZDQFuBk4DOgPdgDsiukwD5gLtgVuA6Wb2vWtMRUQkfmpN/O5e5u7j3X25u1e6+2vA18DgarpfCjzu7gXuvgm4C7gMwMx6AoOA2919u7u/CCwEhsVoX0RE0sZXRaX86a0l7K6ojPm26zzGb2adgJ5AQTWL+wDzIx7PBzqZWfvwsmXuXhK1vE8NzzPazPLMLK+oqKiuYYqIpKxtO3czZsocnvlsJcVlO2O+/TolfjNrBEwFnnT3xdV0aQFsiXhcdb9lNcuqlres7rncfZK757p7bna2RoNEJDO4O7fM/Jyl60t5+KIBdGqVFfPnCJz4zWw/4GlgJzCuhm6lQKuIx1X3S6pZVrW8BBERAWDqpyuZOXc1153WkxN7xOekN1DiNzMDHgc6AcPcfVcNXQuA/hGP+wPr3L04vKybmbWMWl7dkJGISMZZ8M1m7nx1ESf3zOaaUw+N2/MEPeOfCBwOnOPu2/fQ7yngcjPrbWZtCF0BNBnA3QuBecDtZpZlZj8B+gEv7m3wIiLpYlPZTsZMySe7ZRMe+tkA9tvP4vZcQa7j7wxcCQwA1ppZafg20sxywvdzANz9TeAB4D1gJbACuD1icxcBucAm4D7gfHfXN7ciktEqK53rn5/H+pIdTBg5iLbNG8f1+Woty+zuK4A9/elpEdX/QeDBGra1HDgleHgiIulvwntfMmtJEXed14cBh7SJ+/OpZIOISBJ9uHQDD75TyNABBzLqmM4JeU4lfhGRJPl2y3aufXYuPTq24J6fHkHoOpr4U+IXEUmCnbsruXpqPuW7Kpg4ajDNGiduQsSUmHpRRCTd3PvGF+Sv3MyEEYPont2i9hViSGf8IiIJ9tqCNfxj9nJ+fnxXzu53QMKfX4lfRCSBvlxfyk3TFzC4c1t+e1avpMSgxC8ikiBl5aHia1mNGjBhxCAaNUhOCtYYv4hIArg7v5u5kC+LSply+dHs3zr2xdeC0hm/iEgCTPlkBS/PW8MNp/fk+EM7JDUWJX4RkTibt2ozd762iB8cls3VP4hf8bWglPhFROJoU9lOrp6aT8eWWfwlzsXXgtIYv4hInFRWOtc9N4+iknKmjzmWNs3iW3wtKJ3xi4jEyaPvfsn/FhZx+7m96Xdw/IuvBaXELyISB+8XFvHQvwv56cCDGHFUTrLD+Q4lfhGRGFuzeTu/fHYuPTu25A8/SVzxtaCCTr04zszyzKzczCbvod9jERO1lIb7l0Qsn2VmOyKWL4nBPoiI1Bs7d1cydmo+uyqciaMG0bRxg2SH9D1Bv9xdA9wNDAGa1tTJ3a8Crqp6HP4jURnVbZy7/71uYYqIpIZ7/vkF81ZtZuLIQXRLcPG1oAIlfnefAWBmucDBQdYxs+bAMODHex2diEgKeWX+GiZ/tJxfnNCVM49IfPG1oOI5xj8MKALej2q/18w2mNlsMzulppXNbHR4eCmvqEjT8opI/bZ0XQk3v7iA3M5tuenM5BRfCyqeif9S4Cl394i2m4BuwEHAJOBVM+te3cruPsndc909Nzs7O45hiojsm7Ly3YyZmk+zxg2YMDJ5xdeCikt0ZpZDaFL1pyLb3f1Tdy9x93J3fxKYDZwVjxhERBLB3bl5xkKWFZXyyPCBdGqVvOJrQcXrz9LFwGx3X1ZLPwfq13VOIiJ18NTHK3h1/hp+9aPDOK57couvBRX0cs6GZpYFNAAamFmWme3pi+FLgMlR22hjZkOq1jWzkcBJwJt7GbuISFLlr9zE3a8v4rReHRlzcrWj1vVS0DP+W4HtwM3AqPD9W80sJ3w9/v//WZqZHUvoyp8XorbRiNAloUXABuAaYKi7F+7bLoiIJF5xaTlXT81n/9ZZPHhh/Si+FlTQyznHA+NrWPydC1Xd/WOgeTXbKAKOrFt4IiL1T0W4+Fpx2U5mjDmO1s0aJTukOqnfXz2LiNRDD/97KR8s3cAd5/ah70Gtkx1OnSnxi4jUwawl63n03aUMG3QwFx15SLLD2StK/CIiAa3evJ3rnpvHYZ1acvfQvvWu+FpQSvwiIgGU765g7NR8KiqciaMG18via0FpBi4RkQD+8PoXzF+1mcdGDaJrh+9dv5JSdMYvIlKLl+et5qmPV3DFiV05o2/9Lb4WlBK/iMgeFK4r4eYXF3Jkl7b85oz6XXwtKCV+EZEalJbv5qopc2jepCF/HVH/i68FlR57ISISY+7OTS8uYPmGMh4ZPiAliq8FpcQvIlKNyR8t5/UF33LjkF4pU3wtKCV+EZEoc1Zs4g+vf8Hph3fiqpO7JTucmFPiFxGJUFxazrhn8jmwTVP+fGH/lP2R1p7oOn4RkbCKSufaZ+f+X/G1pqlVfC0onfGLiIQ99E4hs78s5q7zUrP4WlBK/CIiwHuL1/Pou19yweCD+dmRObWvkMKCzsA1zszyzKzczCbvod9lZlYRnpyl6nZKxPIuZvaemW0zs8Vmdvq+74KIyL5ZtXEb1z03j8MPaMVdQ/smO5y4CzrGv4bQ7FlDgKa19P3Y3U+oYdk04GNCE6yfBUw3sx7hSVpERBKufHcFVz+TT2WlM3HkILIapW7xtaACnfG7+wx3fwko3tsnMrOewCDgdnff7u4vAguBYXu7TRGRfXXnq4tY8M0W/nRhf7qkePG1oOIxxj/QzDaYWaGZ3RYxKXsfYJm7l0T0nR9u/x4zGx0eXsorKtIHAhGJvRn53zD105VceVI3hvTZP9nhJEysE//7QF+gI6Ez+eHAjeFlLYAtUf23AC2r25C7T3L3XHfPzc7OjnGYIpLpFq/dyu9mLuSoru24cchhyQ4noWKa+N19mbt/7e6V7r4QuBM4P7y4FGgVtUoroAQRkQQq2bGLMVPyaZnViL+OGEjDNCm+FlS899aBqp+9FQDdzCzyDL9/uF1EJCHcnd9MX8DKjdv46/CBdGyZPsXXggp6OWdDM8sCGgANzCwrYuw+st+ZZtYpfL8XcBvwMoC7FwLzgNvD6/8E6Ae8GJtdERGp3eMffs0bn6/lN0MO4+hu7ZMdTlIEPeO/FdgO3AyMCt+/1cxywtfqV/3a4TRggZmVAf8EZgD3RGznIiAX2ATcB5yvSzlFJFHylm/kvjcW86PenRh9UvoVXwvK3D3ZMdQqNzfX8/Lykh2GiKSwDaXlnP3IB2Q1asAr405I2zo8kcxsjrvnRrerSJuIpL2KSufaaXPZvG0XM8celRFJf0+U+EUk7T34ryV89FUxD5zfj94HRl9cmHky6xomEck47y5ex4T3vuJnuYdwYe4hyQ6nXlDiF5G0tWrjNq57dh69D2jFHedVWyQgIynxi0ha2rGrgrFT83HgsVGDM6L4WlAa4xeRtHTHq4tYuHoL/3NJLjntmyU7nHpFZ/wiknZenPMN0z5byVUnd+eHvTslO5x6R4lfRNLK4rVbueWlhRzTrR2//lHPZIdTLynxi0ja2BouvtYqqxGPDh+UccXXgtIYv4ikBXfnNy+Eiq9Nu+IYsls2SXZI9Zb+HIpIWvj7B1/zZsFabj6jF0d1bZfscOo1JX4RSXmffb2R+95czBl99ucXJ3ZNdjj1nhK/iKS09SU7GPdMPjntmvHHC/phZrWvlOGU+EUkZe2uqOTaaXPZumMXE0cNomVWZhdfCyroRCzjwhOfl5vZ5D30u9TM5pjZVjP7xsweiJywxcxmmdmOcA3/UjNbEoN9EJEM9ed/FfLJso3cPfQIeu2v4mtBBT3jXwPcDTxRS79mwHVAB+BoQhOz/Dqqzzh3bxG+ZdYMxyISM/9atI6Js75i+FGHcP7gg5MdTkoJdDmnu88AMLNcoMYj7O4TIx6uNrOpwA/2KUIRkSgri7dxw/Pz6HtQK24/R8XX6ireY/wn8f3J1O81sw1mNtvMTqlpRTMbHR5eyisq0uyMIhKyY1cFY6bOwYCJI1V8bW/ELfGb2c8Jza/7p4jmm4BuwEHAJOBVM+te3fruPsndc909Nzs7O15hikiKGf9KAQVrtvLQRQM4pJ2Kr+2NuCR+MxsK3Auc6e4bqtrd/VN3L3H3cnd/EpgNnBWPGEQk/Tyft4pn/7OKq3/QnVN7qfja3op5yQYzOwP4H+Bsd19YS3cHdNGtiNSqYM0Wbnvpc47r3p4bfqjrQvZF0Ms5G5pZFtAAaGBmWZGXaUb0OxWYCgxz98+ilrUxsyFV65rZSELfAby577shIulsy/ZdjJ2aT5tmjXhk+EAa7KfzxX0RdKjnVmA7cDMwKnz/VjPLCV+PnxPudxvQGvhnxLX6b4SXNSJ0SWgRsAG4Bhjq7oUx2hcRSUPuzo0vzGf1pu1MGDGIDi1UfG1fBb2cczwwvobFLSL61XjpprsXAUfWITYRESa9v4y3F63j1rMPJ7eLiq/Fgko2iEi99emyYh54awlnHbE/l5+g4muxosQvIvXS+q07GDdtLp3bNeP+YSq+FkuaiEVE6p3dFZVcM20upTt2M+Xyo1V8LcaU+EWk3vnj20v49OuN/OVn/Tls/5bJDiftaKhHROqVtwvW8rf/XcbIo3P4yUAVX4sHJX4RqTdWFJfxqxfm0+/g1vz+nN7JDidtKfGLSL2wY1cFY6bks58ZE0YMoklDFV+LF43xi0i98PuXP2fRt1v5x2VHqvhanOmMX0SS7rn/rOT5vG+45tRD+UGvjskOJ+0p8YtIUn2+egu3vVzACYd24LrTeyY7nIygxC8iSVNVfK1ds8Y8fNEAFV9LEI3xi0hSVFY6v3p+Pms2b+e5K4+hvYqvJYzO+EUkKf72/jLe+WIdvzvrcAZ3VvG1RFLiF5GE+/irYv741mLO7ncA/3V8l2SHk3GU+EUkodZv3cE10+bSpUNzFV9LkqAzcI0zszwzKzezybX0vd7M1prZVjN7wsyaRCzrYmbvmdk2M1tsZqfvY/wikkJ2VVQy7pm5lJXv5rFRg2nRRF8zJkPQM/41hGbPemJPncxsCKFZuk4DOgPdgDsiukwD5gLtgVuA6WaWXceYRSRF/fGtJXy2fCP3DTuCnp1UfC1ZAiV+d5/h7i8BxbV0vRR43N0L3H0TcBdwGYCZ9QQGAbe7+3Z3fxFYCAzb2+BFJHW8+flaJr2/jIuP6cx5Aw5KdjgZLdZj/H2A+RGP5wOdzKx9eNkydy+JWt6nug2Z2ejw8FJeUVFRjMMUkURavqGMG1+YT/9D2nDrjw9PdjgZL9aJvwWwJeJx1f2W1SyrWl7t5z13n+Tuue6em52t0SCRVLV9ZwVXTZlDgwbGhBEDVXytHoj1NyulQKuIx1X3S6pZVrW8BBFJS+7ObS9/zpJ1JfzjsiM5uK2Kr9UHsT7jLwD6RzzuD6xz9+Lwsm5m1jJqeUGMYxCReuK5/6xi+pxvuObUHpxymIqv1RdBL+dsaGZZQAOggZllmVl1nxaeAi43s95m1ga4FZgM4O6FwDzg9vD6PwH6AS/GYD9EpJ75fPUWfv9KASf26MAvT+uR7HAkQtAz/luB7YQu1RwVvn+rmeWYWamZ5QC4+5vAA8B7wEpgBXB7xHYuAnKBTcB9wPnurm9uRdLMlm27uGrKHNo3b8zDFw1U8bV6xtw92THUKjc31/Py8pIdhogEUFnpXPFUHu8vLeL5K49lYE7bZIeUscxsjrvnRrerZIOIxNRj73/Fvxev59azeyvp11NK/CISMx99tYE/vbWEc/ofyCXHdk52OFIDJX4RiYm1W3Zw7bS5dO3QnPt+eoSKr9VjqpAkIvssVHwtn207K5h2xTE0V/G1ek3/OyKyz+5/YzF5KzbxyPCB9FDxtXpPQz0isk/eWPgtf//way49tjPn9j8w2eFIAEr8IrLXlhWVcuP0BQw4pA23nN072eFIQEr8IrJXtu+sYOzUfBo1MCaMHETjhkonqUJj/CJSZ+7OLS8tZMm6Ep78r6M4qE3TZIckdaA/0SJSZ9M+W8WM/NX88rQenNRTZdNTjRK/iNTJgm82M/6VAk7qmc21p6r4WipS4heRwDZv28mYKfl0aNGYh342gP1UfC0laYxfRAKprHSuf24e60t28MJVx9GueeNkhyR7SWf8IhLIf8/6kveWFHHbj3sz4JA2yQ5H9oESv4jUavaXG3jwX4Wc2/9ALj5GxddSXdAZuNqZ2UwzKzOzFWY2ooZ+b4QnZqm67TSzhRHLl5vZ9ojlb8dqR0QkPqqKr3XLbsG9Kr6WFoKO8U8AdgKdgAHA62Y2392/M1+uu58Z+djMZgHvRm3rHHd/Z+/CFZFE2lVRydXP5LNjVwWPjRqs4mtpotYzfjNrDgwDbnP3Unf/EHgFuLiW9boAJxKah1dEUtC9/1zMnBWbuP/8fhzasUWyw5EYCTLU0xPYHZ4svcp8oE8t610CfODuy6Pap5pZkZm9bWb9a1rZzEabWZ6Z5RUVaVpekUR7fcG3PDH7ay47rgs/7qfia+kkSOJvAWyNatsC1FZ79RJgclTbSKAL0JnQhOxvmVm1lwe4+yR3z3X33Oxs/TJQJJG+KirlN9PnMyinDb876/BkhyMxFiTxlwKtotpaASU1rWBmJwD7A9Mj2919trtvd/dt7n4vsJnQcJCI1BPbdu5mzJQ5NGnUQMXX0lSQ/9FCoKGZRf42uz9QUEN/gEuBGe5eWsu2HdAlAiL1hLtzy8zPWbq+lIcvGsABrVV8LR3VmvjdvQyYAdxpZs3N7HjgPODp6vqbWVPgQqKGecwsx8yON7PGZpZlZjcCHYDZ+7gPIhIjUz9dycy5q7n+9J6c2ENDrOkq6Ge4sUBTYD0wDRjj7gVmdqKZRZ/VDyU0hPNeVHtLYCKwCVgNnAGc6e7Fexu8iMTO/FWbufPVRZxyWDbjfnBossORODJ3T3YMtcrNzfW8vLxkhyGStjaV7eTHj34IwGvXnEBb1eFJC2Y2x91zo9v1awyRDFdZ6Vz//DyKSsp54apjlfQzgL6uF8lwf33vS2YtKeK2c3rTX8XXMoISv0gG+2BpEX95p5ChAw5k1NE5yQ5HEkSJXyRDrdm8nV8+O48eHVtwj4qvZRQlfpEMtHN3JeOeyad8VwUTRw2mWWN93ZdJ9L8tkoHu+ecX5K/czIQRg+iereJrmUZn/CIZ5tX5a5j80XJ+fnxXzu53QLLDkSRQ4hfJIF+uL+XmFxcwuHNbfntWr2SHI0mixC+SIcrKQ8XXsho1YMKIQTRqoLd/ptIYv0gGcHd+N3MhXxWV8vTlR7N/66xkhyRJpD/5IhlgyicreHneGm74YU+OP7RDssORJFPiF0lzc1du4s7XFnFqr46MPUXF10SJXyStbSzbydVT8+nUKosHL+zPfvvpR1qiMX6RtFVR6Vz33Dw2lO5k+phjadNMxdckJNAZv5m1M7OZZlZmZivMbEQN/cab2S4zK424dYtYPsDM5pjZtvC/A2K1IyLyXY++u5T3C4u4/dze9DtYxdfk/wQd6pkA7AQ6EZowfaKZ9amh73Pu3iLitgzAzBoDLwNTgLbAk8DL4XYRiaH/LSzi4X8v5acDD2LEUSq+Jt9Va+I3s+bAMOA2dy919w+BV4CL6/hcpxAaWnrI3cvd/RFC8+2eWsftiMgerN68neuenUvPji35w09UfE2+L8gZf09gt7sXRrTNB2o64z/HzDaaWYGZjYlo7wMs8O9O+bWgpu2Y2WgzyzOzvKKiogBhikj57grGTs1nV4UzcdQgmjZukOyQpB4KkvhbAFuj2rYQmkM32vPA4UA2cAXwezMbHrGdLQG3g7tPcvdcd8/NztakzyJB/OH1L5i/ajN/uqAf3VR8TWoQJPGXAq2i2loBJdEd3X2Ru69x9wp3/wh4GDi/rtsRkbp7ed5qnvp4Bb84oStn9FXxNalZkMRfCDQ0sx4Rbf2BggDrOqFxfML9+9l3Bxz7BdyOiOzB0nUl/HbGQo7s0pabzlTxNdmzWhO/u5cBM4A7zay5mR0PnAc8Hd3XzM4zs7YWchRwLaEreQBmARXAtWbWxMzGhdvfjcF+iGSs0vLdXDVlDs0aN+DR4Sq+JrUL+goZCzQF1gPTgDHuXmBmJ5pZaUS/i4AvCQ3fPAXc7+5PArj7TmAocAmwGfg5MDTcLiJ7wd25+cUFfL2hjEeGD1TxNQkk0C933X0joaQd3f4BoS9tqx4Pj+4T1X8uMLiOMYpIDZ78aDmvLfiWG4ccxnHdVXxNgtFnQpEUNWfFJu5+/QtO69WRMSd3T3Y4kkKU+EVSUHFpOeOeyeeANlk8eOEAFV+TOlGRNpEUU1V8rbhsJzPGHEfrZo2SHZKkGJ3xi6SYh/+9lA+WbuCOc/vQ96DWyQ5HUpASv0gKmbVkPY++u5Rhgw7moiMPSXY4kqKU+EVSxDebtnHdc/M4rFNL7h7aV8XXZK8p8YukgKriaxUVzsRRg1V8TfaJvtwVSQF3vbaIBd9s4bFRg+naoXmyw5EUpzN+kXrupbmrmfLJSkaf1I0z+u6f7HAkDSjxi9RjheHia0d1aceNQw5LdjiSJpT4ReqpquJrzZs05K8jBqr4msSMXkki9ZC7c9P0BSzfUMajwwfSsZWKr0nsKPGL1EP/mL2c1xd+y41DenFs9/bJDkfSjBK/SD2Tt3wj9/zzC04/vBNXntQt2eFIGlLiF6lHNpSWc/Uz+RzYpil/vrC/iq9JXARK/GbWzsxmmlmZma0wsxE19LvRzD43sxIz+9rMboxavtzMtptZafj2dix2QiQdVFQ6v3x2Lpu27eK/Rw6idVMVX5P4CPoDrgnATqATMAB43czmu3v0fLlGaIatBUB34G0zW+Xuz0b0Ocfd39nHuEXSzl/+VcjsL4u5f9gRKr4mcVXrGb+ZNQeGAbe5e6m7fwi8Alwc3dfdH3D3fHff7e5LCM23e3ysgxZJN+8uXsdf3/uSCwYfzM+OzEl2OJLmggz19AR2u3thRNt8oM+eVrJQBakTgehPBVPNrMjM3jaz/ntYf7SZ5ZlZXlFRUYAwRVLTqo3buP65+Rx+QCvuGto32eFIBgiS+FsAW6PatgAta1lvfHj7/4hoGwl0AToD7wFvmVmb6lZ290nunuvuudnZ2QHCFEk9O3aFiq9VuvPYqEFkNVLxNYm/IIm/FGgV1dYKKKlpBTMbR2is/2x3L69qd/fZ7r7d3be5+73AZkKfCkQy0p2vLWLh6i38+YL+dG6v4muSGEESfyHQ0Mx6RLT15/tDOACY2c+Bm4HT3P2bWrbthL4QFsk4M/K/4ZlPV3LVyd35UR8VX5PEqTXxu3sZMAO408yam9nxwHnA09F9zWwkcA/wQ3dfFrUsx8yON7PGZpYVvtSzAzA7FjsikkoWr93K72Yu5Jhu7fj1j3omOxzJMEF/wDUWaAqsB6YBY9y9wMxONLPSiH53A+2B/0Rcq/9YeFlLYCKwCVgNnAGc6e7FsdgRkVRRsmMXY6bk0yqrEY8MH0hDFV+TBAt0Hb+7bwSGVtP+AaEvf6sed93DNgqAfnsRo0jacHd+M30BKzduY9oVx9CxpYqvSeLpVEMkgR7/8Gve+HwtN51xGEd1bZfscCRDKfGLJMh/lm/k3jcWM6RPJ644UcXXJHmU+EUSoKiknKun5nNI26b88YL+hH7fKJIcmmxdJM52V1Ry7bS5bNm+i8n/dRStslR8TZJLiV8kzh78VyEfLyvmj+f3o/eB0b+FFEk8DfWIxNE7i9bx37O+4qIjD+GC3EOSHY4IoMQvEjcri7dxw/Pz6HNgK8afu8eahiIJpcQvEgc7dlUw9pk5AEwcOVjF16Re0Ri/SBzc8WoBn6/eyt8vySWnfbNkhyPyHTrjF4mx6XO+Ydpnqxh7SndO790p2eGIfI8Sv0gMffHtVm6ZuZBju7Xnhh+q+JrUT0r8IjGydccuxkyZQ+umKr4m9ZvG+EViwN35zQsLWLVpO8+OPobslk2SHZJIjXRKIhIDf//ga94sWMtvz+zFkV1UfE3qNyV+kX306bJi7ntzMWf23Z/LT6ixMrlIvREo8ZtZOzObaWZlZrbCzEbU0M/M7H4zKw7f7reIalRmNsDM5pjZtvC/A2K1IyLJ8MmyYq5+Zi457ZrxwPn9VHxNUkLQM/4JwE6gEzASmGhm1f0UcTShCVv6E5p05RzgSgAzawy8DEwB2gJPAi+H20VSSsmOXdwycyEXTfqEZo0b8LeLB9NSxdckRdT65a6ZNQeGAX3dvRT40MxeAS4mNKl6pEuBP1dNsm5mfwauAB4DTgk/30Pu7sAjZvZr4FTgzdjsznf94sn/sKJ4Wzw2LRluQ2k5W7bv4hcndOVXPzqMpo31y1xJHUGu6ukJ7Hb3woi2+cDJ1fTtE14W2a9PxLIF4aRfZUG4/XuJ38xGE/oEQU5OToAwvy+nXXMaN9TXGBJ7fQ5sxaXHdWFgTttkhyJSZ0ESfwtga1TbFkKTp1fXd0tUvxbhcf7oZXvaDu4+CZgEkJub69X1qc3vz+m9N6uJiKS1IKfDpUB0EfFWQEmAvq2A0vBZfl22IyIicRIk8RcCDc2sR0Rbf6Cgmr4F4WXV9SsA+tl3L3voV8N2REQkTmpN/O5eBswA7jSz5mZ2PHAe8HQ13Z8CbjCzg8zsQOBXwDQPUY4AAAUiSURBVOTwsllABXCtmTUxs3Hh9nf3bRdERKQugn7zORZoCqwHpgFj3L3AzE40s9KIfn8DXgUWAp8Dr4fbcPedhC71vATYDPwcGBpuFxGRBLHvXmRTP+Xm5npeXl6ywxARSSlmNsfdc6Pbda2jiEiGUeIXEckwSvwiIhkmJcb4zawIWLGXq3cANsQwnFhRXHWjuOpGcdVNusbV2d2zoxtTIvHvCzPLq+7LjWRTXHWjuOpGcdVNpsWloR4RkQyjxC8ikmEyIfFPSnYANVBcdaO46kZx1U1GxZX2Y/wiIvJdmXDGLyIiEZT4RUQyjBK/iEiGSavEHy73/LiZrTCzEjObZ2Zn1rLO9Wa21sy2mtkTZtYkTrGNM7M8Mys3s8m19L3MzCrMrDTidkqy4wr3T9TxamdmM82sLPz/OWIPfceb2a6o49UtkXFYyP1mVhy+3R8190TM1SG2uB2fap6rLq/zhLyW6hJXIt974eerU86K1TFLq8RPaCrJVYTmA24N3Ao8b2ZdqutsZkMITRh/GtAZ6AbcEafY1gB3A08E7P+xu7eIuM1KdlwJPl4TgJ1AJ2AkMNHM+uyh/3NRx2tZguMYTajseH9CEwydA1wZoxj2NTaI3/GJFuj1lODXUuC4whL13oM65KyYHjN3T+sboQndh9Ww7BngnojHpwFr4xzP3cDkWvpcBnyY4OMUJK6EHC+gOaGE1jOi7Wngvhr6jwemJDMO4CNgdMTjy4FP4vj/VZfY4nJ89uX1lIz3XsC4Ev7eqyaGanNWLI9Zup3xf4eZdQJ6UvP0jn2A+RGP5wOdzKx9vGMLYKCZbTCzQjO7zcwaJjsgEne8egK73b0w6rn2dMZ/jpltNLMCMxuThDiqOzZ7ijeRsUF8js++0HuvGrXkrJgds7RN/GbWCJgKPOnui2vo1gLYEvG46n7LeMYWwPtAX6AjMAwYDtyY1IhCEnW8WgBbo9q27OF5ngcOB7KBK4Dfm9nwBMdR3bFpEcdx/rrEFq/jsy/03osSIGfF7JilVOI3s1lm5jXcPozotx+hj707gXE1bhBKgVYRj6vul8QjrqDcfZm7f+3ule6+ELgTOL+u24l1XCTueEU/T9VzVfs87r7I3de4e4W7fwQ8zF4cr2rUJY7qjk2phz+Tx0Hg2OJ4fPZFTF5LsRar915dBcxZMTtmKZX43f0Ud7cabidA6OoK4HFCX3gNc/dde9hkAaEv46r0B9a5e3Gs49pHDtT5zDEOcSXqeBUCDc2sR9Rz1TRk972nYC+OVzXqEkd1xyZovPGOLVqsjs++iMlrKQHifqzqkLNidsxSKvEHNJHQx9pz3H17LX2fAi43s95m1obQN+qT4xGUmTU0syygAdDAzLJqGjs0szPDY32YWS/gNuDlZMdFgo6Xu5cBM4A7zay5mR0PnEfojKi6fTjPzNpayFHAtcTgeNUxjqeAG8zsIDM7EPgVcXot1TW2eB2f6tTh9ZSw915d4krkey9C0JwVu2OWzG+vY30jdImTAzsIfSyquo0ML88JP86JWOcGYB2h8dJ/AE3iFNv4cGyRt/HVxQX8KRxTGbCM0MfNRsmOK8HHqx3wUvgYrARGRCw7kdAwStXjaUBxONbFwLXxjqOaGAx4ANgYvj1AuBZWHF/vQWOL2/EJ+npK5mupLnEl8r0Xfr4ac1Y8j5mKtImIZJh0HOoREZE9UOIXEckwSvwiIhlGiV9EJMMo8YuIZBglfhGRDKPELyKSYZT4RUQyzP8DKFSZbge5YtYAAAAASUVORK5CYII=\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "plot_function(F.relu)" - ] - }, - { - "cell_type": "code", - "execution_count": 255, - "metadata": { - "id": "QyuXxFphpokK" - }, - "outputs": [], - "source": [ - "#แน่นอนว่าอ่านง่ายกว่าถ้าใช้ pytorch แทนที่จะเขียนฟังชั่นเอง\n", - "simple_net = nn.Sequential(\n", - " nn.Linear(28*28,30),\n", - " nn.ReLU(),\n", - " nn.Linear(30,1)\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": 256, - "metadata": { - "id": "94urzFOSpokK" - }, - "outputs": [], - "source": [ - "learn = Learner(dls, simple_net, opt_func=SGD,\n", - " loss_func=mnist_loss, metrics=batch_accuracy)" - ] - }, - { - "cell_type": "code", - "execution_count": 257, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 1000 - }, - "id": "Z8WM5ImkpokK", - "outputId": "4773537c-6e5c-410a-e839-14181e3a192d" - }, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n" + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "08BIemoApokA", + "outputId": "a209e1e9-e5ec-4c78-c2ed-3e9842c8974c" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(torch.Size([784]), torch.Size([1]))" + ] + }, + "metadata": {}, + "execution_count": 34 + } ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
epochtrain_lossvalid_lossbatch_accuracytime
00.2019710.0848630.96467100:00
10.0998620.0534450.96712500:00
20.0634570.0442920.97006900:00
30.0466910.0397990.97055900:00
40.0387840.0367570.97252200:00
50.0339280.0347910.97252200:00
60.0303350.0333780.97301300:00
70.0281980.0318290.97546600:00
80.0267690.0309010.97497500:00
90.0251490.0300880.97497500:00
100.0246890.0292900.97595700:00
110.0242790.0287690.97546600:00
120.0232550.0279090.97644800:00
130.0222210.0273520.97644800:00
140.0217220.0267840.97644800:00
150.0207420.0262480.97742900:00
160.0205200.0257570.97890100:00
170.0200880.0254520.97841000:00
180.0192520.0251850.97841000:00
190.0193140.0250370.97890100:00
200.0191190.0245810.97841000:00
210.0187150.0240800.97841000:00
220.0189390.0238080.97841000:00
230.0184360.0235630.97841000:00
240.0185330.0230740.97890100:00
250.0180650.0230190.97988200:00
260.0178940.0227760.97988200:00
270.0172870.0227010.98086400:00
280.0174050.0225140.98135400:00
290.0173150.0221080.98086400:00
300.0167650.0220430.98135400:00
310.0167900.0218320.98135400:00
320.0165870.0217300.98184500:00
330.0159500.0213040.98135400:00
340.0158970.0213510.98233600:00
350.0161210.0210570.98184500:00
360.0160240.0209620.98184500:00
370.0159870.0209560.98233600:00
380.0155440.0207720.98233600:00
390.0155440.0203690.98233600:00
" + "source": [ + "dset = list(zip(train_x,train_y))\n", + "example = dset[0]\n", + "\n", + "#คู่ Inputs　และ Labels\n", + "example[0].shape, example[1].shape" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": { + "id": "Y6ImID5ApokA" + }, + "outputs": [], + "source": [ + "#สร้าง validation set ในแบบเดียวกัน\n", + "valid_3_tens = torch.stack([tensor(Image.open(o)) \n", + " for o in (path/'valid'/'3').ls()])\n", + "valid_3_tens = valid_3_tens.float()/255\n", + "valid_7_tens = torch.stack([tensor(Image.open(o)) \n", + " for o in (path/'valid'/'7').ls()])\n", + "valid_7_tens = valid_7_tens.float()/255\n", + "\n", + "valid_x = torch.cat([valid_3_tens, valid_7_tens]).view(-1, 28*28)\n", + "valid_y = tensor([1]*len(valid_3_tens) + [0]*len(valid_7_tens)).unsqueeze(1)\n", + "valid_dset = list(zip(valid_x,valid_y))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "RR1v1Z2tzHmo" + }, + "source": [ + "#### Initiate `Weights`" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "7slIAUaOI0_d" + }, + "source": [ + "สมมุติว่าเราจะใช้ architecture สุดเรียบง่าย แค่คูณค่า pixels ของรูป `Inputs` ด้วย `W` และบวกด้วย `b`\n", + "\n", + "$$prediction = \\Sigma(xW^T) + b$$\n", + "\n", + "เราสามารถเริ่มตั้ง `Weights` เป็นการ random จาก standard normal distribution" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": { + "id": "WQ9vpYHIpokA" + }, + "outputs": [], + "source": [ + "def init_params(size, std=1.0): return (torch.randn(size)*std).requires_grad_()" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "itXv1TZ9pokA", + "outputId": "cc10560d-27d6-4ff4-b29e-99fca3878da9" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "torch.Size([784, 1])" + ] + }, + "metadata": {}, + "execution_count": 37 + } ], - "text/plain": [ - "" + "source": [ + "#Inputs มี dimension (batch_size, 28*28)\n", + "#เพราะงั้นถ้าเราจะคูณรายตัว (element-wise multiplication) ด้วย W^T, W ต้องมี dimension (28*28,1)\n", + "weights = init_params((28*28,1))\n", + "weights.shape" ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "learn.fit(40, 0.1)" - ] - }, - { - "cell_type": "code", - "execution_count": 258, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 268 - }, - "id": "Q0d__m-FpokL", - "outputId": "846b03cf-5b37-45f8-ef7b-cbe4691e0b7a" - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAD7CAYAAAB9nHO6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxV1bn/8c8TwhzCPCNQGVRACYpDiyhah2pVVLSoVLl6La38rHNbbm+xOLRW7RW1TtdbB7QOoICgVmxVRrUyCAhRCCoBGRNIICQhhiTP74+zY4+Hk+ScmHAyfN+v134lZ6111n7OTrKf7LX2YO6OiIhIrJISHYCIiNQvShwiIhIXJQ4REYmLEoeIiMRFiUNEROKSnOgAalunTp28b9++iQ5DRKReWbFixS537xytrsEnjr59+7J8+fJEhyEiUq+Y2aaK6jRUJSIicVHiEBGRuChxiIhIXGJKHGbWwcxmm1mBmW0ysysqaNfOzKaZWVawTImoTzOzxWa218y2mNnksLqTzOyfZpZjZtlm9oqZdQ+rn2JmB8wsP2w5vJqfW0REqinWI45HgWKgKzAOeNzMBkdpNxVoBfQFTgCuNLOrw+pfBBYBHYBTgYlmdkFQ1x54MnhvH2Af8ExE/9PdPSVs+TLG+EVEpIZUmTjMrDUwBpjs7vnuvgSYC1wZpfn5wH3uXujumcBTwDVh9X2BF9y91N2/AJYAgwHc/S13f8Xd89y9EHgEGFH9jyYiIrUhliOOgUCJu2eEla0m2OFHYRHfDwl7/SBwlZk1NbMjgO8D71TQzylAekTZ+cFQVrqZXVdRwGY2wcyWm9ny7OzsipqJiEg1xHIdRwqQF1G2F2gTpe08YJKZjSc0rHUNoaGrcm8AzwG3AU2AO919WWQnZnYMcDswOqx4BqGhrJ3AicBMM9vj7i9Fvt/dnwzaMnz4cN03XkRq3P7iUl79eAs/Pro7HVo3q9G+3Z231u5g3fbIXW98Ljq2F9/r1LqGovq3WBJHPpAaUZZKaA4i0g3AX4ANwG7gJeByCE2wE0os1xOa6+gGvGpmO939sfIOzKw/8BZwo7svLi9390/D1vOBmT0EXBKsQ0TkkLrrzU958aPNPPzuBv586VBOHRj1Iuu45RYU81+z1jAvfQcAZlW8oRLH9mmfsMSRASSb2QB33xCUDeXgYSTcPYfQ5DkAZvZHYGnw8nCg1N2fC15vMbOXgXOBx4L2fQgNXd3l7s9XEZfz7WExEZFD4h/pO3jxo81cPKwn6dvyGP/0Uv7jB32ZdM6RtGjapNr9LsrI5rZXVpNbWMxvzz2Sa08+nKSkurebqzJxuHuBmc0C7jSza4E0QkNIP4hsa2b9gD3BchYwgdDZUxBKQBacyvsy0AUYC8wP3tsTeA94xN2fiNL3aEJnZO0Bjid0dPPbeD6siMh3lZVXxG9mfsKQnqn8acwxlLlz37z1PP3+RpZ8vosHx6YxpGfbuPosOlDKn95ax7MfZDKgSwrPXH08g3vE18ehFOvpuBOBlkAWoaGh69w93cxGmll+WLvjgDWEhrHuAca5ezqAu+cBFwM3A7nAKmAtcHfw3msJHZVMCb9WI6zvy4DPg76fA+5192nxfmARkeoqK3NufWU1+w+U8uDYYTRLTqJF0ybcfv4gnv/PE9hXdICLHnufJxZ+QWlZbNOrn27L44JHlvDsB5lcPaIvr//y5DqdNACsoT9zfPjw4a6bHIpINO6OxTGJ8NSSjdz1xqf84aIhjDuxz0H1uQXF/Hb2Gt5au4OTDu/A/ZcMpXvbFlH7KnN49oON/PntDNq2alqj8yQ1wcxWuPvwaHUN/u64IiLRrNycy62vrKZLm+b8+dKh9GrfqtL2n23P49631nHGUV254oTeUdu0b92Mx8Ydy6srtjBlbjoj75tfZRxnD+7KPRcfU+NnZtUmHXGISKNSUlrGo/O/4OH3NtClTXP2FZVgwF0XDmF0Wo+oRyBFB0q54JEl5BQc4O2bRtIxpXmV69m8u5A31myjtLTifWz/Lin8aEi3uI56DhUdcYiIAJt2F3DT9FWs3LyHi4b15I7Rg9lbeICbp6/ipumreHddFnePHkLbVk2/9b4/vbWOjJ35PHv18TElDYDeHVsxcVT/2vgYCafEISINnrvzyvItTHk9neQk4+HLh3HB0B4ApLZoyssTTuKJhV/w4DsbWJGZw59/MpQf9OsEwPz1Wd9MXI86oksiP0adoduqi0iDllNQzC/+toJfz/yEob3aMe+mU75JGuWSmyRx/ekDmHndD2jRtAnj/voR9/z9M7bv3c+vXvmEI7q24Tc/OjJBn6Du0RyHiCRM0YFSpi/7ihH9O9G/S0rc79+wcx9zV2+r8NTXMoeZH29hT2Exvzr7iJguqCssLuEPb37GCx9tpnlyEg7MvX4ER3aLvIFGw6Y5DhGpcz7dlsdN01eSsTOfFk2T+O9zj+KnJ/WJaaK4rMyZ9mEm97y1jgOlZSRXkgwGdGnDtKtPYFCP2Hb8rZol84eLjua0I7pw5xufct2ofo0uaVRFiUNEDqmyMuevS7785vqFhy8fxqyPtzB5TjrvrsvivkuOoUub6Nc+AOzMK+K2V1azeMMufnhkF/405hg6t4ltwjoeZwzqyhmDutZ4vw2BEoeIHDLb9uzn1hmr+fDL3d+6fuH8Y7rzt39t4u43P+NHDy7m3jHHcGaUnfa8tduZNGsNRQdK+cNFQ7jihN518lTWhk6JQ0QOibmrt/G72WsoKXPuG3MMlw7v9c1O38y48vt9+X6/jtw0fRU/e245l5/Qm8nnHUWrZsnkf13CHXPTeWXFFo7p1ZapY9Po1zn+ORGpGUocIlKr9u4/wO/nrOW1VdsY1rsdD45No0/H6Lf67t+lDbOuG8HUdzJ4YuEX/OvL3Vw3qh+PvPc5W3IL+eXp/bnhhwNo2kQnhCaSzqoSkVpTWFzCBY+8z8ZdBdxw+gD+32n9SI5xp//Rl7u5ZcZqtu7Zz2EdWjL1J2kM79uhliOWcjqrSkQS4q43PuOL7HyevfqEuG/gd+LhHXnrppG8tWY75x7dnTYtmlb9JjkklDhEpFa8nb6Dl5Zu5hen9qv2XV9TWzRl7PHRbygoiaOBQhGpcTvzipgUPOzoljMHJjocqWFKHCJSo8rKnNsiHnYkDYt+oiJSo55+fyOLN+xi8nmDqnUbEan7lDhEpMZ8tj2P++atr/RhR1L/xZQ4zKyDmc02swIz22RmV1TQrp2ZTTOzrGCZElGfZmaLzWyvmW0xs8kR9T80s3VmVmhm882sT1hdczN72szyzGyHmd1Sjc8rIrWk6EApN768kratmnLvmKN1RXcDFusRx6NAMdAVGAc8bmaDo7SbCrQC+gInAFea2dVh9S8Ci4AOwKnARDO7AMDMOgGzgMlB/XJgeth7pwADgD7AacCvzexHMcYvIrWs/GFHf750aMwPO5L6qcrEYWatgTHAZHfPd/clwFzgyijNzwfuc/dCd88EngKuCavvC7zg7qXu/gWwBChPQBcD6e7+irsXEUoUQ82s/Cb444G73D3X3T8D/g/4j3g+rIjUjvnrQg87umbE96p96q3UH7EccQwEStw9I6xsNf/e4UeyiO+HhL1+ELjKzJqa2RHA94F3grrBQb8AuHsB8AUw2MzaA93D6yuLwcwmmNlyM1uenZ1d1ecTkSoUFpeQ/3X0ZUtuIb96dTVHdmvDr390RKJDlUMglgsAU4C8iLK9QJsobecBk8xsPKFhrWsIDV2VewN4DrgNaALc6e7LwtYTuZcvX09K2OuqYsDdnwSehNAtRyr6YCJStUfnf879b6+vtE2z5CReuPYkWjRtcoiikkSKJXHkA5FPMUkF9kVpewPwF2ADsBt4CbgcQhPshBLL9YTmOroBr5rZTnd/rIr15Ie9LqoiBhGpIcsyc/iff6zntCM6f/MM7miO69ueI7pF/T9OGqBYEkcGkGxmA9x9Q1A2FEiPbOjuOYQmzwEwsz8CS4OXhwOl7v5c8HqLmb0MnAs8FvQ3Puy9rYF+hOY9cs1se7Def1YWg4jUjLyiA9z08ip6tW/FX644lpTmukORhFQ5xxHMNcwC7jSz1mY2AhgNPB/Z1sz6mVlHM2tiZucAE4C7g+qMUBO7wsySzKwbMBb4JKifDQwxszFm1gK4HfjE3dcF9c8BvzOz9sGE+c+AZ6v5uUWkCre/tpYdeUU8eFmakoZ8S6yn404EWgJZhIafrnP3dDMbaWb5Ye2OA9YQGkK6Bxjn7ukA7p5H6Mypm4FcYBWwliCxuHs2obO3/hDUnwhcFtb37wlNlm8CFgL3u/u8eD+wiFTttZVbeW3VNm44fQDH9m6f6HCkjtHzOETkW77KKeTchxYzsFsbpk84KebnZ0jDUtnzOPQbISLfKC1zbpmxCgceHJumpCFRaeBSRL7x+ILPWZaZy9SxQzmsQ6uq3yCNkv6dEBEAVn21h6nvbOCCoT24MK1nosOROkyJQ0Qo+LqEG19eSbfUFtx14RDdoFAqpcQhEigrc2rzZJEDpWW11vd3dcfr6XyVU8jUsWm0balne0vllDhEAr98aSWn/XkBKzbl1Gi/RQdK+f2ctQy+/W2m/jODkjqUQIoOlHLn658yY/kWJo7qzwnf65DokKQeUOIQAdK37eXNNdvZvreIS5/4kAf+mVEjRwhrt+7lvL8sYdqHmxjcM5WH3t3AJU98SOaughqI+rv5bHseox95n6ff38j47/fhxjMGJDokqSeUOESAxxZ8QUrzZN67bRQXDuvJw8EOfmM1d/ClZc7jC77gosfeZ1/RAZ7/zxOYPXEEf7l8GF9m53Puw4uZvmxzrQ6NVaSszPnr4i8Z/cj77C4o5pmrj+eO0UNoqlNvJUa6AFAavY27Cvjh/yzg56f24zc/Cj3+5Y1PtvHfs9dSXFLG7ecP4rLjD4t5wnjrnv3cMn0VH23M4Zwh3fjjRUfTvnWzb+q3793PrTNW88EXuzlrUFf+NOYYOoTV16bwdZ85qCt/uvhoPXRJoqrsAkAlDmn0Js38hNkrt7LkN6fTuc2/d6Lb9+7ntldW8/7nse9k56zayu9eW0tZmXPH6CGMObZn1IRTVuY8tWQj97+9nratmvLnS4fW+gOQ3vhkG7+dtYaSMuf35w/iJ8NjT4bS+ChxKHFIBXbsLWLkfe9x2fG9uevCIQfVl5U5T7+/kfvmrSe1ZVPOGdKNiva1X+UUMn99Nsf1ac/Un6TRu2PVF9B9ui2Pm6avJGNnPj8+ujsdU2rnyGPbnv2881kWaYe148GxafTt1LpW1iMNR2WJQ1eOS6P2f4u/pMxhwimHR61PSjKuHXk4I/p34r9mreGNT7ZV2FdykyRuPXMg143qF/OtOgb1SGXu9Sdz/9vreW3lVspq6R+5JklJ3PjDAVx/en/NZch3piMOabRyC4oZce97/GhwNx4Ym5bocETqFN3kUCSKZz/IpLC4lF+M6pfoUETqFSUOaZTyvy7h2Q8yOXNQVwZ21SNPReKhxCGN0ksfbWbv/gNM1NGGSNyUOKTR+bqklL8u+ZIf9OvIMD3dTiRuShzS6Mz6eCs7875m4qj+iQ5FpF6KKXGYWQczm21mBWa2ycyuqKBdOzObZmZZwTIlrK63meVHLG5mtwb1v42o229mZWbWKah/1syKI9o0qYFtII1ISWkZTyz8gmN6tWVE/46JDkekXor1iONRoBjoCowDHjezwVHaTQVaAX2BE4ArzexqAHff7O4p5QtwNFAGzAzq/xhRfy+wwN13hfV/X3gbdy+N+xNLo/b3tTvYtLuQiaP666ppkWqqMnGYWWtgDDDZ3fPdfQkwF7gySvPzCe3cC909E3gKuKaCrq8CFgXtItdpQf20WD6ESCzcncfmf07/LimcNahrosMRqbdiOeIYCJS4e0ZY2Wog2hEHgEV8f9B9HGJIDCOBLgRHI2EmmlmOma0wszEVBWxmE8xsuZktz87OrqiZNDLvrcti3Y59/OLUfiQl6WhDpLpiSRwpQF5E2V4g2snv84BJZtbGzPoTOtqIdsOekwkNe71awTrHA6+6e35Y2cPAAEIJZTLwrJmNiPZmd3/S3Ye7+/DOnWv3xnFSP+zdf4Db56TzvU6tGZ3WI9HhiNRrsSSOfCA1oiwV2Bel7Q3AfmADMAd4CdgSpd14YGZEYgDAzFoBlxJxNOLuH7v7bncvcfe/Ay8AF8cQvzRy7s7vXlvLjrwipo5N072aRL6jWP6CMoBkMwt/PNhQID2yobvnuPs4d+/m7oOD/peGtzGzlkRJDGEuAnKABVXE5Xx7WEwkqtdWbeX11du4+YwBpB3WLtHhiNR7VSYOdy8AZgF3mlnrYHhoNPB8ZFsz62dmHc2siZmdA0wA7o5odhGQC8yvYJXjgec84u6LZnaJmaWYWZKZnQX8lNAkvUiFvsopZPJr6Rzftz3X6boNkRoR6zH7RKAlkEVo+Ok6d083s5FmFj7cdBywhtAw1j3AOHePPDIZDzwfmRgAzKwncDrwXJQYbgS2AnuA+4GfufuCGOOXRqiktIybpq/CgAd+kkYTTYiL1IiYnsfh7jnAhVHKFxOaPC9/PQOYUUVfZ1dSt7WimNx9ZCyxipR7dP4XrNiUy0OXpXFYh6ofqiQisdEsoTRIKzbl8vB7G7gwrQej03omOhyRBkWJQxqcfUUHuGn6SrqltuDOKI+DFZHvRo+OlQZnytxP2Zq7n+k//z6pLZomOhyRBkdHHNKgvL56GzM/3sL1p/Xn+L4dEh2OSIOkIw6pM9ydaR9kkr4t8kYFsXs7fQdph7Xjlz8cUHVjEakWJQ6pM2av3MqU1z+lc5vmNK3mqbO9O7biQV0dLlKrlDikTti8u5Db56RzQt8OvDThJF1zIVKH6d8ySbjQhXorMYMHxg5V0hCp43TEIQn3yPzP+XjzHh6+fBi92utCPZG6TkccUqPcnf3FsT+YccWmHB5+dwMXD+vJBUN1u3OR+kCJQ2rUpJlrOO7ufzJ92Wai3I7sW0IX6q2iZ/uW3DG6oueCiUhdo8QhNeb11duYvvwr2rVsym9mruEXf1tBTkFxhe3LL9R7cGwabXShnki9ocQhNWLbnv389+w1pB3WjgW/Oo3/Pvco5q/L5uwHF7Ew4+DH935zod7pAziujy7UE6lPlDjkOystc26evorSMuehy9JolpzEz045nNf+3wjat2rK+KeXMmVuOkUHQnMfW4MkM6x3O244Xc/IEKlvlDjkO3ty0Zd8tDGHKRcMpk/H1t+UD+qRytzrT+bqEX159oNMzvvLEtZs2cstQZJ5cGwaybpQT6Te0V9tI/Tch5m8t25njfS1Zste/ucf6/nx0d255LheB9W3aNqE358/mOeuOYG8/Qc4/5ElfLQxhztGD/lWkhGR+kPXcTQyO/OKuH1O6KGMlx1/GJPPG0Tr5tX7NSgsLuHGl1fSuU1z/nDREMwqvnDvlIGdefumU7jrjU9JbdmUMcfqGRki9ZUSRyOzcH1oovqiYT2Zvvwr/vXlbqaOTWNY7/Zx93XXG5+xcXcBL1x7Iu1aNauyffvWzXhgbFrc6xGRuiWmoSoz62Bms82swMw2mdkVFbRrZ2bTzCwrWKaE1fU2s/yIxc3s1qB+lJmVRdSPjzcGqdzCjGy6tGnOAz8Zyss/O4kDpc4lT3zIQ+9soKS0LOZ+3k7fwUtLN/PzU/rxg36dajFiEalrYp3jeBQoBroC44DHzSzaFVtTgVZAX+AE4EozuxrA3Te7e0r5AhwNlAEzw96/LbyNu0+rRgxSgZLSMhZvyObUgZ0xM048vCNv3TSSC4b2YOo7GVz6vx+yaXdBlf3szCti0sxPGNIzlVvOHHgIIheRuqTKoSozaw2MAYa4ez6wxMzmAlcCkyKanw+c4+6FQKaZPQVcAzwTpeurgEXunlnDMUgFVn61h7yiEkYd0eWbstQWTZk6No3TjuzC72av4ZyHFjP5vEGcdHjHCvu5fc5a9h8o5aHLhtEsWedXiDQ2scxxDARK3D0jrGw1cGoF7S3i+4Me+myhWdSrgLsiqrqY2U6gEHgN+J27F8Qbg5lNACYA9O7du4IwG58F67NokmScPODgoaULhvZgeJ/23DJjFf81a02Vff3xoqPp1zmlNsIUkToulsSRAkQ+km0v0CZK23nApGBuoiuho41otzs9Oah/NaxsHZAWfO0DTAMeAH4eZwy4+5PAkwDDhw+v/IZJjcjCjGyO7d2Oti2j396jR7uWvHjtScxfn8W+opIK++mU0pwR/Ss+IhGRhi2WxJEPpEaUpQL7orS9AfgLsAHYDbwEXB6l3XhgZjDsBIC77wB2BC83mtmvgTcIJY54YpAosvYVsXZrHredVfmcRFKS8cOjuh6iqESkPoplgDoDSDaz8Ic4DwXSIxu6e467j3P3bu4+OOh/aXgbM2sJXEroiKIyHhZfzDFIdIsydgF8a35DRKQ6qkwcwRzDLOBOM2ttZiOA0cDzkW3NrJ+ZdTSzJmZ2DqF5hrsjml0E5ALzI957mpn1sZDDgD8Bc+KNQaJbsD6LTinNGdQ98sBNRCQ+sZ4SMxFoCWQRGn66zt3TzWykmeWHtTsOWENoCOkeYJy7Rx4VjAee94Mf1jAM+AAoCL6uITT0VWkMMcbfqJWWOYs37OLUgZ1J0mNZReQ7iunKcXfPAS6MUr6Y0MR1+esZwIwq+jq7gvIHCE2GxxWDVG3VV3vYu/8Ao47onOhQRKQB0En4jcDC9VkkGYyMchquiEi8lDgagQUZ2aQd1i6m+0mJiFRFiaOB25X/NZ9s2auzqUSkxihxNHCLN4Tuhqv5DRGpKUocDdyC9dl0bN2MIT3aJjoUEWkglDgasNIyZ1FGNqfoNFwRqUFKHA3Ymq17yS3UabgiUrOUOBqwBeuzMIORA5Q4RKTmKHE0YAvWZzO0Vzs6tNZpuCJSc5Q4GqicgmJWb9nDqQN1tCEiNUuJo4FavCEbd52GKyI1T4mjgVq4Ppv2rZpyTK92iQ5FRBoYJY4GqKzMWRichttEp+GKSA1T4miA1m7by+6CYs1viEitUOJogBasD91m5BQlDhGpBUocDcw7n+7kmfc3MvSwdnRKaZ7ocESkAYrpQU5S9xUWl3D3m5/x4kebOap7Kv9z6dBEhyQiDZQSRwOw+qs93DR9FZm7C/j5qYdzy5kDaZ7cJNFhiUgDFdNQlZl1MLPZZlZgZpvM7IoK2rUzs2lmlhUsU8LqeptZfsTiZnZrUP9jM1tiZnvMbIeZ/dXM2oS9/1kzK454f6PeO5aUlvGXdzdw8eMf8PWBUl689iT+65yjlDREpFbFesTxKFAMdAXSgDfNbLW7p0e0mwq0AvoCXYB3zWyTuz/j7psJez65mX0P+ByYGRS1Be4GFgHNgReB+4FfhPV/n7v/LvaP13Bt3l3IzTNWsWJTLhcM7cFdFw6hbcumiQ5LRBqBKhOHmbUGxgBD3D0fWGJmc4ErgUkRzc8HznH3QiDTzJ4CrgGeidL1VcAid88EcPcXw+oKzez/gDvi/DwNwqbdBdw7bx0HSj1qvTv868vdmMFDl6UxOq3nIY5QRBqzWI44BgIl7p4RVrYaOLWC9hbx/ZCDGpgZocRxVyXrPQWIPKKZaGYTgY3AH9195sFvAzObAEwA6N27dyWrqJueeT+Tf6TvZEDXNhW2+UG/jtx+/iB6tW91CCMTEYktcaQAeRFle4Foe7V5wCQzG09oWOsaQkNXkU4O6l+NtkIzOxMYD5wYVvwwcGuw7rOA6Wa2w93fj3y/uz8JPAkwfPjw6P+211ElpWW88ck2zhrclcfGHZfocEREDhLL5Hg+kBpRlgrsi9L2BmA/sAGYA7wEbInSbjwwMxj6+hYzO4nQ/MYl4Uc57v6xu+929xJ3/zvwAnBxDPHXK0s+38Wu/GIuGKrhJxGpm2JJHBlAspkNCCsbysHDSLh7jruPc/du7j446H9peBszawlcCkyLfL+ZDQPmAte4+7tVxOV8e1isQZizahupLZI57Uhd9S0idVOVicPdC4BZwJ1m1trMRgCjgecj25pZPzPraGZNzOwcQvMMd0c0uwjIBeZHvHcIoaGuX7r761H6vsTMUswsyczOAn5KKMk0GIXFJbydvoNzj+6uU2pFpM6K9ZYjE4GWQBah4afr3D3dzEaaWfhw03HAGkLDWPcA46KcsjseeN7dI+cebgU6A0+FXacR/t4bga3AHkKn6f7M3RfEGH+98M9Pd1JYXKqzpESkTovpOg53zwEujFK+mLBrM9x9BjCjir7OrqD8auDqSt43MpZY67M5q7bRvW0LTvxeh0SHIiJSId3ksI7IKShmUUY2FwztQZKeoSEidZgSRx3x5prtlJS5hqlEpM5T4qgj5qzcysCuKRzVveKL/kRE6gIljjrgq5xClm/KZXRaT0IX1YuI1F1KHHXA3NXbABid1iPBkYiIVE2JI8HcnddWbuX4vu113ykRqReUOBLs0+15bMjK16S4iNQbShwJNmfVNpKTjB8f3T3RoYiIxESJI4FKy5y5q7Yx6ojOtG/dLNHhiIjERIkjgT7auJsdeUUaphKRekWJI4HmrNxG62ZNOOOorokORUQkZkocCVJ0oJS/r93O2UO60bKZ7oQrIvWHEkeCLFifxb6iEg1TiUi9o8SRIK+t3EanlGaM6Ncx0aGIiMRFiSMB1u/Yx3vrsjjvmB4kN9GPQETql5iexyE1o6zMmfZhJve8tY42LZK56vt9Eh2SiEjclDgOkZ15Rdz2ymoWb9jF6Ud24d4xx9C5TfNEhyUiEjcljkNg3trtTJq1hqIDpdx94RDGndhbd8EVkXorpgF2M+tgZrPNrMDMNpnZFRW0a2dm08wsK1imhNX1DnuWePniZnZrWJsrgv4LzOw1M+sQVhdTDHVJ/tcl/OqV1fzibx9zWPtWvHnDSH56Uh8lDRGp12I94ngUKAa6AmnAm2a22t3TI9pNBVoBfYEuwLtmtsndn3H3zYQ9n9zMvgd8DswMXg8G/hf4MfAx8CTwGHBZnDHUCSs25XLz9FVsyS3k+tP6c+MZA2iqiXARaQCqTBxm1hoYAwxx93xgiZnNBa4EJkU0Px84xyggTg0AAA9hSURBVN0LgUwzewq4BngmStdXAYvcPTN4PQ543d0XBeudDHxmZm2AsjhiSLivS0r5j6eX0rZVU6b//Psc37dD1W8SEaknYvkXeCBQ4u4ZYWWrgcEVtLeI74cc1CA0VnMVMC2seHDQLwDu/gWhI4yB8cZgZhPMbLmZLc/Ozq7oc9WaNVv2su/rEiafN0hJQ0QanFgSRwqQF1G2F4j2cOx5wCQza2Nm/QkdbUR7OtHJhIacXo1Yz94K1hNPDLj7k+4+3N2Hd+7cOVqTWrU0MweA4X3aH/J1i4jUtlgSRz6QGlGWCuyL0vYGYD+wAZgDvARsidJuPDAzGHaKZT3xxJBwyzNz6de5NR1TdLqtiDQ8sSSODCDZzAaElQ0FDpqUdvccdx/n7t3cfXDQ/9LwNmbWEriUbw9TEfQ3NKzd4UDzYP0xx5BoZWXO8swcTviehqhEpGGqMnG4ewEwC7jTzFqb2QhgNPB8ZFsz62dmHc2siZmdA0wA7o5odhGQC8yPKH8BON/MRgYT8ncCs9x9XzwxJNr6nfvIKypheB8lDhFpmGI9P3Qi0BLIIjT8dJ27pwc7+fDhpuOANYSGkO4BxkU5XXY88Ly7e3hh0O4XhBJIFqH5i4lVxRBj/IfM8mB+Q0ccItJQxXQdh7vnABdGKV9M2LUZ7j4DmFFFX2dXUvci8GI8MdQ1SzNz6ZranF7tWyY6FBGRWqEr0mqQu7NsYw7H9+2gq8NFpMFS4qhBW3L3syOvSMNUItKgKXHUoGXfXL+hxCEiDZcSRw1alplLmxbJHNEt6nWJIiINghJHDVqWmcPwPu1pkqT5DRFpuJQ4akhOQTGfZ+UzXPemEpEGTomjhuj6DRFpLJQ4asiyzByaJSdxTK+2iQ5FRKRWKXHUkKWZuQzt1ZbmyU0SHYqISK1S4qgBhcUlpG/dq2dviEijoMRRA1Zt3kNJmStxiEijoMRRA5Zm5mAGx+rBTSLSCChx1IDlmbkc2S2Vti2bJjoUEZFap8TxHZWUlvHx5lyO76ujDRFpHJQ4vqNPt+dRWFyq+Q0RaTSUOL6jpRtDF/4pcYhIY6HE8R0ty8zhsA4t6da2RaJDERE5JJQ4vgN3Z3lmro42RKRRiSlxmFkHM5ttZgVmtsnMrqigXTszm2ZmWcEyJUqbG81sY9DXZ2Y2MCj/rZnlhy37zazMzDoF9c+aWXFEm4Repv3lrgJ2FxRzghKHiDQisR5xPAoUA12BccDjZjY4SrupQCugL3ACcKWZXV1eaWbXAv8J/JjQs8rPA3YBuPsf3T2lfAHuBRa4+66w/u8Lb+PupXF81hq3LJjf0B1xRaQxqTJxmFlrYAww2d3z3X0JMBe4Mkrz8wnt3AvdPRN4Crgm6CcJ+D1ws7t/6iFfuHtOlHUacBUwrZqf65BYlplLh9bN6Ne5daJDERE5ZGI54hgIlLh7RljZaiDaEQeARXw/JPi+V7AMMbOvguGqO4KEEmkk0AWYGVE+0cxyzGyFmY2pKGAzm2Bmy81seXZ2diUf7bspf3BTKM+JiDQOsSSOFCAvomwvEO35qPOASWbWxsz6EzraaBXU9Qq+ngUcDZwGXE5o6CrSeOBVd88PK3sYGEAooUwGnjWzEdECdvcn3X24uw/v3LlzVZ+vWnbmFbE5p1DP3xCRRieWxJEPpEaUpQL7orS9AdgPbADmAC8BW4K6/cHX+9x9TzCU9b/AueEdmFkr4FIihqnc/WN33+3uJe7+d+AF4OIY4q8VyzJ1/YaINE6xJI4MINnMBoSVDQXSIxu6e467j3P3bu4+OOh/aVC9ntAEu4e/Jcr6LgJygAVVxOV8e1jskFq2MYeWTZswqEdkThURadiqTBzuXgDMAu40s9bB8NBo4PnItmbWz8w6mlkTMzsHmADcHfRTCEwHfh0MZfUK6t+I6GY88Jy7e0Tfl5hZipklmdlZwE8JTdInxL++zOHYPu1o2kSXwohI4xLrXm8i0BLIIjT8dJ27p5vZSDMLn4c4DlhDaBjrHmCcu4cfmVxPaOhrG/Ah8CLwdHmlmfUETgeeixLDjcBWYA9wP/Azd18QY/w1avve/azfuY9TBtTO/ImISF2WHEuj4JTZC6OULyY0eV7+egYwo5J+8oDLKqnfWlFM7j4yllgPhYXrQ2dqjTqiS4IjERE59DTOUg0L1mfTLbUFA7umVN1YRKSBUeKI04HSMt7/fBejjuis6zdEpFFS4ojTx5ty2fd1CaOO0PyGiDROShxxWpCRTXKSMaJ/p0SHIiKSEEoccVqwPpvj+rSnTQs9X1xEGicljjjszCvis+15nKphKhFpxJQ44rAwIzgNd6BOwxWRxkuJIw4L12fTNbU5R3WPdn9HEZHGQYkjRiWlZSzekM2pA3Uarog0bkocMVr51R7yiko4VcNUItLIKXHEaOH6bJokGScP0Gm4ItK4KXHEaEFGFsf2bkfbljoNV0QaNyWOGGTtK2Lt1jzd1FBEBCWOmCzK2AXAqQN1/YaIiBJHDBZmZNMppTmDuutpfyIiShxVKC3zb07DTUrSabgiIkocVVj11R72FB7Q3XBFRAJKHFVYuD6LJIOROg1XRASIMXGYWQczm21mBWa2ycyuqKBdOzObZmZZwTIlSpsbzWxj0NdnZjYwKB9lZmVmlh+2jI83hpq2MCObYb3b065Vs0OxOhGROi+mZ44DjwLFQFcgDXjTzFa7e3pEu6lAK6Av0AV418w2ufszAGZ2LfCfwI+Bz4DDgdyw929z917fMYYaszv/az7ZupebzxhYW6sQEal3qjziMLPWwBhgsrvnu/sSYC5wZZTm5wP3uXuhu2cCTwHXBP0kAb8Hbnb3Tz3kC3fPqeEYasyiDdm4o/kNEZEwsQxVDQRK3D0jrGw1MLiC9hbx/ZDg+17BMsTMvgqGq+4IEkq5Lma2M6ibGiSMuGMwswlmttzMlmdnZ8fwEaNbsD6bTinNGNKjbbX7EBFpaGJJHClAXkTZXiDavcXnAZPMrI2Z9Sd0tNEqqCsfgjoLOBo4Dbic0NAVwDpCQ1DdgdOB44AHqhED7v6kuw939+GdO1fvaKG0zFmUkc0pA3QarohIuFgSRz4QeeVbKrAvStsbgP3ABmAO8BKwJajbH3y9z933BENZ/wucC+DuO4IhrDJ33wj8mtDwVLwx1Ig1W/eSW3hAT/sTEYkQS+LIAJLNbEBY2VDgoElpd89x93Hu3s3dBwf9Lw2q1xOa3Pbwt1SyXg+LL+YYasqC9VmYwcgBShwiIuGqTBzuXgDMAu40s9ZmNgIYDTwf2dbM+plZRzNrYmbnABOAu4N+CoHpwK+DoaxeQf0bwXtPM7M+FnIY8CdCRy1xxVBTurdtwaXH9aJDa52GKyISLtYLACcCLYEsQsNP17l7upmNNLP8sHbHAWsIDSHdA4yLOF32ekLDTtuAD4EXgaeDumHAB0BB8HUNoaGvSmOIMf64jT2+N/ddMrS2uhcRqbfMvbLRovpv+PDhvnz58kSHISJSr5jZCncfHq1OtxwREZG4KHGIiEhclDhERCQuShwiIhIXJQ4REYmLEoeIiMRFiUNEROLS4K/jMLNsYFM1394J2FWD4dQkxVY9iq16FFv11OfY+rh71HsuNfjE8V2Y2fKKLoBJNMVWPYqtehRb9TTU2DRUJSIicVHiEBGRuChxVO7JRAdQCcVWPYqtehRb9TTI2DTHISIicdERh4iIxEWJQ0RE4qLEISIicVHiiMLMOpjZbDMrMLNNZnZFomMqZ2YLzKzIzPKDZX0CY7nezJab2ddm9mxE3Q/NbJ2ZFZrZfDPrUxdiM7O+ZuZh2y/fzCYf4tiam9lTwe/WPjNbFTxqubw+YduustjqyLb7m5ltN7M8M8sws2vD6hL9Oxc1trqw3YI4BgT7jr+FlV0R/KwLzOw1M+sQU2furiViIfRo2ulACnAysBcYnOi4gtgWANcmOo4glouBC4HHgWfDyjsF2+xSoAVwP/CvOhJbX8CB5ARut9bAlCCWJOA8Qo9b7pvobVdFbHVh2w0GmgffHwnsIPTI6rrwO1dRbAnfbkFM/wAWA38Li3cfcEqwr3sReDmWvpIrTyuNj5m1BsYAQ9w9H1hiZnOBK4FJCQ2ujnH3WQBmNhzoFVZ1MZDu7q8E9VOAXWZ2pLuvS3BsCefuBYR2zuXeMLONhHYyHUngtqsithW1vf6quHt6+Mtg6UcovkT/zlUU2+5Dsf7KmNllwB7gA6B/UDwOeN3dFwVtJgOfmVkbd99XWX8aqjrYQKDE3TPCylYTys51xT1mtsvM3jezUYkOJorBhLYZ8M3O6Avq1jbcZGZbzOwZM+uUyEDMrCuh37t06ti2i4itXEK3nZk9ZmaFwDpgO/B36sh2qyC2cgnZbmaWCtwJ3BJRFbnNvgCKCf28K6XEcbAUIC+ibC/QJgGxRPMb4HCgJ6ELeF43s36JDekgKYS2Wbi6sg13AccDfQj9l9oGeCFRwZhZ02D904L/jOvMtosSW53Ydu4+MVj3SGAW8DV1ZLtVEFuit9tdwFPuviWivNrbTInjYPlAakRZKqGxwIRz94/cfZ+7f+3u04D3gXMTHVeEOrsN3T3f3Ze7e4m77wSuB84ys0TsmJOA5wn9l3d9UFwntl202OrStnP3UndfQmgY8jrqyHaLFlsit5uZpQFnAFOjVFd7m2mO42AZQLKZDXD3DUHZUL59qF6XOGCJDiJCOjC+/EUwb9SPurkNy2+dcEj/iTIzA54CugLnuvuBoCrh266S2CIlZNtFSObf26eu/c6VxxbpUG63UYQm5zeHfqykAE3MbBAwj9C+DQAzOxxoTmgfWLlEzvLX1QV4mdCZVa2BEdSRs6qAdsDZhM4aSSY0uVUADExQPMlBLPcQ+u+0PK7OwTYbE5Tdy6E/w6Wi2E4EjiD0R9uR0Nlz8xOw7Z4A/gWkRJTXhW1XUWwJ3XZAF+Cy8p1f8LdQAFyQ6O1WRWwJ225AK6Bb2PJn4NVgew0mNCw/MtjX/Y0Yz6o6ZL+M9WkBOgCvBT/4zcAViY4piKszsIzQoeSe4I/7zATGM4V/nz1SvkwJ6s4gNEG4n9ApxH3rQmzA5cDG4Ge7HXgO6HaIY+sTxFNEaLigfBmX6G1XWWyJ3nbB7//C4Hc/D1gD/CysPpHbrcLYEr3dIuKcQnA6bvD6imAfVwDMATrE0o9ucigiInHR5LiIiMRFiUNEROKixCEiInFR4hARkbgocYiISFyUOEREJC5KHCIiEhclDhERicv/ByiFIRh++dzdAAAAAElFTkSuQmCC\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "#ดู accuracy เพิ่มขึ้นเรื่อยๆ\n", - "plt.plot(L(learn.recorder.values).itemgot(2));" - ] - }, - { - "cell_type": "code", - "execution_count": 259, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" }, - "id": "-pnmg5QqpokL", - "outputId": "91382f1d-566d-4cfb-82d9-70e7696a4498" - }, - "outputs": [ { - "data": { - "text/plain": [ - "0.98233562707901" + "cell_type": "code", + "execution_count": 38, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "7Ef5fK0KpokA", + "outputId": "087d1c9a-adf1-4cf2-a3ea-b24fea8066e0" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "tensor([0.3472], requires_grad=True)" + ] + }, + "metadata": {}, + "execution_count": 38 + } + ], + "source": [ + "#bias มีแค่ (1) dimension แล้วจะถูก broadcast ไปทุก dimension ของ batch size เอง\n", + "# (1) -> (batch_size,1)\n", + "bias = init_params(1)\n", + "bias" ] - }, - "execution_count": 259, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "learn.recorder.values[-1][2]" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "fQQNpNZ-pokL" - }, - "source": [ - "## ไปให้ Deep กว่านั้นอีก" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "jq3YclVaOjCk" - }, - "source": [ - "แทนที่เราจะใช้ Architecture ง่ายๆที่เราคิดขึ้นเอง เรามาลองใช้ Architecture ที่ซับซ้อนขึ้นอย่าง `resnet18` กัน" - ] - }, - { - "cell_type": "code", - "execution_count": 260, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 80 - }, - "id": "LB3qg01jpokL", - "outputId": "93503072-abe0-4fb1-a2d0-ff79af9e72fa" - }, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n" + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "pnMvUz7kFoIU", + "outputId": "07d34405-6a39-4b0f-b374-56fe7f0a7272" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(torch.Size([784, 1]),\n", + " torch.Size([1]),\n", + " torch.Size([12396, 784]),\n", + " torch.Size([784]))" + ] + }, + "metadata": {}, + "execution_count": 39 + } + ], + "source": [ + "weights.shape, bias.shape, train_x.shape, train_x[7].shape" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "KXK2LAZqzLYk" + }, + "source": [ + "#### Forward Pass" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "S9md33qTEu6n", + "outputId": "1469e6f5-bd86-45e3-ed3d-831e52ded4ab" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "tensor([[-15.2588]], grad_fn=)" + ] + }, + "metadata": {}, + "execution_count": 40 + } + ], + "source": [ + "#คำนวณ forward pass สำหรับตัวอย่าง 7\n", + "(train_x[7]*weights.T).sum(1)[:,None] + bias" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "22qYPEQDEwjC", + "outputId": "528c7509-1eea-49cf-d5b6-181fd62059ca" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "tensor([[ -6.2330],\n", + " [-10.6388],\n", + " [-20.8865],\n", + " ...,\n", + " [-15.9176],\n", + " [ -1.6866],\n", + " [-11.3568]], grad_fn=)" + ] + }, + "metadata": {}, + "execution_count": 41 + } + ], + "source": [ + "#คำนวณ forward pass สำหรับทุกตัวอย่าง\n", + "(train_x*weights.T).sum(1)[:,None] + bias" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "I65On42BpokB", + "outputId": "d77c1723-af03-485b-b036-c98e1b0f50af" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "tensor([[ -6.2330],\n", + " [-10.6388],\n", + " [-20.8865],\n", + " ...,\n", + " [-15.9176],\n", + " [ -1.6866],\n", + " [-11.3568]], grad_fn=)" + ] + }, + "metadata": {}, + "execution_count": 42 + } + ], + "source": [ + "#เขียนเป็นฟังชั่น; @ คือ matrix multiplication ใน pytorch\n", + "def linear1(xb): return xb@weights + bias\n", + "preds = linear1(train_x)\n", + "preds" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "XJKeTmrPzOr3" + }, + "source": [ + "#### Metric ใช้ Accuracy" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "SLnbggPopokB", + "outputId": "478badc3-ae34-444e-c7ea-ff028b18cb23" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "tensor([[False],\n", + " [False],\n", + " [False],\n", + " ...,\n", + " [ True],\n", + " [ True],\n", + " [ True]])" + ] + }, + "metadata": {}, + "execution_count": 43 + } + ], + "source": [ + "#ถ้าสมมุติว่า Predictions >0 ให้ทายเป็น 1 (เลข 3)\n", + "corrects = (preds>0.0).float() == train_y\n", + "corrects" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "5DVZlAVzJqbV", + "outputId": "299bf0b0-09cf-4822-a55a-9a43537c0888" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "Counter({0.0: 10788, 1.0: 1608})" + ] + }, + "metadata": {}, + "execution_count": 44 + } + ], + "source": [ + "from collections import Counter\n", + "Counter((preds>0.0).float().numpy()[:,0])" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "CiSVOFonpokB", + "outputId": "4f584989-ecc1-47ef-a7e1-b95075af4f18" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "0.5379961133003235" + ] + }, + "metadata": {}, + "execution_count": 45 + } + ], + "source": [ + "#จะเห็นได้ว่ามันถูกเครื่องๆ (เพราะเราเดาสุ่มด้วย weights ที่สุ่มมา)\n", + "corrects.float().mean().item()" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": { + "id": "RwkizTBqpokC" + }, + "outputs": [], + "source": [ + "#ถ้าเราเปลี่ยน weights เล็กน้อย\n", + "with torch.no_grad():\n", + " weights *= 5.0001\n", + " bias+=10" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "d5Kv_vFrpokC", + "outputId": "351fac1c-2a3a-485c-f48e-af448f3141c3" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "0.548402726650238" + ] + }, + "metadata": {}, + "execution_count": 47 + } + ], + "source": [ + "#ความแม่นยำก็เปลี่ยนนิดนึง\n", + "preds = linear1(train_x)\n", + "((preds>0.0).float() == train_y).float().mean().item()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "PTOZYvLRy-Vz" + }, + "source": [ + "#### Loss Function อย่างง่าย" + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "metadata": { + "id": "w7wVT8TjpokC" + }, + "outputs": [], + "source": [ + "trgts = tensor([1,0,1])\n", + "prds = tensor([0.9, 0.4, 0.2])" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": { + "id": "9HgV5HQppokC" + }, + "outputs": [], + "source": [ + "#loss function ที่จะเป็นจะเป็น Predictions ของ class ที่ผิด\n", + "#เพราะงั้นยิ่งทาย class ที่ผิดแบบมั่นใจมาก Loss ก็จะยิ่งสูง\n", + "def mnist_loss(predictions, targets):\n", + " return torch.where(targets==1, 1-predictions, predictions).mean()" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "5xGnIzeHpokC", + "outputId": "29656655-4d49-491a-abfa-b4f3c87a63c7" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "tensor([0.1000, 0.4000, 0.8000])" + ] + }, + "metadata": {}, + "execution_count": 50 + } + ], + "source": [ + "#if target == 1, loss is how far it is from 1\n", + "#if target == 0, loss is how far it is from 0\n", + "torch.where(trgts==1, 1-prds, prds)" + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "dhQH45dYpokC", + "outputId": "2a3c5978-ecea-493d-cb92-02b89b64c610" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "tensor(0.4333)" + ] + }, + "metadata": {}, + "execution_count": 51 + } + ], + "source": [ + "mnist_loss(prds,trgts)" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "uiUjD7aJpokD", + "outputId": "c2fabcb4-0df8-4d71-9196-d110404ea9bb" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "tensor(0.2333)" + ] + }, + "metadata": {}, + "execution_count": 52 + } + ], + "source": [ + "#ถ้าทายแม่นขึ้น loss ก็ลดลง\n", + "mnist_loss(tensor([0.9, 0.4, 0.8]), trgts)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "dqsrmegupokD" + }, + "source": [ + "### Activation Function - Sigmoid" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "-0DXOc8w0YeU" + }, + "source": [ + "ปัญหาอย่างนึงของเราตอนนี้คือโมเดลเราให้ค่า `Predictions` เป็นจำนวนจริง แต่ `Loss Function` ของเราต้องการค่าระหว่าง 0 ถึง 1 เราจึงใช้ `Activation Function` ที่ชื่อ Sigmoid เพื่อเปลี่ยนให้ตัวเลขอยู่ระหว่าง 0 ถึง 1\n", + "\n", + "`Activation Function` นั้นมีมากมายหลายประเภท ที่นิยมใช้ เช่น [ReLU](https://en.wikipedia.org/wiki/Rectifier_(neural_networks))(x) - ฟังชั่นที่บอกว่าถ้าน้อยกว่า 0 ให้เป็น นอกนั้นเป็น x, [tanh](https://mathworld.wolfram.com/HyperbolicTangent.html)(x) - ฟังชั่นที่เปลี่ยนค่าจำนวนจริงให้อยู่ระหว่าง -1 และ 1 เป็นต้น อ่านเพิ่มเกี่ยวกับ [Activation Functions](https://en.wikipedia.org/wiki/Activation_function)" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": { + "id": "sQIUTk5ZpokD" + }, + "outputs": [], + "source": [ + "def sigmoid(x): return 1/(1+torch.exp(-x))" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 265 + }, + "id": "TY9tdedYpokD", + "outputId": "f1b91b5a-9267-4d78-93e5-6fa068f979f2" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhV5b328e+PAAkkzAlTBgISZJQpgnVoHfAccICqVUGrpfqKWqeqPa0ee/StrXayrVqpU6UqteDYyhGsU51RIchMBMIcpgxAQgIJSfbv/SOxb8QoAXay9t65P9fF5R4We9+Sve/rybOetZa5OyIiEv1aBR1ARETCQ4UuIhIjVOgiIjFChS4iEiNU6CIiMaJ1UG+cnJzsmZmZQb29iEhUWrRoUZG7pzT0XGCFnpmZSU5OTlBvLyISlcxs01c9pykXEZEYcchCN7MZZlZgZiu+4nkzswfNLM/MlpnZqPDHFBGRQ2nMCP1JYPzXPD8ByKr7Mw14+OhjiYjI4Tpkobv7e8Cur9lkEvC01/oY6GxmvcIVUEREGiccc+ipwJZ69/PrHhMRkWbUrDtFzWyameWYWU5hYWFzvrWISMwLR6FvBdLr3U+re+xL3P0xd8929+yUlAaXUYqIyBEKxzr0OcD1ZjYbGAuUuPv2MLyuiEhUC4WcovJKdpZUsrO0goK9tf89Y1B3jkvrHPb3O2Shm9ks4FQg2czygbuANgDu/ggwDzgLyAP2Ad8Pe0oRkQhUUVVD/u59bNm9n/zd+9m6ez/b9uxne8l+tu2pYGdpBdWhL19zIqVDfDCF7u5TDvG8A9eFLZGISASprK5hY9E+1heWsaG4nI1F5Wws2semXeXsLK38wrZt41rRs1MCvTsnMLZvV3p2SqBnpwR6dKz9071DPMlJ8bRt3TS7LwM79F9EJJIcqA6RV1DGmp17Wb1zL2t37iWvoIzNu/ZRf5Cd0iGezG7tOSUrhYyu7cno2p70ru1I69KelKR4WrWywP4fVOgi0uKUV1azanspK7aWsGJrKSu3lbCusIyqmtrmbt3K6JucyJDenZg4IpVjUhI5JiWJzOREkuIjtzYjN5mISBiEQs66wjIWbdrN4s17WJq/hzU79/571J2cFM+Q3h059djuDOrVgUG9OpLZLbHJpkWakgpdRGJKVU2I5VtLWLBhF5+sL+bTzXso2V8FQOf2bRie1pn/HNKT49I6MSy1E907JgScOHxU6CIS1dyd1Tv38sHaIj7MK2LBhl2UH6gBoF9KIhOG9mR0ny6M7tOFvsmJmAU3x93UVOgiEnXKKqt5f00h76wu5N01heworQCgX3Ii541K5Rv9khnTtyspHeIDTtq8VOgiEhV2llbw+sodvJFbwMfrijlQE6JDQmtOyUrm1AHdOTkrmd6d2wUdM1AqdBGJWDtKKpi7fDuvLt/Oos27cYe+yYlMPSmTMwZ2Z3SfLrSOi76dl01FhS4iEaVkfxWvLt/Oy0u28fGGYtxhYM8O3DxuABOG9iSrR4egI0YsFbqIBK4m5HyYV8QLi/J5beUOKqtD9E1O5KYzsjh3eG+OSUkKOmJUUKGLSGB2llbw3MItzF64ha179tOpXRsuPj6dC0alcVxap5hekdIUVOgi0qzcnYUbd/Pk/A28tnInNSHnpP7duP2sgZw5uAfxreOCjhi1VOgi0iwOVIf436XbmPHhBlZuK6VjQmuuPLkvU8Zk0Dc5Meh4MUGFLiJNqqyymtkLNvPEBxvYXlJBVvck7j1vGN8e2Zv2bVVB4aR/TRFpEiX7q3hq/kae+GADJfurOKFfV+49fxinDkjR3HgTUaGLSFiVVlTxxPsbmPHhBvZWVDNuUHeuO60/IzO6BB0t5qnQRSQs9h+o4emPNvLwu+vYs6+K/xzSgxtOz2Joaqego7UYKnQROSo1IeelT/P53etr2FFawbcGpPCj/ziWYWkq8uamQheRIzY/r4ifz80ld3spw9M788DkEYzt1y3oWC2WCl1EDlv+7n3cMzeXV1fsILVzOx6cMpJzj+ulnZ0BU6GLSKNVVtfw2Lvrmf5OHgC3njmAq77Zj4Q2OhgoEqjQRaRRFmzYxe0vLWNdYTkThvbkp+cMJrWFn6420qjQReRrlVZU8ct5ucxasIW0Lu34y/eP57RjuwcdSxqgQheRr/T26gJuf3E5BXsruPqb/bhpXJaO7oxg+smIyJfsraji7v9dxfOL8snqnsSjl53E8PTOQceSQ1Chi8gXLNy4i5ufXcK2Pfv5wanHcNO4LJ0BMUqo0EUEgKqaEA+8uZY/vZNHWpf2PH/NiYzuo8P1o4kKXUTYumc/N85azKJNu7lwdBp3TRxCUrzqIdroJybSwr25aic/emEpVdUhHpg8gkkjUoOOJEdIhS7SQlXXhLjv9TU88u46BvfqyPRLR+lCE1FOhS7SAhWXVXLj7MV8mFfMlDEZ3HXuYB3tGQNU6CItzPL8Eq6emUNR+QF+853juCg7PehIEiatGrORmY03s9VmlmdmtzXwfIaZvW1mi81smZmdFf6oInK0Xlm2jQsfnY+Z8eI1J6rMY8whR+hmFgdMB84E8oGFZjbH3VfV2+ynwHPu/rCZDQbmAZlNkFdEjkAo5Nz/1loefGst2X268Mhlo0lOig86loRZY6ZcxgB57r4ewMxmA5OA+oXuQMe6252AbeEMKSJHrqKqhlufX8rcZdu5cHQavzhvqA4UilGNKfRUYEu9+/nA2IO2+b/A62Z2A5AIjAtLOhE5KrvKDzDt6RxyNu3mtgkDufqb/XTO8hjWqDn0RpgCPOnuacBZwEwz+9Jrm9k0M8sxs5zCwsIwvbWINGRTcTkXPDyfZVtLmH7JKK751jEq8xjXmELfCtTfc5JW91h9VwLPAbj7R0ACkHzwC7n7Y+6e7e7ZKSkpR5ZYRA5pxdYSLnh4Pnv2HWDWVWM5+7heQUeSZtCYQl8IZJlZXzNrC0wG5hy0zWbgDAAzG0RtoWsILhKA+XlFTH7sY+Jbx/HCtScyuk/XoCNJMzlkobt7NXA98BqQS+1qlpVmdreZTazb7FbgKjNbCswCprq7N1VoEWnYP1dsZ+pfFtK7cwIvXnsix6QkBR1JmlGjDixy93nULkWs/9id9W6vAk4KbzQRORwvLsrnv15Yyoj0zsyYejyd27cNOpI0Mx0pKhIDZn68if/5xwpO6t+Nxy/P1lWFWij91EWi3OPvreeeebmcMbA70y8dpXOytGAqdJEo9si76/jVq59x9rBe3D95BG3iwrUSWaKRCl0kSv3pnTx+88/VnDu8N3+4aDitVeYtnj4BIlHo8zKfqDKXevQpEIkyf35/Pb/552omjejN71XmUo8+CSJRZOZHG/nF3FzOGtaT312oMpcv0qdBJEo8l7OF/3l5JeMGdef+i0eqzOVL9IkQiQLzlm/ntheXcUpWMg9dMoq2rfXVlS/Tp0Ikwr2/tpCbZi9mVEYXHr1stNaZy1dSoYtEsEWbdjPt6UX0796BJ6YeryNA5Wup0EUi1Nqde7niyYX06BjP01eMoVO7NkFHkginQheJQNtL9vO9GQto27oVM68cS0oHXf9TDk2FLhJhSvZVMXXGQkorqnny+8eT3rV90JEkSqjQRSJIRVUNV83MYUNROY9dPpohvTsFHUmiiPawiESIUMj50fNLWbBhF3+cMpITj/nSVRxFvpZG6CIR4jevreaVZdu5bcJAzh3eO+g4EoVU6CIR4JlPNvHIu+u4dGwGV3+zX9BxJEqp0EUC9u6aQu58eSWnD+zOzyYOwcyCjiRRSoUuEqC1O/dy/TOfMqBHBx6covOzyNHRp0ckIMVllVzx1EIS2sbxxPeySYrXGgU5Oip0kQBUVtdw9cxFFJRW8vjl2fTu3C7oSBIDNCQQaWbuzh1/X0HOpt08dMlIRqR3DjqSxAiN0EWa2RMfbOCFRfncdEYW5xyn5YkSPip0kWb0zuoC7p2Xy4ShPbnpjKyg40iMUaGLNJP1hWXcMGsxx/bsyO8uGk6rVlqeKOGlQhdpBnsrqrjq6RzaxLXi8ctH67zm0iT0qRJpYqGQc/OzS9lYvI+/XjmWtC46e6I0DY3QRZrYg/9ay5u5O/np2YP4xjHdgo4jMUyFLtKE3li1k/vfXMsFo9KYemJm0HEkxqnQRZrI+sIybnl2CcNSO3HPeUN1jhZpcip0kSZQXlnNNX9dROs44+HvjiKhTVzQkaQFaFShm9l4M1ttZnlmdttXbHORma0ys5Vm9rfwxhSJHu7OT15cRl5BGQ9OGamdoNJsDrnKxczigOnAmUA+sNDM5rj7qnrbZAG3Aye5+24z695UgUUi3YwPN/LKsu38ePyxnJKVEnQcaUEaM0IfA+S5+3p3PwDMBiYdtM1VwHR33w3g7gXhjSkSHRZu3MUv5+XyH4N7cO23jgk6jrQwjSn0VGBLvfv5dY/VNwAYYGYfmtnHZjY+XAFFokXh3kque+ZT0rq0476LhmsnqDS7cB1Y1BrIAk4F0oD3zGyYu++pv5GZTQOmAWRkZITprUWCV10T4oZZn1JaUcVTV4yhY0KboCNJC9SYEfpWIL3e/bS6x+rLB+a4e5W7bwDWUFvwX+Duj7l7trtnp6RoblFix32vr+Hj9bu459vDGNSrY9BxpIVqTKEvBLLMrK+ZtQUmA3MO2uYf1I7OMbNkaqdg1ocxp0jEeit3J4+8u44pY9K5YHRa0HGkBTtkobt7NXA98BqQCzzn7ivN7G4zm1i32WtAsZmtAt4G/svdi5sqtEik2LJrH7c8t5QhvTty17lDgo4jLVyj5tDdfR4w76DH7qx324Fb6v6ItAiV1TVc97dPCbnzp0t18JAET2dbFDlC987NZVl+CY98dzR9uiUGHUdEh/6LHIm5y7bz1EebuPLkvowf2jPoOCKACl3ksG0sKucnLy5jRHpnfjJ+YNBxRP5NhS5yGCqqaufN41oZD10ykrat9RWSyKE5dJHDcM/cXFZuK+XPl2frpFsScTS8EGmkV5ZtY+bHm7jqlL6MG9wj6DgiX6JCF2mEjUXl3PbickZmdObHmjeXCKVCFzmEyuoarp9VO2/+xykjaROnr41EJs2hixzCvXNzWbG1lMc1by4RTkMNka/xzxX/f735mZo3lwinQhf5Clt27eO/XljG8LROWm8uUUGFLtKAA9Uhrp+1GICHLhml9eYSFTSHLtKA3772GUu37OHhS0eR3lXz5hIdNOwQOchbuTt5/P0NXHZCHyYM6xV0HJFGU6GL1LO9ZD+3Pr+Uwb06csfZg4KOI3JYVOgidaprQtw4azFV1SEeumSkzm8uUUdz6CJ17n9zLQs37uaBySPol5IUdByRw6YRugjw/tpCpr+Tx0XZaUwakRp0HJEjokKXFq+gtIKbn11C/5QkfjZxaNBxRI6YplykRasJOT98dgllldX87aoTaNdW8+YSvVTo0qI99K885q8r5jffOY4BPToEHUfkqGjKRVqsj9YV88Bba/j2iN5cODot6DgiR02FLi1S4d5Kbpy9mMxuifzivGGYWdCRRI6aplykxQmFnFueW0Lp/iqevmIMSfH6Gkhs0AhdWpw/vZPH+2uLuOvcIQzq1THoOCJho0KXFuXj9cX8/o01nDu8N1PGpAcdRySsVOjSYhSVVXLjrMX06ZbIvecN1by5xBwVurQINSHn5meXULK/iumXjKJDQpugI4mEnfYGSYvwp7dr581/ef4wBvfWvLnEJo3QJebNX1fEH96sXW8++XjNm0vsUqFLTCsoreDGWUvol5LEPVpvLjFOUy4Ss6praq8LWl5ZzayrxpKo9eYS4xo1Qjez8Wa22szyzOy2r9nuAjNzM8sOX0SRI/O7N9awYMMu7j1/KFk6T4u0AIcsdDOLA6YDE4DBwBQzG9zAdh2Am4BPwh1S5HC9uWonD7+zjiljMjhvpM7TIi1DY0boY4A8d1/v7geA2cCkBrb7OfBroCKM+UQO2+bifdz83BKGpnbkrnO/NPYQiVmNKfRUYEu9+/l1j/2bmY0C0t19bhiziRy2iqoarvnrIlqZ8fClo3VdUGlRjnovkZm1An4PTG3EttOAaQAZGRlH+9YiX+Du/M8/VrBqeyl/mXo86V3bBx1JpFk1ZoS+Fai/eDet7rHPdQCGAu+Y2UbgBGBOQztG3f0xd8929+yUlJQjTy3SgNkLt/D8onxuOL0/pw3sHnQckWbXmEJfCGSZWV8zawtMBuZ8/qS7l7h7srtnunsm8DEw0d1zmiSxSAOWbNnDXS+v5JsDUvjhuAFBxxEJxCEL3d2rgeuB14Bc4Dl3X2lmd5vZxKYOKHIoRWWVXPvXRXTvGM8DF48grpUOHpKWqVFz6O4+D5h30GN3fsW2px59LJHGqa4JccPfFrOr/AAvXnsiXRLbBh1JJDA6dE6i2q//+RkfrS/mvguHMzS1U9BxRAKlc7lI1PrH4q08/v4Gpp6YyXd0kWcRFbpEpxVbS/jJi8sY27crd5w9KOg4IhFBhS5Rp7iskqtnLqJbYlumXzqKNnH6GIuA5tAlyhyoDnHtXz+lqKySF645keSk+KAjiUQMFbpEDXfnrjkrWLBxFw9MHsGwNO0EFalPv6tK1Hhq/kZmLdjCdacdw6QRqYf+CyItjApdosIHa4v4+dxcxg3qwa1nHht0HJGIpEKXiJdXUMa1zyyif0oS908eQSsdCSrSIBW6RLRd5Qe48qmFxLduxRNTs0nSZeREvpK+HRKxKqtruGbmIraXVDB72gmkddHpcEW+jkboEpHcndtfXM6Cjbu478LhjMroEnQkkYinQpeI9Ic31/LS4q3ceuYAJg7vHXQckaigQpeI81zOFh58ay0XZ6dz/en9g44jEjVU6BJRPlhbxH+/tJxTspL5xXlDMdOKFpHGUqFLxFixtYSrZ+bQv3sSf9I5WkQOm74xEhE2FZcz9S8L6Ny+LU9dMYYOCW2CjiQSdbRsUQJXVFbJ92YsoDrkzL5iDD06JgQdSSQqaYQugdpbUcX3/7KQHaUVPPG94+nfPSnoSCJRS4UugamoquH/PJVD7vZSHr50NKP7aK25yNHQlIsEoqomxHXPfMqCjbu4/+IRnDawe9CRRKKeRujS7GpCzo+eX8pbnxVw96ShOhWuSJio0KVZhULO7S8t4+Ul2/jx+GO57IQ+QUcSiRkqdGk2tVccWslzOfnceEYWPzhVR4GKhJMKXZqFu3PP3FxmfryJq7/Zj5vHZQUdSSTmqNClyX1e5n/+YANTT8zktgkDdUi/SBPQKhdpUu7Oz1/JZcaHtWV+17mDVeYiTUSFLk3G3fnZ/67iyfkb+f5Jmdx5jspcpCmp0KVJ1IScO/6+nNkLt3DlyX356dmDVOYiTUyFLmFXVRPiR88v5eUl27jh9P7ccuYAlblIM1ChS1hVVNVww6zFvLFqJz8ZP5BrTz0m6EgiLYYKXcKmZH8VVz2dw8KNu7h70hAu/0Zm0JFEWpRGLVs0s/FmttrM8szstgaev8XMVpnZMjN7y8x0+F8LU1BawcWPfsTizbt5YPJIlblIAA5Z6GYWB0wHJgCDgSlmNvigzRYD2e5+HPAC8JtwB5XIlVdQxgWPzGfzrn3MmHq8LuosEpDGjNDHAHnuvt7dDwCzgUn1N3D3t919X93dj4G08MaUSPXJ+mIueHg++w/UMOuqEzglKyXoSCItVmMKPRXYUu9+ft1jX+VK4NWjCSXR4eUlW7nsiQV0S2rL339wEsPTOwcdSaRFC+tOUTP7LpANfOsrnp8GTAPIyMgI51tLMwqFnPvfXMOD/8pjbN+uPHrZaDq3bxt0LJEWrzGFvhVIr3c/re6xLzCzccAdwLfcvbKhF3L3x4DHALKzs/2w00rg9h2o5pZnl/LPlTu4cHQavzhvKPGt44KOJSI0rtAXAllm1pfaIp8MXFJ/AzMbCTwKjHf3grCnlIiwZdc+rp65iM92lPLTswdx5cl9dcCQSAQ5ZKG7e7WZXQ+8BsQBM9x9pZndDeS4+xzgt0AS8HzdF3yzu09swtzSzN5bU8iNsxdTE3KemHo8px2rS8aJRJpGzaG7+zxg3kGP3Vnv9rgw55IIEQo5D7+7jvteX82xPTrwyHdHk5mcGHQsEWmAjhSVr1RcVsmtzy/lndWFTBrRm1+eP4z2bfWREYlU+nZKgxZs2MUNsz5l974qfj5pCN89oY/my0UinApdvqC6JsRDb+fx4Ftr6dMtkRlTj2dI705BxxKRRlChy79tLCrnh88uYcmWPZw3MpW7Jw2hQ0KboGOJSCOp0AV3528LNnPP3FxatzL+OGUk5+p8LCJRR4XewuXv3sdtLy7ng7wiTurfjd9+Zzi9O7cLOpaIHAEVegtVE3Ke+WQTv371MwDuOW8ol4zJ0I5PkSimQm+BcreXcvtLy1myZQ8n90/ml+cPI71r+6BjichRUqG3IGWV1Tz41lpmfLCBju3a8IeLh/PtEakalYvECBV6C+DuvLxkG/fOy6VgbyUXZ6dz24SBdEnUGRJFYokKPcYt2rSbe+au4tPNexie1onHLs9mhM5bLhKTVOgxanPxPn792mfMXbadlA7x/PqCYVw4Op1WrTS9IhKrVOgxZmdpBX/811pmL9hCm7hW3HRGFtO+2Y/EeP2oRWKdvuUxomBvBY+/t56nP9pETciZPCadG07PokfHhKCjiUgzUaFHuR0lFTz63jr+9slmqmpCfHtEKj8cN4CMblqGKNLSqNCj1Nqde3nsvfX8Y8lWQg7nj0zlB6f1p6/OVS7SYqnQo4i782FeMX/5cANvfVZAQptWTBmTwVWn9NOBQSKiQo8GZZXV/H3xVp6ev5G1BWV0S2zLTWdkcfk3+tAtKT7oeCISIVToEcrdWbmtlGc+2cycJVspP1DDkN4due/C4ZxzXC8S2sQFHVFEIowKPcIUlVXyj8VbeWFRPp/t2EtCm1acc1xvLhmbwcj0zjpMX0S+kgo9ApRVVvPGqh28vGQb768toibkDE/rxN2ThjBpRCqd2ukiEyJyaCr0gOytqOJfnxXw6vIdvLOmgIqqEKmd23HVKf04f1QqA3p0CDqiiEQZFXoz2lFSwVuf7eTNVTv5cF0xB6pDdO8Qz0XZ6Uwc3ptRGV10aL6IHDEVehOqqgnx6abdvLOmkHdXF7JqeykAGV3bc9kJfZgwtKdKXETCRoUeRqGQk7ujlI/WFTN/XTGfrC+m/EANrVsZo/t04cfjj+XMQT3o3z1JOzdFJOxU6EehoqqGFVtLyNm0m5yNu1iwYRelFdUA9EtO5PxRaZzUvxsn9k+mY4J2bIpI01KhN1Io5GwsLmdZfglLtuxhyZY9rNpWyoGaEAB9kxM5a1gvxvbryti+3XShZRFpdir0BlRU1bB2Zxm520tZtb2UVdtKWbmthPIDNQC0axPHsLROfP+kTEb36cKoPl1I1hGbIhKwFl3o5ZXVrC8sJ69wL+sKylmzcy9rC8rYVFxOyGu3ad82joE9O/Cd0WkMSe3EsNROZHVPonVcq2DDi4gcJKYL3d3Zva+K/N372FS8j8279rG5eB8bisvZWFROwd7Kf28b18ro0609A3t24NzhvRnYswODenWkT9f2WoUiIlEhagvd3dlbWU1BaQU7SirZUVrBjpL9bN1TwfaS/Wzbs5/83fvZVzdN8rnkpHgyu7XnWwNSyExO5JiURPp3TyKjayJtW2vULSLRK+oK/dmFm5n+9joK9lZQURX60vPdEtvSu3M7MrslcnL/FNK6tCO1SzsyurYno2t7XYpNRGJWo9rNzMYDDwBxwJ/d/VcHPR8PPA2MBoqBi919Y3ij1uqWGM/w9M706BBP947xdO+QQM9OCfTqlECPjgk6C6GItFiHLHQziwOmA2cC+cBCM5vj7qvqbXYlsNvd+5vZZODXwMVNEXjc4B6MG9yjKV5aRCSqNWbSeAyQ5+7r3f0AMBuYdNA2k4Cn6m6/AJxhOhRSRKRZNabQU4Et9e7n1z3W4DbuXg2UAN3CEVBERBqnWZd1mNk0M8sxs5zCwsLmfGsRkZjXmELfCqTXu59W91iD25hZa6ATtTtHv8DdH3P3bHfPTklJObLEIiLSoMYU+kIgy8z6mllbYDIw56Bt5gDfq7v9HeBf7u7hiykiIodyyFUu7l5tZtcDr1G7bHGGu680s7uBHHefAzwBzDSzPGAXtaUvIiLNqFHr0N19HjDvoMfurHe7ArgwvNFERORw6Fh3EZEYYUFNdZtZIbDpCP96MlAUxjjholyHR7kOX6RmU67DczS5+rh7g6tKAiv0o2FmOe6eHXSOgynX4VGuwxep2ZTr8DRVLk25iIjECBW6iEiMiNZCfyzoAF9BuQ6Pch2+SM2mXIenSXJF5Ry6iIh8WbSO0EVE5CAqdBGRGBH1hW5mt5qZm1ly0FkAzOznZrbMzJaY2etm1jvoTABm9lsz+6wu29/NrHPQmQDM7EIzW2lmITMLfHmZmY03s9VmlmdmtwWdB8DMZphZgZmtCDpLfWaWbmZvm9mqup/hTUFnAjCzBDNbYGZL63L9LOhM9ZlZnJktNrNXwv3aUV3oZpYO/AewOegs9fzW3Y9z9xHAK8Cdh/oLzeQNYKi7HwesAW4POM/nVgDnA+8FHaTe1bkmAIOBKWY2ONhUADwJjA86RAOqgVvdfTBwAnBdhPx7VQKnu/twYAQw3sxOCDhTfTcBuU3xwlFd6MAfgB8DEbNn191L691NJEKyufvrdRcfAfiY2tMgB87dc919ddA56jTm6lzNzt3fo/akdxHF3be7+6d1t/dSW1IHX/ym2Xmtsrq7ber+RMT30MzSgLOBPzfF60dtoZvZJGCruy8NOsvBzOweM9sCXErkjNDruwJ4NegQEagxV+eSBphZJjAS+CTYJLXqpjWWAAXAG+4eEbmA+6kdhIaa4sUbdbbFoJjZm0DPBp66A/hvaqdbmt3X5XL3l939DuAOM7sduB64KxJy1W1zB7W/Kj/THJkam0uil5klAS8CPzzoN9TAuHsNMKJuX9HfzWyouwe6D8LMzgEK3H2RmZ3aFO8R0YXu7uMaetzMhgF9gaV116JOAz41szHuviOoXA14htrTDjdLoR8ql5lNBc4BzhUpRp4AAAE4SURBVGjOC5Acxr9X0BpzdS6px8zaUFvmz7j7S0HnOZi77zGzt6ndBxH0TuWTgIlmdhaQAHQ0s7+6+3fD9QZROeXi7svdvbu7Z7p7JrW/Go9qjjI/FDPLqnd3EvBZUFnqM7Px1P6qN9Hd9wWdJ0I15upcUsdqR1NPALnu/vug83zOzFI+X8VlZu2AM4mA76G73+7uaXWdNZnaK7uFrcwhSgs9wv3KzFaY2TJqp4QiYikX8BDQAXijbknlI0EHAjCz88wsH/gGMNfMXgsqS91O48+vzpULPOfuK4PK8zkzmwV8BBxrZvlmdmXQmeqcBFwGnF73mVpSN/oMWi/g7brv4EJq59DDvkQwEunQfxGRGKERuohIjFChi4jECBW6iEiMUKGLiMQIFbqISIxQoYuIxAgVuohIjPh/yzfYtX4+p3UAAAAASUVORK5CYII=\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "plot_function(torch.sigmoid, title='Sigmoid', min=-4, max=4)" + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": { + "id": "htbxxyxlpokD" + }, + "outputs": [], + "source": [ + "def mnist_loss(predictions, targets):\n", + " predictions = predictions.sigmoid() #ใส่ sigmoid ไป; pytorch tensor มี built-in function ให้แล้ว\n", + " return torch.where(targets==1, 1-predictions, predictions).mean()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "UUxkL3_2pokD" + }, + "source": [ + "### SGD and Mini-Batches" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "rxCnTrNRInAz" + }, + "source": [ + "หลายครั้งข้อมูลทั้งชุดของเราใหญ่เกินไปที่จะใส่เข้าไปใน memory ของเครื่อง เราจึงต้องใส่เข้าไปทีละ batch ด้วย `DataLoader`" + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "2-twSiAspokD", + "outputId": "c3b8269f-b27a-4fb0-bda5-cbe2e18f9deb" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "[tensor([ 3, 12, 8, 10, 2]),\n", + " tensor([ 9, 4, 7, 14, 5]),\n", + " tensor([ 1, 13, 0, 6, 11])]" + ] + }, + "metadata": {}, + "execution_count": 56 + } + ], + "source": [ + "#ทดลองใช้ Dataloader สุ่มตัวเลขจาก 0-14; batch ละ 5 ตัวอย่าง; ให้สุ่มด้วย\n", + "coll = range(15)\n", + "dl = DataLoader(coll, batch_size=5, shuffle=True)\n", + "list(dl)" + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "7W43iIU_pokE", + "outputId": "1b46408a-2fdc-40f7-a20d-fa1ab4c988a8" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(#26) [(0, 'a'),(1, 'b'),(2, 'c'),(3, 'd'),(4, 'e'),(5, 'f'),(6, 'g'),(7, 'h'),(8, 'i'),(9, 'j')...]" + ] + }, + "metadata": {}, + "execution_count": 57 + } + ], + "source": [ + "#สมมุติเรามี DataSet　แบบนี้\n", + "ds = L(enumerate(string.ascii_lowercase))\n", + "ds" + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "GOccHc_rpokE", + "outputId": "1d364e87-dedd-455f-8e95-96271d3e9f2e" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "[(tensor([17, 18, 10, 22, 8, 14]), ('r', 's', 'k', 'w', 'i', 'o')),\n", + " (tensor([20, 15, 9, 13, 21, 12]), ('u', 'p', 'j', 'n', 'v', 'm')),\n", + " (tensor([ 7, 25, 6, 5, 11, 23]), ('h', 'z', 'g', 'f', 'l', 'x')),\n", + " (tensor([ 1, 3, 0, 24, 19, 16]), ('b', 'd', 'a', 'y', 't', 'q')),\n", + " (tensor([2, 4]), ('c', 'e'))]" + ] + }, + "metadata": {}, + "execution_count": 58 + } + ], + "source": [ + "#ก็ใส่เข้าไปใน dataloader ได้เช่นกัน\n", + "dl = DataLoader(ds, batch_size=6, shuffle=True)\n", + "list(dl)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "tvWlUYU8pokE" + }, + "source": [ + "## Putting It All Together" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "fjUk_EOS0deu" + }, + "source": [ + "#### Initialize weights" + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "metadata": { + "id": "lK9dlPP4pokE" + }, + "outputs": [], + "source": [ + "weights = init_params((28*28,1))\n", + "bias = init_params(1)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "L4ejZTDi0fds" + }, + "source": [ + "#### Initialize dataloader" + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "XesYQeEUpokE", + "outputId": "b0490d69-191f-49a7-c487-eb96763a85a3" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(torch.Size([256, 784]), torch.Size([256, 1]))" + ] + }, + "metadata": {}, + "execution_count": 60 + } + ], + "source": [ + "#สร้าง dataloader สำหรับ train\n", + "dl = DataLoader(dset, batch_size=256, shuffle=True)\n", + "xb,yb = first(dl)\n", + "xb.shape,yb.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "metadata": { + "id": "hlw0qizVpokE" + }, + "outputs": [], + "source": [ + "#เราจะไม่ shuffle validation set\n", + "valid_dl = DataLoader(valid_dset, batch_size=256, shuffle=False)" + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "opSgyb70pokF", + "outputId": "3f99d56f-92b8-4b35-f887-72298850466e" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "torch.Size([4, 784])" + ] + }, + "metadata": {}, + "execution_count": 62 + } + ], + "source": [ + "#(batch size, 28*28)\n", + "batch = train_x[:4]\n", + "batch.shape" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "d_OZXa2l0iwq" + }, + "source": [ + "#### Forward pass" + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "IQ5eVzE5pokF", + "outputId": "8c2e6586-aa3f-4a20-ef4c-7d0a4b8b0d1b" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "torch.Size([256, 1])" + ] + }, + "metadata": {}, + "execution_count": 63 + } + ], + "source": [ + "def linear1(xb): return xb@weights + bias\n", + "preds = linear1(xb)\n", + "preds.shape" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "P0dawUHM0kpf" + }, + "source": [ + "#### Calculate loss" + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "_zK9HJE7pokF", + "outputId": "5dc8119e-fac6-4fb6-d34f-67632d490ec2" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "tensor(0.7690, grad_fn=)" + ] + }, + "metadata": {}, + "execution_count": 64 + } + ], + "source": [ + "def mnist_loss(predictions, targets):\n", + " predictions = predictions.sigmoid()\n", + " return torch.where(targets==1, 1-predictions, predictions).mean()\n", + "loss = mnist_loss(preds, xb)\n", + "loss" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "tMIHxJcw0mW5" + }, + "source": [ + "#### Backward pass - get gradients and update weights" + ] + }, + { + "cell_type": "code", + "execution_count": 65, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "lMdSQrorpokF", + "outputId": "65a29105-64da-4008-a8a3-37b98344972e" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(torch.Size([784, 1]), tensor(0.0049), tensor([0.0425]))" + ] + }, + "metadata": {}, + "execution_count": 65 + } + ], + "source": [ + "loss.backward()\n", + "weights.grad.shape,weights.grad.mean(), bias.grad" + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "metadata": { + "id": "K4x5HLp4pokF" + }, + "outputs": [], + "source": [ + "#คำนวณ gradient ด้วย .backward()\n", + "def calc_grad(xb, yb, model):\n", + " preds = model(xb)\n", + " loss = mnist_loss(preds, yb)\n", + " loss.backward()" + ] + }, + { + "cell_type": "code", + "execution_count": 67, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "CZzqYNtgpokF", + "outputId": "8c50d09c-6e74-4943-ce7a-a9ec8dee08f7" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(tensor(-0.0007), tensor([0.0070]))" + ] + }, + "metadata": {}, + "execution_count": 67 + } + ], + "source": [ + "calc_grad(batch, train_y[:4], linear1)\n", + "weights.grad.mean(),bias.grad" + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "R5yY7Az-pokF", + "outputId": "9596cc88-13f5-4f32-c0d4-41a9c2de62c4" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(tensor(-0.0064), tensor([-0.0285]))" + ] + }, + "metadata": {}, + "execution_count": 68 + } + ], + "source": [ + "calc_grad(batch, train_y[:4], linear1)\n", + "weights.grad.mean(),bias.grad" + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "metadata": { + "id": "R-Bzu26wpokG" + }, + "outputs": [], + "source": [ + "#เราจำเป็นต้อง reset gradients เพื่อไม่ให้มันบวกเพิ่มขึ้นเรือ่ยๆ\n", + "weights.grad.zero_();\n", + "bias.grad.zero_();" + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "metadata": { + "id": "gKkqYbaqpokG" + }, + "outputs": [], + "source": [ + "#เทรน 1 epoch\n", + "def train_epoch(model, lr, params):\n", + " #โยนทุก batch ให้โมเดลที่ละ batch จนหมด\n", + " for xb,yb in dl:\n", + " #คำนวน loss และ gradients\n", + " calc_grad(xb, yb, model)\n", + " #เปลี่ยน weights ด้วย gradient * learning rate (lr)\n", + " for p in params:\n", + " p.data -= p.grad*lr\n", + " p.grad.zero_()" + ] + }, + { + "cell_type": "code", + "execution_count": 71, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "2XWh1J1ppokG", + "outputId": "cfe8a98d-d53f-45e8-fed6-bb81899f0325" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "tensor(0.5469)" + ] + }, + "metadata": {}, + "execution_count": 71 + } + ], + "source": [ + "#คิด accuracy\n", + "((preds>0.0).float() == yb).float().mean()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4ZfrHoey0w2z" + }, + "source": [ + "### Calculate metric (accuracy in this case)" + ] + }, + { + "cell_type": "code", + "execution_count": 72, + "metadata": { + "id": "5QPI55L4pokG" + }, + "outputs": [], + "source": [ + "def batch_accuracy(xb, yb):\n", + " preds = xb.sigmoid()\n", + " correct = (preds>0.5) == yb\n", + " return correct.float().mean()" + ] + }, + { + "cell_type": "code", + "execution_count": 73, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "8UVCx6RApokG", + "outputId": "fcdbc646-2421-4dfe-bb67-6d51e28d62ef" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "tensor(0.5000)" + ] + }, + "metadata": {}, + "execution_count": 73 + } + ], + "source": [ + "batch_accuracy(linear1(batch), train_y[:4])" + ] + }, + { + "cell_type": "code", + "execution_count": 74, + "metadata": { + "id": "GbJQiOgIpokG" + }, + "outputs": [], + "source": [ + "#ทำการ validate ด้วย validation set\n", + "def validate_epoch(model):\n", + " accs = [batch_accuracy(model(xb), yb) for xb,yb in valid_dl]\n", + " return round(torch.stack(accs).mean().item(), 4)" + ] + }, + { + "cell_type": "code", + "execution_count": 75, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "txrLkdJQpokH", + "outputId": "eeafe072-31d3-438f-e99e-f2377b03eac0" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "0.5484" + ] + }, + "metadata": {}, + "execution_count": 75 + } + ], + "source": [ + "validate_epoch(linear1)" + ] + }, + { + "cell_type": "code", + "execution_count": 76, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "HRIapIkfpokH", + "outputId": "8537b6d8-70fc-4d5b-cca9-40ab62cf8d2e" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "0.9253" + ] + }, + "metadata": {}, + "execution_count": 76 + } + ], + "source": [ + "#เทรนไป 1 epoch; accuracy เพิ่มเกือบเท่าตัว!\n", + "lr = 1.\n", + "\n", + "params = weights, bias\n", + "train_epoch(linear1, lr, params)\n", + "validate_epoch(linear1)" + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "hvBUTwULpokH", + "outputId": "79511794-ad36-4c6b-a2a9-02df71aa8e64" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "0.9554 0.9608 0.9632 0.9657 0.9681 0.9696 0.9716 0.973 0.974 0.974 0.975 0.9775 0.9745 0.976 0.9779 0.9779 0.9784 0.9784 0.9784 0.9789 " + ] + } + ], + "source": [ + "#เทรนไป 20 epoch; ทายถูกเกือบหมด\n", + "for i in range(20):\n", + " train_epoch(linear1, lr, params)\n", + " print(validate_epoch(linear1), end=' ')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "uVt65caepokH" + }, + "source": [ + "### Optimizer as a class" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "gyvY33oqMdZk" + }, + "source": [ + "หลายครั้งเรารวบรวมหน้าที่การ update weights มาเป็น class ชื่อ optimizer" + ] + }, + { + "cell_type": "code", + "execution_count": 78, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "ETZ0sdlXpokH", + "outputId": "acb76f13-b382-48dd-d2a4-f19d8b93b984" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "Linear(in_features=784, out_features=1, bias=True)" + ] + }, + "metadata": {}, + "execution_count": 78 + } + ], + "source": [ + "#nn.Linear ก็คือฟังชั่นที่ x*W.T + b เราเขียนเมื่อกี้เลย\n", + "linear_model = nn.Linear(28*28,1, bias=True)\n", + "linear_model" + ] + }, + { + "cell_type": "code", + "execution_count": 79, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Op_9VxfipokH", + "outputId": "52bc6520-0ddf-4954-9f5e-083a5ab08813" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(torch.Size([1, 784]), torch.Size([1]))" + ] + }, + "metadata": {}, + "execution_count": 79 + } + ], + "source": [ + "w,b = linear_model.parameters()\n", + "w.shape,b.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "metadata": { + "id": "HDPzEqO-pokI" + }, + "outputs": [], + "source": [ + "#optimizer　แบบง่ายที่สุด\n", + "class BasicOptim:\n", + " def __init__(self,params,lr): \n", + " self.params,self.lr = list(params),lr\n", + "\n", + " #step ทำการ update weights\n", + " def step(self, *args, **kwargs):\n", + " for p in self.params: p.data -= p.grad.data * self.lr\n", + "\n", + " #zero grad เพื่อทำการรีเซต gradients\n", + " def zero_grad(self, *args, **kwargs):\n", + " for p in self.params: p.grad = None" + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "metadata": { + "id": "UIAhC389pokI" + }, + "outputs": [], + "source": [ + "opt = BasicOptim(linear_model.parameters(), lr)" + ] + }, + { + "cell_type": "code", + "execution_count": 82, + "metadata": { + "id": "zNokDJCupokI" + }, + "outputs": [], + "source": [ + "def train_epoch(model):\n", + " for xb,yb in dl:\n", + " calc_grad(xb, yb, model)\n", + " #เปลี่ยนโค้ดเป็น optimizer แทน\n", + " opt.step()\n", + " opt.zero_grad()" + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "pUr7KQecpokI", + "outputId": "03eae252-fa47-49ef-dae7-7cf2500497e5" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "0.3567" + ] + }, + "metadata": {}, + "execution_count": 83 + } + ], + "source": [ + "validate_epoch(linear_model)" + ] + }, + { + "cell_type": "code", + "execution_count": 84, + "metadata": { + "id": "wQptVRJfpokI" + }, + "outputs": [], + "source": [ + "def train_model(model, epochs):\n", + " for i in range(epochs):\n", + " train_epoch(model)\n", + " print(validate_epoch(model), end=' ')" + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "kRbgjPURpokI", + "outputId": "8a680b82-56a8-4e0a-8444-e5dde4b97520" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "0.9706 0.975 0.975 0.9779 0.9779 0.9784 0.9794 0.9794 0.9799 0.9804 0.9804 0.9804 0.9804 0.9814 0.9819 0.9823 0.9823 0.9819 0.9823 0.9828 " + ] + } + ], + "source": [ + "#ได้ผลดีเหมือนเดิม\n", + "train_model(linear_model, 20)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "rBjvWLYwNII8" + }, + "source": [ + "## ทำงานง่ายขึ้นด้วย PyTorch" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "60qaqoMbNMA4" + }, + "source": [ + "Pytorch ได้ทำการเปลี่ยนฟังชั่นและ class ต่างๆที่เราใช้งานเป็นฟังชั่นพื้นฐานของ package ให้เราเรียกใช้ได้" + ] + }, + { + "cell_type": "code", + "execution_count": 86, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "DY1kWv7lpokI", + "outputId": "21c7a564-f0be-4d80-e07c-ff0fd34667e1" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "0.9726 0.9755 0.976 0.976 0.977 0.9764 0.9789 0.9799 0.9789 0.9789 0.9799 0.9799 0.9794 0.9803 0.9803 0.9808 0.9803 0.9809 0.9823 0.9813 " + ] + } + ], + "source": [ + "linear_model = nn.Linear(28*28,1,bias=False) #เหมือนกับ init_weights และการคูณ matrix ที่เราเพิ่งทำไป\n", + "opt = SGD(linear_model.parameters(), lr) #เหมือนกับ BasicOptim ที่เราเพิ่งทำไป\n", + "\n", + "#เทรนโมเดลได้ผลดีเหมือนเดิม\n", + "train_model(linear_model, 20)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "6ctjZxC3Nxi8" + }, + "source": [ + "## ใช้ fastai ให้สะดวกยิ่งขึ้น" + ] + }, + { + "cell_type": "code", + "execution_count": 87, + "metadata": { + "id": "jtzVcFkxpokJ" + }, + "outputs": [], + "source": [ + "dls = DataLoaders(dl, valid_dl)\n", + "#รวมทุกอย่างเข้าด้วยกันด้วย class Learner\n", + "learn = Learner(dls, nn.Linear(28*28,1), opt_func=SGD,\n", + " loss_func=mnist_loss, metrics=batch_accuracy)" + ] + }, + { + "cell_type": "code", + "execution_count": 88, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 676 + }, + "id": "0WcORvumpokJ", + "outputId": "0a15116b-a4ca-4557-e01e-0be547013147" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "\n", + "\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
epochtrain_lossvalid_lossbatch_accuracytime
00.0596730.0413070.97203100:00
10.0403680.0348970.97546600:00
20.0326610.0323650.97497500:00
30.0282520.0294630.97742900:00
40.0251780.0284480.97792000:00
50.0236420.0276700.97792000:00
60.0226530.0263400.97841000:00
70.0214880.0252640.97988200:00
80.0209880.0246700.97939200:00
90.0204300.0242370.97939200:00
100.0195260.0239210.98037300:00
110.0190020.0236560.98037300:00
120.0188140.0236990.98135400:00
130.0180700.0226860.98135400:00
140.0174310.0227080.98184500:00
150.0173560.0224090.98184500:00
160.0172960.0219270.98135400:00
170.0169580.0218550.98184500:00
180.0170050.0216770.98184500:00
190.0165770.0217020.98184500:00
" + ] + }, + "metadata": {} + } + ], + "source": [ + "learn.fit(20, lr=lr)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "H_UoKZ24pokJ" + }, + "source": [ + "## สร้าง Architecture ที่ Deep ขึ้น" + ] + }, + { + "cell_type": "code", + "execution_count": 89, + "metadata": { + "id": "4s1EGNszpokK" + }, + "outputs": [], + "source": [ + "#แทนที่จะทำ matrix multipilcation + bias ครั้งเดียว เราใส่ ReLU activation ไปเพิ่ม\n", + "def simple_net(xb): \n", + " res = xb@w1 + b1\n", + " res = res.max(tensor(0.0)) #นี่คือ relu\n", + " res = res@w2 + b2 #แล้วทำ matrix multiplication + bias อีกรอบ\n", + " return res" + ] + }, + { + "cell_type": "code", + "execution_count": 90, + "metadata": { + "id": "bKDW0LWlpokK" + }, + "outputs": [], + "source": [ + "w1 = init_params((28*28,30))\n", + "b1 = init_params(30)\n", + "w2 = init_params((30,1))\n", + "b2 = init_params(1)" + ] + }, + { + "cell_type": "code", + "execution_count": 91, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 265 + }, + "id": "Fza6OpHHpokK", + "outputId": "b23d6134-c2bc-4d4f-d92c-af0c8a2c14df" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU9b3/8ddHVlmUJRHZF0EBRSCkoKJ1q4pUxa6C2KrVUhFqrd2oXrU/7GbtdceF2/KwLbs7tSjiSq2FEkJYZQkgS0AS9n1J8vn9MYd7x5iQSZjJmZm8n4/HPDJzlpn3HMJnTr7nzOeYuyMiIunrpLADiIhIYqnQi4ikORV6EZE0p0IvIpLmVOhFRNJc3bADlCcjI8M7deoUdgwRkZSxYMGCbe6eWd68pCz0nTp1IicnJ+wYIiIpw8zWVzRPQzciImlOhV5EJM2p0IuIpDkVehGRNKdCLyKS5iot9GbW3szeN7PlZrbMzH5UzjJmZk+aWb6ZLTazrKh5N5vZ6uB2c7zfgIiIHF8sp1cWAz9x91wzawosMLPZ7r48apmrgW7BbQDwLDDAzFoADwLZgAfrznD3nXF9FyIiUqFK9+jdfYu75wb39wKfAG3LLDYE+KtHzAWamVlr4CpgtrvvCIr7bGBQXN+BiEga+M+6Hfzpn2tJROv4Ko3Rm1knoC8wr8ystsDGqMebgmkVTS/vuUeYWY6Z5RQVFVUllohISivce4hRk3OZNG8DB4+WxP35Yy70ZtYEeBm42933xDuIu49392x3z87MLPdbvCIiaae4pJQfTl7I3kNHefamLBrVj3/DgpgKvZnVI1LkJ7n7K+UsUgC0j3rcLphW0XQREQH++PYq5q3bwW+u70X3009JyGvEctaNAX8GPnH3RytYbAbw3eDsm/OA3e6+BZgFXGlmzc2sOXBlME1EpNabvXwrz324hmH9O/CNfu0S9jqx/I0wEPgOsMTM8oJp9wIdANz9OWAmMBjIBw4AtwbzdpjZQ8D8YL2x7r4jfvFFRFLT+u37uWd6Hue0PYUHr+2Z0NeqtNC7+0eAVbKMA6MqmDcBmFCtdCIiaejQ0RJGTszFgGeH96NhvToJfb2kbFMsIpLOHnx9Gcu37GHCLdm0b9Eo4a+nFggiIjVoes5GpuVsZPSlXbmse6saeU0VehGRGrJs827uf20pA7u25MdXnFljr6tCLyJSA3YfPMqdk3Jp3qg+TwztS52TjnvoM640Ri8ikmDuzk9fXETBzoNM+8F5ZDRpUKOvrz16EZEEe37OWmYv38q9g3vQr2OLGn99FXoRkQSau3Y7f3hrBV/t1ZpbB3YKJYMKvYhIghTuOcToyQvplNGYh795LpFGAzVPY/QiIglQXFLK6CkL2X+4mMnfH0CTBuGVWxV6EZEEeGTWSv6zbgeP3dCbM1s1DTWLhm5EROJs1rLPeH7OWoYP6MDX+iauWVmsVOhFROLo0237+en0RZzb7lQeSHCzslip0IuIxMmhoyWMnJRLnTrGM8OzaFA3sc3KYqUxehGROLn/taWs+GwPE275Eu2aJ75ZWay0Ry8iEgfT5m/gxQWb+OGlXbn0rNPCjvM5KvQiIidoacFu7n99GRd1y+BHX6m5ZmWxqnToxswmANcAhe5+TjnzfwYMj3q+HkBmcHWpT4G9QAlQ7O7Z8QouIpIMjjUra9m4Po/f0KdGm5XFKpY9+heAQRXNdPdH3L2Pu/cBfgl8WOZygZcG81XkRSStlJY6P5m+iC27DzJueBYta7hZWawqLfTuPgeI9Tqvw4ApJ5RIRCRFPD9nLe98spX7Bvcgq0PzsONUKG5j9GbWiMie/8tRkx1428wWmNmIStYfYWY5ZpZTVFQUr1giIgnx8ZptPDJrBdec25qbL+gUdpzjiufB2GuBf5UZtrnQ3bOAq4FRZvblilZ29/Hunu3u2ZmZmXGMJSISX1v3HOKuKQvpnNGYh78RXrOyWMWz0A+lzLCNuxcEPwuBV4H+cXw9EZEad7SklNGTczlwpITnbupH4xCblcUqLoXezE4FLgZej5rW2MyaHrsPXAksjcfriYiE5Q9vrWD+pzv53dd70S3kZmWxiuX0yinAJUCGmW0CHgTqAbj7c8FiXwPedvf9Uau2Al4N/qSpC0x297fiF11EpGa9tXQL//PPdXznvI4M6dM27Dgxq7TQu/uwGJZ5gchpmNHT1gK9qxtMRCSZrC3ax09fXEzv9s34r2t6hB2nSvTNWBGRShw8UsKdk3Kpl2TNymKV/EcRRERC5O7c99oSVm7dywu39qdts5PDjlRl2qMXETmOqfM38kpuAXdd1o2Lz0zNU79V6EVEKrC0YDcPzog0K7vr8m5hx6k2FXoRkXLsPnCUOyYuIKNxfZ4Y2jcpm5XFSmP0IiJllJY690zPY+ueQ0z/wfm0aFw/7EgnRHv0IiJlPPvhGt5dUch/fbUnfZO4WVmsVOhFRKL8K38b//32Sq7t3Ybvnt8x7DhxoUIvIhL4bHekWVmXzCb8/uu9kr5ZWaw0Ri8iwv81Kzt4tISpw7NSollZrNLnnYiInIDfv7mCnPU7eXJY35RpVhYrDd2ISK03c8kW/vzROm65oBPX9W4Tdpy4U6EXkVptTdE+fv7SYvp2aMa9g1OrWVmsVOhFpNY6cKSYkRMXUL/uSYy7MYv6ddOzJGqMXkRqJXfnvleXsrpwH3/9Xn/apGCzslil58eXiEglJs3bwKsLC7j78jO5qFtqNiuLVaWF3swmmFmhmZV7GUAzu8TMdptZXnB7IGreIDNbaWb5ZjYmnsFFRKpr8aZdjP37ci4+M5MfXtY17DgJF8se/QvAoEqW+ae79wluYwHMrA4wDrga6AkMM7OeJxJWRORE7dx/hJETc8ls2oDHb+jDSSncrCxWlRZ6d58D7KjGc/cH8t19rbsfAaYCQ6rxPCIicVFa6vx4eh6Few8xbngWzVO8WVms4jVGf76ZLTKzN83s7GBaW2Bj1DKbgmnlMrMRZpZjZjlFRUVxiiUi8n+efj+fD1YW8cA1PenTvlnYcWpMPAp9LtDR3XsDTwGvVedJ3H28u2e7e3ZmZnofGBGRmvfR6m089s4qru/ThpvOS49mZbE64ULv7nvcfV9wfyZQz8wygAKgfdSi7YJpIiI1avOug9w1dSFdM5vw2zRqVharEy70Zna6BVvNzPoHz7kdmA90M7POZlYfGArMONHXExGpiiPFpYyanMvhoyU8951+NKpf+74+VOk7NrMpwCVAhpltAh4E6gG4+3PAN4GRZlYMHASGursDxWY2GpgF1AEmuPuyhLwLEZEK/HbmJyzcsItxN2ZxRmaTsOOEotJC7+7DKpn/NPB0BfNmAjOrF01E5MS8sXgzL3z8KbcO7MRXz20ddpzQ6JuxIpKW8gv38YuXFpPVoRm/vDo9m5XFSoVeRNLO/sORZmUN6tVh3PD0bVYWq9p3VEJE0pq7c++rS8gv2sffvjeA1qemb7OyWNXujzkRSTsT567n9bzN3POVM7mwW0bYcZKCCr2IpI28jbsY+8ZyLjkrk1GXpn+zslip0ItIWti5/wijJuVyWtOGtaZZWaw0Ri8iKa+01Ll7Wh5Few/z0sjzadaodjQri5X26EUk5T31Xj4fririgWt7cm672tOsLFYq9CKS0uasKuLxd1fx9b5tGT6gQ9hxkpIKvYikrIJdB/nR1IWceVpTfvO12tesLFYq9CKSko4UlzJqUi5HS5xnb8ri5Pp1wo6UtHQwVkRS0m/+sZy8jbt4ZngWXWpps7JYaY9eRFLOjEWb+cu/13P7hZ0Z3Kv2NiuLlQq9iKSU1Vv3MublxWR3bM4vru4edpyUoEIvIilj/+FiRk7KpVH9SLOyenVUwmJR6VYyswlmVmhmSyuYP9zMFpvZEjP72Mx6R837NJieZ2Y58QwuIrWLuzPmlSWsLdrHk0P70uqUhmFHShmxfBy+AAw6zvx1wMXu3gt4CBhfZv6l7t7H3bOrF1FEBP767/X8fdFmfnLlWVzQVc3KqiKWK0zNMbNOx5n/cdTDuUQuAi4iEje5G3by638s5/LupzHy4jPCjpNy4j3AdRvwZtRjB942swVmNuJ4K5rZCDPLMbOcoqKiOMcSkVS1Y/8RRk/KpdUpDXn022pWVh1xO4/ezC4lUugvjJp8obsXmNlpwGwzW+Huc8pb393HEwz7ZGdne7xyiUjqKil1fjR1Idv2H+GVkRdwaqN6YUdKSXHZozezc4E/AUPcffux6e5eEPwsBF4F+sfj9USkdnjy3dX8c/U2/t91Z3NO21PDjpOyTrjQm1kH4BXgO+6+Kmp6YzNreuw+cCVQ7pk7IiJlfbCykCffW803stox9Evtw46T0iodujGzKcAlQIaZbQIeBOoBuPtzwANAS+CZoKFQcXCGTSvg1WBaXWCyu7+VgPcgImlm084D3D0tj7NaNeXX15+jZmUnKJazboZVMv924PZypq8Fen9xDRGRih0uLmHU5IWUlDjP3tRPzcriQE3NRCSp/PqNT1i0cRfP3ZRF54zGYcdJC/r+sIgkjdfzCvjb3PV8/6LODDpHzcriRYVeRJLCqq17GfPyEr7UqTk/H6RmZfGkQi8iodt3uJiRExfQuEFdnr5RzcriTVtTRELl7vzi5cWs27afp4apWVkiqNCLSKhe+PhT/rF4Cz+7qjvnn9Ey7DhpSYVeREKzYP1OfvOPT/hKj1bccXGXsOOkLRV6EQnF9n2HGT05lzbNTua/v91bX4pKIJ1HLyI1LtKsLI/tx5qVnaxmZYmkPXoRqXFPvLOKj/K38dAQNSurCSr0IlKj3l9RyJPv5fOtfu244Usdwo5TK6jQi0iN2bgj0qysR+tTeOj6c8KOU2uo0ItIjYg0K8ultNR5dngWDeupWVlN0cFYEakRY/++nMWbdvP8d/rRSc3KapT26EUk4V5duIlJ8zbwgy934aqzTw87Tq2jQi8iCbXys73c+8pS+nduwc+uOivsOLVSTIXezCaYWaGZlXspQIt40szyzWyxmWVFzbvZzFYHt5vjFVxEkt/eQ0cZOXEBTRrW5elhfamrZmWhiHWrvwAMOs78q4FuwW0E8CyAmbUgcunBAUQuDP6gmTWvblgRSR3HmpWt33GAp4f15TQ1KwtNTIXe3ecAO46zyBDgrx4xF2hmZq2Bq4DZ7r7D3XcCszn+B4aIpIkJ//qUmUs+4+dXncWALmpWFqZ4/R3VFtgY9XhTMK2i6V9gZiPMLMfMcoqKiuIUS0TCsGD9Dn438xOu7NmKEV9Ws7KwJc2AmbuPd/dsd8/OzMwMO46IVNO2fYe5c1IubZufzCPfUrOyZBCvQl8AtI963C6YVtF0EUlDkWZlC9l14CjPDu+nZmVJIl6Ffgbw3eDsm/OA3e6+BZgFXGlmzYODsFcG00QkDT02exX/yt/OQ9efQ882p4QdRwIxfTPWzKYAlwAZZraJyJk09QDc/TlgJjAYyAcOALcG83aY2UPA/OCpxrr78Q7qikiKem/FVp5+P5+hX2rPt7PbV76C1JiYCr27D6tkvgOjKpg3AZhQ9Wgikio27jjA3VPzOLvNKfzqurPDjiNlJM3BWBFJTYeOljBy0gIAnh3eT83KkpCamonICRn7xnKWFuzhf76bTYeWjcKOI+XQHr2IVNsruZuYPG8Dd1x8Blf0bBV2HKmACr2IVMuKz/Zw76tLGNC5BT+98syw48hxqNCLSJXtOXSUkRNzOaVhPZ66Uc3Kkp3G6EWkStydn7+4mA07DjDl++dxWlM1K0t2+hgWkSr580freGvZZ4wZ1J3+nVuEHUdioEIvIjGb/+kOfvfmCgadfTq3X9Q57DgSIxV6EYlJ0d7DjJqUS/vmJ/OHb52rZmUpRGP0IlKp4pJS7pqykD2HjvKX7/XnlIZqVpZKVOhFpFKPzl7Fv9du54/f6k2P1mpWlmo0dCMixzV7+Vae+WANw/q355v92oUdR6pBhV5EKrRh+wHumZ7HOW1P4cFr1awsVanQi0i5jjUrM9SsLNVpjF5EyvWrGctYtnkPf745m/Yt1KwslWmPXkS+4MWcjUydv5E7LzmDy3uoWVmqi6nQm9kgM1tpZvlmNqac+Y+ZWV5wW2Vmu6LmlUTNmxHP8CISf8s37+G/XlvK+V1acs8ValaWDiodujGzOsA44ApgEzDfzGa4+/Jjy7j7j6OW/yHQN+opDrp7n/hFFpFE2XPoKHdOWkCzRvV4cpialaWLWP4V+wP57r7W3Y8AU4Ehx1l+GDAlHuFEpOa4Oz97cRGbdh5k3I1ZZDZtEHYkiZNYCn1bYGPU403BtC8ws45AZ+C9qMkNzSzHzOaa2fUVvYiZjQiWyykqKoohlojE0//8cy2zlm1lzNXdye6kZmXpJN5/lw0FXnL3kqhpHd09G7gReNzMzihvRXcf7+7Z7p6dmZkZ51gicjzz1m7n4bdWMrjX6dx2oZqVpZtYCn0B0D7qcbtgWnmGUmbYxt0Lgp9rgQ/4/Pi9iISscO8hRk9ZSMcWjXj4G2pWlo5iKfTzgW5m1tnM6hMp5l84e8bMugPNgX9HTWtuZg2C+xnAQGB52XVFJBzFJaX8cPJC9h46yjM3ZdFUzcrSUqVn3bh7sZmNBmYBdYAJ7r7MzMYCOe5+rOgPBaa6u0et3gN43sxKiXyo/D76bB0RCdcf317FvHU7ePTbvel+upqVpauYvhnr7jOBmWWmPVDm8a/KWe9joNcJ5BORBHl72Wc89+EabhzQga9nqVlZOtNJsiK10Prt+/nJi4vo1fZUHrimZ9hxJMFU6EVqmUNHSxg5MZeTzHhmeJaaldUCamomUss88PpSlm/Zw4Rb1KysttAevUgtMn3+RqbnbGL0pV25rLualdUWKvQitcSyzbu5//WlDOzakh+rWVmtokIvUgvsPniUOyfl0rxRfZ4Y2pc6J+lLUbWJxuhF0py789MXF1Gw8yDTfnAeGU3UrKy20R69SJp7fs5aZi/fyr2De9Cvo5qV1UYq9CJpbO7a7fzhrRV89dzW3DqwU9hxJCQq9CJpqnDPIUZPXkinjMZqVlbLaYxeJA0Vl5QyespC9h8uZtLtA2jSQP/VazP964ukoUdmreQ/63bw+A19OOv0pmHHkZBp6EYkzby19DOen7OWm87rwPV9y70YnNQyKvQiaeTTbfv52YuL6N3uVO5XszIJqNCLpImDR0q4Y+IC6tQxxg3PokFdNSuTiJgKvZkNMrOVZpZvZmPKmX+LmRWZWV5wuz1q3s1mtjq43RzP8CIS4e7c//pSVm7dy2M39KFdczUrk/9T6cFYM6sDjAOuADYB881sRjlXiprm7qPLrNsCeBDIBhxYEKy7My7pRQSAafM38tKCTdx1WVcuPeu0sONIkollj74/kO/ua939CDAVGBLj818FzHb3HUFxnw0Mql5UESnP0oLdPDBjGRd1y+BHX1GzMvmiWAp9W2Bj1ONNwbSyvmFmi83sJTNrX8V1MbMRZpZjZjlFRUUxxBKR3QeOcsfEBbRsXJ/Hb+ijZmVSrngdjP070MndzyWy1/6Xqj6Bu49392x3z87MzIxTLJH0VVrq3DM9j617DjFueBYt1axMKhBLoS8A2kc9bhdM+1/uvt3dDwcP/wT0i3VdEameZz9cw7srCrlvcA+yOjQPO44ksVgK/Xygm5l1NrP6wFBgRvQCZtY66uF1wCfB/VnAlWbW3MyaA1cG00TkBHy8Zhv//fZKru3dhpsv6BR2HElylZ514+7FZjaaSIGuA0xw92VmNhbIcfcZwF1mdh1QDOwAbgnW3WFmDxH5sAAY6+47EvA+RGqNz3Yf4q4pC+mc0Zjffb2XmpVJpczdw87wBdnZ2Z6TkxN2DJGkc7SklGHj57J8yx5eHzWQbq3Ux0YizGyBu2eXN09NzURSyMNvriBn/U6eGNpHRV5iphYIIinizSVb+NNH6/ju+R0Z0kfNyiR2KvQiKWBt0T5+9tJierdvxn1f7RF2HEkxKvQiSe7gkRJGTsylXh3jGTUrk2rQGL1IEnN37nttCasK9/LCrf1p2+zksCNJCtIevUgSm/KfjbySW8Bdl3Xj4jP1jXGpHhV6kSS1eNMufhU0K7vr8m5hx5EUpkIvkoR2HTjCyIm5ZDSpzxND+6pZmZwQjdGLJJnSUufH0/Io3HuIF++4gBaN64cdSVKc9uhFkswzH+Tz/soi7r+mJ33aNws7jqQBFXqRJPKv/G08OnsV1/Vuw3fO6xh2HEkTKvQiSeJYs7IumU3UrEziSmP0IkngaEkpoybncvBoCdNuyqJxA/3XlPjRb5NIEvjdzBUsWL+Tp4b1petpalYm8aWhG5GQvbF4MxP+tY5bLujEtb3bhB1H0pAKvUiI8gv38YuXFtO3QzPuHaxmZZIYMRV6MxtkZivNLN/MxpQz/x4zW25mi83sXTPrGDWvxMzygtuMsuuK1FYHjhRz56QFNKhXh3E3ZlG/rva7JDEqHaM3szrAOOAKYBMw38xmuPvyqMUWAtnufsDMRgJ/AG4I5h109z5xzi2S0tyde19ZwurCffz1e/1po2ZlkkCx7EL0B/Ldfa27HwGmAkOiF3D39939QPBwLtAuvjFF0svEeRt4LW8zd19+Jhd1U7MySaxYCn1bYGPU403BtIrcBrwZ9bihmeWY2Vwzu76ilcxsRLBcTlFRUQyxRFLToo27eOjvy7nkrEx+eFnXsONILRDX0yvN7CYgG7g4anJHdy8wsy7Ae2a2xN3XlF3X3ccD4yFycfB45hJJFjv3H+HOSblkNm3AY9/uw0lqViY1IJY9+gKgfdTjdsG0zzGzrwD3Ade5++Fj0929IPi5FvgA6HsCeUVSVmmp8+PpeRTtPcwzw7NormZlUkNiKfTzgW5m1tnM6gNDgc+dPWNmfYHniRT5wqjpzc2sQXA/AxgIRB/EFak1nn4/nw9WFvHAtT3prWZlUoMqHbpx92IzGw3MAuoAE9x9mZmNBXLcfQbwCNAEeDHoz7HB3a8DegDPm1kpkQ+V35c5W0ekVvjn6iIee2cVX+vbluEDOoQdR2oZc0++4fDs7GzPyckJO4ZIXGzedZBrnvqIjCb1eW3UQBrVV+cRiT8zW+Du2eXN0zc0RBLoSHGkWdmR4lKevamfiryEQr91Ign025mfsHDDLsbdmMUZmU3CjiO1lPboRRJkxqLNvPDxp3xvYGe+em7rsONILaZCL5IA+YV7GfPyYvp1bM4vB3cPO47Ucir0InG2/3AxIyfmcnLQrKxeHf03k3BpjF4kjtydX76yhDVF+/jbbQM4/dSGYUcS0R69SDz9be56ZizazD1XnMnArhlhxxEBVOhF4mbhhp089MZyLut+GndeomZlkjxU6EXiYMf+I4yalEurUxry6Ld7q1mZJBWN0YucoJJS5+5peWzbd4SXR15As0ZqVibJRYVe5AQ99d5q5qwq4rdf60WvdqeGHUfkCzR0I3ICPlxVxBPvrubrWW0Z1r995SuIhECFXqSaNu86yN1TF3JWq6b85vpeBJ1bRZKOCr1INRwpLuXOSbkcLXGeGZ7FyfXrhB1JpEIaoxepht/8Yzl5G3fx3E1ZdFGzMkly2qMXqaLX8wr4y7/Xc/uFnRl0jpqVSfKLqdCb2SAzW2lm+WY2ppz5DcxsWjB/npl1ipr3y2D6SjO7Kn7RRWreW0u38MtXlvClTs35xdVqViapodKhGzOrA4wDrgA2AfPNbEaZSwLeBux0965mNhR4GLjBzHoSucbs2UAb4B0zO9PdS+L9RkQSqXDvIR58fRlvLv2Ms9ucwtNqViYpJJYx+v5AvruvBTCzqcAQPn+R7yHAr4L7LwFPW+QUhCHAVHc/DKwzs/zg+f4dn/ifd+1TH3HoqD5DJP627D7EkZJSfj7oLL5/URcVeUkpsRT6tsDGqMebgAEVLRNcTHw30DKYPrfMum3LexEzGwGMAOjQoXoXTz4jszFHSkqrta7I8fRp34wfXHwGXU/TgVdJPUlz1o27jwfGQ+Ti4NV5jseH9o1rJhGRdBDL358FQPRX/toF08pdxszqAqcC22NcV0REEiiWQj8f6GZmnc2sPpGDqzPKLDMDuDm4/03gPXf3YPrQ4KyczkA34D/xiS4iIrGodOgmGHMfDcwC6gAT3H2ZmY0Fctx9BvBn4G/BwdYdRD4MCJabTuTAbTEwSmfciIjULIvseCeX7Oxsz8nJCTuGiEjKMLMF7p5d3jydIyYikuZU6EVE0pwKvYhImlOhFxFJc0l5MNbMioD11Vw9A9gWxzjxolxVo1xVo1xVk465Orp7ZnkzkrLQnwgzy6noyHOYlKtqlKtqlKtqalsuDd2IiKQ5FXoRkTSXjoV+fNgBKqBcVaNcVaNcVVOrcqXdGL2IiHxeOu7Ri4hIFBV6EZE0l/KF3sweMbMVZrbYzF41s2YVLHfcC5wnINe3zGyZmZWaWYWnS5nZp2a2xMzyzCzhndyqkKumt1cLM5ttZquDn80rWK4k2FZ5Zla2XXY88xz3/Qett6cF8+eZWadEZalirlvMrChqG91eA5kmmFmhmS2tYL6Z2ZNB5sVmlpXoTDHmusTMdkdtqwdqKFd7M3vfzJYH/xd/VM4y8d1m7p7SN+BKoG5w/2Hg4XKWqQOsAboA9YFFQM8E5+oBnAV8AGQfZ7lPgYwa3F6V5gppe/0BGBPcH1Pev2Mwb18NbKNK3z9wJ/BccH8oMC1Jct0CPF1Tv0/Ba34ZyAKWVjB/MPAmYMB5wLwkyXUJ8EZNbqvgdVsDWcH9psCqcv4d47rNUn6P3t3fdvfi4OFcIlexKut/L3Du7keAYxc4T2SuT9x9ZSJfozpizFXj2yt4/r8E9/8CXJ/g1zueWN5/dN6XgMvNzJIgV41z9zlErkNRkSHAXz1iLtDMzFonQa5QuPsWd88N7u8FPuGL19KO6zZL+UJfxveIfAqWVd4Fzsu9SHkIHHjbzBYEF0hPBmFsr1buviW4/xnQqoLlGppZjpnNNbNEfRjE8v7/d5lgR2M30DJBeaqSC+AbwZ/7L5lZ+3Lm17Rk/v93vpktMrM3zezsmn7xYMivLzCvzKy4brOkuTj48ZjZO8Dp5cy6z91fDyKcOkgAAAJNSURBVJa5j8hVrCYlU64YXOjuBWZ2GjDbzFYEeyJh54q74+WKfuDubmYVnffbMdheXYD3zGyJu6+Jd9YU9ndgirsfNrMfEPmr47KQMyWrXCK/T/vMbDDwGpHLndYIM2sCvAzc7e57EvlaKVHo3f0rx5tvZrcA1wCXezDAVUZCLlJeWa4Yn6Mg+FloZq8S+fP8hAp9HHLV+PYys61m1trdtwR/ohZW8BzHttdaM/uAyN5QvAt9LO//2DKbzKwucCqwPc45qpzL3aMz/InIsY+wJeT36URFF1d3n2lmz5hZhrsnvNmZmdUjUuQnufsr5SwS122W8kM3ZjYI+DlwnbsfqGCxWC5wXuPMrLGZNT12n8iB5XLPEKhhYWyv6AvM3wx84S8PM2tuZg2C+xnAQCLXI463WN5/dN5vAu9VsJNRo7nKjONeR2T8N2wzgO8GZ5KcB+yOGqYLjZmdfuy4ipn1J1IPE/1hTfCafwY+cfdHK1gsvtuspo84x/sG5BMZy8oLbsfOhGgDzIxabjCRo9triAxhJDrX14iMqx0GtgKzyuYicvbEouC2LFlyhbS9WgLvAquBd4AWwfRs4E/B/QuAJcH2WgLclsA8X3j/wFgiOxQADYEXg9+//wBdEr2NYsz1u+B3aRHwPtC9BjJNAbYAR4PfrduAO4A7gvkGjAsyL+E4Z6HVcK7RUdtqLnBBDeW6kMixucVRdWtwIreZWiCIiKS5lB+6ERGR41OhFxFJcyr0IiJpToVeRCTNqdCLiKQ5FXoRkTSnQi8ikub+PzGlRgmpqkeaAAAAAElFTkSuQmCC\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "plot_function(F.relu)" + ] + }, + { + "cell_type": "code", + "execution_count": 92, + "metadata": { + "id": "QyuXxFphpokK" + }, + "outputs": [], + "source": [ + "#แน่นอนว่าอ่านง่ายกว่าถ้าใช้ pytorch แทนที่จะเขียนฟังชั่นเอง\n", + "simple_net = nn.Sequential(\n", + " nn.Linear(28*28,30),\n", + " nn.ReLU(),\n", + " nn.Linear(30,1)\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 93, + "metadata": { + "id": "94urzFOSpokK" + }, + "outputs": [], + "source": [ + "learn = Learner(dls, simple_net, opt_func=SGD,\n", + " loss_func=mnist_loss, metrics=batch_accuracy)" + ] + }, + { + "cell_type": "code", + "execution_count": 94, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000 + }, + "id": "Z8WM5ImkpokK", + "outputId": "cb7de009-0755-421c-c896-4ecbeee52390" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "\n", + "\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
epochtrain_lossvalid_lossbatch_accuracytime
00.2538390.1014160.96270900:00
10.1182630.0554670.96957800:00
20.0710470.0449550.97055900:00
30.0504350.0399460.97203100:00
40.0399660.0367460.97252200:00
50.0346560.0348830.97399400:00
60.0305980.0330720.97448500:00
70.0284360.0315630.97448500:00
80.0268950.0307310.97595700:00
90.0259610.0297740.97742900:00
100.0250330.0289580.97742900:00
110.0235330.0280510.97742900:00
120.0228950.0276390.97841000:00
130.0217760.0268080.97841000:00
140.0211310.0267610.97742900:00
150.0206410.0260070.97890100:00
160.0200770.0253960.97890100:00
170.0198570.0249650.97890100:00
180.0197740.0249510.97890100:00
190.0189430.0247180.97890100:00
200.0188280.0242100.97890100:00
210.0187040.0238840.97890100:00
220.0186800.0235850.97939200:00
230.0181920.0234340.97988200:00
240.0176120.0228550.97988200:00
250.0176980.0226980.98037300:00
260.0177340.0226160.98037300:00
270.0176730.0222820.98037300:00
280.0169940.0220690.98086400:00
290.0171850.0218720.98086400:00
300.0168300.0216610.98086400:00
310.0167270.0214720.98037300:00
320.0162330.0214040.98135400:00
330.0159870.0211010.98037300:00
340.0156710.0209230.98135400:00
350.0156170.0207930.98135400:00
360.0154030.0206970.98184500:00
370.0152250.0206560.98135400:00
380.0155050.0203050.98233600:00
390.0153150.0203420.98135400:00
" + ] + }, + "metadata": {} + } ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
epochtrain_lossvalid_lossaccuracytime
00.1121400.0218770.99509300:31
" + "source": [ + "learn.fit(40, 0.1)" + ] + }, + { + "cell_type": "code", + "execution_count": 95, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 265 + }, + "id": "Q0d__m-FpokL", + "outputId": "b39fc03d-2bb8-4a3e-8036-e57a88e15c62" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD4CAYAAAAHHSreAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxV1bn/8c/XMMskg8wQlDEiMkScBVErapVBa6HVqj97USvXa3uxSr21t9zrxatYrUNVqlSpcy0i9YqggKB1DPM8g5AECEgSIAlkeH5/nB08ZiCHcOAccp7363Ve7r3W2vs8e7ecJ2vtYcnMcM4558KdFOsAnHPOxR9PDs4558rx5OCcc64cTw7OOefK8eTgnHOunFqxDiAaWrRoYcnJybEOwznnTigLFizYZWYtK6qrEckhOTmZtLS0WIfhnHMnFElbKqvzYSXnnHPleHJwzjlXjicH55xz5USUHCQNkbRG0npJ91dQ30nSbElLJX0sqX1Y3SOSVkhaJelJhTSQ9H+SVgd1D4e1v0VSlqTFwefn0TlU55xzkaoyOUhKAp4BrgRSgFGSUso0mwhMMbPewHhgQrDt+cAFQG+gF3A2MLB0GzPrAfQFLpB0Zdj+3jSzPsHnhWofnXPOuWqJpOcwAFhvZhvN7CDwBjC0TJsUYE6wPDes3oB6QB2gLlAb2GFmeWY2FyDY50KgPc455+JCJMmhHbA1bH1bUBZuCTAiWB4ONJLU3Mw+J5QsMoPPTDNbFb6hpKbANcDssOLrgiGqtyV1iPhonHPORUW0LkiPBQZKWkRo2CgdKJbUBehJqFfQDhgs6aLSjSTVAl4HnjSzjUHxP4DkYIjqQ+Dlir5Q0mhJaZLSsrKyonQYzjkXP77YuJsFW76NyXdHkhzSgfC/3tsHZYeYWYaZjTCzvsADQVk2oV7EF2a2z8z2ATOA88I2nQSsM7Mnwva128wOBKsvAP0rCsrMJplZqpmltmxZ4QN+zjl3wlr0zR5+9uJX/PSFL1mVmXvcvz+S5PA10FVSZ0l1gJHA9PAGklpIKt3XOGBysPwNoR5FLUm1CfUqVgXb/DfQBLinzL7ahK1eW9reOecSRdbeA9z5ykJObVyXJvVrc/tfF5CTV3hcY6gyOZhZETAGmEnoh/otM1shabyka4Nmg4A1ktYCrYCHgvK3gQ3AMkLXJZaY2T+CW10fIHQhe2GZW1bvDm5vXQLcDdwSheN0zrkTQlFxCWNeW8ievIM8f1N//vTT/mTm5PNvby6ipOT4zdypmjBNaGpqqvm7lZxzNcF/v7eSFz7dxOM/PovhfUM3cf71iy38dtpy7r60K7+6vFvUvkvSAjNLrajOn5B2zrljZPX2XJ6es449+w9G1H76kgxe+HQTt5yffCgxANx4Tkeu79+eJ2ev46OVO45VuN/jycE556KsoLCYRz5YzQ+f/JSJs9Zy6R/mMW1ROocbqVm9PZf73l7K2cmn8Juren6vThL/PawXvdo15pdvLmbTrv3H+hA8OTjnXDR9tn4XQ56Yz58+3sDQPu14Y/S5dGzWgHveXMzNf/mard/mldsmJ7+Q2/+6gEb1avHMT/pRp1b5n+Z6tZN47sb+1EoSo6eksf9A0TE9Dk8OzjkXBXv2H2Ts35bwkxe+xIBXf34Oj91wFuee1py/33k+v7/2DBZs/pbLH5/HpPkbKCouAaCkxLjnjUVkZOfz7I39OLVxvUq/o/0pDXhqVD82ZO3j128vPWxP5GjViMl+nHMuVsyMdxdnMP69leTmF/KLQadz96VdqVc76VCbpJPEzecnc3lKKx58dwX/8/5qpi3K4OHrzmT2qp3MXZPFfw09g/6dmlX5fRd2bcGvh/Tg4RmrOeuTJoy++PRjclyeHJxzNdqXG3ezZsdebjq3E5KOaNuSEuPZeRsOO8b/ze48vtr8LX06NGXCiDPp2aZxpW3bNq3Pn3/Wn5krtvPguysY9sw/KTG4rl97bjy3U8Rx3X7xaSzZms3DM1bTq20Tzu/S4oiOKxKeHJxzNdb6nfu47eU09h0oIukk8dNzIv8BBnhqznoe/2gtrRvXI+mkihNLnVon8ftrz+DGcztV2iacJIb0asP5XVowceYaMrLzeWh4ryNKXJJ49EdnsSFrH0vTc45JcvDnHJxzNdLegkKGPfNPsvMK6daqEWlbvuXN28+jX8dTItp+zuod3PZyGsP7tOOxG8464l7H8VBQWPy94asj5c85OOcSipkx9m9L2Lw7j6d/0o9nb+xHmyb1ufOVBWTtPVDl9pt37eeeNxbTs3VjHhp+ZlwmBuCoEkNVPDk452qcZ+dtYOaKHYy7sgfnnd6cpg3q8NyN/cnJL+Su1xZSGNwpVJG8g0Xc8coCJPH8Tf2pX+fY/QDHM7/m4Jw7pvIOFvHcvI2s2V79N4vWqZXELecn079T1UNC89dmMXHmGq45qy23Xdj5UHlK28Y8PKI397y5mAnvr+bBa8pOaBnqcdz/92Ws2bGXl24dQIdmDaod84nOk4Nz7piZtzaLB95ZxrY9+XRr1ZCTqjk8s3PvAd5bmsGN53Ti3iHdaVyvdoXttn6bx91vLKLrqY343+vKDwcN69uOJduymfzPTZzVoQlD+3x/3rLJ/9zM9CUZ3HtFdwZ2S+ypADw5OOeibte+A/zXeyt5d3EGp7c8mbduP48Bnau+h78y+w4U8disNbz02WZmrdzO+KG9uOKM1t9rU1BYzB2vLKC4xHj+pv40qFPxz9tvrurJivRc7vv7Urq1anTo1tMvNu7mf95fxRVntOIXg47NswMnEr9byTkXNWbG3xZs46H/W0X+wWJ+ccnp3DnodOrWis64/eKt2YybuoxVmblccUYrfn9tL1o3qYeZ8e9/W8LUhelMviWVwT1aHXY/O/cWcM1Tn1K3VhLTx1xAfmEx1zz1KY3r1+bduy6gUSU9k5rmcHcreXJwzkXFxqx9PPDOcj7fuJuzk09hwogz6XJqo6h/T2FxCS9+uonHP1xL7aSTuG9Id0oMfjd9Bfdc1pV7LovsldYLtuxh5KTPOf/0FmTnF7J+x17eHXPBMYk5XnlycM4d1lebvuX5eRvILyyu1vZmsOCbPdStdRK/uaonP07twEkRPBB2NLbs3s8D7yzn0/W7ALi0x6n8+WepR/S9r3yxhf+YthyA527sx5BebarYomY5XHLwaw7OJbCc/EIenrGa17/6hlaN69LxKO7OGdanLWN/0P2wL46Lpk7NT+avtw3gnUXpzFubxfihvY44If30nI58u/8gTerXTrjEUJWIeg6ShgB/BJKAF8zs4TL1nQjNG90S+Ba40cy2BXWPAFcTeqbiQ+DfzMwk9QdeAuoD74eVNwPeBJKBzcANZrbncPF5z8G5I2NmvL9sO//5jxXs3neA2y7szC8v71bpRVxXMx3VE9KSkoBngCsJzfk8SlLZG4QnAlPMrDcwHpgQbHs+cAHQG+gFnA0MDLZ5FvgXoGvwGRKU3w/MNrOuwOxg3TkXJenZ+fz85TTuem0hrRrXZfqYC3ng6hRPDO57Ivl/wwBgvZltBJD0BjAUWBnWJgX4VbA8F5gWLBtQD6gDCKgN7JDUBmhsZl8E+5wCDANmBPseFGz/MvAxcN+RH5pzLlxxifHyZ5uZOGsNZvAfV/fklvOTqZXkL0pw5UWSHNoBW8PWtwHnlGmzBBhBaOhpONBIUnMz+1zSXCCTUHJ42sxWSUoN9hO+z9KnUVqZWWawvB2o8J40SaOB0QAdO3aM4DCcq7l25BbwvzNWs27nvkrbZOcfZOu3+Qzq3pL/GtoroZ/+dVWLVj9yLPC0pFuA+UA6UCypC9ATKJ0p+0NJFwH5kew0uAZR4UURM5sETILQNYejC9+5E1NJifHqV9/wyIzVHCwu4bzTm1f6FHKrxnW594oeXNO7Tdy+SM7Fj0iSQzrQIWy9fVB2iJllEOo5IKkhcJ2ZZUv6F+ALM9sX1M0AzgP+yncJo+w+d0hqY2aZwfDTziM/LOdqvrU79jJu6jIWbNnDBV2a89CwM0lucXKsw3I1RCSDjV8DXSV1llQHGAlMD28gqYWk0n2NI3TnEsA3wEBJtSTVJnQxelUwbJQr6VyF/oT5GfBusM104OZg+eawcuccoddE/GHWGq5+8hM2Zu3jsR+dxSu3neOJwUVVlT0HMyuSNAaYSehW1slmtkLSeCDNzKYTuoA8IRgCmg/cFWz+NjAYWEbo4vQHZvaPoO4XfHcr64zgA/Aw8Jak24AtwA1He5DO1RRfbNzNb6YuY+Ou/Yzo244Hru5J84Z1Yx2Wq4H8CWnnTgA5eYX8z/ureDNtKx2bNeCh4b24qGtivzXUHT1/Qtq5E5SZ8d7STH7/jxXsySvk9oGncc+l3RJ2Ahp3/HhycC5ObduTx2+nLWfumizOat+EKf/vHFLaNo51WC5BeHJwLs4UFZfw0mebeWzWWiR48Icp3Hx+MknH+EV2zoXz5OBcHFmensO4qctYlp7DpT1OZfywXrRrWj/WYbkE5MnB1SgrM3L5lylpFFTz1dOxtifvIM1OrsszP+nHVWe29ofVXMx4cnA1ysMfrGbfgSKuOevEfP1ys5PrctsFnWnSIDFmInPxy5ODqzG+2Lib+Wuz+M1VPRh9sc8B7NzR8NcxuhrBzHjkg9W0blyPn52XHOtwnDvheXJwNcKc1TtZ+E02d1/alXq1/RkA546WDyu548rMuP/vy3hnUfph2910Xid++8Oyc0pVrKTEeHTmGpKbN+BHqe2r3sA5VyVPDu64en7+Rt5M28rQPm1pW8ktmpuy9vPip5s4vWVDfnJO1XN1/GNpBqu37+WPI/tQ2yeucS4qPDm44+bTdbt45IPVXH1mG574cZ9Kb9MsLjFufelrfjd9OT3aNKJfx1Mq3WdhcQl/+HAtPds05prebY9V6M4lHP8zyx0X2/bk8a+vL6TLqQ155Preh71/P+kk8eTIPrRuUo9fvLKQrL0HKm37VtpWtuzO494runGSP0HsXNR4cnDHXEFhMXe8soCiYuP5m1I5uW7VHdamDerw/I2pZOcfZMxrCyksLqlwv0/OXkdqp1O4pPupxyJ05xKWJwd3TJkZ/zFtOcvTc3n8x33ofAQT0qS0bcyEEWfy5aZveXjG6nL1L3+2mR25B7j3iu7+JLFzUebXHNwx9eqX3/D2gm3cfWlXLktpdcTbD+/bniVbc3jx0030bt+EoX3aAZBbUMiz8zYwsFtLzjmtebTDdi7hRdRzkDRE0hpJ6yXdX0F9J0mzJS2V9LGk9kH5JZIWh30KJA0L6j4JK8+QNC0oHyQpJ6zuwWgesDt+FmzZw+//sYJLurfknku7Vns/D1zdkwHJzbjv70tZlZkLwAvzN5KdV8i9V3SPVrjOuTBVJgdJScAzwJVACjBKUtkb0CcCU8ysNzAemABgZnPNrI+Z9SE0XWgeMCuouyis7nNgatj+PimtM7PxR3eILhZ27i3gF68uoG3T+jzx475HdbG4dtJJPP3TvjSpX5s7XlnAxqx9vPDpJq7u3YZe7ZpEMWrnXKlIhpUGAOvNbCOApDeAocDKsDYpwK+C5bnAtAr2cz0ww8zywgslNSaUOG49stDdsbT/QBF5B6v3ZlMzY8xri8jNL+KlWwdE5SVypzaqx59+2p+Rkz7n2qf/yYGiEn51ebej3q9zrmKRJId2wNaw9W3AOWXaLAFGAH8EhgONJDU3s91hbUYCf6hg/8OA2WaWG1Z2nqQlQAYw1sxWlN1I0mhgNEDHjlU/KOUi983uPK54Yj75R/na6z+O7EPPNtGbuax/p1P43TVn8B/TlvPj1A6c3rJh1PbtnPu+aF2QHgs8LekWYD6QDhz6ZZHUBjgTmFnBtqOAF8LWFwKdzGyfpKsI9ULKDVib2SRgEkBqaqpF5zAcwOMfrcUwfn/tGdUeDurUrAEXd2sZ5cjgp+d0JLn5yfTr1DTq+3bOfSeS5JAOdAhbbx+UHWJmGYR6DkhqCFxnZtlhTW4A3jGzwvDtJLUgNGw1PGxfuWHL70v6k6QWZrYrskNyR2P19lymLU5n9MWncfP5ybEOpxxJXNi1RazDcK7Gi+Rupa+BrpI6S6pDaHhoengDSS0kle5rHDC5zD5GAa9XsO/rgffMrCBsX60V3LQuaUAQ4+4KtnXHwGOz1tKwbi3uHOjzITiXyKpMDmZWBIwhNCS0CnjLzFZIGi/p2qDZIGCNpLVAK+Ch0u0lJRPqecyrYPcjKZ80rgeWB9ccngRGmpkPGx0HC7/Zw4crd3D7xafRtEGdWIfjnIsh1YTf3dTUVEtLS4t1GCc0M+Mnf/6SdTv3Mu/eSyJ6xYVz7sQmaYGZpVZU56/PcAB8un4Xn2/czV2XdPHE4Jzz5OBCvYZHZ66hXdP6Ec2f4Jyr+Tw5OGau2M7SbTncc1lX6tbyKTadc54cEl5xiTFx1lpOb3kyw/u2i3U4zrk44ckhwU1duI31O/cx9gfdqeVTbDrnAv5rkMAOFBXzxEfr6N2+CUN6tY51OM65OOLJIYG99uU3pGfn+2Q5zrlyPDkkqP0Hinhm7nrOPa0ZF3bx11E4577Pk0OC+ss/N7Fr30F+PaSH9xqcc+V4ckhA2XkHeX7+Ri7r2Yp+HU+JdTjOuTjkySEBPTtvA/sOFDH2Cp8sxzlXMU8OCWZHbgEvf7aZoWe1pUfr6E3E45yrWTw5JJin5qyjqNj4pU+x6Zw7DE8OCWTL7v288dVWRg7oQKfmJ8c6HOdcHPPkkEAe/3AttZLE3YPLzbrqnHPf48khQazensu7SzK45fzOnNq4XqzDcc7FuYiSg6QhktZIWi/p/grqO0maLWmppI8ltQ/KL5G0OOxTIGlYUPeSpE1hdX2Cckl6MviupZL6RfOAE9XEmaHpP+8YeFqsQ3HOnQCqTA6SkoBngCuBFGCUpJQyzSYCU8ysNzAemABgZnPNrI+Z9QEGA3nArLDt7i2tN7PFQdmVQNfgMxp4ttpH5wBYsGUPH63y6T+dc5GLpOcwAFhvZhvN7CDwBjC0TJsUYE6wPLeCegjNDT3DzPKq+L6hhBKNmdkXQFNJbSKI01UgNJHPalo0rMOtF3SOdTjOuRNEJMmhHbA1bH1bUBZuCTAiWB4ONJLUvEybkcDrZcoeCoaOHpdU9wi+D0mjJaVJSsvKyorgMBLTp+t38cXGbxnj0386545AtC5IjwUGSloEDATSgeLSyuAv/zOBmWHbjAN6AGcDzYD7juQLzWySmaWaWWrLli2PMvyaKXz6z1E+/adz7ghEkhzSgQ5h6+2DskPMLMPMRphZX+CBoCw7rMkNwDtmVhi2TWYwdHQA+Auh4auIvs9F5oPlPv2nc656IkkOXwNdJXWWVIfQ8ND08AaSWkgq3dc4YHKZfYyizJBS6XUEhV4JOgxYHlRNB34W3LV0LpBjZplHcEwOKCouYeKsNXQ5tSEj+rWPdTjOuRNMlYPQZlYkaQyhIaEkYLKZrZA0Hkgzs+nAIGCCJAPmA3eVbi8pmVBPYF6ZXb8qqSUgYDFwR1D+PnAVsJ7Q3U23VvfgarItu/czZ/VOSqzi+k279rEhaz/P3diPpJP8ldzOuSMjs0p+XU4gqamplpaWFuswjovC4hImzd/Ik7PXcaCo5LBtUzudwt/uOM/na3DOVUjSAjNLrajOb185gSz8Zg+/mbqM1dv3ctWZrbl/SE+aNKhdafuGdWt5YnDOVYsnhxPA3oJCJs5cw5QvttC6cT3+/LNULk9pFeuwnHM1mCeHODdrxXYefHcFO/YWcPN5yYy9ojsN/XkF59wx5r8ycWrfgSLu/dsSZizfTo/WjXj2xn709Sk9nXPHiSeHOPXU7HV8sGI7vx7SnX+56DRqJ/kLdJ1zx48nhzi0PaeAlz7bzPC+7fjFoC6xDsc5l4D8z9E49NScdZSY8cvLfCpP51xseHKIM1t27+fNr7cyakBHOjRrEOtwnHMJypNDnPnDh2upnXQSYwb7cJJzLnY8OcSRVZm5TF+Swa0XJHNqI5/K0zkXO54c4shjs9bQqG4tbr/49FiH4pxLcJ4c4sSCLd/y0aqd3DHo9MO+EsM5544HTw5xwMx45IM1tGhYl1vOT451OM4558khHsxft4svN33L3Zd2oUEdf/TEORd7nhxiLDSV52o6NKvPyLN9Kk/nXHzw5BBjM5ZvZ3l6Lr+8rBt1avn/HM65+BDRr5GkIZLWSFov6f4K6jtJmi1pqaSPJbUPyi+RtDjsUyBpWFD3arDP5ZImS6odlA+SlBO2zYPRPOB4UjqVZ7dWDRnap12sw3HOuUOqTA6SkoBngCuBFGCUpJQyzSYCU8ysNzAemABgZnPNrI+Z9QEGE5r2c1awzatAD+BMoD7w87D9fVK6nZmNr/bRxbmpC9PZmLWff/9Bd5/K0zkXVyLpOQwA1pvZRjM7CLwBDC3TJgWYEyzPraAe4HpghpnlAZjZ+xYAvgLaV+cATlTf7M7jiY/W0qdDU37gE/c45+JMJMmhHbA1bH1bUBZuCTAiWB4ONJLUvEybkcDrZXceDCfdBHwQVnyepCWSZkg6o6KgJI2WlCYpLSsrK4LDiA9FxSU8P28DP3hiHrkFRfz2hz19Kk/nXNyJ1n2TY4GnJd0CzAfSgeLSSkltCA0fzaxg2z8B883sk2B9IdDJzPZJugqYBnQtu5GZTQImAaSmplqUjuOYWrI1m3FTl7EyM5fLU1oxfugZtGlSP9ZhOedcOZEkh3SgQ9h6+6DsEDPLIOg5SGoIXGdm2WFNbgDeMbPC8O0k/Q5oCdwetq/csOX3Jf1JUgsz2xXZIcWf/QeKeGzWWl76bBMtG9XluRv7M6RX61iH5ZxzlYokOXwNdJXUmVBSGAn8JLyBpBbAt2ZWAowDJpfZx6igPHybnwNXAJcG25WWtwZ2mJlJGkBo6Gv3ER1VHJmzege/nbaCjJx8bjynE/cO6U7jev56DOdcfKsyOZhZkaQxhIaEkoDJZrZC0nggzcymA4OACZKM0LDSXaXbS0om1POYV2bXzwFbgM+DMfepwZ1J1wN3SioC8oGRwUXrE4qZMW7qMt74eivdWjXk7TvOo3+nZrEOyznnIqIT8He3nNTUVEtLS4t1GN/zybosbnrxK269IJlxV/b0B9ycc3FH0gIzS62ozn+xjoHQKzHW0K5pfe6/socnBufcCcd/tY6BD5ZvZ+m2HO65rCt1ayXFOhznnDtinhyirPSVGF1ObciIfgn1XJ9zrgbx5BBlUxelsyFrP2N/0M1fieGcO2F5coiiA0XF/PGjdfRu34QrzvDnGJxzJy5PDlH02pffkJ6dz71XdPdXYjjnTmieHKJk/4Einp6znvNOa86FXVrEOhznnDsqnhyiZPKnm9i9/yD3DvFeg3PuxOfJIQqy8w4yaf5GLk9pRb+Op8Q6HOecO2qeHKLg2Xkb2HewiLE/6B7rUJxzLio8ORylHbkFvPTPzQzr047urRvFOhznnIsKTw5H6cnZ6yguMX55WbdYh+Kcc1HjyeEobNm9nze/3sqoAR3p2LxBrMNxzrmo8eRQTWbGQ/+3ilpJ4l8Hd4l1OM45F1WeHKrpxU83MWvlDn55WTdObVwv1uE451xUeXKohs827GLCjNVc2as1oy8+LdbhOOdc1EWUHCQNkbRG0npJ91dQ30nSbElLJX0sqX1QfomkxWGfAknDgrrOkr4M9vmmpDpBed1gfX1Qnxy9wz16Gdn5jHltEZ1bnMyjPzrLH3hzztVIVSYHSUnAM8CVQAowSlJKmWYTgSlm1hsYD0wAMLO5ZtbHzPoAg4E8YFawzf8Cj5tZF2APcFtQfhuwJyh/PGgXFwoKi7nzlQUcLCrh+Zv607BuJFNwO+fciSeSnsMAYL2ZbTSzg8AbwNAybVKAOcHy3ArqITQ39Awzy1Poz+3BwNtB3cvAsGB5aLBOUH+p4uTP8/+cvoIl23J47IazOL1lw1iH45xzx0wkyaEdsDVsfVtQFm4JMCJYHg40ktS8TJuRwOvBcnMg28yKKtjnoe8L6nOC9t8jabSkNElpWVlZERzG0Xn9q2944+utjLmki7+O2zlX40XrgvRYYKCkRcBAIB0oLq2U1AY4E5gZpe/DzCaZWaqZpbZs2TJau63Qom/28Lt3V3Bxt5b88nJ/2M05V/NFMmieDnQIW28flB1iZhkEPQdJDYHrzCw7rMkNwDtmVhis7waaSqoV9A7C91n6fdsk1QKaBO1jImvvAe58ZSGtmtTlyZF9fHY351xCiKTn8DXQNbi7qA6h4aHp4Q0ktZBUuq9xwOQy+xjFd0NKmJkRujZxfVB0M/BusDw9WCeonxO0P+4Ki0sY89pC9uQd5Lkb+9O0QZ1YhOGcc8ddlckh+Mt+DKEhoVXAW2a2QtJ4SdcGzQYBayStBVoBD5VuH9yK2gGYV2bX9wG/krSe0DWFF4PyF4HmQfmvgHK3zh4vf/18C19u+paHrzuTM9o2iVUYzjl33EV0L6aZvQ+8X6bswbDlt/nuzqOy226m/AVszGwjoTuhypYXAD+KJK5jbUVGLq0b12N43/axDsU5544rf0L6MLbn5tOmqb8awzmXeDw5HEZmdgFtmnhycM4lHk8OlTAzMnLyadOkfqxDcc65486TQyWy8wopKCzxnoNzLiF5cqhERk4+AG2bes/BOZd4PDlUIjO7AMB7Ds65hOTJoRKZ3nNwziUwTw6VyMwpoNZJokXDurEOxTnnjjtPDpXIzCmgVeN6/i4l51xC8uRQiYzsfNr6A3DOuQTlyaESmTkFtPZnHJxzCcqTQwVKSoztOQW09TuVnHMJypNDBXbvP8jBYn8AzjmXuDw5VKD0NtY2fhurcy5BeXKoQEbwAFxbv+bgnEtQnhwqsP1Qz8GHlZxziSmi5CBpiKQ1ktZLKjczm6ROkmZLWirpY0ntw+o6SpolaZWklcHMcEj6RNLi4JMhaVpQPkhSTljdg2W/71jLzCmgTq2TaH6yTwvqnEtMVc4EJykJeAa4HNgGfC1pupmtDGs2EZhiZi9LGgxMAG4K6qYAD5nZh5IaAiUAZnZR2Hf8ne/mkAb4xMx+eBTHdVQyckLzOEj+AJxzLjFF0nMYAKw3swT/P70AAAyDSURBVI1mdhB4Axhapk0KMCdYnltaLykFqGVmHwKY2T4zywvfUFJjYDAwrdpHEWWZ2fl+p5JzLqFFkhzaAVvD1rdRfk7oJcCIYHk40EhSc6AbkC1pqqRFkh4NeiLhhgGzzSw3rOw8SUskzZB0RsRHEyWZOQU+yY9zLqFF64L0WGCgpEXAQCAdKCY0bHVRUH82cBpwS5ltRwGvh60vBDqZ2VnAU1TSo5A0WlKapLSsrKwoHQYUlxjbc316UOdcYoskOaQDHcLW2wdlh5hZhpmNMLO+wANBWTahXsbiYEiqiNAPfb/S7SS1IDRs9X9h+8o1s33B8vtA7aDd95jZJDNLNbPUli1bRna0Ecjae4DiEvNnHJxzCS2S5PA10FVSZ0l1gJHA9PAGklpIKt3XOGBy2LZNJZX+eg8Gwi9kXw+8Z2YFYftqreBKsKQBQYy7j+ywqu/QDHDec3DOJbAqk0PwF/8YYCawCnjLzFZIGi/p2qDZIGCNpLVAK+ChYNtiQkNKsyUtAwT8OWz3I/n+kBKEEsZySUuAJ4GRZmbVPL4jtj2ndAY47zk45xJXlbeywqHhnffLlD0Ytvw28HYl234I9K6kblAFZU8DT0cS17GQkV06A5z3HJxzicufkC4jM6eA+rWTaFK/dqxDcc65mPHkUEZmTj5tmvoDcM65xObJoYyM7AJ/4Z5zLuF5cigjMyef1n6nknMuwXlyCFNYXMLOvQf8NlbnXMLz5BBmR24BZj7Jj3POeXII890zDt5zcM4lNk8OYTKC5NDWew7OuQTnySFMZvAAnPccnHOJzpNDmMycAhrVrUWjev4AnHMusXlyCJORne/zRjvnHJ4cvscn+XHOuRBPDmEyc3x6UOecA08OhxwoKmbXvoPec3DOOTw5HLIj5wCAX3Nwzjk8ORzy3Qxw3nNwzjlPDoHMIDl4z8E55yJMDpKGSFojab2k+yuo7yRptqSlkj6W1D6srqOkWZJWSVopKTkof0nSJkmLg0+foFySngy+a6mkftE51MPLyA6ejvaeg3POVZ0cJCUBzwBXAinAKEkpZZpNBKaYWW9gPDAhrG4K8KiZ9QQGADvD6u41sz7BZ3FQdiXQNfiMBp498sM6cpk5+TRtUJv6dZKOx9c551xci6TnMABYb2Ybzewg8AYwtEybFGBOsDy3tD5IIrWCeaQxs31mllfF9w0llGjMzL4AmkpqE9nhVF9mtj/j4JxzpSJJDu2ArWHr24KycEuAEcHycKCRpOZANyBb0lRJiyQ9GvRESj0UDB09LqnuEXwfkkZLSpOUlpWVFcFhHF5GToHP4+Ccc4FoXZAeCwyUtAgYCKQDxUAt4KKg/mzgNOCWYJtxQI+gvBlw35F8oZlNMrNUM0tt2bLlUR/Adp8BzjnnDokkOaQDHcLW2wdlh5hZhpmNMLO+wANBWTahv/oXB0NSRcA0oF9QnxkMHR0A/kJo+Cqi74u2/IPF7Mkr9Fd1O+dcIJLk8DXQVVJnSXWAkcD08AaSWkgq3dc4YHLYtk0llf5pPxhYGWzTJvivgGHA8qDNdOBnwV1L5wI5ZpZZraOL0KHbWL3n4JxzQGjY57DMrEjSGGAmkARMNrMVksYDaWY2HRgETJBkwHzgrmDbYkljgdlBElgA/DnY9atB0hCwGLgjKH8fuApYD+QBt0blSA8j89AMcN5zcM45iCA5AJjZ+4R+tMPLHgxbfht4u5JtPwR6V1A+uJL2RpBcjpeMYJKftv4AnHPOAf6ENPBdz8EvSDvnXIgnB0LXHFo0rEPdWv4AnHPOgScHIPTqDL/e4Jxz3/HkAGzPKfA7lZxzLownB0Kv6/bk4Jxz30n45LDvQBF7C4po4w/AOefcIQmfHDKz/QE455wrK+GTQ0ZwG6u/OsM5576T8MnBew7OOVdewieHjJwCJGjV2JODc86VSvjkkJmdz6mN6lI7KeFPhXPOHZLwv4jbc/0BOOecKyvhk0NGdr6/cM8558pI6ORgZmTmFNC6sfccnHMuXEInh9z8IvIOFnvPwTnnykjo5JBxaAY47zk451y4iJKDpCGS1khaL+n+Cuo7SZotaamkjyW1D6vrKGmWpFWSVkpKDspfDfa5XNJkSbWD8kGSciQtDj4Plv2+aDk0Paj3HJxz7nuqTA6SkoBngCuBFGCUpJQyzSYCU8ysNzAemBBWNwV41Mx6AgOAnUH5q0AP4EygPvDzsG0+MbM+wWf8kR9WZBrXq80VZ7SiwykNjtVXOOfcCSmSaUIHAOvNbCOApDeAocDKsDYpwK+C5bnAtKBtClArmCoUM9tXukEw9ShBu6+AQ72N4yU1uRmpyc2O99c651zci2RYqR2wNWx9W1AWbgkwIlgeDjSS1BzoBmRLmippkaRHg57IIcFw0k3AB2HF50laImmGpDMqCkrSaElpktKysrIiOAznnHORitYF6bHAQEmLgIFAOlBMqGdyUVB/NnAacEuZbf8EzDezT4L1hUAnMzsLeIqgF1KWmU0ys1QzS23ZsmWUDsM55xxElhzSgQ5h6+2DskPMLMPMRphZX+CBoCybUC9jsZltNLMiQj/0/Uq3k/Q7oCXfDUlhZrmlw0/B0FNtSS2qc3DOOeeqJ5Lk8DXQVVJnSXWAkcD08AaSWkgq3dc4YHLYtk0llf5pP5jgWoWknwNXAKPMrCRsX60lKVgeEMS4uzoH55xzrnqqTA7BX/xjgJnAKuAtM1shabyka4Nmg4A1ktYCrYCHgm2LCQ0pzZa0DBDw52Cb54K2n5e5ZfV6YLmkJcCTwEgzs6M/VOecc5FSTfjdTU1NtbS0tFiH4ZxzJxRJC8wstaK6hH5C2jnnXMU8OTjnnCunRgwrScoCtlRz8xbAriiGE00eW/XEc2wQ3/F5bNVzosbWycwqfBagRiSHoyEprbIxt1jz2KonnmOD+I7PY6uemhibDys555wrx5ODc865cjw5wKRYB3AYHlv1xHNsEN/xeWzVU+NiS/hrDs4558rznoNzzrlyPDk455wrJ6GTQ1XTn8aSpM2SlgXvnYrpu0GCaVx3SloeVtZM0oeS1gX/PSWOYvtPSelhU81eFaPYOkiaG0yPu0LSvwXlMT93h4kt5udOUj1JXwVzuqyQ9PugvLOkL4N/r28GLwKNl9hekrQp7Lz1Od6xhcWYFMyf816wXr3zZmYJ+QGSgA2E5pioQ2jCopRYxxUW32agRazjCGK5mNCr1peHlT0C3B8s3w/8bxzF9p/A2Dg4b22AfsFyI2AtoVkTY37uDhNbzM8doRd0NgyWawNfAucCbxF6ESeEXtx5ZxzF9hJwfaz/PxfE9SvgNeC9YL1a5y2Rew6Hpj81s4NA6fSnrgwzmw98W6Z4KPBysPwyMOy4BhWoJLa4YGaZZrYwWN5L6K3G7YiDc3eY2GLOQkqnFK4dfIzQK//fDspjdd4qiy0uSGoPXA28EKyLap63RE4OkUx/GksGzJK0QNLoWAdTgVZmlhksbyf0+vV4MkbS0mDYKSZDXuEkJQN9Cf2lGVfnrkxsEAfnLhgaWQzsBD4k1MvPttAUAhDDf69lYzOz0vP2UHDeHpdUNxaxAU8AvwZK58hpTjXPWyInh3h3oZn1A64E7pJ0cawDqoyF+qtx89cT8CxwOtAHyAQei2UwkhoCfwfuMbPc8LpYn7sKYouLc2dmxWbWh9DMkwOAHrGIoyJlY5PUi9AkZz0ITYfcDLjveMcl6YfATjNbEI39JXJyqHL601gys/TgvzuBdwj9A4knOyS1AQj+uzPG8RxiZjuCf8AlhCaXitm5k1Sb0I/vq2Y2NSiOi3NXUWzxdO6CeLKBucB5hGaVrBVUxfzfa1hsQ4JhOjOzA8BfiM15uwC4VtJmQsPkg4E/Us3zlsjJocrpT2NF0smSGpUuAz8Alh9+q+NuOnBzsHwz8G4MY/me0h/ewHBidO6C8d4XgVVm9oewqpifu8pii4dzJ6mlpKbBcn3gckLXROYSmikSYnfeKoptdViyF6Ex/eN+3sxsnJm1N7NkQr9nc8zsp1T3vMX6ynosP8BVhO7S2AA8EOt4wuI6jdDdU0uAFbGODXid0BBDIaExy9sIjWXOBtYBHwHN4ii2vwLLgKWEfojbxCi2CwkNGS0FFgefq+Lh3B0mtpifO6A3sCiIYTnwYFB+GvAVsB74G1A3jmKbE5y35cArBHc0xepDaOrm0ruVqnXe/PUZzjnnyknkYSXnnHOV8OTgnHOuHE8OzjnnyvHk4JxzrhxPDs4558rx5OCcc64cTw7OOefK+f/lmSSjovFjeQAAAABJRU5ErkJggg==\n" + }, + "metadata": { + "needs_background": "light" + } + } ], - "text/plain": [ - "" + "source": [ + "#ดู accuracy เพิ่มขึ้นเรื่อยๆ\n", + "plt.plot(L(learn.recorder.values).itemgot(2));" + ] + }, + { + "cell_type": "code", + "execution_count": 96, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "-pnmg5QqpokL", + "outputId": "f6df8472-bbd5-4b94-c0a0-91cc4a363bbb" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "0.981354296207428" + ] + }, + "metadata": {}, + "execution_count": 96 + } + ], + "source": [ + "learn.recorder.values[-1][2]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "fQQNpNZ-pokL" + }, + "source": [ + "## ไปให้ Deep กว่านั้นอีก" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "jq3YclVaOjCk" + }, + "source": [ + "แทนที่เราจะใช้ Architecture ง่ายๆที่เราคิดขึ้นเอง เรามาลองใช้ Architecture ที่ซับซ้อนขึ้นอย่าง `resnet18` กัน" + ] + }, + { + "cell_type": "code", + "execution_count": 97, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 136 + }, + "id": "LB3qg01jpokL", + "outputId": "d2515ddb-c29c-43f5-9612-f04e927985e5" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "/usr/local/lib/python3.7/dist-packages/fastai/vision/learner.py:265: UserWarning: `cnn_learner` has been renamed to `vision_learner` -- please update your code\n", + " warn(\"`cnn_learner` has been renamed to `vision_learner` -- please update your code\")\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "\n", + "\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "" + ], + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
epochtrain_lossvalid_lossaccuracytime
00.0858000.0082050.99754700:23
" + ] + }, + "metadata": {} + } + ], + "source": [ + "dls = ImageDataLoaders.from_folder(path)\n", + "learn = cnn_learner(dls, resnet18, pretrained=False,\n", + " loss_func=F.cross_entropy, metrics=accuracy)\n", + "\n", + "#เทรนแค่ epoch เดียวก็ดีกว่าเทรน 20-40 epoch เมื่อกี้แล้ว\n", + "learn.fit_one_cycle(1, 0.1)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "uxfYQcZfJEsN" + }, + "source": [ + "# Checkpoint ท้ายบท" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "wB1TBtie2VPG" + }, + "source": [ + "### ☑️ ดูวิดีโอ [3Blue1Brown](https://www.youtube.com/watch?v=IHZwWFHWa-w) เกี่ยวกับ SGD (มีซับไทย)\n", + "\n", + "วิดีโอนี้จะอธิบายการทำงานของ SGD ใน Neural Networks ที่เราเรียนกันในบทที่ 1 และ 3 ด้วยกราฟฟิคที่สวยงาม ช่วยเพิ่มความเข้าใจได้มากยิ่งขึ้น\n", + "\n", + "Youtube Link: [https://www.youtube.com/watch?v=IHZwWFHWa-w](https://www.youtube.com/watch?v=IHZwWFHWa-w)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4a0-VMv6Pau1" + }, + "source": [ + "## คำถามชวนคิดเกี่ยวกับบทเรียน" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "t_NIkQBMPgEh" + }, + "source": [ + "1. เราได้เรียนรู้การทำงานเกือบทุกส่วนของ ML Models ในบทเรียนนี้ คุณคิดว่าสำหรับโครงงานของคุณ การปรับแต่งส่วนไหนของโมเดลจะทำให้ได้ผลดีเพิ่มขึ้นมากที่สุด\n", + "\n", + "2. คุณคิดว่าการ initialize weights มีผลต่อคุณภาพของโมเดลที่ถูกเทรนหรือไม่ หากไม่ใช้ weights แบบสุ่ม เราควรใช้ weights อะไรในการเริ่มเทรนโมเดล\n", + "\n", + "3. Loss กับ Metric ต่างกันอย่างไร ทำไมคุณคิดว่าเราถึงเทรนโมเดลเพื่อให้ได้ Loss ที่ดีที่สุด แต่กลับวัดผลมันด้วย Metric\n", + "\n", + "4. คุณคิดว่า `Activation Function` มีความสำคัญอย่างไรกับประสิทธิภาพของโมเดล ML เราควรเลือกใช้อันไหนในกรณีไหน\n", + "\n", + "5. คุณคิดว่าทำไม Architecture อย่าง resnet18 ถึงทำได้ดีกว่า Architecture ที่เราสร้างขึ้นมาเองง่ายๆ ทั้งที่เทรนแค่ 1 epoch เทียบกับที่เราเทรนถึง 20-40 epochs\n", + "\n", + "6. คุณคิดว่ามีการ update weights ที่ดีกว่าเอา gradients มาลบไปทีละ iteration อย่างที่เราทำหรือไม่ คิดว่าวิธีไหนบ้าง หรือเป็นไปได้ไหมที่เราจะเทรนโมเดลโดยไม่ใช้ backpropagation กล่าวคือ update weights โดยไม่ใช้ gradients\n", + "\n", + "\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "HeM7xTrJPc8w" + }, + "source": [ + "## สิ่งที่ควรเตรียมพร้อมสำหรับทำโครงงาน" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "oPvpAnmLJGgU" + }, + "source": [ + "### ☑️ ทบทวนระบบการให้คะแนนโครงงานให้เรียบร้อย\n", + "\n", + "AI Builders จะออกใบประกาศนียบัตรจบการศึกษาให้กับผู้เข้าร่วมโครงการที่ส่งโครงงานได้คะแนนอย่างน้อย 70 จาก 100 คะแนนตามเกณฑ์ต่อไปนี้เท่านั้น\n", + "\n", + "1. problem statement; เหตุผลในการแก้ปัญหาเชิงธุรกิจ/ชีวิตประจำวันด้วย machine learning - 15 คะแนน\n", + "2. metrics and baselines; การให้เหตุผลเชื่อมโยงการแก้ปัญหากับตัวชี้วัดที่เลือก / การวัดผลเทียบกับวิธีแก้ปัญหาในปัจจุบัน - 15 คะแนน\n", + "3. data collection and cleaning; การเก็บและทำความสะอาดข้อมูล - 15 คะแนน\n", + "\n", + "**วันนี้เราจะคิดถึง 2 ข้อนี้เป็นพิเศษ**\n", + "\n", + "**4. exploratory data analysis; การทำความเข้าใจข้อมูล - 20 คะแนน**\n", + "\n", + "**5. modeling, validation and error analysis; การทำโมเดล, ทดสอบโมเดล และวิเคราะห์ข้อผิดพลาดของโมเดล - 20 คะแนน**\n", + "\n", + "6. deployment; การนำโมเดลไปใช้แก้ปัญหาจริง - 15 คะแนน" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9u8rUq1uPSuE" + }, + "source": [ + "### ☑️ เข้าใจชุดข้อมูล\n", + "\n", + "คุณรู้เกี่ยวกับ labels ที่คุณกำลังทำนายแค่ไหน \n", + "* ถ้าเป็น classification มันมี labels กี่ประเภท (class) แต่ละตัวอย่างเป็นได้มากกว่าหนึ่งประเภท (multi-label) หรือได้แค่ประเภทเดียว (multi-class) \n", + "* ถ้าเป็น regression แล้ว labels ของคุณกระจายตัวยังไง\n", + "* คุณมีตัวอย่างพอในแต่ละ class ให้โมเดลเรียนรู้หรือเปล่า\n", + "\n", + "features ของคุณหน้าตาเป็นอย่างไร\n", + "\n", + "* ถ้าเป็นรูปภาพ เป็นรูปจากมุมมองเดียวหรือหลายมุมมอง ขนาดรูป-ความละเอียดเท่ากันไหม\n", + "* ถ้าเป็นข้อความ การกระจายตัวของคำเป็นยังไง ทั้ง word count และ tfidf; คำที่เกิดบ่อยๆในแต่ละ labels คืออะไร\n", + "* ถ้าเป็นข้อมูลตาราง การกระจายตัวของ feature ที่เป็นตัวเลข (numerical) และประเภท (categorical) เป็นอย่างไรบ้าง ลองหา correlation ระหว่าง features ด้วยกัน และ feature กับ labels ดูว่ามีอะไรน่าสนใจไหม\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "xcPUWlA9JGio" + }, + "source": [ + "### ☑️ เทรนโมเดลแรกให้จบจนเห็นผลลัพธ์เร็วที่สุดเท่าที่จะทำได้\n", + "\n", + "แน่นอนว่าการทำความสะอาดข้อมูลและการทำความเข้าใจข้อมูลเป็นสิ่งสำคัญ (คะแนนรวม 15+20=35) แต่วิธีที่ดีที่สุดที่จะรู้ว่าโมเดลของเราทำได้ดีแค่ไหนคือการลองเทรนไปเลยทั้งๆที่ไม่ต้องทำอะไรกับข้อมูลมาก หลังจากนั้นเราอาจจะมาไล่ดูข้อผิดพลาดของโมเดลเพื่อให้มีไอเดียในการจัดการข้อมูลและโมเดลต่อไปได้ \n", + "\n", + "คุณอาจจะคิดว่าการสร้าง ML model มีขั้นตอนแบบนี้:\n", + "\n", + "\n", + "\n", + "แต่จริงๆแล้วมันคือแบบนี้:\n", + "\n", + "\n", + "\n", + "เพราะฉะนั้นอย่ากลัวที่จะลองทำให้ครบขั้นตอนก่อนแล้วจึงย้อนกลับไปกลับมาเพื่อทำใหม่ให้ดียิ่งขึ้น" ] - }, - "metadata": {}, - "output_type": "display_data" } - ], - "source": [ - "dls = ImageDataLoaders.from_folder(path)\n", - "learn = cnn_learner(dls, resnet18, pretrained=False,\n", - " loss_func=F.cross_entropy, metrics=accuracy)\n", - "\n", - "#เทรนแค่ epoch เดียวก็ดีกว่าเทรน 20-40 epoch เมื่อกี้แล้ว\n", - "learn.fit_one_cycle(1, 0.1)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "uxfYQcZfJEsN" - }, - "source": [ - "# Checkpoint ท้ายบท" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "wB1TBtie2VPG" - }, - "source": [ - "### ☑️ ดูวิดีโอ [3Blue1Brown](https://www.youtube.com/watch?v=IHZwWFHWa-w) เกี่ยวกับ SGD (มีซับไทย)\n", - "\n", - "วิดีโอนี้จะอธิบายการทำงานของ SGD ใน Neural Networks ที่เราเรียนกันในบทที่ 1 และ 3 ด้วยกราฟฟิคที่สวยงาม ช่วยเพิ่มความเข้าใจได้มากยิ่งขึ้น\n", - "\n", - "Youtube Link: [https://www.youtube.com/watch?v=IHZwWFHWa-w](https://www.youtube.com/watch?v=IHZwWFHWa-w)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "4a0-VMv6Pau1" - }, - "source": [ - "## คำถามชวนคิดเกี่ยวกับบทเรียน" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "t_NIkQBMPgEh" - }, - "source": [ - "1. เราได้เรียนรู้การทำงานเกือบทุกส่วนของ ML Models ในบทเรียนนี้ คุณคิดว่าสำหรับโครงงานของคุณ การปรับแต่งส่วนไหนของโมเดลจะทำให้ได้ผลดีเพิ่มขึ้นมากที่สุด\n", - "\n", - "2. คุณคิดว่าการ initialize weights มีผลต่อคุณภาพของโมเดลที่ถูกเทรนหรือไม่ หากไม่ใช้ weights แบบสุ่ม เราควรใช้ weights อะไรในการเริ่มเทรนโมเดล\n", - "\n", - "3. Loss กับ Metric ต่างกันอย่างไร ทำไมคุณคิดว่าเราถึงเทรนโมเดลเพื่อให้ได้ Loss ที่ดีที่สุด แต่กลับวัดผลมันด้วย Metric\n", - "\n", - "4. คุณคิดว่า `Activation Function` มีความสำคัญอย่างไรกับประสิทธิภาพของโมเดล ML เราควรเลือกใช้อันไหนในกรณีไหน\n", - "\n", - "5. คุณคิดว่าทำไม Architecture อย่าง resnet18 ถึงทำได้ดีกว่า Architecture ที่เราสร้างขึ้นมาเองง่ายๆ ทั้งที่เทรนแค่ 1 epoch เทียบกับที่เราเทรนถึง 20-40 epochs\n", - "\n", - "6. คุณคิดว่ามีการ update weights ที่ดีกว่าเอา gradients มาลบไปทีละ iteration อย่างที่เราทำหรือไม่ คิดว่าวิธีไหนบ้าง หรือเป็นไปได้ไหมที่เราจะเทรนโมเดลโดยไม่ใช้ backpropagation กล่าวคือ update weights โดยไม่ใช้ gradients\n", - "\n", - "\n", - "\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "HeM7xTrJPc8w" - }, - "source": [ - "## สิ่งที่ควรเตรียมพร้อมสำหรับทำโครงงาน" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "oPvpAnmLJGgU" - }, - "source": [ - "### ☑️ ทบทวนระบบการให้คะแนนโครงงานให้เรียบร้อย\n", - "\n", - "AI Builders จะออกใบประกาศนียบัตรจบการศึกษาให้กับผู้เข้าร่วมโครงการที่ส่งโครงงานได้คะแนนอย่างน้อย 70 จาก 100 คะแนนตามเกณฑ์ต่อไปนี้เท่านั้น\n", - "\n", - "1. problem statement; เหตุผลในการแก้ปัญหาเชิงธุรกิจ/ชีวิตประจำวันด้วย machine learning - 15 คะแนน\n", - "2. metrics and baselines; การให้เหตุผลเชื่อมโยงการแก้ปัญหากับตัวชี้วัดที่เลือก / การวัดผลเทียบกับวิธีแก้ปัญหาในปัจจุบัน - 15 คะแนน\n", - "3. data collection and cleaning; การเก็บและทำความสะอาดข้อมูล - 15 คะแนน\n", - "\n", - "**วันนี้เราจะคิดถึง 2 ข้อนี้เป็นพิเศษ**\n", - "\n", - "**4. exploratory data analysis; การทำความเข้าใจข้อมูล - 20 คะแนน**\n", - "\n", - "**5. modeling, validation and error analysis; การทำโมเดล, ทดสอบโมเดล และวิเคราะห์ข้อผิดพลาดของโมเดล - 20 คะแนน**\n", - "\n", - "6. deployment; การนำโมเดลไปใช้แก้ปัญหาจริง - 15 คะแนน" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "9u8rUq1uPSuE" - }, - "source": [ - "### ☑️ เข้าใจชุดข้อมูล\n", - "\n", - "คุณรู้เกี่ยวกับ labels ที่คุณกำลังทำนายแค่ไหน \n", - "* ถ้าเป็น classification มันมี labels กี่ประเภท (class) แต่ละตัวอย่างเป็นได้มากกว่าหนึ่งประเภท (multi-label) หรือได้แค่ประเภทเดียว (multi-class) \n", - "* ถ้าเป็น regression แล้ว labels ของคุณกระจายตัวยังไง\n", - "* คุณมีตัวอย่างพอในแต่ละ class ให้โมเดลเรียนรู้หรือเปล่า\n", - "\n", - "features ของคุณหน้าตาเป็นอย่างไร\n", - "\n", - "* ถ้าเป็นรูปภาพ เป็นรูปจากมุมมองเดียวหรือหลายมุมมอง ขนาดรูป-ความละเอียดเท่ากันไหม\n", - "* ถ้าเป็นข้อความ การกระจายตัวของคำเป็นยังไง ทั้ง word count และ tfidf; คำที่เกิดบ่อยๆในแต่ละ labels คืออะไร\n", - "* ถ้าเป็นข้อมูลตาราง การกระจายตัวของ feature ที่เป็นตัวเลข (numerical) และประเภท (categorical) เป็นอย่างไรบ้าง ลองหา correlation ระหว่าง features ด้วยกัน และ feature กับ labels ดูว่ามีอะไรน่าสนใจไหม\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "xcPUWlA9JGio" - }, - "source": [ - "### ☑️ เทรนโมเดลแรกให้จบจนเห็นผลลัพธ์เร็วที่สุดเท่าที่จะทำได้\n", - "\n", - "แน่นอนว่าการทำความสะอาดข้อมูลและการทำความเข้าใจข้อมูลเป็นสิ่งสำคัญ (คะแนนรวม 15+20=35) แต่วิธีที่ดีที่สุดที่จะรู้ว่าโมเดลของเราทำได้ดีแค่ไหนคือการลองเทรนไปเลยทั้งๆที่ไม่ต้องทำอะไรกับข้อมูลมาก หลังจากนั้นเราอาจจะมาไล่ดูข้อผิดพลาดของโมเดลเพื่อให้มีไอเดียในการจัดการข้อมูลและโมเดลต่อไปได้ \n", - "\n", - "คุณอาจจะคิดว่าการสร้าง ML model มีขั้นตอนแบบนี้:\n", - "\n", - "\n", - "\n", - "แต่จริงๆแล้วมันคือแบบนี้:\n", - "\n", - "\n", - "\n", - "เพราะฉะนั้นอย่ากลัวที่จะลองทำให้ครบขั้นตอนก่อนแล้วจึงย้อนกลับไปกลับมาเพื่อทำใหม่ให้ดียิ่งขึ้น" - ] - } - ], - "metadata": { - "accelerator": "GPU", - "colab": { - "collapsed_sections": [], - "name": "03_sgd_from_scratch.ipynb", - "provenance": [], - "toc_visible": true - }, - "jupytext": { - "split_at_heading": true - }, - "kernelspec": { - "display_name": "conda_amazonei_pytorch_latest_p36", - "language": "python", - "name": "conda_amazonei_pytorch_latest_p36" + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "collapsed_sections": [], + "name": "03_sgd_from_scratch.ipynb", + "provenance": [], + "toc_visible": true + }, + "jupytext": { + "split_at_heading": true + }, + "kernelspec": { + "display_name": "conda_amazonei_pytorch_latest_p36", + "language": "python", + "name": "conda_amazonei_pytorch_latest_p36" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.13" + } }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.6.13" - } - }, - "nbformat": 4, - "nbformat_minor": 4 -} + "nbformat": 4, + "nbformat_minor": 0 +} \ No newline at end of file