NNFS/02_nnfs_coding_our_first_neurons.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "[![Open in Colab](https://img.shields.io/static/v1?label=&message=Open%20in%20Colab&labelColor=grey&color=blue&logo=google-colab)](https://colab.research.google.com/github/theaveas/DeepLearning/blob/main/NNFS/02_nnfs_coding_our_first_neurons.ipynb#scrollTo=00aa98bb-97e9-4872-b87a-90f12687587f)"
      ],
      "metadata": {
        "id": "tJ6wJbNA3lSW"
      },
      "id": "tJ6wJbNA3lSW"
    },
    {
      "cell_type": "markdown",
      "id": "c0c3976b-bc69-46de-905a-a0423d4eee5c",
      "metadata": {
        "id": "c0c3976b-bc69-46de-905a-a0423d4eee5c"
      },
      "source": [
        "# Coding Our First Neurons\n",
        "- We assume that we all have a basic of programing here.\n",
        "- In this Notebook we focusing on using `python` also `numpy` for some vectorizaton and `matplotlib` for some visualizations."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b08a8e5e-508f-43be-befd-1e4b84956134",
      "metadata": {
        "id": "b08a8e5e-508f-43be-befd-1e4b84956134"
      },
      "outputs": [],
      "source": [
        "import platform\n",
        "import numpy as np\n",
        "import matplotlib\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "print(platform.python_version())\n",
        "print(np.__version__)\n",
        "print(matplotlib. __version__)\n",
        "\n",
        "#python version 3.9.7\n",
        "#numpy version 1.21.2\n",
        "#matplotlib version 3.5.0"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5c1fcba7-982f-498b-8719-7b5ff724dde1",
      "metadata": {
        "id": "5c1fcba7-982f-498b-8719-7b5ff724dde1"
      },
      "source": [
        "## A Single Neuron\n",
        "- As we discussed in previous notebook. \n",
        "- `feature` a group of features makes up a feature set (represented as vectors/arrays), and the values of a feature set can be reffed to as a `sample` (training example)\n",
        "- Let's say we have a single neuron, and there inputs to this neuron"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c470fe8b-3be9-404f-ba74-14946299b842",
      "metadata": {
        "id": "c470fe8b-3be9-404f-ba74-14946299b842"
      },
      "outputs": [],
      "source": [
        "# single neuron with three input features\n",
        "inputs = [1, 2, 3]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b2c58efc-bc86-48d9-877f-ae049e8fc0b2",
      "metadata": {
        "id": "b2c58efc-bc86-48d9-877f-ae049e8fc0b2"
      },
      "source": [
        "- Each input need to have parameters, knob or **weight** associated with it"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "89503de3-38c6-443e-b721-4ede9e3faacf",
      "metadata": {
        "id": "89503de3-38c6-443e-b721-4ede9e3faacf"
      },
      "outputs": [],
      "source": [
        "# initialized weight\n",
        "weights = [0.2, 0.8, -0.5]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3d1c9cd3-e819-4225-aac5-0a63bc483846",
      "metadata": {
        "id": "3d1c9cd3-e819-4225-aac5-0a63bc483846"
      },
      "source": [
        "- Finally we also need the bias, At the moment, we're modeling a single neuron with three input. So we only need one bias"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "bc03049f-0ab2-4e1f-a45e-15a4a1c98b72",
      "metadata": {
        "id": "bc03049f-0ab2-4e1f-a45e-15a4a1c98b72"
      },
      "outputs": [],
      "source": [
        "# initialized bias\n",
        "bias = 2"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ef52f8b6-f7d3-43d4-81a8-f41023129fe6",
      "metadata": {
        "id": "ef52f8b6-f7d3-43d4-81a8-f41023129fe6"
      },
      "source": [
        "- Remember we talk about the output function `yhat`?\n",
        "- `yhat` = **sum(weights * inputs) + bias"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c9e221df-3aa0-4bc4-89fe-31d9e99a17a1",
      "metadata": {
        "id": "c9e221df-3aa0-4bc4-89fe-31d9e99a17a1"
      },
      "outputs": [],
      "source": [
        "# calculate the output layer\n",
        "output = (inputs[0] * weights[0] +\n",
        "          inputs[1] * weights[1] +\n",
        "          inputs[2] * weights[2] + bias)\n",
        "\n",
        "print(f'output layer of this single neuron is {output}')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6590b2e8-78c5-4426-8d9f-51db5e1a7392",
      "metadata": {
        "id": "6590b2e8-78c5-4426-8d9f-51db5e1a7392"
      },
      "source": [
        "- What might we need to change if we have 4 inputs, rather than 3"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "05de527a-b3e9-4d88-a541-2a4378fac74a",
      "metadata": {
        "id": "05de527a-b3e9-4d88-a541-2a4378fac74a"
      },
      "outputs": [],
      "source": [
        "# input features one neuron with four features\n",
        "inputs = [1, 2, 3, 2.5]\n",
        "\n",
        "# weight associate to the inputs \n",
        "weights = [0.2, 0.8, -0.5, 1.0]\n",
        "\n",
        "# one neuron = one bias\n",
        "bias = 2.0\n",
        "\n",
        "output = (inputs[0] * weights[0] +\n",
        "          inputs[1] * weights[1] +\n",
        "          inputs[2] * weights[2] +\n",
        "          inputs[3] * weights[3] + bias)\n",
        "\n",
        "print(f'output layer of this single neuron is {output}')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "46c2efc1-a970-49d3-b6eb-a3074e3af28d",
      "metadata": {
        "id": "46c2efc1-a970-49d3-b6eb-a3074e3af28d"
      },
      "source": [
        "## Layers of Neurons\n",
        "- Same things but with multiple neurons\n",
        "- Let's say we have a scenario with three neurons in layer and 4 inputs"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2210e26e-1326-43aa-8161-cc156eae7221",
      "metadata": {
        "id": "2210e26e-1326-43aa-8161-cc156eae7221"
      },
      "outputs": [],
      "source": [
        "# input features one neuron with four features\n",
        "inputs = [1, 2, 3, 2.5]\n",
        "\n",
        "# three weight associate to the inputs \n",
        "weights1 = [0.2, 0.8, -0.5, 1.0]\n",
        "weights2 = [0.5, -0.91, 0.26, -0.5]\n",
        "weights3 = [-0.26, -0.27, 0.17, 0.87]\n",
        "\n",
        "# because of we have three neurons that right there will be three bias\n",
        "bias1 = 2\n",
        "bias2 = 3\n",
        "bias3 = 0.5\n",
        "\n",
        "output = [\n",
        "    # neuron 1\n",
        "    (inputs[0] * weights1[0] +\n",
        "     inputs[1] * weights1[1] +\n",
        "     inputs[2] * weights1[2] +\n",
        "     inputs[3] * weights1[3] + bias1),\n",
        "    # neuron 2\n",
        "    (inputs[0] * weights2[0] +\n",
        "     inputs[1] * weights2[1] +\n",
        "     inputs[2] * weights2[2] +\n",
        "     inputs[3] * weights2[3] + bias2),\n",
        "    # neuron 3\n",
        "    (inputs[0] * weights3[0] +\n",
        "     inputs[1] * weights3[1] +\n",
        "     inputs[2] * weights3[2] +\n",
        "     inputs[3] * weights3[3] + bias3)]\n",
        "    \n",
        "print(f'output layer of this three neurons is {output}')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "4efe92e7-383b-4512-8da1-091861468a82",
      "metadata": {
        "id": "4efe92e7-383b-4512-8da1-091861468a82"
      },
      "source": [
        "- Let use for loop"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c335da6b-2b4b-4f5a-a662-9c8d680449d9",
      "metadata": {
        "id": "c335da6b-2b4b-4f5a-a662-9c8d680449d9"
      },
      "outputs": [],
      "source": [
        "# input features one neuron with four features\n",
        "inputs = [1, 2, 3, 2.5]\n",
        "weights = [[0.2, 0.8, -0.5, 1.0],\n",
        "           [0.5, -0.91, 0.26, -0.5],\n",
        "           [-0.26, -0.27, 0.17, 0.87]]\n",
        "biases = [2, 3, 0.5]\n",
        "\n",
        "\n",
        "layer_outputs = []\n",
        "# for each neuron\n",
        "for neuron_weights, neuron_bias in zip(weights, biases):\n",
        "    # zeroed output of given neuron\n",
        "    neuron_output = 0\n",
        "    # for each input and weight to the neuron\n",
        "    for neuron_input, neuron_weight in zip (inputs, neuron_weights):\n",
        "        # apply output layer function ( weight * input_features) + bias\n",
        "        neuron_output += neuron_input * neuron_weight\n",
        "        \n",
        "    layer_outputs.append(neuron_output + neuron_bias)\n",
        "    \n",
        "print(f'output layers of this three neurons is {layer_outputs}')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "12077636-d755-4312-8364-096f0dd0a367",
      "metadata": {
        "id": "12077636-d755-4312-8364-096f0dd0a367"
      },
      "source": [
        "## Tensors Arrays Vectors\n",
        "- In Deep learning, Because of the dataset is huge, using for loop is not going to work.\n",
        "- We use `Tensors`, `Arrays`, `Vectors`, and `Matrixes` instead"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e1c3de23-a04d-4996-9037-5f6f98df282d",
      "metadata": {
        "id": "e1c3de23-a04d-4996-9037-5f6f98df282d"
      },
      "outputs": [],
      "source": [
        "# array or list\n",
        "l = [1, 5, 6, 2]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "201aa4f1-7902-461e-a0bc-a0529da9a6a7",
      "metadata": {
        "id": "201aa4f1-7902-461e-a0bc-a0529da9a6a7"
      },
      "outputs": [],
      "source": [
        "# a list of lists\n",
        "lol = [[1, 5, 6, 2],\n",
        "       [3, 4, 1, 3]]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2dcff337-89ba-4485-97e1-b6117191b62f",
      "metadata": {
        "id": "2dcff337-89ba-4485-97e1-b6117191b62f"
      },
      "outputs": [],
      "source": [
        "# list of lists of lists\n",
        "lolol = [[[1,5,6,2],\n",
        "          [3,2,1,3]],\n",
        "         [[5,2,1,2],\n",
        "          [6,4,8,4]],\n",
        "         [[2,8,5,3],\n",
        "          [1,1,9,4]]]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "4102c4d2-456d-4450-8a04-661b3da54216",
      "metadata": {
        "id": "4102c4d2-456d-4450-8a04-661b3da54216"
      },
      "source": [
        "## Dot Product and Vector Addition\n",
        "- To simplyfied the process we use `numpy`\n",
        "- if you not yet install numpy:\n",
        "    - !pip install numpy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2c778149-f6eb-48b1-864f-f695fffefab8",
      "metadata": {
        "id": "2c778149-f6eb-48b1-864f-f695fffefab8"
      },
      "outputs": [],
      "source": [
        "# dot product using pure python\n",
        "a = [1, 2, 3]\n",
        "b = [2, 3, 4]\n",
        "\n",
        "dot_product = a[0] * b[0] + a[1] * b[1] + a[2] * b[2]\n",
        "print(dot_product)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "92d49214-6896-4e96-9bfb-1049751315bc",
      "metadata": {
        "id": "92d49214-6896-4e96-9bfb-1049751315bc"
      },
      "outputs": [],
      "source": [
        "# dot product using numpy\n",
        "a = [1, 2, 3]\n",
        "b = [2, 3, 4]\n",
        "\n",
        "dot_product = np.dot(a, b)\n",
        "print(dot_product)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a4125013-315f-4361-ad5d-0ab6d7108905",
      "metadata": {
        "id": "a4125013-315f-4361-ad5d-0ab6d7108905"
      },
      "source": [
        "## Single neuron with Numpy\n",
        "- You will be surprise how easy to calculate this function"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "67131ac4-8515-41ee-8006-1bddeb44e2c2",
      "metadata": {
        "id": "67131ac4-8515-41ee-8006-1bddeb44e2c2"
      },
      "outputs": [],
      "source": [
        "# input features one neuron with four features\n",
        "inputs = [1, 2, 3, 2.5]\n",
        "\n",
        "weights = [0.2, 0.8, -0.5, 1.0]\n",
        "bias = 2.0\n",
        "\n",
        "output = np.dot(weights, inputs) + bias\n",
        "print(f'output layer of this single neuron is {output}')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "159bf1ff-7061-45b7-a9bb-f6dee79d3064",
      "metadata": {
        "id": "159bf1ff-7061-45b7-a9bb-f6dee79d3064"
      },
      "source": [
        "## A Layer of Neurons with Numpy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ebfb4aa1-4a36-4726-86f1-25a5c8e9484c",
      "metadata": {
        "id": "ebfb4aa1-4a36-4726-86f1-25a5c8e9484c"
      },
      "outputs": [],
      "source": [
        "# input features one neuron with four features\n",
        "inputs = np.array([1, 2, 3, 2.5])\n",
        "weights = np.array([[0.2, 0.8, -0.5, 1.0],\n",
        "           [0.5, -0.91, 0.26, -0.5],\n",
        "           [-0.26, -0.27, 0.17, 0.87]])\n",
        "biases = [2, 3, 0.5]\n",
        "\n",
        "output = np.dot(weights, inputs) + bias\n",
        "print(f'output layer of this single neuron is {output}')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e08cb006-2f32-4cb6-8880-10655a81e30d",
      "metadata": {
        "id": "e08cb006-2f32-4cb6-8880-10655a81e30d"
      },
      "source": [
        "## A Batch of Data\n",
        "- To train, neural netowork tend to recieve data in batch."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7f1b63ca-d109-4f7c-a817-3c8c1a503a32",
      "metadata": {
        "id": "7f1b63ca-d109-4f7c-a817-3c8c1a503a32"
      },
      "outputs": [],
      "source": [
        "batch = np.array([[1,5,6,2],\n",
        "                  [3,2,1,3],\n",
        "                  [5,2,1,2],\n",
        "                  [6,4,8,4],\n",
        "                  [2,8,5,3],\n",
        "                  [1,1,9,4],\n",
        "                  [6,6,0,4],\n",
        "                  [8,7,6,4]])\n",
        "print(batch.shape)\n",
        "print('Simply as we have a 8 training example, and each sample have 4 features')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ecf1c264-1d8a-44da-be39-f22126905496",
      "metadata": {
        "id": "ecf1c264-1d8a-44da-be39-f22126905496"
      },
      "outputs": [],
      "source": [
        "# input features one neuron with four features\n",
        "inputs = np.array([[1, 2, 3, 2.5],\n",
        "                   [2, 5, -1, 2],\n",
        "                   [-1.5, 2.7, 3.3, -0.8]])\n",
        "weights = np.array([[0.2, 0.8, -0.5, 1.0],\n",
        "           [0.5, -0.91, 0.26, -0.5],\n",
        "           [-0.26, -0.27, 0.17, 0.87]])\n",
        "biases = [2, 3, 0.5]\n",
        "\n",
        "output = np.dot(weights, inputs) + bias\n",
        "print(f'output layer of this single neuron is {output}')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ea82a51c-6a20-460e-a360-6cb1438d64a0",
      "metadata": {
        "id": "ea82a51c-6a20-460e-a360-6cb1438d64a0"
      },
      "source": [
        "- You might get **ValueError: shapes (3,4) and (3,4) not aligned: 4 (dim 1) != 3 (dim 0)**\n",
        "- This is the real pain in the ass for me when I start learning DeepLearning\n",
        "- To multiply matrix we need match the `Dimension`\n",
        "- `(a, n)` dot `(n, b)`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "00aa98bb-97e9-4872-b87a-90f12687587f",
      "metadata": {
        "id": "00aa98bb-97e9-4872-b87a-90f12687587f"
      },
      "outputs": [],
      "source": [
        "# input features one neuron with four features\n",
        "inputs = np.array([[1, 2, 3, 2.5],\n",
        "                   [2, 5, -1, 2],\n",
        "                   [-1.5, 2.7, 3.3, -0.8]])\n",
        "weights = np.array([[0.2, 0.8, -0.5, 1.0],\n",
        "           [0.5, -0.91, 0.26, -0.5],\n",
        "           [-0.26, -0.27, 0.17, 0.87]])\n",
        "biases = [2, 3, 0.5]\n",
        "\n",
        "# np.Transpose is the function allow you to swap columns to rows and rows to columns\n",
        "output = np.dot(weights, inputs.T) + bias\n",
        "print(f'output layer of this single neuron is \\n{output}')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3eeba053-8720-48d5-9196-4b77f36fc5a2",
      "metadata": {
        "id": "3eeba053-8720-48d5-9196-4b77f36fc5a2"
      },
      "source": [
        "[**Learn more about Linear algebra**](https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab) A Youtube playlist by **3blue1brown**"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "XLpBYXL926o9"
      },
      "id": "XLpBYXL926o9",
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "torch110",
      "language": "python",
      "name": "torch110"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.7"
    },
    "colab": {
      "name": "02_nnfs_coding_our_first_neurons.ipynb",
      "provenance": []
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}