Part08_Time_To_Get_TRIGGERED.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 8: Time to Get TRIGGERED\n",
    "Let's take a detour into what's inside our network. We've been ignoring this so far so we can get a feel for what the network does, how to train it, how to perturb data to make things better, and we got a glimpse of some of the numbers (*hyperparameters*) that tune the network.\n",
    "\n",
    "We discussed briefly that the network is made up of *perceptrons*. It's time to look at a single perceptron. But first let's see where perceptrons come from."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Enter the Neuron\n",
    "Image a brain. Could be yours, or someone else's, or an animal brain, particularly if you've taken biology recently and done some dissection. If you take a strong microscope and zoom in on the contents of the brain, a lot of what you'll find are *neurons*. A neuron is a cell that listens to connections from other neurons, which pass along small amounts of electricity using chemical reactions. The neuron listens harder - gives more *weight* - to some of the neurons that are sending to it, and less to other connections, and the neuron adds up how much signal it's hearing overall. If what it's hearing is enough, the neuron will get triggered and send signals to other neurons it's connected to."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Easy-Going Neuron\n",
    "Let's pretend you're Bruce, and you are not easily triggered - you're a calm person and don't spread rumors or excitement very easily. Maybe you don't like noise very much, or think that most rumors are dumb... But let's say you have two classmates who are spreading rumors. Now you know that Cathy is very chatty and spreads rumors all the time. You don't usually spend much time listening to Cathy: you give her less *weight* in your decision to pass on a rumor. But Bob rarely passes rumors on, so when Bob speaks you tend to listen harder, to give him more *weight*. And let's say you might spread a rumor to your friend Yvette if you decide it's good enough to pass on.\n",
    "\n",
    "Now let's say Cathy is talking and spreading her usual rumors in her usual excitable way. She's saying a lot but you tend to tune her out. You don't get triggered - you don't pass on any of her rumors.\n",
    "\n",
    "Now let's say Bob quietly mentions something someone did earlier in class. Bob's not very excited so you decide not to get triggered here either. You don't think you need to tell Yvette about what Bob is saying.\n",
    "\n",
    "But now Bob and Cathy both start talking at the same time, Cathy more softly but Bob a bit more loudly than usual, and when you put the information together you decide to to tell Yvette. You just got triggered.\n",
    "\n",
    "Or let's say another day Bob is quiet but Cathy is really excited, way beyond her usual. She manages to get your attention and you decide to pass on her rumor. She overcame the small weight you gave her by being excited enough. Triggered again!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Excitable Neuron\n",
    "Now let's say you're Cathy the Chatty Neuron. You like spreading rumors and talking a lot, which already makes you easily triggered. You're listening to three friends John, Mary, and Joan. But your weight for each of them is high. So high that you get triggered on anything your friends say above a whisper, and tell Yvette and Bruce.\n",
    "\n",
    "You're easily triggered, it doesn't take much to get you going.\n",
    "\n",
    "Let's say John is whispering a new rumor, but Mary and Joan are quiet right now. Not quite enough to get excited about, so you stay quiet.\n",
    "\n",
    "But let's say Mary joins in. Now it's more interesting... and you decide to tell Yvette and Bruce. *Triggered.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making Squishy Brains Into Math\n",
    "OK so we get the idea of a neuron just a bit. But of course, at least at this point in time, our computers are great at math and passing electrical signals around a CPU chip, but aren't so good at being squishy, neuron-filled brains. We need a computer-neuron. So a bunch of years ago was created the *perceptron*.\n",
    "\n",
    "A perceptron keeps track of which other perceptrons are connected to it, how hard it listens to each one of them (the *weights*), and a number, the *bias*, that indicates how soon it gets triggered, with a low bias meaning excitable like Cathy, and a high bias meaning laid back like Bruce. When the perceptrons it listens to give it enough signal, and the neuron cares enough to give them enough weight to get bigger than its bias, it triggers and passes a number greater than zero to the perceptrons it sends signal to. When not triggered, the perceptron sends zero to its friends."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Time to Give It a Try\n",
    "The code below simulates a perceptron with 2 inputs 'Bob' and 'Mary', and their 2 weights, a bias, and 1 output to Yvette. If you want to act more like Bruce, set your bias closer to 1.0 or even higher. If you want to be more like Cathy, try 0.1. And you can give your friends different weights as well, listening harder to one or the other, then try setting how loud your friends are speaking and see if you get TRIGGERED."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9309d430d7d044be96e86eeceee26c31",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(FloatSlider(value=0.1, description='bob', max=1.0), FloatSlider(value=0.0, description='…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from ipywidgets import interact\n",
    "\n",
    "prefix=\" \"\n",
    "@interact(bob=(0.0,1.0,0.1), bobWeight=(0.0,1.0,0.1), mary=(0.0,1.0,0.1), maryWeight=(0.0,1.0,0.1), bias=(0.0,2.0,0.1))\n",
    "def amITriggered(bob=0.1, mary=0.0, bobWeight=0.2, maryWeight=0.3, bias=0.5):\n",
    "    global prefix\n",
    "    bobInput = (bob * bobWeight)\n",
    "    maryInput = (mary * maryWeight)\n",
    "    sumOfInputs = bobInput + maryInput \n",
    "    if sumOfInputs >= bias:\n",
    "        output = \"TRIGGERED! ==output==> 1.0\"\n",
    "        topMarquee = \"\\\\||||||||/\"\n",
    "        bottomMarquee = \"/||||||||\\\\\"\n",
    "    else:\n",
    "        output = \"Not triggered ==output==> 0.0\"\n",
    "        topMarquee = \"\"\n",
    "        bottomMarquee = prefix + \". . . . . .\"\n",
    "        if prefix == \"\":\n",
    "            prefix = \" \"\n",
    "        else:\n",
    "            prefix = \"\"\n",
    "    print(\"Bob  = %1.1f * (w=%1.1f) = %1.2f                     %s\" % (bob, bobWeight, bobInput, topMarquee))\n",
    "    print(\"                           + --> %1.2f >= %1.2f ? %s\" % (sumOfInputs, bias, output))\n",
    "    print(\"Mary = %1.1f * (w=%1.1f) = %1.2f                     %s\" % (mary, maryWeight, maryInput, bottomMarquee))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Coming up...\n",
    "We'll start tuning the hyperparameters to save a bit of training time at the cost of some accuracy, add some more wiggling (perturbation) to get even more data, and start splitting our data into training parts and testing parts..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}