16-CodeTesting.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Code Testing\n",
    "\n",
    "- smoke tests\n",
    "- unit tests\n",
    "- `pytest`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Writing Good Code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "All in all, write code that is:\n",
    "\n",
    "- Well organized (follows a style guide)\n",
    "- Documented\n",
    "- **Tested**\n",
    "\n",
    "And you will have understandable, maintainable, and trustable code. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Class Question #1\n",
    "\n",
    "Given the following code, which assert will fail?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def extend(input_arg):\n",
    "    output = input_arg.copy()\n",
    "    for element in input_arg:\n",
    "        output.append(element)\n",
    "    return output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# test here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- A) `assert type(extend([1, 2])) == list`\n",
    "- B) `assert extend([1, 2]) == [1, 2, 1, 2]`\n",
    "- C) `assert extend((1, 2)) == (1, 2, 1, 2)` \n",
    "- D) `assert extend(['a', 'b', 'c']) == ['a', 'b', 'c', 'a', 'b', 'c']`\n",
    "- E) `assert extend([]) == []`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Clicker Question - Asserts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Check that extend returns a list\n",
    "assert type(extend([1, 2])) == list   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Check that an input list returns the expected result\n",
    "assert extend([1, 2]) == [1, 2, 1, 2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Check if the function works on tuples\n",
    "assert extend((1, 2)) == (1, 2, 1, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Check that a different input list (different lengths / contents) returns expected result\n",
    "assert extend(['a', 'b', 'c']) == ['a', 'b', 'c', 'a', 'b', 'c'] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Check that an empty list executes, executing an empty list\n",
    "assert extend([]) == []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Code Testing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<div class=\"alert alert-success\">\n",
    "Code tests are code that run and check other code, to make sure it does what it is expected to do. \n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Levels of Code Testing:\n",
    "\n",
    "- Smoke Tests\n",
    "- **Unit Tests**\n",
    "- Integration Tests\n",
    "- System Tests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "#### Four general types\n",
    "\n",
    "1. **Smoke tests** - preliminary tests to basic functionality; checks if something runs (but not necessarily if it does the right thing) (gut check) \n",
    "2. **Unit tests** - test functions & objects to ensure that they code is behaving as expected\n",
    "3. **Integration tests** - tests functions, classes & modules interacting\n",
    "4. **System tests** - tests end-to-end behavior "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Unit Tests\n",
    "\n",
    "- one test for each \"piece\" of your code (each function, each class, each module, etc)\n",
    "- passes silently if true\n",
    "- error if it fails\n",
    "- consider \"edge cases\"\n",
    "- help you resist the urge to assume computers will act how you think it will work \n",
    "- functions used with pytest start with `test_`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Not tests, but related\n",
    "- `assert`s - make a statement of fact about code \n",
    "- static checks - check your code as you go that it behaves as expected\n",
    "- argument validation - checks to see if the input given to a function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Why Write Tests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- To ensure code does what it is supposed to\n",
    "- To have a system for checking things when you change things in the code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Tests, when run, help identify code that will give an error if something has gone wrong. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## The Best (Laziest) Argument for Writing Tests\n",
    "\n",
    "Whenever you write new code, you will find yourself using little snippets of code to check it. \n",
    "\n",
    "Collect these snippets into a test function, and you get re-runnable tests for free."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![](img/code_testing.png)\n",
    "\n",
    "Source: https://twitter.com/jimhester_/status/1361697676832739328"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## How to Write Tests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Given a function or class you want to test:\n",
    "- You need to have an expectation for what it should do\n",
    "- Write out some example cases, with known answers\n",
    "- Use `assert` to check that your example cases do run as expected\n",
    "- Collect these examples into test functions, stored in test files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Example Test Code\n",
    "\n",
    "What the function should do: add two inputs together"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def add(num1, num2):\n",
    "    return num1 + num2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "def test_add():\n",
    "    \"\"\"Tests for the `add` function.\"\"\"\n",
    "    \n",
    "    # Test adding positve numbers\n",
    "    assert add(2, 2) == 4\n",
    "     \n",
    "    # Test adding negative numbers\n",
    "    assert add(-2, -2) == -4\n",
    "    \n",
    "    # Test adding floats\n",
    "    #assert add(2.7, 1.2) == 3.9\n",
    "    assert math.isclose(add(2.7, 1.2), 3.9)\n",
    "    \n",
    "    # Test adding with 0\n",
    "    assert add(2, 0) == 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Run our test function\n",
    "test_add()\n",
    "\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Class Question #2\n",
    "\n",
    "If you were asked to write a function `remove_punctuation` that removed all the punctuation from a given input string...what are some things that would be `True` of the output of that function (and therefore what `assert` statements would you write)?\n",
    "\n",
    "- A) I've got some ideas!\n",
    "- B) I tried but I'm stuck.\n",
    "- C) I'm lost/don't understand what we're supposed to be doing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "Brainstorm here..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# assert statements here\n",
    "assert remove_punctuation(\"Hello!\") == \"Hello\"\n",
    "assert remove_punctuation(\"!Hello\") == \"Hello\"\n",
    "assert remove_punctuation(\"He?llo\") == \"Hello\"\n",
    "assert remove_punctuation(\"He,ll>o\") == \"Hello\"\n",
    "assert remove_punctuation(\"Hello\") == \"Hello\"\n",
    "assert remove_punctuation(\"#$%^$%\") == \"\"\n",
    "assert remove_punctuation(\"Hello!\" + \" There!\") == \"Hello There\"\n",
    "assert remove_punctuation(\"\") == \"\"\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "# test function here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# function here\n",
    "def test_remove_punctuation(input_string):\n",
    "    assert remove_punctuation(\"Hello!\") == \"Hello\"\n",
    "    assert remove_punctuation(\"He'llo!\") == \"Hello\"\n",
    "    assert remove_punctuation(\"He\\\"llo!\") == \"Hello\"\n",
    "    assert remove_punctuation(\"!Hello\") == \"Hello\"\n",
    "    assert remove_punctuation(\"He?llo\") == \"Hello\"\n",
    "    assert remove_punctuation(\"He,ll>o\") == \"Hello\"\n",
    "    assert remove_punctuation(\"Hello\") == \"Hello\"\n",
    "    assert remove_punctuation(\"#$%^$%\") == \"\"\n",
    "    assert remove_punctuation(\"Hello!\" + \" There!\") == \"Hello There\"\n",
    "    assert remove_punctuation(\"\") == \"\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Class Question #3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Given the following function:\n",
    "def divide_list(in_list):    \n",
    "    output = []\n",
    "    \n",
    "    for el1, el2 in zip(in_list[1:], in_list[0:-1]):\n",
    "        output.append(el1 / el2)\n",
    "    \n",
    "    return output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# And the following test function:\n",
    "def test_divide_list():\n",
    "    assert callable(divide_list)\n",
    "    assert type(divide_list([1, 2])) == list\n",
    "    assert divide_list([1, 2, 4]) == [2, 2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "test_divide_list()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- A) These tests will pass, and this function is well tested\n",
    "- B) These tests will pass, but this function needs more tests\n",
    "- C) These tests will fail, but they cover the needed cases\n",
    "- D) These tests will fail, and we should also have more tests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "divide_list((0,2,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Test Driven Development"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<div class=\"alert alert-success\">\n",
    "In software development, <b>test-driven development</b> is an approach in which you write tests first -  and then write code to pass the tests. \n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Test Driven Development"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Ensures you go into writing code with a good plan / outline\n",
    "- Ensures that you have a test suite, as you can not decide to neglect test code after the fact\n",
    "- Note: when you complete (or at least write) assignments for this class, you are effectively doing test-driven development"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Test Coverage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<div class=\"alert alert-success\">\n",
    "<b>Test coverage</b> is the proportion of a software project that is run by the test suite. \n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Class Question #4\n",
    "\n",
    "Write a test function that checks the following piece of code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "def sum_list(input_list):\n",
    "    \"\"\"add all values in a list - return sum\"\"\"\n",
    "    \n",
    "    output = 0\n",
    "    \n",
    "    for val in input_list:\n",
    "        output += val\n",
    "        \n",
    "    return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "- A) I did it!\n",
    "- B) I think I did it!\n",
    "- C) I'm lost."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "Thought process:\n",
    "1. Define test function `def test_...`\n",
    "2. make `assert`ion within the test function\n",
    "    - check that the function is callable\n",
    "    - check the output is expected output / expected type\n",
    "    - check that function sums the list (which was our expectation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def test_sum_list():\n",
    "    # YOUR CODE HERE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_sum_list()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "### POSSIBLE TEST\n",
    "def test_sum_list():\n",
    "    \n",
    "    # write multiple asserts\n",
    "    assert callable(sum_list)\n",
    "    assert type(sum_list([1, 2, 3, 4])) == int\n",
    "    assert sum_list([1, 2, 3, 4]) == 10\n",
    "    \n",
    "test_sum_list()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Testing Code: when `input()` is used"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "def get_input():\n",
    "    \"\"\"ask user for an input message\n",
    "    \n",
    "    Returns\n",
    "    -------\n",
    "    msg : str\n",
    "        text specified on input by user\n",
    "    out_msg : None\n",
    "        always returns None; subsequent functions would return a more specific out_msg\n",
    "    \"\"\"\n",
    "    \n",
    "    msg = input('INPUT :\\t')\n",
    "    out_msg = None\n",
    "    \n",
    "    return msg, out_msg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "get_input()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "import mock\n",
    "import builtins\n",
    "\n",
    "def test_get_input():\n",
    "    # specify after lambda what you want function to use as \"input\"\n",
    "    with mock.patch.object(builtins, 'input', lambda _: 'Hello'):\n",
    "\n",
    "        # execute function and specify something that asserts True\n",
    "        msg, out_msg = get_input()\n",
    "        assert msg == 'Hello'\n",
    "        assert out_msg == None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "test_get_input()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<div class=\"alert alert-danger\">\n",
    "If your function only has <code>print</code> statements as its output, it will *not* be testable. Consider this during development/planning!\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## PyTest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<div class=\"alert alert-info\">\n",
    "<b><a href = 'https://docs.pytest.org/en/latest/'> PyTest </a></b> is a module that for writing and running test code. It is available in Anaconda and datahub.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### `pytest`\n",
    "\n",
    "- check if error raised when expected to be raised\n",
    "- autorun all of your tests\n",
    "- formal testing to your code/projects\n",
    "\n",
    "#### Executing `pytest`\n",
    "\n",
    "1. Look for any file called `test_...`\n",
    "2. If everything works, silently moves along. \n",
    "4. For anything that fails, will alert you.\n",
    "\n",
    "**Available from Anaconda and on datahub**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pytest test_sums.py\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  },
  "rise": {
   "scroll": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}