talk-solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"http://codigodelsur.com/wp-content/uploads/2017/11/1_IvCDlfi3vQfgyKO1eFv4jA.png\" \n",
    "alt=\"IMAGE ALT TEXT HERE\" width=\"240\" height=\"180\" /></img>\n",
    "\n",
    "\n",
    "## Spokane Python  --  Using GraphQL in Python\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The goal of this talk is to help people understand what GraphQL is and why it's useful. I will also discuss best practices when using GraphQL and how to use GraphQL in Python.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "#### First, in order to better understand the need for GraphQL we first need to look at the structure of a simple REST API. Let's use the Star Wars API (SWAPI).\n",
    "\n",
    "You could even go back further and look at the [landscape before REST](https://blog.readme.io/the-history-of-rest-apis/). Remember SOAP? Did you ever use CORBA?\n",
    "\n",
    "### Let's use the [Star Wars REST API](https://swapi.co/) as an example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'birth_year': '19BBY',\n",
      " 'created': '2014-12-09T13:50:51.644000Z',\n",
      " 'edited': '2014-12-20T21:17:56.891000Z',\n",
      " 'eye_color': 'blue',\n",
      " 'films': ['https://swapi.co/api/films/2/',\n",
      "           'https://swapi.co/api/films/6/',\n",
      "           'https://swapi.co/api/films/3/',\n",
      "           'https://swapi.co/api/films/1/',\n",
      "           'https://swapi.co/api/films/7/'],\n",
      " 'gender': 'male',\n",
      " 'hair_color': 'blond',\n",
      " 'height': '172',\n",
      " 'homeworld': 'https://swapi.co/api/planets/1/',\n",
      " 'mass': '77',\n",
      " 'name': 'Luke Skywalker',\n",
      " 'skin_color': 'fair',\n",
      " 'species': ['https://swapi.co/api/species/1/'],\n",
      " 'starships': ['https://swapi.co/api/starships/12/',\n",
      "               'https://swapi.co/api/starships/22/'],\n",
      " 'url': 'https://swapi.co/api/people/1/',\n",
      " 'vehicles': ['https://swapi.co/api/vehicles/14/',\n",
      "              'https://swapi.co/api/vehicles/30/']}\n"
     ]
    }
   ],
   "source": [
    "import pprint\n",
    "import requests\n",
    "\n",
    "# Query Luke Skywalker\n",
    "resp = requests.get('https://swapi.co/api/people/1/')\n",
    "pprint.pprint(resp.json())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above we are doing a simple REST request for __/people/1/__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### How do we load Luke's films and star ships?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Luke's Films...\n",
      "\n",
      "[{'characters': ['https://swapi.co/api/people/1/',\n",
      "                 'https://swapi.co/api/people/2/',\n",
      "                 'https://swapi.co/api/people/3/',\n",
      "                 'https://swapi.co/api/people/4/',\n",
      "                 'https://swapi.co/api/people/5/',\n",
      "                 'https://swapi.co/api/people/10/',\n",
      "                 'https://swapi.co/api/people/13/',\n",
      "                 'https://swapi.co/api/people/14/',\n",
      "                 'https://swapi.co/api/people/18/',\n",
      "                 'https://swapi.co/api/people/20/',\n",
      "                 'https://swapi.co/api/people/21/',\n",
      "                 'https://swapi.co/api/people/22/',\n",
      "                 'https://swapi.co/api/people/23/',\n",
      "                 'https://swapi.co/api/people/24/',\n",
      "                 'https://swapi.co/api/people/25/',\n",
      "                 'https://swapi.co/api/people/26/'],\n",
      "  'created': '2014-12-12T11:26:24.656000Z',\n",
      "  'director': 'Irvin Kershner',\n",
      "  'edited': '2017-04-19T10:57:29.544256Z',\n",
      "  'episode_id': 5,\n",
      "  'opening_crawl': 'It is a dark time for the\\r\\n'\n",
      "                   'Rebellion. Although the Death\\r\\n'\n",
      "                   'Star has been destroyed,\\r\\n'\n",
      "                   'Imperial troops have driven the\\r\\n'\n",
      "                   'Rebel forces from their hidden\\r\\n'\n",
      "                   'base and pursued them across\\r\\n'\n",
      "                   'the galaxy.\\r\\n'\n",
      "                   '\\r\\n'\n",
      "                   'Evading the dreaded Imperial\\r\\n'\n",
      "                   'Starfleet, a group of freedom\\r\\n'\n",
      "                   'fighters led by Luke Skywalker\\r\\n'\n",
      "                   'has established a new secret\\r\\n'\n",
      "                   'base on the remote ice world\\r\\n'\n",
      "                   'of Hoth.\\r\\n'\n",
      "                   '\\r\\n'\n",
      "                   'The evil lord Darth Vader,\\r\\n'\n",
      "                   'obsessed with finding young\\r\\n'\n",
      "                   'Skywalker, has dispatched\\r\\n'\n",
      "                   'thousands of remote probes into\\r\\n'\n",
      "                   'the far reaches of space....',\n",
      "  'planets': ['https://swapi.co/api/planets/4/',\n",
      "              'https://swapi.co/api/planets/5/',\n",
      "              'https://swapi.co/api/planets/6/',\n",
      "              'https://swapi.co/api/planets/27/'],\n",
      "  'producer': 'Gary Kurtz, Rick McCallum',\n",
      "  'release_date': '1980-05-17',\n",
      "  'species': ['https://swapi.co/api/species/6/',\n",
      "              'https://swapi.co/api/species/7/',\n",
      "              'https://swapi.co/api/species/3/',\n",
      "              'https://swapi.co/api/species/2/',\n",
      "              'https://swapi.co/api/species/1/'],\n",
      "  'starships': ['https://swapi.co/api/starships/15/',\n",
      "                'https://swapi.co/api/starships/10/',\n",
      "                'https://swapi.co/api/starships/11/',\n",
      "                'https://swapi.co/api/starships/12/',\n",
      "                'https://swapi.co/api/starships/21/',\n",
      "                'https://swapi.co/api/starships/22/',\n",
      "                'https://swapi.co/api/starships/23/',\n",
      "                'https://swapi.co/api/starships/3/',\n",
      "                'https://swapi.co/api/starships/17/'],\n",
      "  'title': 'The Empire Strikes Back',\n",
      "  'url': 'https://swapi.co/api/films/2/',\n",
      "  'vehicles': ['https://swapi.co/api/vehicles/8/',\n",
      "               'https://swapi.co/api/vehicles/14/',\n",
      "               'https://swapi.co/api/vehicles/16/',\n",
      "               'https://swapi.co/api/vehicles/18/',\n",
      "               'https://swapi.co/api/vehicles/19/',\n",
      "               'https://swapi.co/api/vehicles/20/']},\n",
      " {'characters': ['https://swapi.co/api/people/1/',\n",
      "                 'https://swapi.co/api/people/2/',\n",
      "                 'https://swapi.co/api/people/3/',\n",
      "                 'https://swapi.co/api/people/4/',\n",
      "                 'https://swapi.co/api/people/5/',\n",
      "                 'https://swapi.co/api/people/6/',\n",
      "                 'https://swapi.co/api/people/7/',\n",
      "                 'https://swapi.co/api/people/10/',\n",
      "                 'https://swapi.co/api/people/11/',\n",
      "                 'https://swapi.co/api/people/12/',\n",
      "                 'https://swapi.co/api/people/13/',\n",
      "                 'https://swapi.co/api/people/20/',\n",
      "                 'https://swapi.co/api/people/21/',\n",
      "                 'https://swapi.co/api/people/33/',\n",
      "                 'https://swapi.co/api/people/46/',\n",
      "                 'https://swapi.co/api/people/51/',\n",
      "                 'https://swapi.co/api/people/52/',\n",
      "                 'https://swapi.co/api/people/53/',\n",
      "                 'https://swapi.co/api/people/54/',\n",
      "                 'https://swapi.co/api/people/55/',\n",
      "                 'https://swapi.co/api/people/56/',\n",
      "                 'https://swapi.co/api/people/58/',\n",
      "                 'https://swapi.co/api/people/63/',\n",
      "                 'https://swapi.co/api/people/64/',\n",
      "                 'https://swapi.co/api/people/67/',\n",
      "                 'https://swapi.co/api/people/68/',\n",
      "                 'https://swapi.co/api/people/75/',\n",
      "                 'https://swapi.co/api/people/78/',\n",
      "                 'https://swapi.co/api/people/79/',\n",
      "                 'https://swapi.co/api/people/80/',\n",
      "                 'https://swapi.co/api/people/81/',\n",
      "                 'https://swapi.co/api/people/82/',\n",
      "                 'https://swapi.co/api/people/83/',\n",
      "                 'https://swapi.co/api/people/35/'],\n",
      "  'created': '2014-12-20T18:49:38.403000Z',\n",
      "  'director': 'George Lucas',\n",
      "  'edited': '2015-04-11T09:45:44.862122Z',\n",
      "  'episode_id': 3,\n",
      "  'opening_crawl': 'War! The Republic is crumbling\\r\\n'\n",
      "                   'under attacks by the ruthless\\r\\n'\n",
      "                   'Sith Lord, Count Dooku.\\r\\n'\n",
      "                   'There are heroes on both sides.\\r\\n'\n",
      "                   'Evil is everywhere.\\r\\n'\n",
      "                   '\\r\\n'\n",
      "                   'In a stunning move, the\\r\\n'\n",
      "                   'fiendish droid leader, General\\r\\n'\n",
      "                   'Grievous, has swept into the\\r\\n'\n",
      "                   'Republic capital and kidnapped\\r\\n'\n",
      "                   'Chancellor Palpatine, leader of\\r\\n'\n",
      "                   'the Galactic Senate.\\r\\n'\n",
      "                   '\\r\\n'\n",
      "                   'As the Separatist Droid Army\\r\\n'\n",
      "                   'attempts to flee the besieged\\r\\n'\n",
      "                   'capital with their valuable\\r\\n'\n",
      "                   'hostage, two Jedi Knights lead a\\r\\n'\n",
      "                   'desperate mission to rescue the\\r\\n'\n",
      "                   'captive Chancellor....',\n",
      "  'planets': ['https://swapi.co/api/planets/2/',\n",
      "              'https://swapi.co/api/planets/5/',\n",
      "              'https://swapi.co/api/planets/8/',\n",
      "              'https://swapi.co/api/planets/9/',\n",
      "              'https://swapi.co/api/planets/12/',\n",
      "              'https://swapi.co/api/planets/13/',\n",
      "              'https://swapi.co/api/planets/14/',\n",
      "              'https://swapi.co/api/planets/15/',\n",
      "              'https://swapi.co/api/planets/16/',\n",
      "              'https://swapi.co/api/planets/17/',\n",
      "              'https://swapi.co/api/planets/18/',\n",
      "              'https://swapi.co/api/planets/19/',\n",
      "              'https://swapi.co/api/planets/1/'],\n",
      "  'producer': 'Rick McCallum',\n",
      "  'release_date': '2005-05-19',\n",
      "  'species': ['https://swapi.co/api/species/19/',\n",
      "              'https://swapi.co/api/species/33/',\n",
      "              'https://swapi.co/api/species/2/',\n",
      "              'https://swapi.co/api/species/3/',\n",
      "              'https://swapi.co/api/species/36/',\n",
      "              'https://swapi.co/api/species/37/',\n",
      "              'https://swapi.co/api/species/6/',\n",
      "              'https://swapi.co/api/species/1/',\n",
      "              'https://swapi.co/api/species/34/',\n",
      "              'https://swapi.co/api/species/15/',\n",
      "              'https://swapi.co/api/species/35/',\n",
      "              'https://swapi.co/api/species/20/',\n",
      "              'https://swapi.co/api/species/23/',\n",
      "              'https://swapi.co/api/species/24/',\n",
      "              'https://swapi.co/api/species/25/',\n",
      "              'https://swapi.co/api/species/26/',\n",
      "              'https://swapi.co/api/species/27/',\n",
      "              'https://swapi.co/api/species/28/',\n",
      "              'https://swapi.co/api/species/29/',\n",
      "              'https://swapi.co/api/species/30/'],\n",
      "  'starships': ['https://swapi.co/api/starships/48/',\n",
      "                'https://swapi.co/api/starships/59/',\n",
      "                'https://swapi.co/api/starships/61/',\n",
      "                'https://swapi.co/api/starships/32/',\n",
      "                'https://swapi.co/api/starships/63/',\n",
      "                'https://swapi.co/api/starships/64/',\n",
      "                'https://swapi.co/api/starships/65/',\n",
      "                'https://swapi.co/api/starships/66/',\n",
      "                'https://swapi.co/api/starships/74/',\n",
      "                'https://swapi.co/api/starships/75/',\n",
      "                'https://swapi.co/api/starships/2/',\n",
      "                'https://swapi.co/api/starships/68/'],\n",
      "  'title': 'Revenge of the Sith',\n",
      "  'url': 'https://swapi.co/api/films/6/',\n",
      "  'vehicles': ['https://swapi.co/api/vehicles/33/',\n",
      "               'https://swapi.co/api/vehicles/50/',\n",
      "               'https://swapi.co/api/vehicles/53/',\n",
      "               'https://swapi.co/api/vehicles/56/',\n",
      "               'https://swapi.co/api/vehicles/60/',\n",
      "               'https://swapi.co/api/vehicles/62/',\n",
      "               'https://swapi.co/api/vehicles/67/',\n",
      "               'https://swapi.co/api/vehicles/69/',\n",
      "               'https://swapi.co/api/vehicles/70/',\n",
      "               'https://swapi.co/api/vehicles/71/',\n",
      "               'https://swapi.co/api/vehicles/72/',\n",
      "               'https://swapi.co/api/vehicles/73/',\n",
      "               'https://swapi.co/api/vehicles/76/']},\n",
      " {'characters': ['https://swapi.co/api/people/1/',\n",
      "                 'https://swapi.co/api/people/2/',\n",
      "                 'https://swapi.co/api/people/3/',\n",
      "                 'https://swapi.co/api/people/4/',\n",
      "                 'https://swapi.co/api/people/5/',\n",
      "                 'https://swapi.co/api/people/10/',\n",
      "                 'https://swapi.co/api/people/13/',\n",
      "                 'https://swapi.co/api/people/14/',\n",
      "                 'https://swapi.co/api/people/16/',\n",
      "                 'https://swapi.co/api/people/18/',\n",
      "                 'https://swapi.co/api/people/20/',\n",
      "                 'https://swapi.co/api/people/21/',\n",
      "                 'https://swapi.co/api/people/22/',\n",
      "                 'https://swapi.co/api/people/25/',\n",
      "                 'https://swapi.co/api/people/27/',\n",
      "                 'https://swapi.co/api/people/28/',\n",
      "                 'https://swapi.co/api/people/29/',\n",
      "                 'https://swapi.co/api/people/30/',\n",
      "                 'https://swapi.co/api/people/31/',\n",
      "                 'https://swapi.co/api/people/45/'],\n",
      "  'created': '2014-12-18T10:39:33.255000Z',\n",
      "  'director': 'Richard Marquand',\n",
      "  'edited': '2015-04-11T09:46:05.220365Z',\n",
      "  'episode_id': 6,\n",
      "  'opening_crawl': 'Luke Skywalker has returned to\\r\\n'\n",
      "                   'his home planet of Tatooine in\\r\\n'\n",
      "                   'an attempt to rescue his\\r\\n'\n",
      "                   'friend Han Solo from the\\r\\n'\n",
      "                   'clutches of the vile gangster\\r\\n'\n",
      "                   'Jabba the Hutt.\\r\\n'\n",
      "                   '\\r\\n'\n",
      "                   'Little does Luke know that the\\r\\n'\n",
      "                   'GALACTIC EMPIRE has secretly\\r\\n'\n",
      "                   'begun construction on a new\\r\\n'\n",
      "                   'armored space station even\\r\\n'\n",
      "                   'more powerful than the first\\r\\n'\n",
      "                   'dreaded Death Star.\\r\\n'\n",
      "                   '\\r\\n'\n",
      "                   'When completed, this ultimate\\r\\n'\n",
      "                   'weapon will spell certain doom\\r\\n'\n",
      "                   'for the small band of rebels\\r\\n'\n",
      "                   'struggling to restore freedom\\r\\n'\n",
      "                   'to the galaxy...',\n",
      "  'planets': ['https://swapi.co/api/planets/5/',\n",
      "              'https://swapi.co/api/planets/7/',\n",
      "              'https://swapi.co/api/planets/8/',\n",
      "              'https://swapi.co/api/planets/9/',\n",
      "              'https://swapi.co/api/planets/1/'],\n",
      "  'producer': 'Howard G. Kazanjian, George Lucas, Rick McCallum',\n",
      "  'release_date': '1983-05-25',\n",
      "  'species': ['https://swapi.co/api/species/1/',\n",
      "              'https://swapi.co/api/species/2/',\n",
      "              'https://swapi.co/api/species/3/',\n",
      "              'https://swapi.co/api/species/5/',\n",
      "              'https://swapi.co/api/species/6/',\n",
      "              'https://swapi.co/api/species/8/',\n",
      "              'https://swapi.co/api/species/9/',\n",
      "              'https://swapi.co/api/species/10/',\n",
      "              'https://swapi.co/api/species/15/'],\n",
      "  'starships': ['https://swapi.co/api/starships/15/',\n",
      "                'https://swapi.co/api/starships/10/',\n",
      "                'https://swapi.co/api/starships/11/',\n",
      "                'https://swapi.co/api/starships/12/',\n",
      "                'https://swapi.co/api/starships/22/',\n",
      "                'https://swapi.co/api/starships/23/',\n",
      "                'https://swapi.co/api/starships/27/',\n",
      "                'https://swapi.co/api/starships/28/',\n",
      "                'https://swapi.co/api/starships/29/',\n",
      "                'https://swapi.co/api/starships/3/',\n",
      "                'https://swapi.co/api/starships/17/',\n",
      "                'https://swapi.co/api/starships/2/'],\n",
      "  'title': 'Return of the Jedi',\n",
      "  'url': 'https://swapi.co/api/films/3/',\n",
      "  'vehicles': ['https://swapi.co/api/vehicles/8/',\n",
      "               'https://swapi.co/api/vehicles/16/',\n",
      "               'https://swapi.co/api/vehicles/18/',\n",
      "               'https://swapi.co/api/vehicles/19/',\n",
      "               'https://swapi.co/api/vehicles/24/',\n",
      "               'https://swapi.co/api/vehicles/25/',\n",
      "               'https://swapi.co/api/vehicles/26/',\n",
      "               'https://swapi.co/api/vehicles/30/']},\n",
      " {'characters': ['https://swapi.co/api/people/1/',\n",
      "                 'https://swapi.co/api/people/2/',\n",
      "                 'https://swapi.co/api/people/3/',\n",
      "                 'https://swapi.co/api/people/4/',\n",
      "                 'https://swapi.co/api/people/5/',\n",
      "                 'https://swapi.co/api/people/6/',\n",
      "                 'https://swapi.co/api/people/7/',\n",
      "                 'https://swapi.co/api/people/8/',\n",
      "                 'https://swapi.co/api/people/9/',\n",
      "                 'https://swapi.co/api/people/10/',\n",
      "                 'https://swapi.co/api/people/12/',\n",
      "                 'https://swapi.co/api/people/13/',\n",
      "                 'https://swapi.co/api/people/14/',\n",
      "                 'https://swapi.co/api/people/15/',\n",
      "                 'https://swapi.co/api/people/16/',\n",
      "                 'https://swapi.co/api/people/18/',\n",
      "                 'https://swapi.co/api/people/19/',\n",
      "                 'https://swapi.co/api/people/81/'],\n",
      "  'created': '2014-12-10T14:23:31.880000Z',\n",
      "  'director': 'George Lucas',\n",
      "  'edited': '2015-04-11T09:46:52.774897Z',\n",
      "  'episode_id': 4,\n",
      "  'opening_crawl': 'It is a period of civil war.\\r\\n'\n",
      "                   'Rebel spaceships, striking\\r\\n'\n",
      "                   'from a hidden base, have won\\r\\n'\n",
      "                   'their first victory against\\r\\n'\n",
      "                   'the evil Galactic Empire.\\r\\n'\n",
      "                   '\\r\\n'\n",
      "                   'During the battle, Rebel\\r\\n'\n",
      "                   'spies managed to steal secret\\r\\n'\n",
      "                   \"plans to the Empire's\\r\\n\"\n",
      "                   'ultimate weapon, the DEATH\\r\\n'\n",
      "                   'STAR, an armored space\\r\\n'\n",
      "                   'station with enough power\\r\\n'\n",
      "                   'to destroy an entire planet.\\r\\n'\n",
      "                   '\\r\\n'\n",
      "                   \"Pursued by the Empire's\\r\\n\"\n",
      "                   'sinister agents, Princess\\r\\n'\n",
      "                   'Leia races home aboard her\\r\\n'\n",
      "                   'starship, custodian of the\\r\\n'\n",
      "                   'stolen plans that can save her\\r\\n'\n",
      "                   'people and restore\\r\\n'\n",
      "                   'freedom to the galaxy....',\n",
      "  'planets': ['https://swapi.co/api/planets/2/',\n",
      "              'https://swapi.co/api/planets/3/',\n",
      "              'https://swapi.co/api/planets/1/'],\n",
      "  'producer': 'Gary Kurtz, Rick McCallum',\n",
      "  'release_date': '1977-05-25',\n",
      "  'species': ['https://swapi.co/api/species/5/',\n",
      "              'https://swapi.co/api/species/3/',\n",
      "              'https://swapi.co/api/species/2/',\n",
      "              'https://swapi.co/api/species/1/',\n",
      "              'https://swapi.co/api/species/4/'],\n",
      "  'starships': ['https://swapi.co/api/starships/2/',\n",
      "                'https://swapi.co/api/starships/3/',\n",
      "                'https://swapi.co/api/starships/5/',\n",
      "                'https://swapi.co/api/starships/9/',\n",
      "                'https://swapi.co/api/starships/10/',\n",
      "                'https://swapi.co/api/starships/11/',\n",
      "                'https://swapi.co/api/starships/12/',\n",
      "                'https://swapi.co/api/starships/13/'],\n",
      "  'title': 'A New Hope',\n",
      "  'url': 'https://swapi.co/api/films/1/',\n",
      "  'vehicles': ['https://swapi.co/api/vehicles/4/',\n",
      "               'https://swapi.co/api/vehicles/6/',\n",
      "               'https://swapi.co/api/vehicles/7/',\n",
      "               'https://swapi.co/api/vehicles/8/']},\n",
      " {'characters': ['https://swapi.co/api/people/1/',\n",
      "                 'https://swapi.co/api/people/3/',\n",
      "                 'https://swapi.co/api/people/5/',\n",
      "                 'https://swapi.co/api/people/13/',\n",
      "                 'https://swapi.co/api/people/14/',\n",
      "                 'https://swapi.co/api/people/27/',\n",
      "                 'https://swapi.co/api/people/84/',\n",
      "                 'https://swapi.co/api/people/85/',\n",
      "                 'https://swapi.co/api/people/86/',\n",
      "                 'https://swapi.co/api/people/87/',\n",
      "                 'https://swapi.co/api/people/88/'],\n",
      "  'created': '2015-04-17T06:51:30.504780Z',\n",
      "  'director': 'J. J. Abrams',\n",
      "  'edited': '2015-12-17T14:31:47.617768Z',\n",
      "  'episode_id': 7,\n",
      "  'opening_crawl': 'Luke Skywalker has vanished.\\r\\n'\n",
      "                   'In his absence, the sinister\\r\\n'\n",
      "                   'FIRST ORDER has risen from\\r\\n'\n",
      "                   'the ashes of the Empire\\r\\n'\n",
      "                   'and will not rest until\\r\\n'\n",
      "                   'Skywalker, the last Jedi,\\r\\n'\n",
      "                   'has been destroyed.\\r\\n'\n",
      "                   ' \\r\\n'\n",
      "                   'With the support of the\\r\\n'\n",
      "                   'REPUBLIC, General Leia Organa\\r\\n'\n",
      "                   'leads a brave RESISTANCE.\\r\\n'\n",
      "                   'She is desperate to find her\\r\\n'\n",
      "                   'brother Luke and gain his\\r\\n'\n",
      "                   'help in restoring peace and\\r\\n'\n",
      "                   'justice to the galaxy.\\r\\n'\n",
      "                   ' \\r\\n'\n",
      "                   'Leia has sent her most daring\\r\\n'\n",
      "                   'pilot on a secret mission\\r\\n'\n",
      "                   'to Jakku, where an old ally\\r\\n'\n",
      "                   'has discovered a clue to\\r\\n'\n",
      "                   \"Luke's whereabouts....\",\n",
      "  'planets': ['https://swapi.co/api/planets/61/'],\n",
      "  'producer': 'Kathleen Kennedy, J. J. Abrams, Bryan Burk',\n",
      "  'release_date': '2015-12-11',\n",
      "  'species': ['https://swapi.co/api/species/3/',\n",
      "              'https://swapi.co/api/species/2/',\n",
      "              'https://swapi.co/api/species/1/'],\n",
      "  'starships': ['https://swapi.co/api/starships/77/',\n",
      "                'https://swapi.co/api/starships/10/'],\n",
      "  'title': 'The Force Awakens',\n",
      "  'url': 'https://swapi.co/api/films/7/',\n",
      "  'vehicles': []}]\n",
      "\n",
      "Luke's Star Ships...\n",
      "\n",
      "[{'MGLT': '100',\n",
      "  'cargo_capacity': '110',\n",
      "  'consumables': '1 week',\n",
      "  'cost_in_credits': '149999',\n",
      "  'created': '2014-12-12T11:19:05.340000Z',\n",
      "  'crew': '1',\n",
      "  'edited': '2014-12-22T17:35:44.491233Z',\n",
      "  'films': ['https://swapi.co/api/films/2/',\n",
      "            'https://swapi.co/api/films/3/',\n",
      "            'https://swapi.co/api/films/1/'],\n",
      "  'hyperdrive_rating': '1.0',\n",
      "  'length': '12.5',\n",
      "  'manufacturer': 'Incom Corporation',\n",
      "  'max_atmosphering_speed': '1050',\n",
      "  'model': 'T-65 X-wing',\n",
      "  'name': 'X-wing',\n",
      "  'passengers': '0',\n",
      "  'pilots': ['https://swapi.co/api/people/1/',\n",
      "             'https://swapi.co/api/people/9/',\n",
      "             'https://swapi.co/api/people/18/',\n",
      "             'https://swapi.co/api/people/19/'],\n",
      "  'starship_class': 'Starfighter',\n",
      "  'url': 'https://swapi.co/api/starships/12/'},\n",
      " {'MGLT': '50',\n",
      "  'cargo_capacity': '80000',\n",
      "  'consumables': '2 months',\n",
      "  'cost_in_credits': '240000',\n",
      "  'created': '2014-12-15T13:04:47.235000Z',\n",
      "  'crew': '6',\n",
      "  'edited': '2014-12-22T17:35:44.795405Z',\n",
      "  'films': ['https://swapi.co/api/films/2/', 'https://swapi.co/api/films/3/'],\n",
      "  'hyperdrive_rating': '1.0',\n",
      "  'length': '20',\n",
      "  'manufacturer': 'Sienar Fleet Systems',\n",
      "  'max_atmosphering_speed': '850',\n",
      "  'model': 'Lambda-class T-4a shuttle',\n",
      "  'name': 'Imperial shuttle',\n",
      "  'passengers': '20',\n",
      "  'pilots': ['https://swapi.co/api/people/1/',\n",
      "             'https://swapi.co/api/people/13/',\n",
      "             'https://swapi.co/api/people/14/'],\n",
      "  'starship_class': 'Armed government transport',\n",
      "  'url': 'https://swapi.co/api/starships/22/'}]\n"
     ]
    }
   ],
   "source": [
    "resp = requests.get('https://swapi.co/api/people/1/')\n",
    "\n",
    "# As you can see this requires multiple round trips to the REST API server in order to get the data I need\n",
    "lukes_films = [requests.get(film_url).json() for film_url in resp.json().get('films', [])]\n",
    "lukes_starships = [requests.get(starship_url).json() for starship_url in resp.json().get('starships', [])]\n",
    "\n",
    "print(\"Luke's Films...\\n\")\n",
    "pprint.pprint(lukes_films)\n",
    "\n",
    "print(\"\\nLuke's Star Ships...\\n\")\n",
    "pprint.pprint(lukes_starships)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Is there a better/more efficient way to access the data that I need through REST? Maybe HTTP/2 Multiplexing?\n",
    "\n",
    "Without creating an explosion of custom REST endoints, there really isn't a better way to achieve these kind of queries via REST. Designing a RESTful API requires careful coordination between the backend developers and the frontend developers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GraphQL to the Rescue\n",
    "\n",
    "![alt text](http://guimperarnau.com/files/blog/Fantastic-GANs-and-where-to-find-them/hype_train.jpg \"Credit: \")\n",
    "\n",
    "\n",
    "#### So what is GraphQL?\n",
    "> GraphQL is a query language for your API. GraphQL isn't tied to any specific database or storage engine and is instead backed by your existing code and data.\n",
    "> -- [graphql.org](http://graphql.org/learn/)\n",
    "\n",
    "##### Awesome Resources\n",
    "\n",
    "* https://graphql.org\n",
    "* https://www.howtographql.com/\n",
    "* https://dev-blog.apollodata.com/\n",
    "* https://github.com/chentsulin/awesome-graphql\n",
    "\n",
    "Yes calling it GraphQL is confusing.\n",
    "\n",
    "Here GraphQL example that simplifies the above REST API queries...\n",
    "\n",
    "[Example Star Wars API in GraphQL](http://graphql.org/swapi-graphql/)\n",
    "\n",
    "Please try to look past the semantics of \"*Connection\", \"edges\" and \"node\". These are [Relay semantics](https://facebook.github.io/relay/graphql/connections.htm).\n",
    "\n",
    "```bash\n",
    "{\n",
    "  person(id: \"cGVvcGxlOjE=\") {\n",
    "    name\n",
    "    birthYear\n",
    "    eyeColor\n",
    "    films {\n",
    "      id\n",
    "      title\n",
    "    }\n",
    "    starships {\n",
    "      id\n",
    "      name\n",
    "      model\n",
    "    }\n",
    "  }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### GraphQL's Killer App GraphiQL\n",
    "\n",
    "Do people really keep up with docs? Nope. Ugh, Swagger docs.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* https://developer.github.com/v4/explorer/   (Yelp uses Graphene!)\n",
    "* https://www.yelp.com/developers/graphiql"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [Python Graphene](http://graphene-python.org/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[GraphQLError('Cannot query field \"fosss\" on type \"Query\".',)]\n",
      "null\n"
     ]
    }
   ],
   "source": [
    "# GraphQL is not bound to a protocol, yet people \n",
    "\n",
    "import json\n",
    "import graphene\n",
    "\n",
    "\n",
    "\n",
    "class Query(graphene.ObjectType):\n",
    "    hello = graphene.String()\n",
    "    foo = graphene.String()\n",
    "    baz = graphene.Int()\n",
    "    \n",
    "    def resolve_hello(self, info):\n",
    "        return 'World'\n",
    "\n",
    "    def resolve_foo(self, info):\n",
    "        return 'Bar'\n",
    "\n",
    "    \n",
    "\n",
    "schema = graphene.Schema(query=Query)\n",
    "\n",
    "resp = schema.execute('''\n",
    "  query {\n",
    "    hello\n",
    "    foo\n",
    "    baz\n",
    "    fo\n",
    "  }\n",
    "''')\n",
    "\n",
    "print(json.dumps(resp.data, indent=4))\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### More Complex Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/site-packages/ipykernel_launcher.py:59: DeprecationWarning: Please use assertEqual instead.\n",
      "..\n",
      "----------------------------------------------------------------------\n",
      "Ran 2 tests in 0.005s\n",
      "\n",
      "OK\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<unittest.runner.TextTestResult run=2 errors=0 failures=0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# React's Jest framework has popularized Snapshot testing\n",
    "# https://facebook.github.io/jest/docs/en/snapshot-testing.html\n",
    "# You should look at Snapshot testing in Python using https://github.com/syrusakbary/snapshottest\n",
    "import json\n",
    "import unittest\n",
    "\n",
    "unittest.TestCase.maxDiff = None\n",
    "\n",
    "import graphene\n",
    "\n",
    "\n",
    "class Gender(graphene.Enum):\n",
    "    MALE = 1\n",
    "    FEMALE = 2\n",
    "    NA = 3\n",
    "\n",
    "\n",
    "class Person(graphene.ObjectType):\n",
    "    id = graphene.ID()\n",
    "    name = graphene.String()\n",
    "    birth_year = graphene.String()\n",
    "    gender = graphene.Field(Gender)\n",
    "\n",
    "\n",
    "class Query(graphene.ObjectType):\n",
    "    person = graphene.Field(Person, id=graphene.ID(required=True))\n",
    "    \n",
    "    def resolve_person(self, info, id):\n",
    "        # TODO: Implement method\n",
    "        return Person(id=1, name=\"Luke Skywalker\", birth_year=\"19BBY\", gender=1)\n",
    "    \n",
    "    \n",
    "class CreatePerson(graphene.Mutation):\n",
    "    class Arguments:\n",
    "        id = graphene.ID()\n",
    "        name = graphene.String()\n",
    "        birth_year = graphene.String()\n",
    "        gender = graphene.Argument(Gender, default_value=Gender.NA.value)\n",
    "\n",
    "    ok = graphene.Boolean()\n",
    "    person = graphene.Field(Person)\n",
    "\n",
    "    def mutate(self, info, name, birth_year, gender):\n",
    "        # TODO: Implement method\n",
    "        return CreatePerson(ok=True, person=Person(id=2, name=name, birth_year=birth_year, gender=gender))\n",
    "\n",
    "\n",
    "class Mutation(graphene.ObjectType):\n",
    "    create_person = CreatePerson.Field()\n",
    "\n",
    "\n",
    "schema = graphene.Schema(query=Query, mutation=Mutation)\n",
    "\n",
    "\n",
    "class TestGrapheneQuery(unittest.TestCase):\n",
    "    \n",
    "    def assertResponseEqual(self, resp, expected):\n",
    "        self.assertIsNone(resp.errors)\n",
    "        self.assertEquals(json.dumps(resp.data, indent=4), json.dumps(expected, indent=4))\n",
    "        \n",
    "    def test_retrieve_person(self):\n",
    "        resp = schema.execute('''\n",
    "          query {\n",
    "            person(id: \"1\"){\n",
    "                id\n",
    "                name\n",
    "                birthYear\n",
    "                gender\n",
    "            }\n",
    "          }\n",
    "        ''')\n",
    "        expected = {\n",
    "            \"person\": {\n",
    "                \"id\": \"1\",\n",
    "                \"name\": \"Luke Skywalker\",\n",
    "                \"birthYear\": \"19BBY\",\n",
    "                \"gender\": \"MALE\"\n",
    "             }\n",
    "        }\n",
    "        self.assertResponseEqual(resp, expected)\n",
    "        \n",
    "    def test_create_person(self):\n",
    "        resp = schema.execute('''\n",
    "          mutation {\n",
    "            createPerson(name: \"C-3PO\", birthYear: \"112BBY\", gender: NA){\n",
    "                person{\n",
    "                    id\n",
    "                    name\n",
    "                    birthYear\n",
    "                    gender\n",
    "                }\n",
    "                ok\n",
    "            }\n",
    "          }\n",
    "        ''')\n",
    "        expected = {\n",
    "            \"createPerson\": {\n",
    "                \"person\": {\n",
    "                    \"id\": \"2\",\n",
    "                    \"name\": \"C-3PO\",\n",
    "                    \"birthYear\": \"112BBY\",\n",
    "                    \"gender\": \"NA\"\n",
    "                },\n",
    "                \"ok\": True\n",
    "             }\n",
    "        }\n",
    "        self.assertResponseEqual(resp, expected)\n",
    "        \n",
    "\n",
    "suite = unittest.TestLoader().loadTestsFromModule(TestGrapheneQuery())\n",
    "unittest.TextTestRunner().run(suite)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### NCAA March Madness 2018\n",
    "\n",
    "https://en.wikipedia.org/wiki/March_Madness_pools"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "    {\n",
      "        \"abbrev\": \"LIU\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"LIU-Brooklyn\",\n",
      "        \"bracket_challenge_id\": \"101-visitor\",\n",
      "        \"seed\": \"16\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21316\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"RADFRD\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Radford\",\n",
      "        \"bracket_challenge_id\": \"217-visitor\",\n",
      "        \"seed\": \"16\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21186\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"STBON\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"St. Bona.\",\n",
      "        \"bracket_challenge_id\": \"221-visitor\",\n",
      "        \"seed\": \"11\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21145\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"UCLA\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"UCLA\",\n",
      "        \"bracket_challenge_id\": \"102-home\",\n",
      "        \"seed\": \"11\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21341\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"ARIZST\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Arizona St.\",\n",
      "        \"bracket_challenge_id\": \"104-home\",\n",
      "        \"seed\": \"11\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21336\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"CUSE\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Syracuse\",\n",
      "        \"bracket_challenge_id\": \"229-visitor\",\n",
      "        \"seed\": \"11\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21170\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"NCCU\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"N.C. Central\",\n",
      "        \"bracket_challenge_id\": \"103-home\",\n",
      "        \"seed\": \"16\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"160973\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"TEXSO\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Texas So.\",\n",
      "        \"bracket_challenge_id\": \"209-visitor\",\n",
      "        \"seed\": \"16\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21396\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"UVA\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Virginia\",\n",
      "        \"bracket_challenge_id\": \"201-home\",\n",
      "        \"seed\": \"1\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21136\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"CINCY\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Cincinnati\",\n",
      "        \"bracket_challenge_id\": \"208-home\",\n",
      "        \"seed\": \"2\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21235\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"TENN\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Tennessee\",\n",
      "        \"bracket_challenge_id\": \"206-home\",\n",
      "        \"seed\": \"3\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21362\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"ARIZ\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Arizona\",\n",
      "        \"bracket_challenge_id\": \"204-home\",\n",
      "        \"seed\": \"4\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21335\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"UK\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Kentucky\",\n",
      "        \"bracket_challenge_id\": \"203-home\",\n",
      "        \"seed\": \"5\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21357\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"MIAMI\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Miami (Fla.)\",\n",
      "        \"bracket_challenge_id\": \"205-home\",\n",
      "        \"seed\": \"6\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21163\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"NEVADA\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Nevada\",\n",
      "        \"bracket_challenge_id\": \"207-home\",\n",
      "        \"seed\": \"7\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21219\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"CREIGH\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Creighton\",\n",
      "        \"bracket_challenge_id\": \"202-home\",\n",
      "        \"seed\": \"8\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21305\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"KSTATE\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Kansas St.\",\n",
      "        \"bracket_challenge_id\": \"202-visitor\",\n",
      "        \"seed\": \"9\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21205\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"TEXAS\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Texas\",\n",
      "        \"bracket_challenge_id\": \"207-visitor\",\n",
      "        \"seed\": \"10\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21210\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"LOYCHI\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Loyola-Chi.\",\n",
      "        \"bracket_challenge_id\": \"205-visitor\",\n",
      "        \"seed\": \"11\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21300\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"DAVID\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Davidson\",\n",
      "        \"bracket_challenge_id\": \"203-visitor\",\n",
      "        \"seed\": \"12\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21368\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"BUFF\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Buffalo\",\n",
      "        \"bracket_challenge_id\": \"204-visitor\",\n",
      "        \"seed\": \"13\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21267\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"WRIGHT\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Wright St.\",\n",
      "        \"bracket_challenge_id\": \"206-visitor\",\n",
      "        \"seed\": \"14\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21303\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"GAST\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Georgia St.\",\n",
      "        \"bracket_challenge_id\": \"208-visitor\",\n",
      "        \"seed\": \"15\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21408\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"UMBC\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"UMBC\",\n",
      "        \"bracket_challenge_id\": \"201-visitor\",\n",
      "        \"seed\": \"16\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21323\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"XAVIER\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Xavier\",\n",
      "        \"bracket_challenge_id\": \"209-home\",\n",
      "        \"seed\": \"1\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21149\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"UNC\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"N. Carolina\",\n",
      "        \"bracket_challenge_id\": \"216-home\",\n",
      "        \"seed\": \"2\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21134\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"MICH\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Michigan\",\n",
      "        \"bracket_challenge_id\": \"214-home\",\n",
      "        \"seed\": \"3\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21193\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"GONZAG\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Gonzaga\",\n",
      "        \"bracket_challenge_id\": \"212-home\",\n",
      "        \"seed\": \"4\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21415\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"OHIOST\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Ohio St.\",\n",
      "        \"bracket_challenge_id\": \"211-home\",\n",
      "        \"seed\": \"5\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21197\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"HOU\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Houston\",\n",
      "        \"bracket_challenge_id\": \"213-home\",\n",
      "        \"seed\": \"6\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21237\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"TEXAM\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Texas A&amp;M\",\n",
      "        \"bracket_challenge_id\": \"215-home\",\n",
      "        \"seed\": \"7\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21211\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"MIZZOU\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Missouri\",\n",
      "        \"bracket_challenge_id\": \"210-home\",\n",
      "        \"seed\": \"8\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21206\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"FSU\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Florida St.\",\n",
      "        \"bracket_challenge_id\": \"210-visitor\",\n",
      "        \"seed\": \"9\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21131\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"PROV\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Providence\",\n",
      "        \"bracket_challenge_id\": \"215-visitor\",\n",
      "        \"seed\": \"10\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21166\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"SDGST\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"San Diego St\",\n",
      "        \"bracket_challenge_id\": \"213-visitor\",\n",
      "        \"seed\": \"11\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21431\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"SDAKST\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"S. Dak. St.\",\n",
      "        \"bracket_challenge_id\": \"211-visitor\",\n",
      "        \"seed\": \"12\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"532957\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"NCGRN\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"UNC-Green.\",\n",
      "        \"bracket_challenge_id\": \"212-visitor\",\n",
      "        \"seed\": \"13\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21372\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"MNTNA\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Montana\",\n",
      "        \"bracket_challenge_id\": \"214-visitor\",\n",
      "        \"seed\": \"14\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21175\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"LPSCMB\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Lipscomb\",\n",
      "        \"bracket_challenge_id\": \"216-visitor\",\n",
      "        \"seed\": \"15\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"161302\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"PLAY4\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"NCCU/TXSO\",\n",
      "        \"bracket_challenge_id\": null,\n",
      "        \"seed\": \"16\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"67\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"NOVA\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Villanova\",\n",
      "        \"bracket_challenge_id\": \"217-home\",\n",
      "        \"seed\": \"1\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21171\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"PURDUE\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Purdue\",\n",
      "        \"bracket_challenge_id\": \"224-home\",\n",
      "        \"seed\": \"2\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21199\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"TXTECH\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Texas Tech\",\n",
      "        \"bracket_challenge_id\": \"222-home\",\n",
      "        \"seed\": \"3\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21212\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"WICHST\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Wichita St.\",\n",
      "        \"bracket_challenge_id\": \"220-home\",\n",
      "        \"seed\": \"4\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21313\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"WVU\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"W. Virginia\",\n",
      "        \"bracket_challenge_id\": \"219-home\",\n",
      "        \"seed\": \"5\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21172\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"FLA\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Florida\",\n",
      "        \"bracket_challenge_id\": \"221-home\",\n",
      "        \"seed\": \"6\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21355\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"ARK\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Arkansas\",\n",
      "        \"bracket_challenge_id\": \"223-home\",\n",
      "        \"seed\": \"7\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21353\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"VATECH\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Va. Tech\",\n",
      "        \"bracket_challenge_id\": \"218-home\",\n",
      "        \"seed\": \"8\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21148\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"BAMA\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Alabama\",\n",
      "        \"bracket_challenge_id\": \"218-visitor\",\n",
      "        \"seed\": \"9\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21352\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"BUTLER\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Butler\",\n",
      "        \"bracket_challenge_id\": \"223-visitor\",\n",
      "        \"seed\": \"10\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21296\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"PLAY2\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"STBON/UCLA\",\n",
      "        \"bracket_challenge_id\": null,\n",
      "        \"seed\": \"11\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"65\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"MURYST\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Murray St.\",\n",
      "        \"bracket_challenge_id\": \"219-visitor\",\n",
      "        \"seed\": \"12\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21330\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"MRSHL\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Marshall\",\n",
      "        \"bracket_challenge_id\": \"220-visitor\",\n",
      "        \"seed\": \"13\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21272\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"SFA\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"SF Austin\",\n",
      "        \"bracket_challenge_id\": \"222-visitor\",\n",
      "        \"seed\": \"14\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21383\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"CSFULL\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"CSFullerton\",\n",
      "        \"bracket_challenge_id\": \"224-visitor\",\n",
      "        \"seed\": \"15\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21216\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"PLAY1\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"LIU/RAD\",\n",
      "        \"bracket_challenge_id\": null,\n",
      "        \"seed\": \"16\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"64\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"KANSAS\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Kansas\",\n",
      "        \"bracket_challenge_id\": \"225-home\",\n",
      "        \"seed\": \"1\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21204\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"DUKE\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Duke\",\n",
      "        \"bracket_challenge_id\": \"232-home\",\n",
      "        \"seed\": \"2\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21130\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"MICHST\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Michigan St.\",\n",
      "        \"bracket_challenge_id\": \"230-home\",\n",
      "        \"seed\": \"3\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21194\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"AUBURN\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Auburn\",\n",
      "        \"bracket_challenge_id\": \"228-home\",\n",
      "        \"seed\": \"4\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21354\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"CLEM\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Clemson\",\n",
      "        \"bracket_challenge_id\": \"227-home\",\n",
      "        \"seed\": \"5\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21129\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"TCU\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"TCU\",\n",
      "        \"bracket_challenge_id\": \"229-home\",\n",
      "        \"seed\": \"6\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21434\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"RI\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Rhode Island\",\n",
      "        \"bracket_challenge_id\": \"231-home\",\n",
      "        \"seed\": \"7\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21144\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"SETON\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Seton Hall\",\n",
      "        \"bracket_challenge_id\": \"226-home\",\n",
      "        \"seed\": \"8\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21168\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"NCST\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"NC State\",\n",
      "        \"bracket_challenge_id\": \"226-visitor\",\n",
      "        \"seed\": \"9\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21135\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"OKLA\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Oklahoma\",\n",
      "        \"bracket_challenge_id\": \"231-visitor\",\n",
      "        \"seed\": \"10\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21208\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"PLAY3\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"ASU/SYR\",\n",
      "        \"bracket_challenge_id\": null,\n",
      "        \"seed\": \"11\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"66\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"NMEXST\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"New Mex. St.\",\n",
      "        \"bracket_challenge_id\": \"227-visitor\",\n",
      "        \"seed\": \"12\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21220\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"CHARLS\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Charleston\",\n",
      "        \"bracket_challenge_id\": \"228-visitor\",\n",
      "        \"seed\": \"13\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21367\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"BUCK\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Bucknell\",\n",
      "        \"bracket_challenge_id\": \"230-visitor\",\n",
      "        \"seed\": \"14\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21346\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"IONA\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Iona\",\n",
      "        \"bracket_challenge_id\": \"232-visitor\",\n",
      "        \"seed\": \"15\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21256\"\n",
      "    },\n",
      "    {\n",
      "        \"abbrev\": \"PENN\",\n",
      "        \"team_url\": \"\",\n",
      "        \"name\": \"Penn\",\n",
      "        \"bracket_challenge_id\": \"225-visitor\",\n",
      "        \"seed\": \"16\",\n",
      "        \"regionId\": \"\",\n",
      "        \"image_url\": \"\",\n",
      "        \"id\": \"21251\"\n",
      "    }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "# Let's scrape some data!\n",
    "import requests\n",
    "\n",
    "resp = requests.get('https://www.cbssports.com/data/collegebasketball/ncaa-tournament/brackets/update/2017/NCAAB?as=json')\n",
    "tournament = resp.json().get('tournament')\n",
    "bracket = tournament.get('bracket')\n",
    "locations = bracket.get('locations')\n",
    "regions = bracket.get('regions').get('region')\n",
    "rounds_by_region = [round.get('rounds').get('round') for round in regions]\n",
    "teams = bracket.get('teams', {}).get('team')\n",
    "\n",
    "\n",
    "print(json.dumps(teams, indent=4))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "."
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Preparing to import teams...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "."
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Importing teams 1/1...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "----------------------------------------------------------------------\n",
      "Ran 2 tests in 0.810s\n",
      "\n",
      "OK\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<unittest.runner.TextTestResult run=2 errors=0 failures=0>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's add some data to redis and elasticsearch!\n",
    "import arrow\n",
    "import unittest\n",
    "import mock\n",
    "import copy\n",
    "\n",
    "unittest.TestCase.maxDiff = None\n",
    "\n",
    "\n",
    "from requests_html import HTMLSession\n",
    "from elasticsearch import Elasticsearch\n",
    "es = Elasticsearch(['es'])\n",
    "import redis\n",
    "r = redis.StrictRedis(host='redis', port=6379, db=0)\n",
    "\n",
    "\n",
    "class MarchMadnessDataImporter(object):\n",
    "    def __init__(self, r, es):\n",
    "        \"\"\"Writes rows to Redis and ElasticSearch.\n",
    "        \n",
    "        Args:\n",
    "            r: A redis instance\n",
    "            es: A elasticsearch instance\n",
    "        \"\"\"\n",
    "        self.r = r\n",
    "        self.es = es\n",
    "        \n",
    "    def __retrieve_roster(self, abbrev):\n",
    "        session = HTMLSession()\n",
    "        resp = session.get('https://www.cbssports.com/collegebasketball/teams/roster/{0}'.format(abbrev))\n",
    "        table = resp.html.find('.data')\n",
    "\n",
    "        for elem in table[1:-1]:\n",
    "            player_rows = [row.text for row in elem.find('tr')[3:]]\n",
    "            for num, name, position, height, weight, univ_class, hometown in [player_row.split('\\n') for player_row in player_rows]:\n",
    "                yield {\n",
    "                    'abbrev': abbrev,\n",
    "                    'num': num,\n",
    "                    'name': name,\n",
    "                    'position': position,\n",
    "                    'height': height,\n",
    "                    'class': univ_class,\n",
    "                    'hometown': hometown\n",
    "                }\n",
    "\n",
    "        \n",
    "    def import_teams(self, teams):\n",
    "        total_teams = len(teams)\n",
    "        print(\"Preparing to import teams...\")\n",
    "        for team_idx, team in enumerate(teams):\n",
    "            key = 'team:{0}'.format(team.get('abbrev'))\n",
    "            self.r.set(key, json.dumps(team))\n",
    "            self.es.index(index='teams', doc_type='team', id=key, body=team)\n",
    "            for idx, player in enumerate(self.__retrieve_roster(team.get('abbrev'))):\n",
    "                key = 'player:{0}:{1}'.format(team.get('abbrev'), idx)\n",
    "                self.r.set(key, json.dumps(player))\n",
    "                self.es.index(index='players', doc_type='player', id=key, body=player)\n",
    "            print(\"Importing teams {0}/{1}...\".format(team_idx+1, total_teams))\n",
    "            \n",
    "    def __get_games_by_round(self, round_num):\n",
    "        south, west, east, midwest = rounds_by_region[1:-1]\n",
    "        games = []\n",
    "        games.extend(south[round_num].get('games').get('game'))\n",
    "        games.extend(west[round_num].get('games').get('game'))\n",
    "        games.extend(east[round_num].get('games').get('game'))\n",
    "        games.extend(midwest[round_num].get('games').get('game'))\n",
    "        return games\n",
    "\n",
    "    def import_rounds_by_regions(self, rounds_by_region):\n",
    "        all_games = []\n",
    "        round_1 = self.__get_games_by_round(0)\n",
    "        round_2 = self.__get_games_by_round(1)\n",
    "        sweet_16 = self.__get_games_by_round(2)\n",
    "        elite_8 = self.__get_games_by_round(3)\n",
    "        final_4 = rounds_by_region[-1][0].get('games').get('game')\n",
    "        championship = rounds_by_region[-1][1].get('games').get('game')\n",
    "        all_games.extend(round_1 + round_2 + sweet_16 + elite_8 + final_4 + championship)\n",
    "        for idx, game in enumerate(all_games):\n",
    "            if game.get('game_abbrev'):\n",
    "                away, home = game.get('game_abbrev').split('_')[2].split('@')\n",
    "                game.get('game_participant')[0]['abbrev'] = home \n",
    "                game.get('game_participant')[1]['abbrev'] = away\n",
    "            key = 'game:{0}'.format(idx)\n",
    "            self.r.set(key, json.dumps(game))\n",
    "            self.es.index(index='games', doc_type='game', id=key, body=game)\n",
    "\n",
    "            \n",
    "class TestMarchMadnessImporter(unittest.TestCase):\n",
    "    \n",
    "    def test_import_teams(self):\n",
    "        r = mock.Mock()\n",
    "        es = mock.Mock()\n",
    "        importer = MarchMadnessDataImporter(r, es)\n",
    "        importer.import_teams([{'abbrev': 'Gonz', 'name': 'Gonzaga', 'seed': 4}])\n",
    "        r.set.assert_called_with('team:Gonz', '{\"abbrev\": \"Gonz\", \"name\": \"Gonzaga\", \"seed\": 4}')\n",
    "        es.index.assert_called_with(index='teams', doc_type='team', id='team:Gonz', body={\"abbrev\": \"Gonz\", \"name\": \"Gonzaga\", \"seed\": 4})\n",
    "        \n",
    "    def test_import_rounds_by_regions(self):\n",
    "        pass\n",
    "        #TODO: implement test\n",
    "    \n",
    "    \n",
    "suite = unittest.TestLoader().loadTestsFromModule(TestMarchMadnessImporter())\n",
    "unittest.TextTestRunner().run(suite)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Preparing to import teams...\n",
      "Importing teams 1/72...\n",
      "Importing teams 2/72...\n",
      "Importing teams 3/72...\n",
      "Importing teams 4/72...\n",
      "Importing teams 5/72...\n",
      "Importing teams 6/72...\n",
      "Importing teams 7/72...\n",
      "Importing teams 8/72...\n",
      "Importing teams 9/72...\n",
      "Importing teams 10/72...\n",
      "Importing teams 11/72...\n",
      "Importing teams 12/72...\n",
      "Importing teams 13/72...\n",
      "Importing teams 14/72...\n",
      "Importing teams 15/72...\n",
      "Importing teams 16/72...\n",
      "Importing teams 17/72...\n",
      "Importing teams 18/72...\n",
      "Importing teams 19/72...\n",
      "Importing teams 20/72...\n",
      "Importing teams 21/72...\n",
      "Importing teams 22/72...\n",
      "Importing teams 23/72...\n",
      "Importing teams 24/72...\n",
      "Importing teams 25/72...\n",
      "Importing teams 26/72...\n",
      "Importing teams 27/72...\n",
      "Importing teams 28/72...\n",
      "Importing teams 29/72...\n",
      "Importing teams 30/72...\n",
      "Importing teams 31/72...\n",
      "Importing teams 32/72...\n",
      "Importing teams 33/72...\n",
      "Importing teams 34/72...\n",
      "Importing teams 35/72...\n",
      "Importing teams 36/72...\n",
      "Importing teams 37/72...\n",
      "Importing teams 38/72...\n",
      "Importing teams 39/72...\n",
      "Importing teams 40/72...\n",
      "Importing teams 41/72...\n",
      "Importing teams 42/72...\n",
      "Importing teams 43/72...\n",
      "Importing teams 44/72...\n",
      "Importing teams 45/72...\n",
      "Importing teams 46/72...\n",
      "Importing teams 47/72...\n",
      "Importing teams 48/72...\n",
      "Importing teams 49/72...\n",
      "Importing teams 50/72...\n",
      "Importing teams 51/72...\n",
      "Importing teams 52/72...\n",
      "Importing teams 53/72...\n",
      "Importing teams 54/72...\n",
      "Importing teams 55/72...\n",
      "Importing teams 56/72...\n",
      "Importing teams 57/72...\n",
      "Importing teams 58/72...\n",
      "Importing teams 59/72...\n",
      "Importing teams 60/72...\n",
      "Importing teams 61/72...\n",
      "Importing teams 62/72...\n",
      "Importing teams 63/72...\n",
      "Importing teams 64/72...\n",
      "Importing teams 65/72...\n",
      "Importing teams 66/72...\n",
      "Importing teams 67/72...\n",
      "Importing teams 68/72...\n",
      "Importing teams 69/72...\n",
      "Importing teams 70/72...\n",
      "Importing teams 71/72...\n",
      "Importing teams 72/72...\n"
     ]
    }
   ],
   "source": [
    "importer = MarchMadnessDataImporter(r, es)\n",
    "importer.import_rounds_by_regions(rounds_by_region)\n",
    "importer.import_teams(teams)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"took\": 8,\n",
      "    \"timed_out\": false,\n",
      "    \"_shards\": {\n",
      "        \"total\": 5,\n",
      "        \"successful\": 5,\n",
      "        \"skipped\": 0,\n",
      "        \"failed\": 0\n",
      "    },\n",
      "    \"hits\": {\n",
      "        \"total\": 1,\n",
      "        \"max_score\": 1.0,\n",
      "        \"hits\": [\n",
      "            {\n",
      "                \"_index\": \"players\",\n",
      "                \"_type\": \"player\",\n",
      "                \"_id\": \"player:CREIGH:5\",\n",
      "                \"_score\": 1.0,\n",
      "                \"_source\": {\n",
      "                    \"abbrev\": \"CREIGH\",\n",
      "                    \"num\": \"32\",\n",
      "                    \"name\": \"Toby Hegner\",\n",
      "                    \"position\": \"F\",\n",
      "                    \"height\": \"6-10\",\n",
      "                    \"class\": \"Sr.\",\n",
      "                    \"hometown\": \"Berlin, WI\"\n",
      "                }\n",
      "            }\n",
      "        ]\n",
      "    }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "res = es.search(index=\"players\", body={\n",
    "    \"query\": {\n",
    "        \"query_string\" : {\n",
    "            \"default_field\" : \"name\",\n",
    "            \"query\" : \"*toby*\"\n",
    "        }\n",
    "    }\n",
    "})\n",
    "print(json.dumps(res, indent=4))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's bulid the GraphQL schema  http://localhost\n",
    "\n",
    "Using gunicorn and gevent, let's build a GraphQL server based on green threads.\n",
    "\n",
    "Look at demo/application.py, demo/gunicorn_config.py and demo/schema.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dataloader Pattern\n",
    "\n",
    "Helps prevent recursive lookups sending a deluge of queries to your backend plus solves the [N+1 problem with ORMs](https://use-the-index-luke.com/sql/join/nested-loops-join-n1-problem).\n",
    "\n",
    "* https://github.com/facebook/dataloader\n",
    "* http://docs.graphene-python.org/en/latest/execution/dataloader/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# %load /code/demo/loaders.py\n",
    "\n",
    "from promise import Promise\n",
    "from promise.dataloader import DataLoader\n",
    "import redis\n",
    "\n",
    "from promise.schedulers.gevent import GeventScheduler\n",
    "from promise import set_default_scheduler\n",
    "from schema import Player\n",
    "\n",
    "set_default_scheduler(GeventScheduler())\n",
    "\n",
    "r = redis.StrictRedis(host='redis', port=6379, db=0)\n",
    "\n",
    "# http://docs.graphene-python.org/en/latest/execution/dataloader/\n",
    "class PlayerLoader(DataLoader):\n",
    "\n",
    "    def batch_load_fn(self, keys):\n",
    "        pipe = r.pipeline()\n",
    "        for key in keys:\n",
    "            pipe.get(key)\n",
    "        pipeline_resp = pipe.execute()\n",
    "        return Promise.resolve([Player.from_redis_obj(key, obj) for key, obj in zip(keys, pipeline_resp)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Advanced Topics\n",
    "* https://dev-blog.apollodata.com/graphql-at-facebook-by-dan-schafer-38d65ef075af\n",
    "* https://github.com/facebook/graphql/pull/267  GraphQL Subscriptions\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}