aggregator.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Information Aggregator"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Author: Iain Muir, iam9ez@virginia.edu  \n",
    "*Date: January 20th, 2021  \n",
    "Project: Information Aggregator Application — powered by Datapane*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sources"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* **[Robinhood](https://robin-stocks.readthedocs.io/en/latest/index.html)**\n",
    "* **[Finnhub](https://finnhub.io/docs/api/)**\n",
    "* **[ESPN](https://gist.github.com/akeaswaran/b48b02f1c94f873c6655e7129910fc3b)**\n",
    "* **[The New York Times](https://developer.nytimes.com/apis)**\n",
    "* **[CNN Money](https://money.cnn.com/data/markets)**\n",
    "* **[FiveThirtyEight](https://data.fivethirtyeight.com/)**\n",
    "* **[COVID Act Now](https://apidocs.covidactnow.org/api)**\n",
    "* **[Lichess](https://lichess.org/api)**\n",
    "* **[Spotify](https://developer.spotify.com/documentation/)**\n",
    "* **[YouTube]()**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Table of Contents <a class=\"anchor\" id=\"toc\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-01T17:36:00.730516Z",
     "iopub.status.busy": "2022-02-01T17:36:00.729491Z",
     "iopub.status.idle": "2022-02-01T17:36:00.743532Z",
     "shell.execute_reply": "2022-02-01T17:36:00.744113Z"
    }
   },
   "source": [
    "* **[0. Import Libraries and Secrets](#import)**  \n",
    "* **[1. Build Report Components](#build)**  \n",
    "    * [1.1 Header](#header)  \n",
    "    * [1.2 Main Selector](#selector)  \n",
    "        * *[1.2.1 Sports Results and Analytics](#espn)*  \n",
    "        * *[1.2.2 Top World News](#nyt)*  \n",
    "        * *[1.2.3 Stock Market](#finnhub)*  \n",
    "        * *[1.2.4 Data Exploration](#data)*  \n",
    "        * *[1.2.5 Compiled Selector](#compiled)*  \n",
    "    * [1.3 Credits](#credits)  \n",
    "* **[2. Datapane Report](#report)**  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 0. Import Libraries and Secrets <a class=\"anchor\" id=\"import\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Table of Contents](#toc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Importing Standard Libraries..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:11.214677Z",
     "iopub.status.busy": "2022-02-14T16:20:11.203708Z",
     "iopub.status.idle": "2022-02-14T16:20:12.429035Z",
     "shell.execute_reply": "2022-02-14T16:20:12.428080Z"
    }
   },
   "outputs": [],
   "source": [
    "from itertools import zip_longest\n",
    "from bs4 import BeautifulSoup\n",
    "from functools import partial\n",
    "import datapane as dp\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import webbrowser\n",
    "import warnings\n",
    "import datetime\n",
    "import requests\n",
    "import time\n",
    "import json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Importing Modules..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:12.437849Z",
     "iopub.status.busy": "2022-02-14T16:20:12.436504Z",
     "iopub.status.idle": "2022-02-14T16:20:12.939245Z",
     "shell.execute_reply": "2022-02-14T16:20:12.940012Z"
    }
   },
   "outputs": [],
   "source": [
    "from robinhood import ticker_toggle, make_header, authenticate_, load_portfolio, get_scroll_objects, robinhood_news\n",
    "from espn import format_news, news, parse_team, format_scores, scores, group_sport, standings\n",
    "from finnhub import quote, candles, candlestick, name_search, big_number, profile, news\n",
    "from nyt import top_stories, semantics, format_article, format_sections\n",
    "from cnn_money import cnn_big_numbers, format_modules, group_modules\n",
    "from fivethirtyeight import active_teams, make_button, plot_elo\n",
    "from covid import county_choropleth, plot_state, altair_line\n",
    "from spotify import get_spotify_embed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:12.958370Z",
     "iopub.status.busy": "2022-02-14T16:20:12.955812Z",
     "iopub.status.idle": "2022-02-14T16:20:12.967216Z",
     "shell.execute_reply": "2022-02-14T16:20:12.968513Z"
    }
   },
   "outputs": [],
   "source": [
    "from errors import ErrorHandler, Logging, get_error_info "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:12.976213Z",
     "iopub.status.busy": "2022-02-14T16:20:12.974239Z",
     "iopub.status.idle": "2022-02-14T16:20:12.982020Z",
     "shell.execute_reply": "2022-02-14T16:20:12.983455Z"
    }
   },
   "outputs": [],
   "source": [
    "from constants import ROOT, API_LOGOS, ESPN_SPORTS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Checking Datapane version..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:12.993900Z",
     "iopub.status.busy": "2022-02-14T16:20:12.989472Z",
     "iopub.status.idle": "2022-02-14T16:20:15.030648Z",
     "shell.execute_reply": "2022-02-14T16:20:15.031178Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "14.0"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "version = !datapane --version\n",
    "version_num = float(version[0].split()[2][2:])\n",
    "version_num"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:15.036951Z",
     "iopub.status.busy": "2022-02-14T16:20:15.036060Z",
     "iopub.status.idle": "2022-02-14T16:20:15.039835Z",
     "shell.execute_reply": "2022-02-14T16:20:15.040248Z"
    }
   },
   "outputs": [],
   "source": [
    "assert version_num >= 13.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:15.045891Z",
     "iopub.status.busy": "2022-02-14T16:20:15.045047Z",
     "iopub.status.idle": "2022-02-14T16:20:18.162075Z",
     "shell.execute_reply": "2022-02-14T16:20:18.162689Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32mConnected successfully to https://datapane.com as iam9ez@virginia.edu\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!datapane login --token=55010cebc170ecfbeddb82838c360776bf36f6be"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Open and Unpack Secrets File..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:18.170503Z",
     "iopub.status.busy": "2022-02-14T16:20:18.168745Z",
     "iopub.status.idle": "2022-02-14T16:20:18.176458Z",
     "shell.execute_reply": "2022-02-14T16:20:18.175908Z"
    }
   },
   "outputs": [],
   "source": [
    "with open('secrets.json') as s:\n",
    "    secrets = json.loads(s.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:18.184758Z",
     "iopub.status.busy": "2022-02-14T16:20:18.184005Z",
     "iopub.status.idle": "2022-02-14T16:20:18.186650Z",
     "shell.execute_reply": "2022-02-14T16:20:18.187420Z"
    }
   },
   "outputs": [],
   "source": [
    "# Datapane\n",
    "DATAPANE_KEY = secrets['datapane']\n",
    "    \n",
    "# Finnhub\n",
    "FINNHUB_KEY = secrets['finnhub']\n",
    "\n",
    "# Robinhood\n",
    "ROBIN_USERNAME, ROBIN_PASSWORD = secrets['robinhood'].values()\n",
    "\n",
    "# New York Times\n",
    "NYT_KEY,  NYT_SECRET, NYT_ID = secrets['nyt'].values()\n",
    "\n",
    "# Spotify\n",
    "SPOTIFY_SECRET, SPOTIFY_ID = secrets['spotify'].values()\n",
    "\n",
    "# COVID Act Now\n",
    "COVID_KEY = secrets['covid']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:18.196017Z",
     "iopub.status.busy": "2022-02-14T16:20:18.195117Z",
     "iopub.status.idle": "2022-02-14T16:20:18.348533Z",
     "shell.execute_reply": "2022-02-14T16:20:18.349215Z"
    }
   },
   "outputs": [],
   "source": [
    "AUTH_URL = 'https://accounts.spotify.com/api/token'\n",
    "auth_response = requests.post(\n",
    "    AUTH_URL, \n",
    "    {\n",
    "        'grant_type': 'client_credentials',\n",
    "        'client_id': SPOTIFY_ID,\n",
    "        'client_secret': SPOTIFY_SECRET\n",
    "    }\n",
    ")\n",
    "auth_response_data = auth_response.json()\n",
    "access_token = auth_response_data['access_token']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:18.359397Z",
     "iopub.status.busy": "2022-02-14T16:20:18.358230Z",
     "iopub.status.idle": "2022-02-14T16:20:18.480172Z",
     "shell.execute_reply": "2022-02-14T16:20:18.479432Z"
    }
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    robinhood = authenticate_(ROBIN_USERNAME, ROBIN_PASSWORD)\n",
    "except KeyError:\n",
    "    print(\n",
    "        ErrorHandler(\"Failed Robinhood Authentication\", *get_error_info())\n",
    "    )\n",
    "    robinhood = None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:18.485166Z",
     "iopub.status.busy": "2022-02-14T16:20:18.484157Z",
     "iopub.status.idle": "2022-02-14T16:20:18.486634Z",
     "shell.execute_reply": "2022-02-14T16:20:18.487068Z"
    }
   },
   "outputs": [],
   "source": [
    "assert robinhood is not None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Preferences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "warnings.simplefilter(action=\"ignore\", category=pd.core.common.SettingWithCopyWarning)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Build Report Compenents <a class=\"anchor\" id=\"build\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Table of Contents](#toc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:18.493662Z",
     "iopub.status.busy": "2022-02-14T16:20:18.492838Z",
     "iopub.status.idle": "2022-02-14T16:20:18.496102Z",
     "shell.execute_reply": "2022-02-14T16:20:18.496543Z"
    }
   },
   "outputs": [],
   "source": [
    "TODAY = datetime.date.today()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 Header <a class=\"anchor\" id=\"header\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:18.501410Z",
     "iopub.status.busy": "2022-02-14T16:20:18.500617Z",
     "iopub.status.idle": "2022-02-14T16:20:18.504074Z",
     "shell.execute_reply": "2022-02-14T16:20:18.503393Z"
    }
   },
   "outputs": [],
   "source": [
    "datapane, nyt, espn, spotify, _538, lichess = API_LOGOS.values()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:18.511587Z",
     "iopub.status.busy": "2022-02-14T16:20:18.510772Z",
     "iopub.status.idle": "2022-02-14T16:20:18.513647Z",
     "shell.execute_reply": "2022-02-14T16:20:18.514081Z"
    }
   },
   "outputs": [],
   "source": [
    "header_logo = dp.HTML(\n",
    "    \"\"\"\n",
    "    <html>\n",
    "        <style type='text/css'>\n",
    "            .images {\n",
    "                display:flex;\n",
    "                justify-content:center;\n",
    "                align-items:center;\n",
    "            }\n",
    "            .images img {\n",
    "                margin-left:5px;\n",
    "                margin-right:5px;\n",
    "            }\n",
    "        </style>\n",
    "        \n",
    "        <center>\n",
    "            <div class='images'>\n",
    "                <img src='\"\"\" + datapane + \"\"\"' width=\"75\"/>\n",
    "                <img src='\"\"\" + nyt + \"\"\"' width=\"100\"/>\n",
    "                <img src='\"\"\" + espn + \"\"\"' width=\"90\"/>\n",
    "                <img src='\"\"\" + spotify + \"\"\"' width=\"90\"/>\n",
    "                <img src='\"\"\" + _538 + \"\"\"' width=\"90\"/>\n",
    "            </div>\n",
    "        </center>\n",
    "    </html>\n",
    "    \"\"\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:18.519112Z",
     "iopub.status.busy": "2022-02-14T16:20:18.518415Z",
     "iopub.status.idle": "2022-02-14T16:20:18.524779Z",
     "shell.execute_reply": "2022-02-14T16:20:18.523672Z"
    }
   },
   "outputs": [],
   "source": [
    "header_text = dp.HTML(\n",
    "    \"\"\"\n",
    "    <html>\n",
    "        <style type='text/css'>\n",
    "            @keyframes rotate {\n",
    "                0%   {color: #EEE;}\n",
    "                25%  {color: #EC4899;}\n",
    "                50%  {color: #8B5CF6;}\n",
    "                100% {color: #EF4444;}\n",
    "            }\n",
    "            h1 {\n",
    "                color:#eee;\n",
    "                animation-name: rotate;\n",
    "                animation-duration: 4s;\n",
    "                animation-iteration-count: infinite;\n",
    "            }\n",
    "        </style>\n",
    "        <center>\n",
    "            <h1>Morning Scoop</h1>\n",
    "            <i>\"\"\" + TODAY.strftime('%A, %B %d, %Y') + \"\"\"<i>\n",
    "        </center>\n",
    "    </html>\n",
    "    \"\"\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:18.529484Z",
     "iopub.status.busy": "2022-02-14T16:20:18.528779Z",
     "iopub.status.idle": "2022-02-14T16:20:18.531506Z",
     "shell.execute_reply": "2022-02-14T16:20:18.531936Z"
    }
   },
   "outputs": [],
   "source": [
    "header_description = dp.Text(\n",
    "    \"\"\"\n",
    "Welcome to the Morning Scoop! This project is aimed to aggregate and display an eclectic and wide-ranging mix of information, data, and visualizations. Information sources include The New York Times, ESPN, FiveThirtyEight, and many more. Enjoy!\n",
    "\n",
    "This report is powered by Datapane.\n",
    "\n",
    "    \"\"\".strip()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Main Selector <a class=\"anchor\" id=\"selector\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Table of Contents](#toc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.2.1 Sports Results and Analytics <a class=\"anchor\" id=\"espn\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Scores/Analytics/YouTube Highlights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:18.539161Z",
     "iopub.status.busy": "2022-02-14T16:20:18.537617Z",
     "iopub.status.idle": "2022-02-14T16:20:20.248102Z",
     "shell.execute_reply": "2022-02-14T16:20:20.248695Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for KC @ DET; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for MIA @ WSH; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for MIL @ PIT; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for TB @ TOR; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for ATL @ CIN; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for NYY @ CLE; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for TEX @ NYM; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for LAA @ HOU; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for BAL @ MIN; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for BOS @ CHC; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for ARI @ COL; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for CHW @ SF; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for OAK @ SEA; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for SD @ LAD; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for STL @ PHI; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for ATL @ NYC; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for PHI @ CLB; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for CIN @ NE; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for RSL @ MIN; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for NY @ SKC; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for CLT @ HOU; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for POR @ NSH; OK.\n",
      "espn.py (Line: ~400) || CustomError: No Stats/Leaders for CHI @ SJ; OK.\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    espn_results = list(map(\n",
    "        scores, ESPN_SPORTS.items()\n",
    "    ))\n",
    "    espn_results = list(map(\n",
    "        lambda result: [dp.Text('No games played yesterday...')] if result == [] else result, espn_results\n",
    "    ))\n",
    "except:\n",
    "    print(\n",
    "        ErrorHandler(\"Failed to get ESPN Results\", *get_error_info())\n",
    "    )\n",
    "    exit(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Upcoming Schedule"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "try:   \n",
    "    espn_upcoming = list(map(\n",
    "        lambda sport: scores(sport, upcoming=True), ESPN_SPORTS.items()\n",
    "    ))\n",
    "    espn_upcoming = list(map(\n",
    "        lambda game: [dp.Text('No upcoming games today...')] if game == [] else game, espn_upcoming\n",
    "    ))\n",
    "except:\n",
    "    print(\n",
    "        ErrorHandler(\"Failed to get ESPN Schedule\", *get_error_info())\n",
    "    )\n",
    "    exit(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Standings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# records = list(map(\n",
    "#     standings, ESPN_SPORTS.items()\n",
    "# ))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Compiled ESPN Group"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Order Results and Schedules by Active Leagues"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "active = [\n",
    "    1 if not isinstance(result[0], dp.client.api.report.blocks.Text) else 0\n",
    "    for result in espn_results\n",
    "]\n",
    "active = pd.DataFrame({\n",
    "    'results': espn_results,\n",
    "    'league': ESPN_SPORTS.keys(),\n",
    "    'active': active\n",
    "}).sort_values(\n",
    "    by='active',\n",
    "    ascending=False\n",
    ")\n",
    "espn_results = active.results.tolist()\n",
    "league_results = active.league.tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "active = [\n",
    "    1 if not isinstance(upcoming[0], dp.client.api.report.blocks.Text) else 0\n",
    "    for upcoming in espn_upcoming\n",
    "]\n",
    "active = pd.DataFrame({\n",
    "    'upcoming': espn_upcoming,\n",
    "    'league': ESPN_SPORTS.keys(),\n",
    "    'active': active\n",
    "}).sort_values(\n",
    "    by='active',\n",
    "    ascending=False\n",
    ")\n",
    "espn_upcoming = active.upcoming.tolist()\n",
    "league_upcoming = active.league.tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "assert league_upcoming == league_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "espn_groups = list(map(\n",
    "    group_sport, league_results, espn_results, espn_upcoming\n",
    "))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.2.2 Top World News <a class=\"anchor\" id=\"nyt\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:20.278035Z",
     "iopub.status.busy": "2022-02-14T16:20:20.277002Z",
     "iopub.status.idle": "2022-02-14T16:20:20.830119Z",
     "shell.execute_reply": "2022-02-14T16:20:20.829502Z"
    }
   },
   "outputs": [],
   "source": [
    "articles = top_stories(NYT_KEY)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:20.846941Z",
     "iopub.status.busy": "2022-02-14T16:20:20.846367Z",
     "iopub.status.idle": "2022-02-14T16:20:20.850650Z",
     "shell.execute_reply": "2022-02-14T16:20:20.851214Z"
    }
   },
   "outputs": [],
   "source": [
    "articles = articles.loc[articles['item_type'] != 'Promo']\n",
    "articles = articles.loc[articles['multimedia'].str.len() >= 1]\n",
    "articles['multimedia'] = articles['multimedia'].str[0].str['url']\n",
    "articles['subsection'] = articles['subsection'].replace('', 'miscellaneous')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:20.862063Z",
     "iopub.status.busy": "2022-02-14T16:20:20.861172Z",
     "iopub.status.idle": "2022-02-14T16:20:20.864250Z",
     "shell.execute_reply": "2022-02-14T16:20:20.864763Z"
    }
   },
   "outputs": [],
   "source": [
    "grouped_articles = articles.groupby(\n",
    "    'subsection',\n",
    "    axis=0\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:20.869416Z",
     "iopub.status.busy": "2022-02-14T16:20:20.868775Z",
     "iopub.status.idle": "2022-02-14T16:20:20.917888Z",
     "shell.execute_reply": "2022-02-14T16:20:20.918381Z"
    }
   },
   "outputs": [],
   "source": [
    "ordered_groups = grouped_articles.size().sort_values(ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:20.923961Z",
     "iopub.status.busy": "2022-02-14T16:20:20.922896Z",
     "iopub.status.idle": "2022-02-14T16:20:20.943251Z",
     "shell.execute_reply": "2022-02-14T16:20:20.943781Z"
    }
   },
   "outputs": [],
   "source": [
    "article_sections = list(map(\n",
    "    partial(format_sections, grouped_articles), ordered_groups.index\n",
    "))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:20.948829Z",
     "iopub.status.busy": "2022-02-14T16:20:20.947703Z",
     "iopub.status.idle": "2022-02-14T16:20:20.951871Z",
     "shell.execute_reply": "2022-02-14T16:20:20.951381Z"
    }
   },
   "outputs": [],
   "source": [
    "# TODO Article Links"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.2.3 Stock Market <a class=\"anchor\" id=\"finnhub\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Ticker Scroll"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:20.962489Z",
     "iopub.status.busy": "2022-02-14T16:20:20.961075Z",
     "iopub.status.idle": "2022-02-14T16:20:24.979975Z",
     "shell.execute_reply": "2022-02-14T16:20:24.980812Z"
    }
   },
   "outputs": [],
   "source": [
    "t100 = robinhood.markets.get_top_100()\n",
    "t100 = pd.DataFrame(t100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:24.993819Z",
     "iopub.status.busy": "2022-02-14T16:20:24.992413Z",
     "iopub.status.idle": "2022-02-14T16:20:24.995772Z",
     "shell.execute_reply": "2022-02-14T16:20:24.996201Z"
    }
   },
   "outputs": [],
   "source": [
    "scroll_objects = t100[['last_trade_price', 'previous_close', 'symbol']].apply(\n",
    "    get_scroll_objects, axis=1\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:25.002837Z",
     "iopub.status.busy": "2022-02-14T16:20:25.001945Z",
     "iopub.status.idle": "2022-02-14T16:20:25.007167Z",
     "shell.execute_reply": "2022-02-14T16:20:25.008310Z"
    }
   },
   "outputs": [],
   "source": [
    "scroll = dp.HTML(\n",
    "    \"\"\"\n",
    "    <html>\n",
    "        <style type='text/css'>\n",
    "            body {\n",
    "                background: #FFFFFF;\n",
    "            }\n",
    "            \n",
    "            .news-scroll a {\n",
    "                font-size: 16px;\n",
    "                text-decoration: none;\n",
    "                color: #000000;\n",
    "            }\n",
    "            \n",
    "            .price {\n",
    "                font-size: 14px;\n",
    "            }\n",
    "            \n",
    "            .up {\n",
    "                font-size: 12px;\n",
    "                color: #228B22;\n",
    "            }\n",
    "            \n",
    "            .down {\n",
    "                font-size: 12px;\n",
    "                color: #D2042D;\n",
    "            }\n",
    "        </style>\n",
    "    \n",
    "        <div class=\"container mt-5\">\n",
    "            <div class=\"row\">\n",
    "                <div class=\"col-md-12\">\n",
    "                    <div class=\"d-flex justify-content-between align-items-center breaking-news bg-white\">\n",
    "                        <marquee class=\"news-scroll\" behavior=\"scroll\" direction=\"left\" onmouseover=\"this.stop();\" onmouseout=\"this.start();\">\"\"\" + \n",
    "                            ' '.join(scroll_objects) + \n",
    "                        \"\"\"</marquee>\n",
    "                    </div>\n",
    "                </div>\n",
    "            </div>\n",
    "        </div>\n",
    "    \n",
    "    </html>\n",
    "    \"\"\".strip()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### FAANG and ETF Prices and Candlestick"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:25.013420Z",
     "iopub.status.busy": "2022-02-14T16:20:25.012830Z",
     "iopub.status.idle": "2022-02-14T16:20:25.016784Z",
     "shell.execute_reply": "2022-02-14T16:20:25.017499Z"
    }
   },
   "outputs": [],
   "source": [
    "COLUMNS = 5\n",
    "TICKERS = [\n",
    "    'SPY', 'QQQ', 'XLF', 'IWM', 'BND',\n",
    "    'FB', 'AAPL', 'AMZN', 'NFLX', 'GOOG',\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:25.027592Z",
     "iopub.status.busy": "2022-02-14T16:20:25.026424Z",
     "iopub.status.idle": "2022-02-14T16:20:31.826753Z",
     "shell.execute_reply": "2022-02-14T16:20:31.827191Z"
    }
   },
   "outputs": [],
   "source": [
    "big_numbers = []\n",
    "figures = []\n",
    "\n",
    "for i, t in enumerate(TICKERS):\n",
    "    close, delta, delta_pct, high, low, open_, p_close, _ = quote(FINNHUB_KEY, t)\n",
    "    df = candles(FINNHUB_KEY, t)\n",
    "    \n",
    "    bn = dp.BigNumber(\n",
    "        heading=t,\n",
    "        value=f\"${round(close, 2)}\",\n",
    "        change=f\"{round(delta_pct, 2)}%\",\n",
    "        is_upward_change=True if delta_pct > 0 else False\n",
    "    )\n",
    "    big_numbers.append(bn)\n",
    "    \n",
    "    figure = candlestick(df, t)\n",
    "    figures.append(figure)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:31.833408Z",
     "iopub.status.busy": "2022-02-14T16:20:31.832340Z",
     "iopub.status.idle": "2022-02-14T16:20:31.834850Z",
     "shell.execute_reply": "2022-02-14T16:20:31.835438Z"
    }
   },
   "outputs": [],
   "source": [
    "ticker_groups = list(zip_longest(*(iter(big_numbers),) * COLUMNS, fillvalue=''))\n",
    "ticker_groups = [\n",
    "    dp.Group(columns=COLUMNS, *g) for g in ticker_groups\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Scrape CNN Market"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:31.842481Z",
     "iopub.status.busy": "2022-02-14T16:20:31.841737Z",
     "iopub.status.idle": "2022-02-14T16:20:32.059483Z",
     "shell.execute_reply": "2022-02-14T16:20:32.059013Z"
    }
   },
   "outputs": [],
   "source": [
    "cnn = 'https://money.cnn.com/data/markets'\n",
    "with requests.get(cnn) as page:\n",
    "    soup = BeautifulSoup(page.content, 'html.parser')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:32.070445Z",
     "iopub.status.busy": "2022-02-14T16:20:32.068013Z",
     "iopub.status.idle": "2022-02-14T16:20:32.073855Z",
     "shell.execute_reply": "2022-02-14T16:20:32.074557Z"
    }
   },
   "outputs": [],
   "source": [
    "modules = soup.find_all('div', class_='module')\n",
    "key_stats = soup.find('ul', class_='module-body wsod key-stats')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:32.083844Z",
     "iopub.status.busy": "2022-02-14T16:20:32.082859Z",
     "iopub.status.idle": "2022-02-14T16:20:32.085358Z",
     "shell.execute_reply": "2022-02-14T16:20:32.085823Z"
    }
   },
   "outputs": [],
   "source": [
    "pairs = list(map(format_modules, modules))\n",
    "pairs = list(filter(None, pairs))\n",
    "pairs = list(sum(pairs, ()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:32.091137Z",
     "iopub.status.busy": "2022-02-14T16:20:32.090382Z",
     "iopub.status.idle": "2022-02-14T16:20:32.093472Z",
     "shell.execute_reply": "2022-02-14T16:20:32.094149Z"
    }
   },
   "outputs": [],
   "source": [
    "headers = pairs[::2]\n",
    "modules = pairs[1::2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:32.099442Z",
     "iopub.status.busy": "2022-02-14T16:20:32.098756Z",
     "iopub.status.idle": "2022-02-14T16:20:32.101495Z",
     "shell.execute_reply": "2022-02-14T16:20:32.101959Z"
    }
   },
   "outputs": [],
   "source": [
    "module_groups = list(map(group_modules, headers, modules))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "headers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Free Stock Search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:32.108551Z",
     "iopub.status.busy": "2022-02-14T16:20:32.107222Z",
     "iopub.status.idle": "2022-02-14T16:20:32.111164Z",
     "shell.execute_reply": "2022-02-14T16:20:32.111616Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 3 µs, sys: 1 µs, total: 4 µs\n",
      "Wall time: 7.87 µs\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "# sp500_figures = free_stock_search(FINNHUB_KEY)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.2.4 Data Exploration and Visualization <a class=\"anchor\" id=\"data\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### COVID"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "US KPIs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:56.165696Z",
     "iopub.status.busy": "2022-02-14T16:20:56.164700Z",
     "iopub.status.idle": "2022-02-14T16:20:56.553542Z",
     "shell.execute_reply": "2022-02-14T16:20:56.554225Z"
    }
   },
   "outputs": [],
   "source": [
    "usa_link = f'https://api.covidactnow.org/v2/country/US.json?apiKey={COVID_KEY}'\n",
    "with requests.get(usa_link) as r:\n",
    "    usa = r.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:56.560908Z",
     "iopub.status.busy": "2022-02-14T16:20:56.560249Z",
     "iopub.status.idle": "2022-02-14T16:20:56.562836Z",
     "shell.execute_reply": "2022-02-14T16:20:56.563269Z"
    }
   },
   "outputs": [],
   "source": [
    "stats = usa['actuals']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:56.568560Z",
     "iopub.status.busy": "2022-02-14T16:20:56.567743Z",
     "iopub.status.idle": "2022-02-14T16:20:56.569970Z",
     "shell.execute_reply": "2022-02-14T16:20:56.570397Z"
    }
   },
   "outputs": [],
   "source": [
    "c, n_c, d, n_d = stats['cases'], stats['newCases'], stats['deaths'], stats['newDeaths']\n",
    "vax_rat = usa['metrics']['vaccinationsCompletedRatio']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:56.579908Z",
     "iopub.status.busy": "2022-02-14T16:20:56.579225Z",
     "iopub.status.idle": "2022-02-14T16:20:56.582716Z",
     "shell.execute_reply": "2022-02-14T16:20:56.582111Z"
    }
   },
   "outputs": [],
   "source": [
    "covid_kpi = dp.Group(\n",
    "    dp.BigNumber(\n",
    "        heading='Total Cases',\n",
    "        value=\"{:,}\".format(c),\n",
    "        change=f\"{round(n_c / c * 100, 2)}%\",\n",
    "        is_upward_change=True if n_c / c - 1 > 0 else False\n",
    "    ),\n",
    "    dp.BigNumber(\n",
    "        heading='Total Deaths',\n",
    "        value=\"{:,}\".format(d),\n",
    "        change=f\"{round(n_d / d * 100, 2)}%\",\n",
    "        is_upward_change=True if n_d / d - 1 > 0 else False\n",
    "    ),\n",
    "    dp.BigNumber(\n",
    "        heading='Vaccine Completion',\n",
    "        value=f\"{round(vax_rat * 100, 2)}%\"\n",
    "    ),\n",
    "    columns=3\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "County Choropleth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:56.588591Z",
     "iopub.status.busy": "2022-02-14T16:20:56.587668Z",
     "iopub.status.idle": "2022-02-14T16:20:57.167120Z",
     "shell.execute_reply": "2022-02-14T16:20:57.167558Z"
    }
   },
   "outputs": [],
   "source": [
    "geo_link = 'https://raw.githubusercontent.com/plotly/datasets/master/geojson-counties-fips.json'\n",
    "with requests.get(geo_link) as r:\n",
    "    geo_json = r.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:57.170856Z",
     "iopub.status.busy": "2022-02-14T16:20:57.170157Z",
     "iopub.status.idle": "2022-02-14T16:20:57.176663Z",
     "shell.execute_reply": "2022-02-14T16:20:57.178272Z"
    }
   },
   "outputs": [],
   "source": [
    "counties = f'https://api.covidactnow.org/v2/counties.json?apiKey={COVID_KEY}'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:57.186397Z",
     "iopub.status.busy": "2022-02-14T16:20:57.185200Z",
     "iopub.status.idle": "2022-02-14T16:20:58.585088Z",
     "shell.execute_reply": "2022-02-14T16:20:58.585517Z"
    }
   },
   "outputs": [],
   "source": [
    "with requests.get(counties) as r:\n",
    "    counties = pd.DataFrame(r.json())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:58.603819Z",
     "iopub.status.busy": "2022-02-14T16:20:58.590833Z",
     "iopub.status.idle": "2022-02-14T16:20:58.606476Z",
     "shell.execute_reply": "2022-02-14T16:20:58.607221Z"
    }
   },
   "outputs": [],
   "source": [
    "counties['cases'] = counties.actuals.str['cases']\n",
    "counties['case_ratio'] = counties.cases / counties.population\n",
    "counties['deaths'] = counties.actuals.str['deaths']\n",
    "counties['death_ratio'] = counties.deaths / counties.population\n",
    "counties['vaccine_ratio'] = counties.metrics.str['vaccinationsCompletedRatio']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:58.615029Z",
     "iopub.status.busy": "2022-02-14T16:20:58.614161Z",
     "iopub.status.idle": "2022-02-14T16:21:01.690242Z",
     "shell.execute_reply": "2022-02-14T16:21:01.690812Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.82 s, sys: 258 ms, total: 2.08 s\n",
      "Wall time: 2.88 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "covid_fig = county_choropleth(counties, geo_json)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Individual State Plots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:21:01.697356Z",
     "iopub.status.busy": "2022-02-14T16:21:01.696644Z",
     "iopub.status.idle": "2022-02-14T16:21:01.699831Z",
     "shell.execute_reply": "2022-02-14T16:21:01.699360Z"
    }
   },
   "outputs": [],
   "source": [
    "states = f'https://api.covidactnow.org/v2/states.timeseries.json?apiKey={COVID_KEY}'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:21:01.705904Z",
     "iopub.status.busy": "2022-02-14T16:21:01.705119Z",
     "iopub.status.idle": "2022-02-14T16:21:03.936767Z",
     "shell.execute_reply": "2022-02-14T16:21:03.937200Z"
    }
   },
   "outputs": [],
   "source": [
    "with requests.get(states) as r:\n",
    "    states = pd.DataFrame(r.json())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:21:03.948417Z",
     "iopub.status.busy": "2022-02-14T16:21:03.947792Z",
     "iopub.status.idle": "2022-02-14T16:21:30.131551Z",
     "shell.execute_reply": "2022-02-14T16:21:30.132075Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 24.1 s, sys: 552 ms, total: 24.6 s\n",
      "Wall time: 25.2 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "covid_figures = list(map(\n",
    "    plot_state, states.state, states.actualsTimeseries\n",
    "))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Five Thirty Eight"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:36.589228Z",
     "iopub.status.busy": "2022-02-14T16:20:36.587464Z",
     "iopub.status.idle": "2022-02-14T16:20:36.592405Z",
     "shell.execute_reply": "2022-02-14T16:20:36.593083Z"
    }
   },
   "outputs": [],
   "source": [
    "NBA_ELO = 'https://projects.fivethirtyeight.com/nba-model/nba_elo.csv'\n",
    "NFL_ELO = 'https://projects.fivethirtyeight.com/nfl-api/nfl_elo.csv'\n",
    "MLB_ELO = 'https://projects.fivethirtyeight.com/mlb-api/mlb_elo.csv'\n",
    "ELO_LINKS = [\n",
    "    NBA_ELO, NFL_ELO, MLB_ELO\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:36.598518Z",
     "iopub.status.busy": "2022-02-14T16:20:36.597873Z",
     "iopub.status.idle": "2022-02-14T16:20:36.600633Z",
     "shell.execute_reply": "2022-02-14T16:20:36.601093Z"
    }
   },
   "outputs": [],
   "source": [
    "YEAR = 2021\n",
    "NCAA = f'https://projects.fivethirtyeight.com/march-madness-api/{YEAR}/fivethirtyeight_ncaa_forecasts.csv'\n",
    "POLLS = 'https://github.com/fivethirtyeight/data/tree/master/polls'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:36.608651Z",
     "iopub.status.busy": "2022-02-14T16:20:36.607647Z",
     "iopub.status.idle": "2022-02-14T16:20:56.103681Z",
     "shell.execute_reply": "2022-02-14T16:20:56.104139Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 14.3 s, sys: 898 ms, total: 15.2 s\n",
      "Wall time: 20.3 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "_538_figures = list(map(\n",
    "    plot_elo, ELO_LINKS\n",
    "))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:56.109709Z",
     "iopub.status.busy": "2022-02-14T16:20:56.106581Z",
     "iopub.status.idle": "2022-02-14T16:20:56.114234Z",
     "shell.execute_reply": "2022-02-14T16:20:56.113717Z"
    }
   },
   "outputs": [],
   "source": [
    "# TODO Clean and Style Graphs\n",
    "    # Restyle Color\n",
    "# TODO Header Blocks --> Intro ELO and By Sport\n",
    "# Incorporate March Madness and ~Polls~"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Spotify"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:56.119908Z",
     "iopub.status.busy": "2022-02-14T16:20:56.118866Z",
     "iopub.status.idle": "2022-02-14T16:20:56.121654Z",
     "shell.execute_reply": "2022-02-14T16:20:56.122206Z"
    }
   },
   "outputs": [],
   "source": [
    "TOP50 = '37i9dQZEVXbLRQDuF5jeBp'\n",
    "MIX1 = '37i9dQZF1E35bNzojVbMHG'\n",
    "MIX2 = '37i9dQZF1E38sJU7OsGSQi'\n",
    "MIX3 = '37i9dQZF1E37xnKwvD1GJE'\n",
    "\n",
    "PLAYLISTS = [\n",
    "    TOP50, MIX1, MIX2, MIX3\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:56.128292Z",
     "iopub.status.busy": "2022-02-14T16:20:56.126670Z",
     "iopub.status.idle": "2022-02-14T16:20:56.131638Z",
     "shell.execute_reply": "2022-02-14T16:20:56.132156Z"
    }
   },
   "outputs": [],
   "source": [
    "HEADERS = {\n",
    "    'Authorization': 'Bearer {token}'.format(token=access_token)\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:56.136652Z",
     "iopub.status.busy": "2022-02-14T16:20:56.136074Z",
     "iopub.status.idle": "2022-02-14T16:20:56.138753Z",
     "shell.execute_reply": "2022-02-14T16:20:56.139217Z"
    }
   },
   "outputs": [],
   "source": [
    "playlists_html = list(map(get_spotify_embed, PLAYLISTS))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:56.145126Z",
     "iopub.status.busy": "2022-02-14T16:20:56.144466Z",
     "iopub.status.idle": "2022-02-14T16:20:56.149413Z",
     "shell.execute_reply": "2022-02-14T16:20:56.149845Z"
    }
   },
   "outputs": [],
   "source": [
    "# TODO Correct Embedding..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Lichess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:20:56.153554Z",
     "iopub.status.busy": "2022-02-14T16:20:56.152138Z",
     "iopub.status.idle": "2022-02-14T16:20:56.156698Z",
     "shell.execute_reply": "2022-02-14T16:20:56.157406Z"
    }
   },
   "outputs": [],
   "source": [
    "# TODO Top Player Matches"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Data Exploration Compilation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:21:30.139628Z",
     "iopub.status.busy": "2022-02-14T16:21:30.138939Z",
     "iopub.status.idle": "2022-02-14T16:21:31.771210Z",
     "shell.execute_reply": "2022-02-14T16:21:31.770369Z"
    }
   },
   "outputs": [],
   "source": [
    "covid_group = dp.Toggle(\n",
    "    covid_kpi,\n",
    "    dp.Divider(),\n",
    "    dp.Plot(\n",
    "        covid_fig\n",
    "    ),\n",
    "    dp.Divider(),\n",
    "    dp.Select(\n",
    "        *covid_figures\n",
    "    ),\n",
    "    label = 'COVID'\n",
    ")\n",
    "_538_group = dp.Toggle(\n",
    "    dp.Select(\n",
    "        *_538_figures\n",
    "    ),\n",
    "    label = 'Five Thirty Eight'\n",
    ")\n",
    "spotify_group = dp.Toggle(\n",
    "    dp.Group(\n",
    "        *playlists_html,\n",
    "        columns=4\n",
    "    ),\n",
    "    label = 'Spotify'\n",
    ")\n",
    "# lichess_group = dp.Toggle(\n",
    "#     dp.Text(\"Lichess\", label='A'),\n",
    "#     label = 'Lichess'\n",
    "# )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:21:31.778164Z",
     "iopub.status.busy": "2022-02-14T16:21:31.776894Z",
     "iopub.status.idle": "2022-02-14T16:21:31.784450Z",
     "shell.execute_reply": "2022-02-14T16:21:31.784979Z"
    }
   },
   "outputs": [],
   "source": [
    "data_groups = [\n",
    "    covid_group,\n",
    "    _538_group,\n",
    "    spotify_group\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.2.5 Compiled Selector <a class=\"anchor\" id=\"compiled\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:21:31.796769Z",
     "iopub.status.busy": "2022-02-14T16:21:31.794861Z",
     "iopub.status.idle": "2022-02-14T16:21:31.799461Z",
     "shell.execute_reply": "2022-02-14T16:21:31.799938Z"
    }
   },
   "outputs": [],
   "source": [
    "sports = dp.Group(\n",
    "    dp.Select(\n",
    "        *espn_groups,\n",
    "        type=dp.SelectType.DROPDOWN\n",
    "    ),\n",
    "    label='Sports: Results and Analysis',\n",
    ")\n",
    "news = dp.Group(\n",
    "    dp.Select(*article_sections),\n",
    "    label='Top World News'\n",
    ")\n",
    "market = dp.Group(\n",
    "    scroll,\n",
    "    dp.Divider(),\n",
    "    *ticker_groups,\n",
    "    dp.Divider(),\n",
    "    dp.Select(\n",
    "        *figures\n",
    "    ),\n",
    "    dp.Divider(),\n",
    "    *module_groups[:2],\n",
    "    dp.Group(\n",
    "        *module_groups[2:],\n",
    "        columns=4\n",
    "    ),\n",
    "#     dp.Select(\n",
    "#         *sp500_figures\n",
    "#     ),\n",
    "    label='Stock Market'\n",
    ")\n",
    "data = dp.Group(\n",
    "    *data_groups,\n",
    "    label='Data Exploration'\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:21:31.806230Z",
     "iopub.status.busy": "2022-02-14T16:21:31.805558Z",
     "iopub.status.idle": "2022-02-14T16:21:31.807370Z",
     "shell.execute_reply": "2022-02-14T16:21:31.807873Z"
    }
   },
   "outputs": [],
   "source": [
    "main_select = dp.Select(\n",
    "    blocks=[\n",
    "        sports, news, market, data\n",
    "\n",
    "    ],\n",
    "    type=dp.SelectType.TABS,\n",
    "    label='selector'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 Credits <a class=\"anchor\" id=\"credits\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Table of Contents](#toc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:21:31.816549Z",
     "iopub.status.busy": "2022-02-14T16:21:31.815173Z",
     "iopub.status.idle": "2022-02-14T16:21:31.817861Z",
     "shell.execute_reply": "2022-02-14T16:21:31.818438Z"
    }
   },
   "outputs": [],
   "source": [
    "credits = dp.Text(\n",
    "    \"Report built by Iain Muir.\",\n",
    "    name='credits'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Datapane Report <a class=\"anchor\" id=\"report\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Table of Contents](#toc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:21:31.824158Z",
     "iopub.status.busy": "2022-02-14T16:21:31.823225Z",
     "iopub.status.idle": "2022-02-14T16:21:31.825989Z",
     "shell.execute_reply": "2022-02-14T16:21:31.826728Z"
    }
   },
   "outputs": [],
   "source": [
    "BLOCKS = [\n",
    "    header_logo,\n",
    "    header_text,\n",
    "    header_description,\n",
    "    dp.Divider(),\n",
    "    main_select,\n",
    "    dp.Divider(),\n",
    "    credits\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:21:31.834672Z",
     "iopub.status.busy": "2022-02-14T16:21:31.833667Z",
     "iopub.status.idle": "2022-02-14T16:21:31.836108Z",
     "shell.execute_reply": "2022-02-14T16:21:31.836560Z"
    }
   },
   "outputs": [],
   "source": [
    "report = dp.Report(\n",
    "    blocks=BLOCKS\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:21:31.849799Z",
     "iopub.status.busy": "2022-02-14T16:21:31.849050Z",
     "iopub.status.idle": "2022-02-14T16:21:48.081463Z",
     "shell.execute_reply": "2022-02-14T16:21:48.082055Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Uploading report and associated data - *please wait...*"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Uploading files\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Report successfully uploaded. View and share your report <a href='https://datapane.com/reports/BAmGp47/information-aggregator/' target='_blank'>here</a>, or edit your report <a href='https://datapane.com/reports/BAmGp47/information-aggregator/edit/' target='_blank'>here</a>."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "try:\n",
    "    report.upload(\n",
    "        name='Information Aggregator', \n",
    "        open=False\n",
    "    )\n",
    "except requests.exceptions.HTTPError:\n",
    "    print(\n",
    "        ErrorHandler(\"Report Upload Error; FATAL.\", *get_error_info())\n",
    "    )\n",
    "    time.sleep(5)\n",
    "    \n",
    "    report.upload(\n",
    "        name='Information Aggregator', \n",
    "        open=False\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:21:48.089600Z",
     "iopub.status.busy": "2022-02-14T16:21:48.088787Z",
     "iopub.status.idle": "2022-02-14T16:21:56.726328Z",
     "shell.execute_reply": "2022-02-14T16:21:56.726762Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Report saved to .//Users/iainmuir/PycharmProjects/Desktop/PersonalProjects/Information-Aggregator/Output/Information-Aggregator.html"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "report.save(\n",
    "    path=f'{ROOT}/Output/Information-Aggregator.html'\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:21:56.745113Z",
     "iopub.status.busy": "2022-02-14T16:21:56.741341Z",
     "iopub.status.idle": "2022-02-14T16:21:57.376944Z",
     "shell.execute_reply": "2022-02-14T16:21:57.378366Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "webbrowser.open(\n",
    "    report.web_url\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-14T16:21:57.409242Z",
     "iopub.status.busy": "2022-02-14T16:21:57.408030Z",
     "iopub.status.idle": "2022-02-14T16:21:57.551349Z",
     "shell.execute_reply": "2022-02-14T16:21:57.550378Z"
    }
   },
   "outputs": [],
   "source": [
    "Logging.write_success_to_log()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Info-Agg",
   "language": "python",
   "name": "information-aggregator"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}