Continuous_Control.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Continuous Control\n",
    "\n",
    "---\n",
    "\n",
    "You are welcome to use this coding environment to train your agent for the project.  Follow the instructions below to get started!\n",
    "\n",
    "### 1. Start the Environment\n",
    "\n",
    "Run the next code cell to install a few packages.  This line will take a few minutes to run!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31mtensorflow 1.7.1 has requirement numpy>=1.13.3, but you'll have numpy 1.12.1 which is incompatible.\u001b[0m\r\n",
      "\u001b[31mipython 6.5.0 has requirement prompt-toolkit<2.0.0,>=1.0.15, but you'll have prompt-toolkit 3.0.5 which is incompatible.\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!pip -q install ./python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The environments corresponding to both versions of the environment are already saved in the Workspace and can be accessed at the file paths provided below.  \n",
    "\n",
    "Please select one of the two options below for loading the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import torch\n",
    "import time\n",
    "from unityagents import UnityEnvironment\n",
    "import numpy as np\n",
    "from collections import deque\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "from ddpg_agent import Agent, ReplayBuffer\n",
    "\n",
    "random_seed = random.randint(1,25)\n",
    "plt.ion()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:unityagents:\n",
      "'Academy' started successfully!\n",
      "Unity Academy name: Academy\n",
      "        Number of Brains: 1\n",
      "        Number of External Brains : 1\n",
      "        Lesson number : 0\n",
      "        Reset Parameters :\n",
      "\t\tgoal_speed -> 1.0\n",
      "\t\tgoal_size -> 5.0\n",
      "Unity brain name: ReacherBrain\n",
      "        Number of Visual Observations (per agent): 0\n",
      "        Vector Observation space type: continuous\n",
      "        Vector Observation space size (per agent): 33\n",
      "        Number of stacked Vector Observation: 1\n",
      "        Vector Action space type: continuous\n",
      "        Vector Action space size (per agent): 4\n",
      "        Vector Action descriptions: , , , \n"
     ]
    }
   ],
   "source": [
    "\n",
    "# select this option to load version 1 (with a single agent) of the environment\n",
    "env = UnityEnvironment(file_name='/data/Reacher_One_Linux_NoVis/Reacher_One_Linux_NoVis.x86_64')\n",
    "\n",
    "# select this option to load version 2 (with 20 agents) of the environment\n",
    "# env = UnityEnvironment(file_name='/data/Reacher_Linux_NoVis/Reacher.x86_64')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Environments contain **_brains_** which are responsible for deciding the actions of their associated agents. Here we check for the first brain available, and set it as the default brain we will be controlling from Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the default brain\n",
    "brain_name = env.brain_names[0]\n",
    "brain = env.brains[brain_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Examine the State and Action Spaces\n",
    "\n",
    "Run the code cell below to print some information about the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of agents: 1\n",
      "Size of each action: 4\n",
      "There are 1 agents. Each observes a state with length: 33\n",
      "The state for the first agent looks like: [  0.00000000e+00  -4.00000000e+00   0.00000000e+00   1.00000000e+00\n",
      "  -0.00000000e+00  -0.00000000e+00  -4.37113883e-08   0.00000000e+00\n",
      "   0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00\n",
      "   0.00000000e+00   0.00000000e+00  -1.00000000e+01   0.00000000e+00\n",
      "   1.00000000e+00  -0.00000000e+00  -0.00000000e+00  -4.37113883e-08\n",
      "   0.00000000e+00   0.00000000e+00   0.00000000e+00   0.00000000e+00\n",
      "   0.00000000e+00   0.00000000e+00   5.75471878e+00  -1.00000000e+00\n",
      "   5.55726671e+00   0.00000000e+00   1.00000000e+00   0.00000000e+00\n",
      "  -1.68164849e-01]\n"
     ]
    }
   ],
   "source": [
    "# reset the environment\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "\n",
    "# number of agents\n",
    "num_agents = len(env_info.agents)\n",
    "print('Number of agents:', num_agents)\n",
    "\n",
    "# size of each action\n",
    "action_size = brain.vector_action_space_size\n",
    "print('Size of each action:', action_size)\n",
    "\n",
    "# examine the state space \n",
    "states = env_info.vector_observations\n",
    "state_size = states.shape[1]\n",
    "print('There are {} agents. Each observes a state with length: {}'.format(states.shape[0], state_size))\n",
    "print('The state for the first agent looks like:', states[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Take Random Actions in the Environment\n",
    "\n",
    "In the next code cell, you will learn how to use the Python API to control the agent and receive feedback from the environment.\n",
    "\n",
    "Note that **in this coding environment, you will not be able to watch the agents while they are training**, and you should set `train_mode=True` to restart the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total score (averaged over agents) this episode: 0.0\n"
     ]
    }
   ],
   "source": [
    "env_info = env.reset(train_mode=True)[brain_name]      # reset the environment    \n",
    "states = env_info.vector_observations                  # get the current state (for each agent)\n",
    "scores = np.zeros(num_agents)                          # initialize the score (for each agent)\n",
    "while True:\n",
    "    actions = np.random.randn(num_agents, action_size) # select an action (for each agent)\n",
    "    actions = np.clip(actions, -1, 1)                  # all actions between -1 and 1\n",
    "    env_info = env.step(actions)[brain_name]           # send all actions to tne environment\n",
    "    next_states = env_info.vector_observations         # get next state (for each agent)\n",
    "    rewards = env_info.rewards                         # get reward (for each agent)\n",
    "    dones = env_info.local_done                        # see if episode finished\n",
    "    scores += env_info.rewards                         # update the score (for each agent)\n",
    "    states = next_states                               # roll over states to next time step\n",
    "    if np.any(dones):                                  # exit loop if episode finished\n",
    "        break\n",
    "print('Total score (averaged over agents) this episode: {}'.format(np.mean(scores)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Train the Policy \n",
    "\n",
    "\n",
    "Now it's time to train the agent to solve the environment!  A few **important notes**:\n",
    "- When training the environment, set `train_mode=True`, so that the line for resetting the environment looks like the following:\n",
    "```python\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "```\n",
    "- To structure your work, you're welcome to work directly in this Jupyter notebook, or you might like to start over with a new file!  You can see the list of files in the workspace by clicking on **_Jupyter_** in the top left corner of the notebook.\n",
    "- In this coding environment, you will not be able to watch the agents while they are training.  However, **_after training the agents_**, you can download the saved model weights to watch the agents on your own machine! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_mode = True\n",
    "agent = Agent(state_size=state_size, action_size=action_size, random_seed=random_seed) #Creating an agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def ddpg(n_episodes=2000, max_t=1000, print_every=10, learn_every=20, num_learn=10, goal_score=30):\n",
    "    total_scores_deque = deque(maxlen=100)\n",
    "    total_scores = []\n",
    "    \n",
    "    for i_episode in range(1, n_episodes+1):\n",
    "        #Reset the env and the agent\n",
    "        env_info = env.reset(train_mode=train_mode)[brain_name]   # reset the environment \n",
    "        states = env_info.vector_observations # get the crrrent states\n",
    "        scores = np.zeros(num_agents) # initialize the score for each agent\n",
    "        agent.reset()\n",
    "        \n",
    "        start_time = time.time()\n",
    "        \n",
    "        for t in range(max_t):\n",
    "            actions = agent.act(states)\n",
    "            \n",
    "            env_info = env.step(actions)[brain_name]            # send all actions to the environment\n",
    "            next_states = env_info.vector_observations          # get next state (for each agent)\n",
    "            rewards = env_info.rewards                          # get reward (for each agent)\n",
    "            \n",
    "            dones = env_info.local_done                         # see if episode finished\n",
    "            \n",
    "            for state, action, reward, next_state, done in zip(states, actions, rewards, next_states, dones):\n",
    "                agent.step(state, action, reward, next_state, done) # send actions to the agent\n",
    "            \n",
    "            scores += env_info.rewards                           # update the score (for each agent)\n",
    "            states = next_states                                # roll over states to next time step\n",
    "            \n",
    "            if t%learn_every == 0:         \n",
    "                for _ in range(num_learn):\n",
    "                    agent.start_learn()\n",
    "                    \n",
    "            if np.any(dones):                                   # exit loop if episode finished\n",
    "                break\n",
    "    \n",
    "        mean_score = np.mean(scores)\n",
    "        min_score = np.min(scores)\n",
    "        max_score = np.max(scores)\n",
    "        total_scores_deque.append(mean_score)\n",
    "        total_scores.append(mean_score)\n",
    "        total_average_score = np.mean(total_scores_deque)\n",
    "        duration = time.time() - start_time\n",
    "        \n",
    "        print('\\rEpisode {}\\tTotal Average Score: {:.2f}\\tMean: {:.2f}\\tMin: {:.2f}\\tMax: {:.2f}\\tDuration: {:.2f}'\n",
    "              .format(i_episode, total_average_score, mean_score, min_score, max_score, duration))\n",
    "\n",
    "        if i_episode % print_every == 0:\n",
    "            torch.save(agent.actor_local.state_dict(), 'checkpoint_actor.pth')\n",
    "            torch.save(agent.critic_local.state_dict(), 'checkpoint_critic.pth')\n",
    "            print('\\rEpisode {}\\tTotal Average Score: {:.2f}'.format(i_episode, total_average_score))  \n",
    "            \n",
    "        if total_average_score >= goal_score and i_episode >= 100:\n",
    "            print('Problem Solved after {} epsisodes!! Total Average score: {:.2f}'.format(i_episode, total_average_score))\n",
    "            torch.save(agent.actor_local.state_dict(), 'checkpoint_actor.pth')\n",
    "            torch.save(agent.critic_local.state_dict(), 'checkpoint_critic.pth')\n",
    "            break\n",
    "    \n",
    "    return total_scores\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 1\tTotal Average Score: 0.43\tMean: 0.43\tMin: 0.43\tMax: 0.43\tDuration: 7.46\n",
      "Episode 2\tTotal Average Score: 0.78\tMean: 1.14\tMin: 1.14\tMax: 1.14\tDuration: 7.98\n",
      "Episode 3\tTotal Average Score: 0.57\tMean: 0.15\tMin: 0.15\tMax: 0.15\tDuration: 8.17\n",
      "Episode 4\tTotal Average Score: 0.50\tMean: 0.29\tMin: 0.29\tMax: 0.29\tDuration: 8.04\n",
      "Episode 5\tTotal Average Score: 0.46\tMean: 0.27\tMin: 0.27\tMax: 0.27\tDuration: 8.05\n",
      "Episode 6\tTotal Average Score: 0.52\tMean: 0.84\tMin: 0.84\tMax: 0.84\tDuration: 7.96\n",
      "Episode 7\tTotal Average Score: 0.56\tMean: 0.80\tMin: 0.80\tMax: 0.80\tDuration: 8.08\n",
      "Episode 8\tTotal Average Score: 0.49\tMean: 0.00\tMin: 0.00\tMax: 0.00\tDuration: 8.05\n",
      "Episode 9\tTotal Average Score: 0.57\tMean: 1.18\tMin: 1.18\tMax: 1.18\tDuration: 8.02\n",
      "Episode 10\tTotal Average Score: 0.57\tMean: 0.63\tMin: 0.63\tMax: 0.63\tDuration: 8.21\n",
      "Episode 10\tTotal Average Score: 0.57\n",
      "Episode 11\tTotal Average Score: 0.61\tMean: 0.93\tMin: 0.93\tMax: 0.93\tDuration: 8.10\n",
      "Episode 12\tTotal Average Score: 0.62\tMean: 0.82\tMin: 0.82\tMax: 0.82\tDuration: 8.13\n",
      "Episode 13\tTotal Average Score: 0.67\tMean: 1.20\tMin: 1.20\tMax: 1.20\tDuration: 8.23\n",
      "Episode 14\tTotal Average Score: 0.74\tMean: 1.75\tMin: 1.75\tMax: 1.75\tDuration: 8.23\n",
      "Episode 15\tTotal Average Score: 0.74\tMean: 0.71\tMin: 0.71\tMax: 0.71\tDuration: 8.19\n",
      "Episode 16\tTotal Average Score: 0.81\tMean: 1.77\tMin: 1.77\tMax: 1.77\tDuration: 8.56\n",
      "Episode 17\tTotal Average Score: 0.82\tMean: 0.99\tMin: 0.99\tMax: 0.99\tDuration: 8.18\n",
      "Episode 18\tTotal Average Score: 0.93\tMean: 2.83\tMin: 2.83\tMax: 2.83\tDuration: 8.18\n",
      "Episode 19\tTotal Average Score: 0.95\tMean: 1.24\tMin: 1.24\tMax: 1.24\tDuration: 8.17\n",
      "Episode 20\tTotal Average Score: 0.95\tMean: 1.00\tMin: 1.00\tMax: 1.00\tDuration: 8.18\n",
      "Episode 20\tTotal Average Score: 0.95\n",
      "Episode 21\tTotal Average Score: 0.95\tMean: 0.95\tMin: 0.95\tMax: 0.95\tDuration: 8.17\n",
      "Episode 22\tTotal Average Score: 1.07\tMean: 3.70\tMin: 3.70\tMax: 3.70\tDuration: 8.14\n",
      "Episode 23\tTotal Average Score: 1.04\tMean: 0.33\tMin: 0.33\tMax: 0.33\tDuration: 8.18\n",
      "Episode 24\tTotal Average Score: 1.04\tMean: 1.05\tMin: 1.05\tMax: 1.05\tDuration: 8.28\n",
      "Episode 25\tTotal Average Score: 1.02\tMean: 0.62\tMin: 0.62\tMax: 0.62\tDuration: 8.22\n",
      "Episode 26\tTotal Average Score: 1.06\tMean: 1.88\tMin: 1.88\tMax: 1.88\tDuration: 8.13\n",
      "Episode 27\tTotal Average Score: 1.09\tMean: 1.88\tMin: 1.88\tMax: 1.88\tDuration: 8.15\n",
      "Episode 28\tTotal Average Score: 1.09\tMean: 1.12\tMin: 1.12\tMax: 1.12\tDuration: 8.17\n",
      "Episode 29\tTotal Average Score: 1.09\tMean: 1.21\tMin: 1.21\tMax: 1.21\tDuration: 8.53\n",
      "Episode 30\tTotal Average Score: 1.10\tMean: 1.30\tMin: 1.30\tMax: 1.30\tDuration: 8.39\n",
      "Episode 30\tTotal Average Score: 1.10\n",
      "Episode 31\tTotal Average Score: 1.12\tMean: 1.80\tMin: 1.80\tMax: 1.80\tDuration: 8.33\n",
      "Episode 32\tTotal Average Score: 1.14\tMean: 1.69\tMin: 1.69\tMax: 1.69\tDuration: 8.19\n",
      "Episode 33\tTotal Average Score: 1.14\tMean: 1.10\tMin: 1.10\tMax: 1.10\tDuration: 8.15\n",
      "Episode 34\tTotal Average Score: 1.15\tMean: 1.58\tMin: 1.58\tMax: 1.58\tDuration: 8.31\n",
      "Episode 35\tTotal Average Score: 1.17\tMean: 1.85\tMin: 1.85\tMax: 1.85\tDuration: 8.23\n",
      "Episode 36\tTotal Average Score: 1.20\tMean: 2.28\tMin: 2.28\tMax: 2.28\tDuration: 8.30\n",
      "Episode 37\tTotal Average Score: 1.25\tMean: 3.00\tMin: 3.00\tMax: 3.00\tDuration: 8.23\n",
      "Episode 38\tTotal Average Score: 1.24\tMean: 0.82\tMin: 0.82\tMax: 0.82\tDuration: 8.25\n",
      "Episode 39\tTotal Average Score: 1.24\tMean: 1.11\tMin: 1.11\tMax: 1.11\tDuration: 8.27\n",
      "Episode 40\tTotal Average Score: 1.25\tMean: 1.84\tMin: 1.84\tMax: 1.84\tDuration: 8.25\n",
      "Episode 40\tTotal Average Score: 1.25\n",
      "Episode 41\tTotal Average Score: 1.26\tMean: 1.42\tMin: 1.42\tMax: 1.42\tDuration: 8.22\n",
      "Episode 42\tTotal Average Score: 1.27\tMean: 1.92\tMin: 1.92\tMax: 1.92\tDuration: 8.32\n",
      "Episode 43\tTotal Average Score: 1.34\tMean: 4.13\tMin: 4.13\tMax: 4.13\tDuration: 8.22\n",
      "Episode 44\tTotal Average Score: 1.37\tMean: 2.82\tMin: 2.82\tMax: 2.82\tDuration: 8.36\n",
      "Episode 45\tTotal Average Score: 1.38\tMean: 1.65\tMin: 1.65\tMax: 1.65\tDuration: 8.21\n",
      "Episode 46\tTotal Average Score: 1.37\tMean: 0.96\tMin: 0.96\tMax: 0.96\tDuration: 8.21\n",
      "Episode 47\tTotal Average Score: 1.43\tMean: 4.17\tMin: 4.17\tMax: 4.17\tDuration: 8.26\n",
      "Episode 48\tTotal Average Score: 1.47\tMean: 3.56\tMin: 3.56\tMax: 3.56\tDuration: 8.16\n",
      "Episode 49\tTotal Average Score: 1.50\tMean: 2.72\tMin: 2.72\tMax: 2.72\tDuration: 8.22\n",
      "Episode 50\tTotal Average Score: 1.57\tMean: 4.95\tMin: 4.95\tMax: 4.95\tDuration: 8.26\n",
      "Episode 50\tTotal Average Score: 1.57\n",
      "Episode 51\tTotal Average Score: 1.62\tMean: 4.15\tMin: 4.15\tMax: 4.15\tDuration: 8.24\n",
      "Episode 52\tTotal Average Score: 1.68\tMean: 4.80\tMin: 4.80\tMax: 4.80\tDuration: 8.22\n",
      "Episode 53\tTotal Average Score: 1.71\tMean: 3.25\tMin: 3.25\tMax: 3.25\tDuration: 8.49\n",
      "Episode 54\tTotal Average Score: 1.74\tMean: 3.47\tMin: 3.47\tMax: 3.47\tDuration: 8.57\n",
      "Episode 55\tTotal Average Score: 1.79\tMean: 4.22\tMin: 4.22\tMax: 4.22\tDuration: 8.33\n",
      "Episode 56\tTotal Average Score: 1.81\tMean: 3.36\tMin: 3.36\tMax: 3.36\tDuration: 8.22\n",
      "Episode 57\tTotal Average Score: 1.83\tMean: 2.41\tMin: 2.41\tMax: 2.41\tDuration: 8.24\n",
      "Episode 58\tTotal Average Score: 1.86\tMean: 4.08\tMin: 4.08\tMax: 4.08\tDuration: 8.28\n",
      "Episode 59\tTotal Average Score: 1.92\tMean: 5.11\tMin: 5.11\tMax: 5.11\tDuration: 8.26\n",
      "Episode 60\tTotal Average Score: 1.94\tMean: 3.29\tMin: 3.29\tMax: 3.29\tDuration: 8.34\n",
      "Episode 60\tTotal Average Score: 1.94\n",
      "Episode 61\tTotal Average Score: 2.00\tMean: 5.43\tMin: 5.43\tMax: 5.43\tDuration: 8.38\n",
      "Episode 62\tTotal Average Score: 2.06\tMean: 5.72\tMin: 5.72\tMax: 5.72\tDuration: 8.32\n",
      "Episode 63\tTotal Average Score: 2.08\tMean: 3.51\tMin: 3.51\tMax: 3.51\tDuration: 8.42\n",
      "Episode 64\tTotal Average Score: 2.16\tMean: 7.01\tMin: 7.01\tMax: 7.01\tDuration: 8.35\n",
      "Episode 65\tTotal Average Score: 2.21\tMean: 5.16\tMin: 5.16\tMax: 5.16\tDuration: 8.40\n",
      "Episode 66\tTotal Average Score: 2.21\tMean: 2.70\tMin: 2.70\tMax: 2.70\tDuration: 8.47\n",
      "Episode 67\tTotal Average Score: 2.26\tMean: 5.17\tMin: 5.17\tMax: 5.17\tDuration: 8.36\n",
      "Episode 68\tTotal Average Score: 2.26\tMean: 2.42\tMin: 2.42\tMax: 2.42\tDuration: 8.67\n",
      "Episode 69\tTotal Average Score: 2.30\tMean: 4.78\tMin: 4.78\tMax: 4.78\tDuration: 8.72\n",
      "Episode 70\tTotal Average Score: 2.34\tMean: 5.70\tMin: 5.70\tMax: 5.70\tDuration: 8.89\n",
      "Episode 70\tTotal Average Score: 2.34\n",
      "Episode 71\tTotal Average Score: 2.36\tMean: 3.74\tMin: 3.74\tMax: 3.74\tDuration: 8.85\n",
      "Episode 72\tTotal Average Score: 2.39\tMean: 4.22\tMin: 4.22\tMax: 4.22\tDuration: 8.71\n",
      "Episode 73\tTotal Average Score: 2.40\tMean: 3.26\tMin: 3.26\tMax: 3.26\tDuration: 8.74\n",
      "Episode 74\tTotal Average Score: 2.40\tMean: 2.17\tMin: 2.17\tMax: 2.17\tDuration: 8.64\n",
      "Episode 75\tTotal Average Score: 2.43\tMean: 4.82\tMin: 4.82\tMax: 4.82\tDuration: 8.72\n",
      "Episode 76\tTotal Average Score: 2.50\tMean: 7.80\tMin: 7.80\tMax: 7.80\tDuration: 8.84\n",
      "Episode 77\tTotal Average Score: 2.52\tMean: 3.62\tMin: 3.62\tMax: 3.62\tDuration: 8.74\n",
      "Episode 78\tTotal Average Score: 2.53\tMean: 3.78\tMin: 3.78\tMax: 3.78\tDuration: 8.70\n",
      "Episode 79\tTotal Average Score: 2.56\tMean: 4.59\tMin: 4.59\tMax: 4.59\tDuration: 8.41\n",
      "Episode 80\tTotal Average Score: 2.57\tMean: 3.50\tMin: 3.50\tMax: 3.50\tDuration: 8.31\n",
      "Episode 80\tTotal Average Score: 2.57\n",
      "Episode 81\tTotal Average Score: 2.59\tMean: 3.93\tMin: 3.93\tMax: 3.93\tDuration: 8.36\n",
      "Episode 82\tTotal Average Score: 2.62\tMean: 5.20\tMin: 5.20\tMax: 5.20\tDuration: 8.32\n",
      "Episode 83\tTotal Average Score: 2.65\tMean: 5.13\tMin: 5.13\tMax: 5.13\tDuration: 8.23\n",
      "Episode 84\tTotal Average Score: 2.66\tMean: 3.98\tMin: 3.98\tMax: 3.98\tDuration: 8.40\n",
      "Episode 85\tTotal Average Score: 2.72\tMean: 7.46\tMin: 7.46\tMax: 7.46\tDuration: 8.29\n",
      "Episode 86\tTotal Average Score: 2.78\tMean: 7.49\tMin: 7.49\tMax: 7.49\tDuration: 8.28\n",
      "Episode 87\tTotal Average Score: 2.83\tMean: 7.34\tMin: 7.34\tMax: 7.34\tDuration: 8.33\n",
      "Episode 88\tTotal Average Score: 2.88\tMean: 6.89\tMin: 6.89\tMax: 6.89\tDuration: 8.39\n",
      "Episode 89\tTotal Average Score: 2.90\tMean: 5.21\tMin: 5.21\tMax: 5.21\tDuration: 8.35\n",
      "Episode 90\tTotal Average Score: 3.00\tMean: 11.52\tMin: 11.52\tMax: 11.52\tDuration: 8.73\n",
      "Episode 90\tTotal Average Score: 3.00\n",
      "Episode 91\tTotal Average Score: 3.01\tMean: 4.21\tMin: 4.21\tMax: 4.21\tDuration: 8.55\n",
      "Episode 92\tTotal Average Score: 3.06\tMean: 7.33\tMin: 7.33\tMax: 7.33\tDuration: 8.28\n",
      "Episode 93\tTotal Average Score: 3.04\tMean: 1.79\tMin: 1.79\tMax: 1.79\tDuration: 8.37\n",
      "Episode 94\tTotal Average Score: 3.08\tMean: 6.70\tMin: 6.70\tMax: 6.70\tDuration: 8.38\n",
      "Episode 95\tTotal Average Score: 3.13\tMean: 7.53\tMin: 7.53\tMax: 7.53\tDuration: 8.45\n",
      "Episode 96\tTotal Average Score: 3.11\tMean: 1.62\tMin: 1.62\tMax: 1.62\tDuration: 8.38\n",
      "Episode 97\tTotal Average Score: 3.16\tMean: 7.87\tMin: 7.87\tMax: 7.87\tDuration: 8.42\n",
      "Episode 98\tTotal Average Score: 3.17\tMean: 4.20\tMin: 4.20\tMax: 4.20\tDuration: 8.50\n",
      "Episode 99\tTotal Average Score: 3.24\tMean: 9.41\tMin: 9.41\tMax: 9.41\tDuration: 8.33\n",
      "Episode 100\tTotal Average Score: 3.26\tMean: 5.36\tMin: 5.36\tMax: 5.36\tDuration: 8.40\n",
      "Episode 100\tTotal Average Score: 3.26\n",
      "Episode 101\tTotal Average Score: 3.33\tMean: 7.83\tMin: 7.83\tMax: 7.83\tDuration: 8.48\n",
      "Episode 102\tTotal Average Score: 3.37\tMean: 4.57\tMin: 4.57\tMax: 4.57\tDuration: 8.63\n",
      "Episode 103\tTotal Average Score: 3.43\tMean: 6.65\tMin: 6.65\tMax: 6.65\tDuration: 8.35\n",
      "Episode 104\tTotal Average Score: 3.51\tMean: 8.57\tMin: 8.57\tMax: 8.57\tDuration: 8.17\n",
      "Episode 105\tTotal Average Score: 3.63\tMean: 12.16\tMin: 12.16\tMax: 12.16\tDuration: 8.35\n",
      "Episode 106\tTotal Average Score: 3.71\tMean: 8.44\tMin: 8.44\tMax: 8.44\tDuration: 8.23\n",
      "Episode 107\tTotal Average Score: 3.75\tMean: 4.59\tMin: 4.59\tMax: 4.59\tDuration: 8.42\n",
      "Episode 108\tTotal Average Score: 3.83\tMean: 7.91\tMin: 7.91\tMax: 7.91\tDuration: 8.20\n",
      "Episode 109\tTotal Average Score: 3.91\tMean: 10.06\tMin: 10.06\tMax: 10.06\tDuration: 8.21\n",
      "Episode 110\tTotal Average Score: 3.98\tMean: 7.07\tMin: 7.07\tMax: 7.07\tDuration: 8.31\n",
      "Episode 110\tTotal Average Score: 3.98\n",
      "Episode 111\tTotal Average Score: 4.04\tMean: 6.63\tMin: 6.63\tMax: 6.63\tDuration: 8.12\n",
      "Episode 112\tTotal Average Score: 4.08\tMean: 5.19\tMin: 5.19\tMax: 5.19\tDuration: 8.27\n",
      "Episode 113\tTotal Average Score: 4.15\tMean: 8.10\tMin: 8.10\tMax: 8.10\tDuration: 8.22\n",
      "Episode 114\tTotal Average Score: 4.23\tMean: 9.79\tMin: 9.79\tMax: 9.79\tDuration: 8.29\n",
      "Episode 115\tTotal Average Score: 4.27\tMean: 4.75\tMin: 4.75\tMax: 4.75\tDuration: 8.18\n",
      "Episode 116\tTotal Average Score: 4.32\tMean: 6.32\tMin: 6.32\tMax: 6.32\tDuration: 8.28\n",
      "Episode 117\tTotal Average Score: 4.37\tMean: 6.73\tMin: 6.73\tMax: 6.73\tDuration: 8.29\n",
      "Episode 118\tTotal Average Score: 4.46\tMean: 11.85\tMin: 11.85\tMax: 11.85\tDuration: 8.30\n",
      "Episode 119\tTotal Average Score: 4.55\tMean: 9.75\tMin: 9.75\tMax: 9.75\tDuration: 8.28\n",
      "Episode 120\tTotal Average Score: 4.62\tMean: 7.93\tMin: 7.93\tMax: 7.93\tDuration: 8.25\n",
      "Episode 120\tTotal Average Score: 4.62\n",
      "Episode 121\tTotal Average Score: 4.76\tMean: 15.13\tMin: 15.13\tMax: 15.13\tDuration: 8.22\n",
      "Episode 122\tTotal Average Score: 4.81\tMean: 8.89\tMin: 8.89\tMax: 8.89\tDuration: 8.34\n",
      "Episode 123\tTotal Average Score: 4.88\tMean: 7.40\tMin: 7.40\tMax: 7.40\tDuration: 8.17\n",
      "Episode 124\tTotal Average Score: 4.96\tMean: 8.91\tMin: 8.91\tMax: 8.91\tDuration: 8.25\n",
      "Episode 125\tTotal Average Score: 5.09\tMean: 13.68\tMin: 13.68\tMax: 13.68\tDuration: 8.13\n",
      "Episode 126\tTotal Average Score: 5.17\tMean: 9.57\tMin: 9.57\tMax: 9.57\tDuration: 8.10\n",
      "Episode 127\tTotal Average Score: 5.33\tMean: 18.43\tMin: 18.43\tMax: 18.43\tDuration: 8.30\n",
      "Episode 128\tTotal Average Score: 5.43\tMean: 10.37\tMin: 10.37\tMax: 10.37\tDuration: 8.55\n",
      "Episode 129\tTotal Average Score: 5.47\tMean: 5.34\tMin: 5.34\tMax: 5.34\tDuration: 8.20\n",
      "Episode 130\tTotal Average Score: 5.54\tMean: 8.69\tMin: 8.69\tMax: 8.69\tDuration: 8.39\n",
      "Episode 130\tTotal Average Score: 5.54\n",
      "Episode 131\tTotal Average Score: 5.61\tMean: 9.01\tMin: 9.01\tMax: 9.01\tDuration: 8.22\n",
      "Episode 132\tTotal Average Score: 5.71\tMean: 11.58\tMin: 11.58\tMax: 11.58\tDuration: 8.35\n",
      "Episode 133\tTotal Average Score: 5.78\tMean: 7.97\tMin: 7.97\tMax: 7.97\tDuration: 8.38\n",
      "Episode 134\tTotal Average Score: 5.86\tMean: 9.89\tMin: 9.89\tMax: 9.89\tDuration: 8.49\n",
      "Episode 135\tTotal Average Score: 5.96\tMean: 11.27\tMin: 11.27\tMax: 11.27\tDuration: 8.37\n",
      "Episode 136\tTotal Average Score: 6.06\tMean: 12.46\tMin: 12.46\tMax: 12.46\tDuration: 8.40\n",
      "Episode 137\tTotal Average Score: 6.11\tMean: 7.85\tMin: 7.85\tMax: 7.85\tDuration: 8.26\n",
      "Episode 138\tTotal Average Score: 6.20\tMean: 10.35\tMin: 10.35\tMax: 10.35\tDuration: 8.44\n",
      "Episode 139\tTotal Average Score: 6.32\tMean: 12.59\tMin: 12.59\tMax: 12.59\tDuration: 8.52\n",
      "Episode 140\tTotal Average Score: 6.47\tMean: 16.72\tMin: 16.72\tMax: 16.72\tDuration: 8.47\n",
      "Episode 140\tTotal Average Score: 6.47\n",
      "Episode 141\tTotal Average Score: 6.56\tMean: 10.43\tMin: 10.43\tMax: 10.43\tDuration: 8.47\n",
      "Episode 142\tTotal Average Score: 6.60\tMean: 6.49\tMin: 6.49\tMax: 6.49\tDuration: 8.23\n",
      "Episode 143\tTotal Average Score: 6.66\tMean: 10.03\tMin: 10.03\tMax: 10.03\tDuration: 8.32\n",
      "Episode 144\tTotal Average Score: 6.71\tMean: 7.52\tMin: 7.52\tMax: 7.52\tDuration: 8.42\n",
      "Episode 145\tTotal Average Score: 6.83\tMean: 14.23\tMin: 14.23\tMax: 14.23\tDuration: 8.44\n",
      "Episode 146\tTotal Average Score: 6.97\tMean: 14.14\tMin: 14.14\tMax: 14.14\tDuration: 8.27\n",
      "Episode 147\tTotal Average Score: 7.07\tMean: 14.83\tMin: 14.83\tMax: 14.83\tDuration: 8.49\n",
      "Episode 148\tTotal Average Score: 7.16\tMean: 12.01\tMin: 12.01\tMax: 12.01\tDuration: 8.37\n",
      "Episode 149\tTotal Average Score: 7.30\tMean: 17.36\tMin: 17.36\tMax: 17.36\tDuration: 8.47\n",
      "Episode 150\tTotal Average Score: 7.38\tMean: 12.21\tMin: 12.21\tMax: 12.21\tDuration: 8.37\n",
      "Episode 150\tTotal Average Score: 7.38\n",
      "Episode 151\tTotal Average Score: 7.46\tMean: 12.28\tMin: 12.28\tMax: 12.28\tDuration: 8.30\n",
      "Episode 152\tTotal Average Score: 7.54\tMean: 12.85\tMin: 12.85\tMax: 12.85\tDuration: 8.32\n",
      "Episode 153\tTotal Average Score: 7.64\tMean: 13.34\tMin: 13.34\tMax: 13.34\tDuration: 8.33\n",
      "Episode 154\tTotal Average Score: 7.78\tMean: 17.45\tMin: 17.45\tMax: 17.45\tDuration: 8.36\n",
      "Episode 155\tTotal Average Score: 7.85\tMean: 10.84\tMin: 10.84\tMax: 10.84\tDuration: 8.50\n",
      "Episode 156\tTotal Average Score: 7.95\tMean: 13.79\tMin: 13.79\tMax: 13.79\tDuration: 8.47\n",
      "Episode 157\tTotal Average Score: 8.11\tMean: 18.59\tMin: 18.59\tMax: 18.59\tDuration: 8.33\n",
      "Episode 158\tTotal Average Score: 8.33\tMean: 25.76\tMin: 25.76\tMax: 25.76\tDuration: 8.47\n",
      "Episode 159\tTotal Average Score: 8.43\tMean: 15.09\tMin: 15.09\tMax: 15.09\tDuration: 8.44\n",
      "Episode 160\tTotal Average Score: 8.54\tMean: 14.77\tMin: 14.77\tMax: 14.77\tDuration: 8.36\n",
      "Episode 160\tTotal Average Score: 8.54\n",
      "Episode 161\tTotal Average Score: 8.59\tMean: 10.05\tMin: 10.05\tMax: 10.05\tDuration: 8.20\n",
      "Episode 162\tTotal Average Score: 8.68\tMean: 15.25\tMin: 15.25\tMax: 15.25\tDuration: 8.31\n",
      "Episode 163\tTotal Average Score: 8.81\tMean: 16.14\tMin: 16.14\tMax: 16.14\tDuration: 8.40\n",
      "Episode 164\tTotal Average Score: 8.92\tMean: 17.87\tMin: 17.87\tMax: 17.87\tDuration: 8.53\n",
      "Episode 165\tTotal Average Score: 9.05\tMean: 18.72\tMin: 18.72\tMax: 18.72\tDuration: 8.79\n",
      "Episode 166\tTotal Average Score: 9.07\tMean: 4.44\tMin: 4.44\tMax: 4.44\tDuration: 8.36\n",
      "Episode 167\tTotal Average Score: 9.16\tMean: 13.59\tMin: 13.59\tMax: 13.59\tDuration: 8.34\n",
      "Episode 168\tTotal Average Score: 9.33\tMean: 20.28\tMin: 20.28\tMax: 20.28\tDuration: 8.33\n",
      "Episode 169\tTotal Average Score: 9.44\tMean: 15.53\tMin: 15.53\tMax: 15.53\tDuration: 8.52\n",
      "Episode 170\tTotal Average Score: 9.52\tMean: 13.86\tMin: 13.86\tMax: 13.86\tDuration: 8.41\n",
      "Episode 170\tTotal Average Score: 9.52\n",
      "Episode 171\tTotal Average Score: 9.64\tMean: 15.63\tMin: 15.63\tMax: 15.63\tDuration: 8.42\n",
      "Episode 172\tTotal Average Score: 9.73\tMean: 13.08\tMin: 13.08\tMax: 13.08\tDuration: 8.38\n",
      "Episode 173\tTotal Average Score: 9.83\tMean: 13.06\tMin: 13.06\tMax: 13.06\tDuration: 8.44\n",
      "Episode 174\tTotal Average Score: 10.01\tMean: 20.27\tMin: 20.27\tMax: 20.27\tDuration: 8.41\n",
      "Episode 175\tTotal Average Score: 10.07\tMean: 10.39\tMin: 10.39\tMax: 10.39\tDuration: 8.65\n",
      "Episode 176\tTotal Average Score: 10.12\tMean: 13.01\tMin: 13.01\tMax: 13.01\tDuration: 8.47\n",
      "Episode 177\tTotal Average Score: 10.18\tMean: 9.71\tMin: 9.71\tMax: 9.71\tDuration: 8.50\n",
      "Episode 178\tTotal Average Score: 10.24\tMean: 10.20\tMin: 10.20\tMax: 10.20\tDuration: 8.40\n",
      "Episode 179\tTotal Average Score: 10.34\tMean: 13.78\tMin: 13.78\tMax: 13.78\tDuration: 8.39\n",
      "Episode 180\tTotal Average Score: 10.49\tMean: 18.59\tMin: 18.59\tMax: 18.59\tDuration: 8.45\n",
      "Episode 180\tTotal Average Score: 10.49\n",
      "Episode 181\tTotal Average Score: 10.68\tMean: 22.84\tMin: 22.84\tMax: 22.84\tDuration: 8.52\n",
      "Episode 182\tTotal Average Score: 10.76\tMean: 13.62\tMin: 13.62\tMax: 13.62\tDuration: 8.43\n",
      "Episode 183\tTotal Average Score: 10.92\tMean: 20.99\tMin: 20.99\tMax: 20.99\tDuration: 8.54\n",
      "Episode 184\tTotal Average Score: 11.10\tMean: 22.12\tMin: 22.12\tMax: 22.12\tDuration: 8.63\n",
      "Episode 185\tTotal Average Score: 11.24\tMean: 21.64\tMin: 21.64\tMax: 21.64\tDuration: 8.47\n",
      "Episode 186\tTotal Average Score: 11.35\tMean: 18.66\tMin: 18.66\tMax: 18.66\tDuration: 8.51\n",
      "Episode 187\tTotal Average Score: 11.51\tMean: 23.00\tMin: 23.00\tMax: 23.00\tDuration: 8.44\n",
      "Episode 188\tTotal Average Score: 11.67\tMean: 23.16\tMin: 23.16\tMax: 23.16\tDuration: 8.51\n",
      "Episode 189\tTotal Average Score: 11.79\tMean: 17.39\tMin: 17.39\tMax: 17.39\tDuration: 8.42\n",
      "Episode 190\tTotal Average Score: 11.85\tMean: 17.48\tMin: 17.48\tMax: 17.48\tDuration: 8.55\n",
      "Episode 190\tTotal Average Score: 11.85\n",
      "Episode 191\tTotal Average Score: 12.05\tMean: 23.99\tMin: 23.99\tMax: 23.99\tDuration: 8.52\n",
      "Episode 192\tTotal Average Score: 12.25\tMean: 27.33\tMin: 27.33\tMax: 27.33\tDuration: 8.60\n",
      "Episode 193\tTotal Average Score: 12.46\tMean: 22.58\tMin: 22.58\tMax: 22.58\tDuration: 8.64\n",
      "Episode 194\tTotal Average Score: 12.57\tMean: 17.87\tMin: 17.87\tMax: 17.87\tDuration: 8.44\n",
      "Episode 195\tTotal Average Score: 12.70\tMean: 20.19\tMin: 20.19\tMax: 20.19\tDuration: 8.58\n",
      "Episode 196\tTotal Average Score: 12.91\tMean: 23.15\tMin: 23.15\tMax: 23.15\tDuration: 8.55\n",
      "Episode 197\tTotal Average Score: 12.98\tMean: 14.31\tMin: 14.31\tMax: 14.31\tDuration: 8.48\n",
      "Episode 198\tTotal Average Score: 13.07\tMean: 13.85\tMin: 13.85\tMax: 13.85\tDuration: 8.56\n",
      "Episode 199\tTotal Average Score: 13.19\tMean: 21.16\tMin: 21.16\tMax: 21.16\tDuration: 8.58\n",
      "Episode 200\tTotal Average Score: 13.34\tMean: 20.06\tMin: 20.06\tMax: 20.06\tDuration: 8.46\n",
      "Episode 200\tTotal Average Score: 13.34\n",
      "Episode 201\tTotal Average Score: 13.48\tMean: 21.61\tMin: 21.61\tMax: 21.61\tDuration: 8.77\n",
      "Episode 202\tTotal Average Score: 13.54\tMean: 11.37\tMin: 11.37\tMax: 11.37\tDuration: 8.62\n",
      "Episode 203\tTotal Average Score: 13.68\tMean: 19.89\tMin: 19.89\tMax: 19.89\tDuration: 8.44\n",
      "Episode 204\tTotal Average Score: 13.81\tMean: 22.37\tMin: 22.37\tMax: 22.37\tDuration: 8.39\n",
      "Episode 205\tTotal Average Score: 13.90\tMean: 20.77\tMin: 20.77\tMax: 20.77\tDuration: 8.62\n",
      "Episode 206\tTotal Average Score: 14.02\tMean: 19.97\tMin: 19.97\tMax: 19.97\tDuration: 8.56\n",
      "Episode 207\tTotal Average Score: 14.13\tMean: 16.32\tMin: 16.32\tMax: 16.32\tDuration: 8.44\n",
      "Episode 208\tTotal Average Score: 14.24\tMean: 18.84\tMin: 18.84\tMax: 18.84\tDuration: 8.39\n",
      "Episode 209\tTotal Average Score: 14.35\tMean: 20.49\tMin: 20.49\tMax: 20.49\tDuration: 8.35\n",
      "Episode 210\tTotal Average Score: 14.54\tMean: 26.56\tMin: 26.56\tMax: 26.56\tDuration: 8.64\n",
      "Episode 210\tTotal Average Score: 14.54\n",
      "Episode 211\tTotal Average Score: 14.70\tMean: 22.46\tMin: 22.46\tMax: 22.46\tDuration: 8.76\n",
      "Episode 212\tTotal Average Score: 14.75\tMean: 10.10\tMin: 10.10\tMax: 10.10\tDuration: 8.35\n",
      "Episode 213\tTotal Average Score: 14.90\tMean: 23.29\tMin: 23.29\tMax: 23.29\tDuration: 8.57\n",
      "Episode 214\tTotal Average Score: 14.99\tMean: 18.89\tMin: 18.89\tMax: 18.89\tDuration: 8.45\n",
      "Episode 215\tTotal Average Score: 15.20\tMean: 25.81\tMin: 25.81\tMax: 25.81\tDuration: 8.42\n",
      "Episode 216\tTotal Average Score: 15.33\tMean: 19.27\tMin: 19.27\tMax: 19.27\tDuration: 8.55\n",
      "Episode 217\tTotal Average Score: 15.44\tMean: 17.52\tMin: 17.52\tMax: 17.52\tDuration: 8.57\n",
      "Episode 218\tTotal Average Score: 15.56\tMean: 23.94\tMin: 23.94\tMax: 23.94\tDuration: 8.50\n",
      "Episode 219\tTotal Average Score: 15.71\tMean: 24.96\tMin: 24.96\tMax: 24.96\tDuration: 8.29\n",
      "Episode 220\tTotal Average Score: 15.87\tMean: 23.40\tMin: 23.40\tMax: 23.40\tDuration: 8.39\n",
      "Episode 220\tTotal Average Score: 15.87\n",
      "Episode 221\tTotal Average Score: 16.00\tMean: 28.62\tMin: 28.62\tMax: 28.62\tDuration: 8.49\n",
      "Episode 222\tTotal Average Score: 16.13\tMean: 21.55\tMin: 21.55\tMax: 21.55\tDuration: 8.62\n",
      "Episode 223\tTotal Average Score: 16.26\tMean: 20.56\tMin: 20.56\tMax: 20.56\tDuration: 8.50\n",
      "Episode 224\tTotal Average Score: 16.43\tMean: 26.30\tMin: 26.30\tMax: 26.30\tDuration: 8.85\n",
      "Episode 225\tTotal Average Score: 16.55\tMean: 24.82\tMin: 24.82\tMax: 24.82\tDuration: 8.60\n",
      "Episode 226\tTotal Average Score: 16.65\tMean: 19.72\tMin: 19.72\tMax: 19.72\tDuration: 8.56\n",
      "Episode 227\tTotal Average Score: 16.59\tMean: 13.14\tMin: 13.14\tMax: 13.14\tDuration: 8.54\n",
      "Episode 228\tTotal Average Score: 16.73\tMean: 24.35\tMin: 24.35\tMax: 24.35\tDuration: 8.58\n",
      "Episode 229\tTotal Average Score: 16.84\tMean: 16.10\tMin: 16.10\tMax: 16.10\tDuration: 8.62\n",
      "Episode 230\tTotal Average Score: 16.94\tMean: 18.06\tMin: 18.06\tMax: 18.06\tDuration: 8.45\n",
      "Episode 230\tTotal Average Score: 16.94\n",
      "Episode 231\tTotal Average Score: 17.13\tMean: 28.87\tMin: 28.87\tMax: 28.87\tDuration: 8.44\n",
      "Episode 232\tTotal Average Score: 17.35\tMean: 33.54\tMin: 33.54\tMax: 33.54\tDuration: 8.49\n",
      "Episode 233\tTotal Average Score: 17.58\tMean: 30.68\tMin: 30.68\tMax: 30.68\tDuration: 8.70\n",
      "Episode 234\tTotal Average Score: 17.73\tMean: 24.30\tMin: 24.30\tMax: 24.30\tDuration: 8.57\n",
      "Episode 235\tTotal Average Score: 17.86\tMean: 25.16\tMin: 25.16\tMax: 25.16\tDuration: 8.56\n",
      "Episode 236\tTotal Average Score: 18.00\tMean: 25.63\tMin: 25.63\tMax: 25.63\tDuration: 8.64\n",
      "Episode 237\tTotal Average Score: 18.18\tMean: 26.37\tMin: 26.37\tMax: 26.37\tDuration: 8.86\n",
      "Episode 238\tTotal Average Score: 18.30\tMean: 22.71\tMin: 22.71\tMax: 22.71\tDuration: 8.86\n",
      "Episode 239\tTotal Average Score: 18.46\tMean: 27.75\tMin: 27.75\tMax: 27.75\tDuration: 8.77\n",
      "Episode 240\tTotal Average Score: 18.61\tMean: 32.10\tMin: 32.10\tMax: 32.10\tDuration: 8.56\n",
      "Episode 240\tTotal Average Score: 18.61\n",
      "Episode 241\tTotal Average Score: 18.72\tMean: 21.69\tMin: 21.69\tMax: 21.69\tDuration: 8.60\n",
      "Episode 242\tTotal Average Score: 18.87\tMean: 21.18\tMin: 21.18\tMax: 21.18\tDuration: 8.71\n",
      "Episode 243\tTotal Average Score: 19.02\tMean: 25.51\tMin: 25.51\tMax: 25.51\tDuration: 8.62\n",
      "Episode 244\tTotal Average Score: 19.24\tMean: 29.24\tMin: 29.24\tMax: 29.24\tDuration: 8.75\n",
      "Episode 245\tTotal Average Score: 19.35\tMean: 24.91\tMin: 24.91\tMax: 24.91\tDuration: 8.68\n",
      "Episode 246\tTotal Average Score: 19.51\tMean: 29.94\tMin: 29.94\tMax: 29.94\tDuration: 8.89\n",
      "Episode 247\tTotal Average Score: 19.56\tMean: 20.43\tMin: 20.43\tMax: 20.43\tDuration: 8.59\n",
      "Episode 248\tTotal Average Score: 19.67\tMean: 22.74\tMin: 22.74\tMax: 22.74\tDuration: 8.54\n",
      "Episode 249\tTotal Average Score: 19.88\tMean: 38.67\tMin: 38.67\tMax: 38.67\tDuration: 8.58\n",
      "Episode 250\tTotal Average Score: 20.03\tMean: 27.13\tMin: 27.13\tMax: 27.13\tDuration: 8.56\n",
      "Episode 250\tTotal Average Score: 20.03\n",
      "Episode 251\tTotal Average Score: 20.12\tMean: 20.62\tMin: 20.62\tMax: 20.62\tDuration: 8.70\n",
      "Episode 252\tTotal Average Score: 20.18\tMean: 19.61\tMin: 19.61\tMax: 19.61\tDuration: 8.64\n",
      "Episode 253\tTotal Average Score: 20.32\tMean: 26.63\tMin: 26.63\tMax: 26.63\tDuration: 8.68\n",
      "Episode 254\tTotal Average Score: 20.42\tMean: 27.97\tMin: 27.97\tMax: 27.97\tDuration: 8.71\n",
      "Episode 255\tTotal Average Score: 20.64\tMean: 33.03\tMin: 33.03\tMax: 33.03\tDuration: 8.48\n",
      "Episode 256\tTotal Average Score: 20.74\tMean: 23.94\tMin: 23.94\tMax: 23.94\tDuration: 8.71\n",
      "Episode 257\tTotal Average Score: 20.87\tMean: 30.66\tMin: 30.66\tMax: 30.66\tDuration: 8.56\n",
      "Episode 258\tTotal Average Score: 20.84\tMean: 23.57\tMin: 23.57\tMax: 23.57\tDuration: 8.75\n",
      "Episode 259\tTotal Average Score: 20.92\tMean: 22.49\tMin: 22.49\tMax: 22.49\tDuration: 8.54\n",
      "Episode 260\tTotal Average Score: 21.09\tMean: 31.60\tMin: 31.60\tMax: 31.60\tDuration: 8.58\n",
      "Episode 260\tTotal Average Score: 21.09\n",
      "Episode 261\tTotal Average Score: 21.28\tMean: 29.31\tMin: 29.31\tMax: 29.31\tDuration: 8.64\n",
      "Episode 262\tTotal Average Score: 21.37\tMean: 24.86\tMin: 24.86\tMax: 24.86\tDuration: 8.68\n",
      "Episode 263\tTotal Average Score: 21.57\tMean: 35.40\tMin: 35.40\tMax: 35.40\tDuration: 8.56\n",
      "Episode 264\tTotal Average Score: 21.67\tMean: 27.73\tMin: 27.73\tMax: 27.73\tDuration: 8.57\n",
      "Episode 265\tTotal Average Score: 21.81\tMean: 33.11\tMin: 33.11\tMax: 33.11\tDuration: 8.61\n",
      "Episode 266\tTotal Average Score: 22.04\tMean: 27.86\tMin: 27.86\tMax: 27.86\tDuration: 8.57\n",
      "Episode 267\tTotal Average Score: 22.16\tMean: 25.35\tMin: 25.35\tMax: 25.35\tDuration: 8.53\n",
      "Episode 268\tTotal Average Score: 22.30\tMean: 33.68\tMin: 33.68\tMax: 33.68\tDuration: 8.63\n",
      "Episode 269\tTotal Average Score: 22.47\tMean: 33.11\tMin: 33.11\tMax: 33.11\tDuration: 8.76\n",
      "Episode 270\tTotal Average Score: 22.58\tMean: 24.71\tMin: 24.71\tMax: 24.71\tDuration: 8.64\n",
      "Episode 270\tTotal Average Score: 22.58\n",
      "Episode 271\tTotal Average Score: 22.75\tMean: 32.68\tMin: 32.68\tMax: 32.68\tDuration: 8.59\n",
      "Episode 272\tTotal Average Score: 22.92\tMean: 29.75\tMin: 29.75\tMax: 29.75\tDuration: 8.59\n",
      "Episode 273\tTotal Average Score: 23.02\tMean: 23.77\tMin: 23.77\tMax: 23.77\tDuration: 8.90\n",
      "Episode 274\tTotal Average Score: 23.08\tMean: 26.12\tMin: 26.12\tMax: 26.12\tDuration: 8.85\n",
      "Episode 275\tTotal Average Score: 23.27\tMean: 28.96\tMin: 28.96\tMax: 28.96\tDuration: 8.71\n",
      "Episode 276\tTotal Average Score: 23.48\tMean: 33.90\tMin: 33.90\tMax: 33.90\tDuration: 8.56\n",
      "Episode 277\tTotal Average Score: 23.70\tMean: 32.48\tMin: 32.48\tMax: 32.48\tDuration: 8.80\n",
      "Episode 278\tTotal Average Score: 23.94\tMean: 33.96\tMin: 33.96\tMax: 33.96\tDuration: 8.72\n",
      "Episode 279\tTotal Average Score: 24.15\tMean: 34.78\tMin: 34.78\tMax: 34.78\tDuration: 8.70\n",
      "Episode 280\tTotal Average Score: 24.31\tMean: 34.31\tMin: 34.31\tMax: 34.31\tDuration: 8.80\n",
      "Episode 280\tTotal Average Score: 24.31\n",
      "Episode 281\tTotal Average Score: 24.41\tMean: 33.03\tMin: 33.03\tMax: 33.03\tDuration: 8.95\n",
      "Episode 282\tTotal Average Score: 24.60\tMean: 32.25\tMin: 32.25\tMax: 32.25\tDuration: 8.85\n",
      "Episode 283\tTotal Average Score: 24.73\tMean: 34.44\tMin: 34.44\tMax: 34.44\tDuration: 8.65\n",
      "Episode 284\tTotal Average Score: 24.85\tMean: 33.79\tMin: 33.79\tMax: 33.79\tDuration: 8.74\n",
      "Episode 285\tTotal Average Score: 24.91\tMean: 27.96\tMin: 27.96\tMax: 27.96\tDuration: 8.75\n",
      "Episode 286\tTotal Average Score: 25.09\tMean: 36.82\tMin: 36.82\tMax: 36.82\tDuration: 8.76\n",
      "Episode 287\tTotal Average Score: 25.18\tMean: 32.03\tMin: 32.03\tMax: 32.03\tDuration: 8.87\n",
      "Episode 288\tTotal Average Score: 25.17\tMean: 21.61\tMin: 21.61\tMax: 21.61\tDuration: 8.76\n",
      "Episode 289\tTotal Average Score: 25.20\tMean: 20.08\tMin: 20.08\tMax: 20.08\tDuration: 8.71\n",
      "Episode 290\tTotal Average Score: 25.38\tMean: 35.94\tMin: 35.94\tMax: 35.94\tDuration: 8.65\n",
      "Episode 290\tTotal Average Score: 25.38\n",
      "Episode 291\tTotal Average Score: 25.32\tMean: 17.63\tMin: 17.63\tMax: 17.63\tDuration: 8.73\n",
      "Episode 292\tTotal Average Score: 25.39\tMean: 34.94\tMin: 34.94\tMax: 34.94\tDuration: 8.71\n",
      "Episode 293\tTotal Average Score: 25.51\tMean: 34.64\tMin: 34.64\tMax: 34.64\tDuration: 8.73\n",
      "Episode 294\tTotal Average Score: 25.71\tMean: 37.36\tMin: 37.36\tMax: 37.36\tDuration: 8.83\n",
      "Episode 295\tTotal Average Score: 25.85\tMean: 34.49\tMin: 34.49\tMax: 34.49\tDuration: 8.73\n",
      "Episode 296\tTotal Average Score: 25.87\tMean: 25.49\tMin: 25.49\tMax: 25.49\tDuration: 8.79\n",
      "Episode 297\tTotal Average Score: 26.08\tMean: 34.61\tMin: 34.61\tMax: 34.61\tDuration: 8.85\n",
      "Episode 298\tTotal Average Score: 26.32\tMean: 37.72\tMin: 37.72\tMax: 37.72\tDuration: 8.86\n",
      "Episode 299\tTotal Average Score: 26.47\tMean: 37.04\tMin: 37.04\tMax: 37.04\tDuration: 8.73\n",
      "Episode 300\tTotal Average Score: 26.65\tMean: 37.51\tMin: 37.51\tMax: 37.51\tDuration: 8.77\n",
      "Episode 300\tTotal Average Score: 26.65\n",
      "Episode 301\tTotal Average Score: 26.67\tMean: 23.74\tMin: 23.74\tMax: 23.74\tDuration: 8.81\n",
      "Episode 302\tTotal Average Score: 26.92\tMean: 35.89\tMin: 35.89\tMax: 35.89\tDuration: 8.85\n",
      "Episode 303\tTotal Average Score: 27.08\tMean: 36.46\tMin: 36.46\tMax: 36.46\tDuration: 8.78\n",
      "Episode 304\tTotal Average Score: 27.23\tMean: 36.75\tMin: 36.75\tMax: 36.75\tDuration: 8.78\n",
      "Episode 305\tTotal Average Score: 27.30\tMean: 28.37\tMin: 28.37\tMax: 28.37\tDuration: 8.86\n",
      "Episode 306\tTotal Average Score: 27.45\tMean: 35.24\tMin: 35.24\tMax: 35.24\tDuration: 8.93\n",
      "Episode 307\tTotal Average Score: 27.55\tMean: 25.58\tMin: 25.58\tMax: 25.58\tDuration: 8.92\n",
      "Episode 308\tTotal Average Score: 27.69\tMean: 33.04\tMin: 33.04\tMax: 33.04\tDuration: 8.92\n",
      "Episode 309\tTotal Average Score: 27.83\tMean: 34.31\tMin: 34.31\tMax: 34.31\tDuration: 9.22\n",
      "Episode 310\tTotal Average Score: 27.92\tMean: 35.83\tMin: 35.83\tMax: 35.83\tDuration: 8.87\n",
      "Episode 310\tTotal Average Score: 27.92\n",
      "Episode 311\tTotal Average Score: 28.00\tMean: 30.31\tMin: 30.31\tMax: 30.31\tDuration: 8.67\n",
      "Episode 312\tTotal Average Score: 28.26\tMean: 36.66\tMin: 36.66\tMax: 36.66\tDuration: 8.89\n",
      "Episode 313\tTotal Average Score: 28.39\tMean: 35.47\tMin: 35.47\tMax: 35.47\tDuration: 8.74\n",
      "Episode 314\tTotal Average Score: 28.41\tMean: 21.30\tMin: 21.30\tMax: 21.30\tDuration: 8.85\n",
      "Episode 315\tTotal Average Score: 28.48\tMean: 32.92\tMin: 32.92\tMax: 32.92\tDuration: 8.85\n",
      "Episode 316\tTotal Average Score: 28.66\tMean: 37.35\tMin: 37.35\tMax: 37.35\tDuration: 9.25\n",
      "Episode 317\tTotal Average Score: 28.84\tMean: 35.23\tMin: 35.23\tMax: 35.23\tDuration: 8.81\n",
      "Episode 318\tTotal Average Score: 28.94\tMean: 34.11\tMin: 34.11\tMax: 34.11\tDuration: 8.90\n",
      "Episode 319\tTotal Average Score: 29.05\tMean: 35.93\tMin: 35.93\tMax: 35.93\tDuration: 8.89\n",
      "Episode 320\tTotal Average Score: 29.15\tMean: 33.12\tMin: 33.12\tMax: 33.12\tDuration: 8.79\n",
      "Episode 320\tTotal Average Score: 29.15\n",
      "Episode 321\tTotal Average Score: 29.19\tMean: 32.47\tMin: 32.47\tMax: 32.47\tDuration: 8.80\n",
      "Episode 322\tTotal Average Score: 29.33\tMean: 36.05\tMin: 36.05\tMax: 36.05\tDuration: 8.97\n",
      "Episode 323\tTotal Average Score: 29.46\tMean: 33.44\tMin: 33.44\tMax: 33.44\tDuration: 8.98\n",
      "Episode 324\tTotal Average Score: 29.57\tMean: 37.31\tMin: 37.31\tMax: 37.31\tDuration: 8.92\n",
      "Episode 325\tTotal Average Score: 29.68\tMean: 36.07\tMin: 36.07\tMax: 36.07\tDuration: 9.04\n",
      "Episode 326\tTotal Average Score: 29.80\tMean: 31.70\tMin: 31.70\tMax: 31.70\tDuration: 9.21\n",
      "Episode 327\tTotal Average Score: 29.83\tMean: 16.05\tMin: 16.05\tMax: 16.05\tDuration: 8.95\n",
      "Episode 328\tTotal Average Score: 29.91\tMean: 31.94\tMin: 31.94\tMax: 31.94\tDuration: 9.01\n",
      "Episode 329\tTotal Average Score: 30.13\tMean: 38.39\tMin: 38.39\tMax: 38.39\tDuration: 9.12\n",
      "Problem Solved after 329 epsisodes!! Total Average score: 30.13\n"
     ]
    }
   ],
   "source": [
    "scores = ddpg() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Visualizing the scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzsvXmcXVWZ7/1be+8z1lyVypwQSBgCAgEigqgIgiKg0s7adre+duP14uvQtt0O7bXfVvvag4p2O6HY0rdxwAH1CiozAWVKmBIyAJmHSmqezriH9f6x91p77X32ObVPpU7Vqcrz/Xzyqap99lSQrGc9z+8ZGOccBEEQxImLNtcvQBAEQcwtZAgIgiBOcMgQEARBnOCQISAIgjjBIUNAEARxgkOGgCAI4gSHDAFBEMQJDhkCgiCIExwyBARBECc4RqMfwBjTAWwGcJhzfi1j7GQAPwbQDeBJAH/GOS/XuseiRYv4mjVrGv2qBEEQC4otW7YMcs57pzqv4YYAwEcA7ADQ7v38zwC+yjn/MWPs2wDeD+BbtW6wZs0abN68ubFvSRAEscBgjO2Pc15DQ0OMsZUArgHwPe9nBuByAD/zTrkFwHWNfAeCIAiiNo3WCG4E8LcAHO/nHgCjnHPL+/kQgBVRFzLGrmeMbWaMbR4YGGjwaxIEQZy4NMwQMMauBdDPOd+iHo44NbL9Kef8Js75Rs75xt7eKUNcBEEQxDRppEZwCYA3MsauBpCGqxHcCKCTMWZ4XsFKAEca+A4EQRDEFDTMI+Ccf4pzvpJzvgbAOwHcxzn/UwD3A3ird9pfAPhVo96BIAiCmJq5qCP4OwB/zRh7Ea5mcPMcvANBEAThMRvpo+CcPwDgAe/7PQAunI3nEgRBEFNDlcUEQcwqQ5Ml/HZr31y/BqFAhoAgiFnl9qcO44O3Pol82Zr65BMY03YwNFmCaTtTn3yckCEgCGJWKVnuwmZakZnjhMe2w2O44Av34OEXBxv+LDIEBEHMKo7jGgDLafxOdz5T9gxmSm/8Mk2GgCCIWcWzA7Ad8ghqITynpEGGgCCIBYbNhUcwvwzBdzftwfYj47P2POkRGHrDnzUr6aMEQRACERqaTx6BZTv44p07AAB/f816vPn8lehuSTb0mWWbPAKCIBYo88UjuO2Jg/jOg7sBAJMlP8PpC3fswH89sq/u+w3nyvjYT57GRNGM/Pz+nf147siY/LlMoSGCIBYqvkfQ3GLxL58+jJ9uOQQAmCgGU11721J13++mTXtw+1OH8cPHDkR+/r4fPIFrvv6w/JkMAUEQCxbbmR8ewVjBxFjB3b2PV9nF14NY0HNlO9b5Jcs9L0lZQwRBLDRkaMieW0MwOFnCmk/egV89fTjy87GCiXHPEAiP4PpXnQLA92p2HZ3AF36zHZxH/y79E0W5s29JuqJvvlS7kE4YSsoaIghiwdIsYvG+wRwA4L8eiZ7mOJY3UbIcFE1bGoJL1i0C4Hozpu3gdTduwvce3ouByVLF9ZxzXPjFe3HDD58EACS8nX3erPQI1OrhwyMFAL5YnCJDQBDEQqNZxGJdc+dkWREtHGyHY8LbuU8ULSnwdmUT8vP/+4w/SiXKuzG9Y3dvPwYAKHqhniiPIK+Ei/YOuQZKagQUGiIIYqEh1t259ggMzV3+ogySCAkBrj4gPILOjJsy6nCOO7celedEGYJyyMAUvMV+MmQIHIcHMon2Dky611sOEjqDpkUNdpxZyBAQBDGrNEuLCY6gVjGcK+MnT7gZPWOKIRgrmHKh7vA8gomihYdeGECPV0sQXvQBoKSEgByHS0MwlCujf7woPYVP/WIr3vufT8hzDwy7oaGS5cyKNwCQISAIYpZxeHNoBCJ0IwzSb7f14e9+vhUDE6WAIRgvuB5B0tCQSbiC75b9IyhZDl571hLvXkFD8LqvbsKnb98qfz48WpDawNBkGbc8sg8f+D+bUbYc7BmcxIv9k/LcnOcxlC1nVoRioIGVxYyxNIBNAFLec37GOf8cY+wHAC4FICon3ss5f7pR70EQRHPRLBqB0AbEe5heTN52eNAQFC2MFy20pw0YXphGpID2tKS8e/m/C+ccu45NYNexCXls72AOReERTJbQN1aEw4GJoonJUlA8FgajbDmz0l4CaKxHUAJwOef8XAAbAFzFGLvI++wTnPMN3h8yAgRxAiGzhupIH3320Cg+9YutVdM0p4P0CLyv4nVszis8gsmShbZ0QsbrhZCb8VJC1dDQeKFSDN4/lJOCcK5s4+BwHoAbdsqFNIOCN6ehbM+eR9DI4fWccy78nYT3p7krSAiCaDhiwa3HI3johUH86PEDsYux4mA6vgcA+AbKCXkEQiNoS7sBFF1jKHsZQGkvVPSTJw7g//mBG+c/PFqoeFbJclBQNIOdfa63MF60AoagK5uQ581maKihT2GM6YyxpwH0A7ibc/6Y99EXGWPPMsa+yhirv1abIIh5y3TqCMSuXYRvZgIrpBGIkFXfWBG/2+ZmBGnMzxpSDYEo9kon3CV08/4RPLpnCEC0IbAVsRiATE0d87wNQU9rSnoOJcteGGIx59zmnG8AsBLAhYyxlwD4FIAzALwUQDeAv4u6ljF2PWNsM2Ns88DAQCNfkyCIWcSeRtaQ6Es0nbGNW/YP47YnDlYcFxqBCBGJ9/rcr5+TU8G6W1IYL7h1BG0pN2NIZ0yGhtJeDD9XsqSHcyTKEHCOgmnjJSvaA8dHcmVpVACgpyUpDUZpoXgEAs75KIAHAFzFOe/zwkYlAP8J4MIq19zEOd/IOd/Y29s7G69JEMQsYE8ja0gsslFpmlPxlm89gr/9+bPoHy8Gjpshz0ToD2KXDwDtGaPCIzA0Jt9DhIZyJVsalkiPwObIly2s7s6iPe3n6BwZC567qDW1sEJDjLFexlin930GwBUAdjLGlnnHGIDrAGxr1DsQBNF8+HUE9RsCs87+RI7yjJ896XYS/dufPYNbH9svw0wyNOTZGLHL/9o7N6AtncB4wUTBtKUwrGlMEYtFIzkLDnefF2kIuBsayiQMnLqkTR5XvYekrqE9Y8jQUNl2ZqW9BNBYj2AZgPsZY88CeAKuRvAbALcyxrYC2ApgEYAvNPAdCIJoMqblEQiNoE6PYM+gn59/59Y+cM5x2+ZD+Mzt26QB8LOG/JBVT0sSb9qwAilDQ9lyYNlc9grStcrQkEhmMh0Hh0YqDYHjuKGhbFLHqYtb5fG+Ud9LyaZ0ZBKGTDN100fneR0B5/xZAOdFHL+8Uc8kCKL5mU4barFol2OKxZxzmDbHk/tHAQCXntaLJ/YNB7KOjo6VAu/hKF6HG7AAEjpD0XRg2g4M3T2ma0xek04G8/wtm2O/1yso+P4c+bJrCFZ2tcLQGBK6FvAeWpIGMkkNedMG53zhaQQEQRACWVlcx+7eDw3Fu+bWxw7gtL//LR54vh+tKQOXnd6LfNnGrqP+zOH7dvUHrlE9ApGsY2gaLNuB5XAkvN5EOvN7/6RDBV+DkyWM5k0k9GB/IMtxF/Z0Qsc7Xroad37klVjemUbfmOIRJHVkkwZsxzViZWoxQRBEM/Ni/8TUJ1VhOh6BbVdqBH95yxOyxXMYkf5513PHsLIrg5N73XCM8BAA4LnDY4FrpEdgcbnYJ3SGss1hOzzgEQhUYRmAbBVx+tK2wHGRIppN6kgaGk5b0oaOTCJQr5BNGbKFRaFsLwyxmCCIhckDu/pxxVc24ZdPRQ90mQqx/tejEZgR6aP37OjHHc/2RXoJi1rdZnCWw7GyK4uTe1oAuD2CBKohMm1Hvo/pODI0ZGgail4Wj6oRCDKh0JA0BEv8NFHGgEmve6l6fnsm4d8noaMlqcvP86blicXzv8UEQRALkN0Dbgz86YOjU5wZzXSyhuwa6aOikEtlUatfp7qyK4PlnWkYGsOTB0YqzgWAomkHJqeJxd7QmczrF32GAh6BUWkIGAPWKYJwNqHL6mGx4weADsUQfPg1p+LdL1uNbNL3CEqmTR4BQRDNiVicplPcBVTPGvrhYwfw+N7hyGtEZs9X734eb/nWHwEA3V4L6Pt3Vhacqj38V3RmYOgaVnVn0T/hCsSd2UTg/IJp+wbKduRin9Q1mddvxPAIXuifxPKODNozbh4OY0AqoctKYvX8rmxSfv++S9bg2nOWS0ORL9sLo9cQQRALk6QebLxWL9U8gk/fvhVv/84jkdeIrKGdRyew2xvconnhm6jB8mo7h5VdGQDAaUvcXXrS0NCtLMIAUDIdaaBMh0Os9apHIARgVSwOi7kDEyUsaktJTyGpa9AYi/QINqzqlN+LNFEZGirbMG1OYjFBEM2JzJk/bo+gjqwhzyMoWw5KZrAQrGjauPbfH8I93qAXcUywwjMEG0/q9t6fI5UI7uQLpi0LykzbkUbG0DUZjhITzYS3oTH3ezVDKF+2kNI1pDwROWVoMDRWoTMAwCtPXSS/F5qECA2JCWnkERAE0ZSIRa3eKl+BWHCnU1kMuLN/OeeyMngkX8a2w+P46E/8jvZqp88Vna4huGBNl3zvcKFWoayGhnyNIKGEgUTWkNAKhGEQXwF3J58wmO8RGLpbgCaNiX+/ntbKfpuiZcVIvgxgdgbXAw0sKCMIYmFS8HbkpeMMDanzCKIGyKuoegLnrmgsDJHwENTFv2g6yCR0fOzKU6WW8JLlHfLz8AKrisVhj0Agdv5aSDRO6AwiC1SMl1Q9Ak3zw2h6aP7wD973UhxRq4uT7pIs0krJEBAE0ZSIBbdkRc8G+OB/b8Hyzgw+e+2ZkZ9HTSgrTmFUwmGoounIlFJRLawai5JlY/2yNlz/qrXyWNLQ8JHXnIpV3Vn8+pkjFb+Tql2ITb6hhH18DyDoGSRCcfyErsm0z5ShgQOYsKyK+wHAq09fHPhZhIbGKDREEEQzI0JDE8XKSVwA8NttR3Hzw3urXh81j0CN6UcRzjDKly2pVYiJXiqFsi3DLCofu/I0vPWClREegRPIZhKCcDLgEQQri3W9iiEwNFloljQ0aEz1CGovueKdR/Nm5L0bBRkCgiDqQmTRRGXrxCHSI4gwBJxzPLZnyNUDQoZANUL5iKllRSvaEAjChsB2eMDYaCEdAFBDQ+7PsvrYCO7yk4pHkDS0QJM6IxQaCiM8AlGJHA4lNQoyBARB1IUIDY0XpmkInMqsoaJZGRr6zbN9eMdNj+Knmw9VZBiphqAQYQjcls+1DIH7mVhoLceBOg7Z1whUsTgoDvuCcnAZTeq+R5Ay3PRRYfSmWtiFByDCbmQICIJoSnxDYE1rmLy4ZCqPYMAr/treNy7TRwXqeMdcRGioaDpSsI1CfCaMhWUHPQK115BAZBBpU2kEBgt4BEGdYeqF3fUgeOA9Gg0ZAoIg6qKoDE4JZw45MVJC7ZgagZgINl4wK1JNJ5SwVNQjS9ZUHoG79Inwke1wGbIC/PCPGhqSlcXe2iw0grAA7IrFnkaga4HFPM4OX2dMiuMaeQQEQTQjappmODxUmEL0BappBJWhodaUZwiKVoVYPFlFqBaGqJpYLBA7dhHCsRweMGKajP+rhkCkjQZDRGGPwA0NiawhPbCYG1OIxYBrhIQhII+AIIimRF3sx6ZhCKLqCFSP4OdbDqFvrCB3+hNFsyJ9VGgE4XVysuyGq4qWU9EiWkW2dBChIccJeARRBWVyHoEWOqeWR+BVFsv76vV5BPNeI2CMpRljjzPGnmGMPccY+/+84yczxh5jjL3AGPsJYyw51b0IgmgeCmVbLn65kFAbJdyGia4j8K/7+E+fwe1PHZYtJCYiPALRxC0b2vWPF0yYXry/ZmgoEQwNhTWCqIIyv7I4mEZa4REYGjSNIalrXvpofRqBprkzEMT3s0EjPYISgMs55+cC2ADgKsbYRQD+GcBXOeenAhgB8P4GvgNBEDNM0bTRnna7d4Ybz0WlcoaJkzXkjod0zxsvmhXtLPz+/sGa2ImiJY1KnNCQ6hE4UR6BKhZXqSw2IgrK3GdoMn00fN9a6BqT7TPmfWiIu4jJ0QnvDwdwOYCfecdvAXBdo96BIIiZp2Da6PDaOIeri+sJDakeQfg603Zk2wnXIwiHhtyQVDZZ6RGIMFOcOgIxc9gK1xGI7qOqWKwFxWLhISQrQkPuz//zsnV447nLA4t/rKyhgFg85ekzQkMfwxjTGWNPA+gHcDeA3QBGOedC6TkEYEUj34EgiJmlULblUJVKjyBaxD0yWsAr/+U+HBzOR84jKIUMQdlyZBHZRLEya0gd/agyXrRQLLvvFMcQZIRYbHOotia6jiAoFutV0kdFW4gPvnotLjqlp26PQNPYwhKLOec253wDgJUALgSwPuq0qGsZY9czxjYzxjYPDFQOniAIYm4omk5VQ1BNI/jp5kM4OFzAbZsPygW3Vh2B6hE4HBV1BBMRox8BzyPwvJTaGoGf1SPeJUosjmwxIVNLo0ND4RkCekAjiJE1xPzOrvNeLFbhnI8CeADARQA6GWMisLcSwJEq19zEOd/IOd/Y29s7G69JEMQU2A5H2fYNQbiOoFpoSIR2DE2L9AjCGkHZcgKZQuH7TlTxCAqmrYSGps4aSuhuVo+lzCwGoj0C3xBogXOisoZUtHo1Aua3rZ73YjFjrJcx1ul9nwFwBYAdAO4H8FbvtL8A8KtGvQNBEDOLWGQ7q4aGKg3Bi/0TyrhHJhfcWh6B2mY6CqERZBJBsbhkOdIriRMaSuhMvpMTKCir7DXkh4aCP4dbTCRCfYyMOjWCuQgNNbIN9TIAtzDGdLgG5zbO+W8YY9sB/Jgx9gUATwG4uYHvQBDEDCIWdOkR2LVDQyO5Mq668SF0eTMB1FTKQNaQVakRhMNBKiJrKOwRlCxbtrSOkzVk6AyGpsmUU4HY5AdbTIS6j4qCsoqmc8GfhVFhLN4OP5A1NEseQcMMAef8WQDnRRzfA1cvIAhiniEW+vZqGkFoZz+SL8NyuOwbpK5r6kJfKFeGhqwaoyyricXlgEdQPeAhBF1D0zyPoFpoqFZlcaXXAFSGhoThiOMNiPOFUK4tBLGYIIj5A+e86rAZgfi8PROdPipCQ2L9ypWCn6vrml2loAxwxeJaoSHxHFUs1jWGkuVII9GWSlS9Plz5a1UNDVWGdcKVxeHhMeGfdXldvEVdU9pWL4j0UYIg5g93bj2KC794b80hMWJxbvEKucIegbiWc9ewhDuDBkNDNdJHlayhMG0pP5Ah3oMxt8q4ZDqy/1F7pnrAQ3gLhuaGhiybB5rXycE0hn+eGDAfNhJ+F9LodFK9iudQjUCTOvIICIKYTfYP5zBWMAMtnsOIxVvspGvVEVgORy50L3XxN2tUFpsWrzrcXhSzAb5HkPDmBJcsWw7MaUvX8giERuBW/lY0natIEa30DDQtuPALTaJaaKgejyDq+0ZChoAgCAD+Yhxu8KYiFmdDY0gZWkX6qJo1ZNm8oheRajiqNZ0DXBG62nt0KoZAaARiKljZcjBWMNGWMmouvDJrSHOzhsJN57RQHyE1Mygc83/jhuX47LVnBt5FJSrMVAv1cvIICIKYVUR4JrzLV7GUrphJQ6saGgLc/j35kEegGo5qTecAwPSyhloUDUCEaUTGEuAbgoTuvk/JcjBesKSGUQ3VIxAaQSBrKOQJqB5BuLJ4bW8r3v+Kk2XoJ6wRGHVqBPXOL5gJyBAQBAHAX8RrGgLFI4gyBGGPIBxmUsVldeHtHy8Fdsxl24HpOIHwTiahQ2NBEVg0nROtn0uWjbGCOaUhaM8Y+MtXnIzLz1jsaQRO5DwCsbir2UPhymKBrxEEj+t1egRzERpqZB0BQRDzCBEaKtcIDYnF2/BaLFc0nVMNgcMrCsyER5DQ/Tm+hbKN/okSTl/Shl3HJtx38DwCNSvo4lN6wBhk51PAb0PtGwIH+bKN9nTtpY0xhr+/9kzvd2GVE8pCVcOJiMVZD4m/wlhUVBYLjSDGLAKAxGKCIELkShZufWz/tGYD14sIz8TxCHTNncsbNhpFVQOIEItLnrFJ6po0KgeG8wCAtYtb5HmmpxGou+hrzlmGb73nAqkRiPAU4C7YKcPPGprKI1AxNOYVlPnHxFouNQJ1UlkojTR8vDJ9VHweb7kNegSxLjluyBAQRBNz9/Zj+Mzt27B7INfwZ4nQUK38faERGN7glQqNQPEATNupSB8VHkTS0GTBmDAE63pb5XluryEeCMmI3Xln1q1Sth3uL9Seh1K2XUPQUY8h8IxSZB2BXhnWCYeN/PtUSx8NagpTQR4BQRABRCpktfbOM4kMDcXRCBRxNnCPkAYQLigrKe0fRGXx/iHXyJ2iGgLbrSxORIi0ataQmrsvNILxohUIH02F7vX2iaosFtlCiYBGEC3+VhOLq2kKtd4n6vtGQoaAIJoY0W45arj7TCN262W7ekGZ1Ag0LTJ9tGjaMtPHcpyK0JAwMpmELnP3Dwzn0ZYy0NuWCpxn2TxY2est+l0BQ+CHblIJDfmSjcmSVZdHkBBN5wK9hvxaAY1F1xHEFoupjoAgiONBZN3EmfxVjacPjuJ32/qmPM/3CKqHhswp0kcLZRutnlBbSywW8wDKtoODw3ms6s4GFlapEeiVsfmOjD/mXJ0SljJ0DEy6PY1qVRWH0TWtYh6BFjBAWiAMFB5V6b9fZc2BuL/6/lO+D1O/J0NAECc8ostmnKHw1fj+w3vxhTt2THmeTB+1HYwXTfSPFyvOER6BK876huDpg6O47YmDKFoOWr0WEFHpo75H4C49JcvBRNHdwauLvsPdzxI6qwjFdLX4u31RvGVoGpK6Jj2oekJDCc0rKIsYVSmekYjhEbidTFnFLj7cm2gq5iI0ROmjBNHEiIW0Vv+fqSiadqyh8mK3bloOXn/jQzg8WsC+L10TOCecNSTCSdd94w/yHGkIHF6hbYjzRVpoybJh2g5aUkZFSCVfttCZTcjQjQgDdQY8gmBoSFBPaEjXmNdrqDI05D6DBT2CUM8heZ6SxaQSNdugFmo/Juo+ShCE3OEeT2jIza2fWmxWPYLDowUACMTNgaBGILJ07NA5MjRkO1XFYjFGsmw5KFmO17souBzlSjYMTVMmg4msoUqxOOl5KIJ60kcTulY5vD7QdVQLVRZX8wi0iowhoH6NQD1vlhwCMgQEMRUPvTCAPQOTc/LsyZKbNXQ8HkHJslE0/QWbc47P/2Y7njowEjjPTx/14/4i5i6Q6aO6nz66/ch44BzVI8iVrcDsYDVrCPBHUiZ1DcnQgJeCabutI0L9ftSBM2r6qGgbAQDdLfV6BE6g+6i6E0/orErWUHgOAYs2BBGN62pR72jLmaCRoypXMcbuZ4ztYIw9xxj7iHf8HxhjhxljT3t/rm7UOxDETPCJnz6L7z60d06ePRNisYjLi3tYDsfND+/FfTv7A+dFpY8eGikEzolqMfHInsHAOa1eCwjLdgvK1Gwg0c9I9Agq2w7KdjWPwArssqMWxXAdgaArm6w4txqGzipqJ9T1XMT+/c+qeASaVjGdTD2/3joCxiBbXzeaRmoEFoCPc86fZIy1AdjCGLvb++yrnPN/a+CzCWLGKJh2zdz6RiLE4uJxiMViF54vW2hNGfJ3UX8nzrmsAShZDlqSOnJlG4dG8rjgpC55nh3QCNz00V89fSTwvNaUu8gXTBumzdHTmpRFY1GhobLlegThHXPJcpDQmBwFGbWjVpvCqaGhugrKvIE2KqpH0JZKBHoeVWsid/Hansjn1ttrSBqOWTICQGNHVfYB6PO+n2CM7QCwolHPI4hGMdXYxEYy6cXYj1cjAPzMI2EA1MWvbDsQWqlpO1jUlkJuKC+1AoHYOQuNIF+28dyRcVx8Sg8e2TMEwNcIxID5nhbFIxChoaQaGuJIGNFhFUMJDanL4kk9WTec5H2W9ArK/OviBzsMXUM51DNJNQT/8e7zZLhL/SxsmN514erI+9c9oayKGN1IZiVriDG2Bu784scAXALgQ4yxPwewGa7XMFL9aoKYW0zbqTokpdEIjWAmQkNCuBX9gcRX2+H4zTN9gfPFonU4FBoSA+dFZbHgXS9bLQ1Bi7doiklhi1r9MI3MGkqIrCHhEeiRO2Y1NKT2NXrwE5fJdwfEYBo9cO+4RHkE6qKtVjyrn8XN6KlfLA5eNxs0XCxmjLUC+DmAj3LOxwF8C8BaABvgegxfrnLd9YyxzYyxzQMDA41+TYKIxHHcSVl2jf47jcK0HRm3Dw93rwex+BZMN8wUDg39/rmj+PhPnwk8VzSHq6YR6IxJcfaknixWdKblOWKUpNA3VnRmlHu71wdCQ7aDhMHkLl4dOp/QGBa3u/dmqFwYdY2BMU8j8K5vSdVnCHSNVTTPq7Ubn25b6bC4PNX9Z0soBhpsCBhjCbhG4FbO+S8AgHN+jHNuc84dAN8FcGHUtZzzmzjnGznnG3t7exv5mgRRFbFAzEVoSG3PEB7cUg++RhDyCLzj4awfN6XTPXckXw58ZtkcGnMXN5G6uX5pu/QCADU05L7/aUvb8OsPXYKLT+mR5/h1BK5HkFKKtlqVeQOGruHGd2zA595wJtYva4v8/drTCbRnDFlHkE3WF+hI6BrCzV1r7canK/7GNhwiNDR7dqChWUMMwM0AdnDOv6IcX6ac9icAtjXqHQjiePENwex7BGIhBY5PLK4IDYU8gp1HQ4ZA8QhESumBoTwmSxYsx+8IesyrPF6/rF0OkQf8hVw0zMsmdZyzsjMQShIpoKK+Qc0aak0F00O7W5J43yUnV82gue0DF+N9l5wsPZRssn6PIEytRVivohFUQ5xXr0Ywmx5BIzWCSwD8GYCtjLGnvWOfBvAuxtgGABzAPgAfaOA7EMRxYVp+HH22UdszzIhY7IWGzJBGsPPoROD8ssXlNSI09fbvPIK3b1wJ23GUrBl34d64pivgEYjQzLhnyNRxkgIRGhK/Y0L1CJShMuFq4yhOX+p6CkIsVt8lDokoQxAjNBQ31KPV6RHMRWiokVlDDwMRQT3gzkY9kyBmGrFY1hro3ijEIqmx6RsCS6n8zVdkDdmYKJoVOkDRsuXvLTyC0UIZI3kThtL758OvWYcLTurCJesWBSaViVGSwqPJJNxlRq0TEMZBnJM0NDDm5uur3kXctgwAZNO4+j1B39MZAAAgAElEQVSC6tXA0efXl9457ayhhSQWE8R8xvQ6cc6FRyA0gu6W5LSbzqkiaD4iNLR3sHLgTUCb8AyB7XBYjtsaWmTxZJMGrjxzCQB3GHx4Ry/SR4UeYEzhEQBAV0sSSzt84Tlu+EX9/Vrq1AiinlHL/tStEdRdRxC8bjagpnMEUQPRm38uNAIRnunIJKbdYqKkzDGIEovDvYAYC+7SxehJy+Fue2qDV12gskkDYwVT5tyL9FE/NKRoBN4xYXSEfvCLD74c+bIti9TihIYE6xa7aZ6vP3tp7GuA6AW61m5c9DGKW7Tmp4/WN6pyNj0CMgQEUQPRm9+ag/TRsmIIjoxWtoSOg5ofnw+lj5a81E0VjTG5k+/IJDAwUYLlFZuZtgNdq76zbU0ZGC+aFWEf6RFoER6Bd46I76/qzmJI6W9UT2jo9KVt2PoPrw1UAcchqvis1m58bW8r7vjwK3DmsvZY96+311C9dQczAYWGCKIGc5E1dHi0AMt2AoZAaATlmJ1EBWobiXzJxod++CS+ft8L7mfKM/7HpWsBACcvapGLs9jxCq9BhIaqLWgtKR0pw28V4WsEIjTkLzeuJgBMhEJDgBsKE9TjEQCo2wgA9XsEAHDW8o7YfYCqDbKpxoKrIyCI+Y4pq29nRyz+/sN7ccmX7sPPthySRkg1BH/yzT/gzP/1+9j3U0XcfNnGb57tw7bDbrqoWi/w5vNXYN+XrsGyjrRcwDs9QzDhVTeXLbe4rtouPZs0kE7o8nPRPdRvDKc0bmNu6wgZGlIMAWMMK7vcIrR6WkVMl0iNYAbDMtUG2VRDPHsWI0NkCAiiFmLHPNOhoRf7J/Grpw8HjnHO8Y+/2Q4AODpeDHgEZcvN/nkuVPw1FYHQUMiTEA3fAH8hThma3KWHPQIx4L3aTrU1ZSCT0GW1LxBs9xDss++2qBBGJxEa6HLyohYA8RfP4yHqGTO5G6+3LmAums6RISCIGjQqNHTrY/vxmduDtZTqou1w3wgJcXI6gnHQEASvV0NDoipXDdEIQyAyeyzHbb5XSyOQYSDvHLXKN9DT35sxPBnhEQC+IRjzBOdGEuXhzKT9mXb3UcoaIojmoFEFZflSZWtrdaEuWbbsbd9RhyEYyZXR1VLZ5E1jlXOPy4pYLLt4Rkz5EuEb0+KwjOoawQdfvRZDOVfoNTQNpm1Lodg95l/XnjaQMjTfEIQ8ApEBNDARHIzTCKLTR2duEZ5uARrVERBEk9CogrKCacN0wobAD92UTH+3LgaxT1VUtmdgEhd84W5s2e838xUeQVs6UdGvqGz5rSTEQhzlEQhDUPa6sFZb0M5d1YnLz3DrCsSir4aGDEUraE0ZSBqaFKbDHsHbN67Ce1++Bn/1qlNq/s4zQZRHMJNhGaojIIh5ji8Wz6xHUDBtcO7e919/vwuMAW853x/XUbIclLzJXWJXPZVHcGA4D4e7vYPEMBlhTFqSesXvYDlc3lMYgmTEcBc1NGQ7PNaCJnbZapWvaOXQnk6AeWJxIfR8QTqh4x/eeNaUz5kJonsNzaAhqFMjWLDzCAhivlK2GqMRqPOBv/3gbgDAJWsXyc9Llo2ypSGla3JXHS7+CjOad+PpfUrNgZwIFmEIAD99Ux3wIgh7BKbFYdrVNQIV4TUEQkPevdu8ymN18Q8bgtlE/X0MjcFyeM3K4noR94pdRyDF4pl7h6mg0BBB1KBsi4KymQ0NCT1gXBFDf7L5oPxeDmxRPAI1Xq4u6vsGcyiUbdky+siY3zvInxFsRBuCoiX7/AC1PQJTeAQxViiRKpoJDJp3j4lcf/VZ9dYLzCTq7yPnI89o+mj1mctRNHUdAWPsFYyx93nf9zLGTm7caxFEc9Aoj0AIt2rnz3t3HJPfC40gaWhyUEu/YghMRbt49b89gI/+5CmMRHgEQuOo5hFMliyk1EIv1SPIijoCv2tpLY1AZVV3FkAwNCR23tIj0JvDI4gySDO5CItfs/55BE1mCBhjnwPwdwA+5R1KAPjvRr0UQTQLjdIIRGhoR59bF9CVTUgvoSWpu6EhWxgCdzHtn/AXeGGYjo65x7bsH8GY5xH0BTwCXyOIMmaTRTOwEK7q9qeJRYWGbIdHtm0O88ZzlwMA9g7l5bGaoaFZKByrxtJ2v8md8AjiVg3HQauz11AzewR/AuCNAHIAwDk/AiB6XBBBLCBUj4CHx1gdB0Ik3Xl0Agmd4SUrOuRnndmkP7nL8DWCY+O+R2DZDt70jT/gPTc/BgBY3Jb2PYKxonxXoRFkk0Zk5tNE0QoMfV+v9M/pjGgx4fYbmnqBuuZsd/5Ue7py6HtUaGguPYLlyihNGRqawUV4ZVcW121Yjped3B3r/LnoNRRXLC5zzjljjAMAY6ylge9EEE2DunjGjY/HQez+X+ifwOruLLqyfu5/ZzbhZg2FNIL+cd8jMG2OZw6Oyp8Xt6ekRlCyHAznyuhpTfnD4pN6Rd0C4IaG1EX4tCX+/q49pBGI6uY4/w26WpL45Q2XyFYRgO9ZiHTYVJN4BOlAiuvMV/UmDQ03vvO82OfPRffRuP/1b2OMfQdAJ2PsrwDcA3feMEEsaNTFcyZ1AuERjBVMtKUTMlwCAF3ZJEqmW3CWVLKGojQCweK2FEbzpqyIPTxakO9vaG47hyhDIMRigbooioVahoZs7rWYiLdsbFjViUWtKfmzGF0ZDg0xNru731okdf+d5oq5qCOI9X+Uc/5vAH4GdxD96QD+F+f832tdwxhbxRi7nzG2gzH2HGPsI97xbsbY3YyxF7yvXcf7SxBEo1DbNM+UIbAdLhflfMlG0tDk7hvwewv5YrEIDSkaQaj3kaFrGC2Ucd5q95/Ts4fGALjeQcrQYGgs0G5CEPYIVBhjSBl+YzjLcWA6TiyNIArRV0j8ruqcgpmMyU8H2RiuAWJxvTSlWMwY0xlj93DO7+acf4Jz/jec87tj3NsC8HHO+XoAFwG4gTF2JoBPAriXc34qgHu9nwmiKVF30XaNxnN7Bibxx92DFcc3PT9Q0S9HLQzLl20kdS0QLskkdaWgzG3tzFjIIwhVJVu2g9GciXNWdmBRawpPetXFE0UTLSkDGmMVswfE5+GwzF0fexW++afnA3A9hEnFI7Ds6k3npuJsTwfZsMr9esoit41ElKcy2yzxBON6Uz0bgS5DQ7P3zCkNAefcBpBnjHVMdW7ouj7O+ZPe9xMAdgBYAeBNAG7xTrsFwHV1vTFBzCJmwCOovmB9+8Hd+MRPnw0cy5ctvPc/H8fPtxwKHfcNgWjVLMIl2aS78BeV0BBjDGkjOIc37BEUTQcTJQtd2SQuOKkTWw64hqBvrIhlHemK1EWRkupwd8ykymlL2nC1J/amE34/IPc59rR1kjefvwJ//OTluOAkVzQ9Y1nz5Jss88Zj5sr+nOi5opkH0xQBbGWM3cwY+7r4E/chjLE1AM4D8BiAJZzzPsA1FgAW1/fKBDF7xNUI8mU7sGACbq2Awyt7BIVbRaihoWzSQMrQvawhW8bpM6GB7GGNQDR768omcMFJXdg/lMfQZAlHx4pY2pGGHlq8xThJ8fxqpBN6oKK5YNp1TQ1TYYwFMnTOWBpvwtdscMNl6wC408eA2Q3LhJFicRMagjsAfBbAJgBblD9TwhhrhastfJRzHruZOmPsesbYZsbY5oGBgbiXEcSMYio771qGoGw5FQu+KauSg9eFz0vomvQI0gkNqYQm6wikIUjokff48GtOxSm9LRjJueGn1rSBlV1uMVf/RMnzCDIVHkFsQ6C0igZcz2OmdqrqJLK55rIzFmPfl67BUs8zmNPQkPAImm1mMef8FsZYEsBp3qFdnPMpG4UzxhJwjcCtnPNfeIePMcaWcc77GGPLAPRXeeZNAG4CgI0bN87+wFiCQLCffy2NQPT2Vwe3+DUI1dtNAwhoBK5HoMG0OQplRy7SIpTDGMC5n8nTktSR1DW/eZyuo8Vb5I+OFzFZsrCsI41c6Jmt6bgeQeVnszEsZq4Qv9pcegRNO4+AMfZquPH8fQAYgFWMsb/gnG+qcQ0DcDOAHZzzrygf/RrAXwD4kvf1V9N6c4KYBdQQTFigVfEHwttyGIvfwjrkEYQNgaGhI+Nek0nqMmY/oVT9itDQ4rYUjo2X5D2S3ozg0bz7s6EztKbcc3f3TwIAlnaksW/Qr/AFgh5BqoYhiDISMzk+ctMnLqsIqc0l+hx0/gwzF3UEcQvKvgzgtZzzXQDAGDsNwI8AXFDjmksA/BlcbeFp79in4RqA2xhj7wdwAMDbpvPiBDEbBLKGaoSGhMEolH1DUG3ecVgjcENDrkeQSehyYS55YrE4DrjZLcfGS3KHn9A1GJrSzlnXpEewe8A1BMs6Mjg04redAPyiLmAahmAGF8nVPdkZu9dMwOYgLBNG/OedzRq7uIYgIYwAAHDOn/fCPlXhnD8M13uI4jUxn0sQc0rAI6jRgVQYDDX+L45VeAQRhsAPDemBoi6xEIvd4eK2NIAxOcQmqbs1AuKehs7Q4hmiF/uFIUhXhBkWtaVkkVmtqt6oz5ql+KsRyNTNOezL3MxZQ5u9jKFXe3++i5hiMUHMZ8p2dY9gLG/iNq91tNAS1N2+MBxTaQQJgyGd0JDQmRcaquzBU/Tuv7g9FbiHCA0Jo5MIeAQ5eU14F39SdxZLvHvV0giiPpvLltGNphk0gmYODX0QwA0APgx3l78JwDcb9VIE0SyI6t6y5VRkDb3vB4/jyQOjeOWpi6TBKJpO4Fqg0oCEPYKUVyuwtCON3raUHCQP+Dn+RW/hX9zmGQIvri5CQ4KEztDiaQTDuTLaUm46qrqofPs95+N1Zy3FPTuO4eBwoaKOQCUZ8VncFhPzEbEIz2VoqGnFYu+8rwnRlzGmA0jVvoQgZp6v3LULzx+bxLf/rJY8NXOULAetKQPDVrkiDfTJA27TN4dXCQ1VEYuLYY/AC7/c9oGL0ZZO4JHdQ/IzsSMX9xUVsL5GwCoGq6QMHQmdwbQ5OlvckJN6jqG5hkf0AarpEUSEhhZ21lATiMXN2GLC414AGeXnDNzGcwQxq+w6NoGdR2OXoxw3RdOWOf5qiEdtSe0ovYPUjCCZPhrSFio0Am8hXtaRQWvKiAwNiWukR6CGhrTgIg9ACtadGTdXX91diu87vY6n9YaGFrRGIBfhOXyHJp5HkOacT4ofvO+bS+4nTggsm1fssBtJwbSl+KqGeER3T8AtNBO7/0JAI+Dyc5WoOgIV1RCI6WF+aCjt3UMViysnbIn00E5vypgRYQi6vM9q9fqJyig6ETSCpigoa0JDkGOMnS9+YIxtBFCocT5BNISy7UQ2T2sUhbIti6/U0NC2w75XonYTLQZCQ3bFdeIc9d94IrTYpiKyhl52Sg8AoNfzCETbh4ShBdpHiDCT0AnErl8NM/iGwP1MzDGIIsoj6G5ZuFHhuRBqK98Bs/4OcTWCjwL4KWPsCAAOYDmAdzTsrQiiCq5HMDuGwHE4SpYjp2ypO/sjikfg8GhDYFrCIwiFhso22tIJ2ZU0GdphR4WGvv6uDTgyWpT1BAXT9wjUttBCCxCZQ2LXr2oE0hB4LR5G89WbBERpBItam6c1xEzjtsSm7qMBGGMvZYwt5Zw/AeAMAD+B2176dwD2zsL7EUQAy3EqdtjHQ9ly8N1NewLhkZs27caff/9xmRIqwixqYdixwLQwP6NI1QhK1SqLFd0BqNx1q4ZALOjZpIF1i1uRMNzVIa8UlKlZPGLhbpEagWsI1HPEQvPSNe7sgtesr973McojEF7JQuQt56/Eje/YID2ruaAZR1V+B8AV3vcXw60M/n8BbIDbB+itjXs1gqjEtPmMhoY27x/GF+/cgTOXt+OSdYsAAM8dGcdzh8dkvF+EhtQF/ahiCFQvoKCkj5pV0kfznkcgoqvhRWdlVxZvPm8Fzl3ViVd47yQQekC+JMRiFojZG1VCQ1EawUk9Ldj9T1fXXHCiNIKFbAiWdqTxpg0r5vQdmrGOQOecD3vfvwPATZzznwP4udI2giBmDdN2B6hzzmdkqpXYwas7+XzZnQUgFvjWlLurVhd01SMolP3FvxBRUBYOZRVNG9mkDkNjsBxeYQiShoavvGND5PuKRT8nxWI9lD4aDA0JsTiQNRShF1QjyiNQ+xQRM08zisU6Y0z8X38NgPuUz+hvAzHrWDYH57X7/qiMF00cGslX/VxWBFuqIbBQsv220m0RGsGx8RLavAVRXfxLES0mwllDBdNGJuEv4PUMbmeMQdeYNFwJg4WyhtzvW6VG4KWP1rH4q0QZgrkeK7nQacb00R8BeJAx9iu4fuxDAMAYWwdgrMHvRhAViA6gcecHf+2eF/Ce7z1W9XOx61crgoVHIBZbsaiKegDOOY6OFeWQlUIgNBTVYqKy+2g6oSPhLeC18vijSOhMegQJPVhHIAyBqCPoEB5BhFgch3qMFDEzNF1oiHP+RcbYvQCWAbiL+1U0GlytgCBmFSEUl20n0JytGgMTJQznqqdHCgMQiPN7BkAMW5eGwFvQx4sWCqaN5Z1p7Do2gULZqrgW8MXiqIKybNL3COoVJhOahgmvxURSSR9Vs11EK+quCI2gnsrgeo0Ucfz4oaHZe+aU4R3O+aMRx55vzOsQRG1k3D3mwPN82a4pLpcs4RH4C7jYbYv0TiEWi3CU0AdWdHkeQbmKRyDSRyPmEbihIfdfer0FWqom4KaPevdRQkTLOjJIGZpM9VS9gHraJ9RqUU00hmauIyCIpkBk7sStLs6XLZSt6uKy8AjUSWRiYR8vuoZAaAHCCAnNQYyELCrXRhaUhesITBuZpC7z/+vddauDYdz0UeFZ+L/fmzYsx8vX9cg5B6pGcDwewYVruut6V6J+hOYzmz2dyBAQ8wqxqMYtKst7A+Td7JwoQxDhEXipmeNVPIJnD42BMeCs5e7wddUjKAbSR6NbTBRMTyPwFtl64/DifF1zhWPxe6kGwtA1LOvIKD9PVyPww2//90OvwMm9LXW9K1E/XdkEPnP1elz1kmWz9kwyBMS8wpIeQTxDIBbpsuVExuKLodCQ43AZ3pGhoZBGsPXQGNb1tsphMnIojDcg5r8e2Ycl7WkZklJDQ6IdhUgfBerXCHxtQcSSRYip+n2iCsrioHoEZ6/sqOs9ienBGMNfveqUWX1mwwKAjLHvM8b6GWPblGP/wBg7zBh72vtzdaOeTyxMqrV2rkbea8VQqqIplKRYXJlGGvYI3NRVjmcPj+HslR1yQRXGpjObQK5k4VsP7MZtTxz0DYFTWWeQSehy4a47NCRCSiGNoZbWEFVQFgcSi08MGvl/+QcAroo4/lXO+Qbvz50NfD4xzxE7chWrSpFWNVSPIIqwWCzCQurzW5IGGHNbTBwbL2FgooRzVnRIMU8s7is6M+gbK6J/ooSRfFlpQ+0bLfE+6ePJGgoZED2GZxFoOleHCEnpoycGDfu/zDnfBGB4yhMJIoLdA5M47x/vwvYjfpdPx+EQ4fa4hkAs7NUMge8JVM4TEIZA5PyPFkyZMbSqO+t7BJ4hWN3TgrGCCdvhGC2YkZXFRcUjEKJgvYttQg+Gggz5tYZHEBpMExfyCE4M5uL/8ocYY896oaOuaicxxq5njG1mjG0eGBiYzfcjmoBjY0U4HOgb87t8mkqIJU5oSI33l5SQD+DG6m+853nsG3Ln+ooFWoSSALdeIGm4WTmvWb8YP9tyCAeG3YyhdEKXhkDMCjip2x/RMZY3IwvKRLO4bFL3QzrG9NJHxSIdDhVFEUwfjf8sSh89MZjt/8vfArAWbtO6PgBfrnYi5/wmzvlGzvnG3t7e2Xo/okkQ8XV1J68u/nE8AjXeH9YI7t/ZjxvveQFPeeMmq4WG0t5C+LdXnYF82ZbD6tMJP21TLO6rFUMwWjCl/qAagsJMeAQhcVgYgpoeQcQUsziQITgxmNX/y5zzY5xzm3PuAPgugAtn8/nE/EEYAHUBVyt043QgVSeBhc9/6uBI4GexaIdDQ5mkmz4pdvuDk26VcsrQZazdDw35hsB2OIa9gS/qe0uNwOs1NJ3e92FtQfxca4FXNYJ6PAIKDZ0YzOr/ZcaYmhj7JwC2VTuXOLERu/9qHkGcmQR5ZXdfUvL7f/X0YXzj/t2Bc4X3kFfaRYwXTDkIRtMY0gkNo97ink7ockEV3oTqEQDAwHgJgDvc3vG8AqkRJN2sIXcQSn2GQCzOfmhoas+CNAKiFo1MH/0RgEcAnM4YO8QYez+Af2GMbWWMPQvgMgAfa9TzifmNqMpVY/tWQCOI4REo8X7VI/jIjys7qEuNoBwMJ6n9jDIJXY51TCf8WcHCI+jKJgMDZ0Q/IPfdeeD+oo4gNY2snJd61b2HR1z9JE5oKKAR1GF3KGvoxKBhBWWc83dFHL65Uc8jFhaiKlcNDYljQExDoC7q3mIt+iaet7oT/eMlOYReZA+FB8tXGgI/k8jhwcU9aWhY2p7GZGkSPOSwWI6DJLSKOoLwvOI4vO2ClfjX3+/C4KTrcRihLKIohNHSNVaXB2KQITghoMpioikRO3hhCLbsH8a3H9zjfx6j6VwhQiMQ4aUr1i/BA7v6FUNQGRoCIENDQNAopBO6NC7Fsi3bPbzjpauwbyiH/370QOA+4rmiU6nQCOptOAcAi9vTeP8rTsbS9jSAykrjKGSPe5olQERAhoBoSsRCL77e9dwx3L39mPw8zjyCnBKakffzDEJS19DhzfMFokNDANCe8f+JBAyBoUkRuGDaMoTyl688BQMTpQpDIPoUTZb8GQenLWmT1cv18tlrz5Tfx2lVMRfDToj5AxkCoikxQx6B2t5Z/bwWgclhIcOSNDS0q4bAOz6SL4MxyNCO6N4JQGYQ6RqDoXT9tByOTNJfYFUDkzQ0lC1HGo1cyYLGXI3hhsvWTfk7xEF2q6wZGiJDQFSHAoBEUxL2CIohQxAnNBRIH40yBMoiX7Yc2A7H3duP4eJTeuRx9RwRJkqHWjsAwd140tCQ9YyG+Lp5/wgch2OyZKElZczouMe6QkNkCIgIyBAQTYnvEXjD5U0n9HmM9NFahkDXZIaPKJp66IUBHBop4O0bV8nrokJD4qtWo8f/W85fiZVdGZnh8z9vfRIPvziIXMma8eHvMjRUIy30eAai//j6i3Dvxy+d3ssR8wIKDRFNSSlUWVwIxe7D4x+jUEdICoMi0lKThiYXdLE4/nTLIWQSOl531lJ5XVRoKHwdUBmf//x1LwHg1iwIbWNwsoRc2fUIZhI/NFR9kde06RWvAcBFiodELEzIIyCaEpEqKsTdcGiomkZQNG3c+th+Lwxjy3BJuFI5oWtS4BVL4x3P9uGyM3rlgg8A7UpdQCbhnp/yvqoZONXCMmrx1kTRwmTJnnlDELOLqaExyhoiIiGPgGhKZEGZGS0Wl6uEhv7pzh34r0f2Y2VXFmOFMjqzSYwXzAqxOGVockFX4/WqNwAgICjL0JDhVxsLYbmaUKvu0idLlhca0iPPnS5+1lDtRV6kuBJEGPIIiIZRNG287qub8PALg3VfO12PYOvhMQDu4jicK6Mrm0DS0CKzhoQ2IHb9i1qTuPyMxYH7RYrFCWXaFxN9fqIXWHVxnii6hiCbnGmPYOqCMsD1TsgQEFGQR0A0jKFcGbuOTeC5I2N4xamLYl2zed8wTl3SFiEWx9MI+kbdeQGWwzGSN9GVTWI4V/a7mdq+ITh1SRsA4H+8ei1evrYHJy9qrVgo1ZYRYbEYcL0COLzqIqyOiJwsmZhsoFg8VRWwxmZ3IDoxfyBDQDQMUdCVCwm91TBtB2/99iPYsKoTK7rcwesyfbQcLzR01BscUyjbGMmVsW5xKw6N6DLEpGYNnbuqE/d9/FKcvKilajqnWhMQFosBd2Eto7pQm1AW3knPI2iZ6dCQmE8wRWjI0DXXcBFECAoNEbH4yt3PY5sXdonLpDAEJWuKM13Erv/pg6MwQ+JutYIy2+G48isP4o5n+2T1LuCGkkbyJjqzSbeoy64MDQHAKb2tNXP644aGqqVuqrt0VyOYebFYj+kR6Bojj4CIhDwCYkos28HX730BjsPxkhUdsa8TBiDcv6caqg4QXriLSh1BOqFJQzCaL+OF/kns6BvHOSv9d8uXbYzky+huSSCpayjL9NGgIZiK1kDWUFAsBiB32NU8AjXUJEJUrTOsEaR0932mGiJjaCxQ+0AQAjIExJT4DdvizQkWyNBQKV5oSJ0ZoLaY4JwHPIJs0pDzCEa9Xj2TJQt7B3PynP6JImyHoyubRCoRIRbH7KqpLuRpLzSUUkJDU+3GVbH4mDefYKY9go5sAl9+27m49PTak/zc1hhkCIhKyBAQUxI1LSwOosFa3NBQwCNQFu7wczMJXRqnUa8tdK5k4eBIXp5zxOsq2pVNeh5BUCyezghG0VoiEBqSVb1T1xGIwfczLRYDwFsuWDnlOTp5BEQVSCMgpkQsotP2CGKHhtRRlGIegV1RVZxN6vJdxgpl+YwDw3m50z/iZQ91t1TxCKYwBJ3ZRMWxKLFYpo9WKyhTjouOqTPtEcSFNAKiGg37G8kY+z6AawH0c85f4h3rBvATAGsA7APwds75SLV7EM1BaZqGYFJqBPFCQ+qw+bx3bclyAseBoCEQHsFkycbB4TxWdmVwbLwoPYLOrKsRjBfc+8U1BA/97WUV/YyiNALpEVQJDdkR7bJnOmsoLobGKGuIiKSRHsEPAFwVOvZJAPdyzk8FcK/3M9HkhHfTccnVmTWkhoaOjhXls8MeQTqhy4KzQGhouIBV3VlkkgaOjLmGoLsliZSh+72GlBYTtWhLJ9Ddkqx4rvvVv1ZEfuoxBI0IDcVB1zTyCIhIGmYIOOebAAyHDr8JwC3e97cAuK5RzydmDrGIxun4qRIWix2HY/9Qrur5amhIzPstW06FR5FK+Au7EItzJTc0tLo7i0xSk/fqzCaRTerIl204DkfJcsCmWe4M7AcAAB2KSURBVFglQkOqviAbvlW531nL2/GF616Cv3ntafLY3IWGqA01Ec1sawRLOOd9AOB9XTzF+UQTEBZa4yLFYk8juGv7UVz+5QfR74mmYcJtJATjxeAUr0zCX+jHvGHyR0YLGCuYWNWdkSEcXWNoTxvIpnQUyjbef8sT+I/7X0RS16Y1D6Azk4CuMfS0puQxsa5WyxpijOE9F52E5Z0ZeWxtb2vdz54J1i9tx+leNTVBqDRt1hBj7HoA1wPA6tWr5/htTmzCvfzjklM0As459g/lYTsc/RMlLPbm7f7r73fCsjk+dfX66oYgNM4xndClbiCGyY8X3Wet7MpKQ9CVTYIxhmzSQK5sYfeA643ErSEI09Oawu8/+iqs6cnKY75GUNuwiAjRxpO6pv384+Vf33bunDyXaH5m+2/kMcbYMgDwvvZXO5FzfhPnfCPnfGNvb+38aKKxTFcsFp6A7YVkhnPu7l1d2L9x/258Z5M7lL4YMjRikR0LGwJDl0ZjNPTZkvaUjOV3eZk/2aSOoulIz2I6qaOCdYtbA7t/TTadq33PDas6AQCfunr9tJ9NEI1itj2CXwP4CwBf8r7+apafT0yDOOmjt/xxH3YeHcf/fvM58tikIhLnShYGJz1DUIwWj0shj6A1ZWCsYEpD8Lk3nIl82Ub/eLEiNCTobU3LWH6XJ/a2eJW8QliOW0wWhzhjIgHXgOz70jUz9lyCmEka5hEwxn4E4BEApzPGDjHG3g/XAFzJGHsBwJXez0STE273EMWje4bwwK6BwDE1WyhftjGccytrxc48fD+RHbTSazgnsmuEIbhi/RLccNk6pJO6rDQOewSL2pJKaMjzCELpmjMZmpmqjoAg5gMN8wg45++q8tFrGvVMojH4Yx6rZw0VTbsiTTRXsmXGTq5sYcgLDU14HkH/RFA0Llo2dI1hWUcah0YKFYZAHQzjVhzbGJosI5PwDUM2aUhD0B3yCAQRGZ3TRvYamiI0RBDNDP3tJabEF4urF4YVTaeiQ+hkycLiNjfDJleyMDQZ1AiOhbKHiqaDtKGhK+su4KIF9NEx15MQhkEYhO8/vA+TJQuvPzs4VUz0BBL3UUdPAvGb4MVBdh8lj4CYx5AhIKbE1whqeASWDdPm8lzOOXIlC4vb3OygyZKNoVBoqG8sbAhspBO6XMBXdrshoqcOjKAzm1BaPLh/bb/94G5ceHI33nDucgD+YqxmDQGVHkHcSuc4xG0BTRDNDP3tJaYkTtaQEG9FnD9ftmE5HMs6XUMwOFGS54jQ0FHFEFi243oECR2dLa4n0Ot5E0O5MlYoefhioR8rmDhzWbv0FMRxaQi80FBYI2iEIZiqUpkgmhn620tMSZwWEyLjJ2+6i/zgpLv7P6mnBQBwYNjvDCpCQ6ohKHo9hVIJPzTkOBxt3iKvGgK16VtrypA7fjELWGYNeWJx2COYSeLWERBEM0OGgJiSOJXFIq9ftJMYmPAMQbdbfHVQMQS+WFwKXF8ybaQNXS7gI3lTjqwUX4Fgr5/WtCEzdsTOPx32CJKNa/IWt46AIJoZ+tt7AvLv975Q19jJWqGhr9y1C2s+eYcUiv/5dzvxn3/YKz2CNYtcQ3BoxG0C19OSlBqBWihWNG0vNKRh45puAMClp/VKT0D1CFIhj2BVVxbLO9L43BvOAgC0eAt/zywYAmOKCWUEMR9o2hYTRGMwbQdfvvt5TJatmmMncyULKUODoQ51UUJDtsOx/cg4/uP+FwH4RWJ3bz+GiaKJa85xBdxVXVlozB8qv6IrIwu71B5CRdORYvHa3lbs/PxVSCd0bN7n9i2M0ggAoC1tIJPU8cdP+VnJrz97GQBgteeNNLLJm0ahIWIBQB7BCYYIy4RbO6t884EXcdbnfo9L//UB7OgbR9n2uodyYPuRcdgOxxfv2IE3/MfDMidfbbc8WbIwOFECY24uf0vSkKmiyzsy0gCMF0xZ3FU0bRQtW2n17H6NDg0FPYIwHZkE3nnhatlYLmVoUJtunl3H3OWp0Ck0RCwAyCM4wRBCba3MmWcPumEjy3HwsZ88HVg4r/76Q/jrK0/DT7ccrHr9RNHC4GQJXdkkDF1DNqVjYtw1QMs7M5goWuCcY7zo1hkcGikEQkMqrz59MR7bM4zTlK6ZAY0gxm6fMYaWpIGiZePxT19RUVdwPJBYTCwEaBuzALAdjn/53c6KAq0o4ngEubKF81d34qNXnIadRyfw6N6hwOfPHhqT94li0jMEi1q9PH5vsdaY2xTOdjhG8ibGC6YsOFNDQyqnLWnDze99aeC4OiGsNR1vL5NN6WhJGehqSVY843jQqbKYWADQ394FwNbDY/jmA7vx17c9PeW5IixTq7p2omihJWXgjecuR0tSx8HhQuDzUo0KY8AdKjMwUZJ1ACJ9szVl4OVrFwEAfrblIEqWIwvOfI9g6kVa3dG3pSpnC0fRkjQakkaqk1hMLADIECwAxBI0oKRjViNOaChXstz8/JSBM5a1V3z+Yv9kzWeULQd9Y0Us8ga4iKydtnQCZ6/swLkrO/CtB3YDcD0EwE0lHS+YaIuxw5+OR5BJ6g0ZEakxKigj5j/0t3cBIBZ1kcNfCxkaqjIExr2PJRfNpd4Amajn1UI1BCI0JO75+rOXyYEyYkDNLX/ch7Lt4A1etlEtUopGEHcQfHs6IXsXzSRi/adZwMR8hsTiBYDo+19rcRf4oaHq506ULLl4L+2oNAROzPadFYbA272LtE7AbyOx69gELlzTXTOlVZAyNDDm7sJTRjxD8Jlr1oPPYNdRge5pA4k5mjpGEDMBGYIFgD8kfuqumiI0VE0sFs3ixO59WYQhsGIaAl8jcBdrcc+VSiqoEIsB4PyTumLdlzGGtKHXVSgWx8BMB+ERJEgsJuYx9Ld3nmM7XHoEpRgzhUXhV66KWFwwbTjc38UviQgNxfE8AMisIdEDSHgEK7t8j0CIxQCwvLPyWdVIJ7SGForFhQbTEAsBMgTzmKHJEtZ++k58Z9NueUxtA/GtB3bjqhs3Ba6ZKjQkjIpYtKM8griI0FCrF8cXDeRELyEA6PGMBeAWm8UlnWiM+FsvVFlMLATm5F8SY2wfgAkANgCLc75xLt5jvvO/f7sTAALpnYOTJSzzFtR//p37uWk7MqtFiMVly4HtcJn+KBCCs1i8ozSCuIjQUDYkFouKXwABAXdZHR5BJqHHzhhqJAbVERALgLn8l3QZ53xwDp8/7/n9c0crjh0bL6FkOjipxw+/DE2W5YI+rjR6y5cttKXdhXiyZOHhFwbQng62blZDNyqGxqbUCvxRkZ5GELFwpxSRVe0nNBUd2YQMPc0lGtUREAuAud9SERUUynbNNggf+fFTSOpaZHXvbZsP4kePH8BPP3CxPDYwUfINgXJNoWxLQ/CTJw7i87/ZLj8Ti3bS0KBrDC87uRt/3O1XGLekDEwUzcD837s+9iq3aOxL96G7JSm9kHD6KAC89swluGv7sarewVR89e0bAmmkc4VOdQTEAmCuDAEHcBdjjAP4Duf8pvAJjLHrAVwPAKtXr57l15s7BidL2PiFe/DZa8/E+qVtuHhtT2CxBIBfPX2k4rqUoaFkOdh2eAycA3fvOCY/G5gsAnCzZiaKJnSNwXZ4QCfYPRAsElMX7d3/dDWePzaB137V1xsyCXeAvCocL2lLoz1jIKGzwG5diMVqsdi33nMBLCcobod/z1qsWdQS+9xG4reYII+AmL/M1TbmEs75+QBeD+AGxtirwidwzm/inG/knG/s7e2d/TecI8TUrs//Zjve/b3HcPf2Y4HPh3PlwM8iDi9SMvcO5AAAT+wdlucMTvjXjBVMuUjnyza2HxkH5xz7BnOB+4YzcpKhHW86oVXsyFMJDYwxtKYMKRS79/IriwW6xmLXADQzNLOYWAjMyd9ezvkR72s/gNsBXDgX79GMhBf6wcngzzuPjgd+Pm1JKwBX1DU0hgkv62fPYE62eB7whsTYDsdE0ZJi8iN7hnD11x/CA7sGsH8ojyvWL5b3bQsZgnDBVDqhB+L7jPnx/iXtaaxSUkSXdWTAWHUN4L6PX4qH/+6yyM+aHT80RB4BMX+ZdUPAGGthjLWJ7wG8FsC22X6PZqNo2jBtByP54MLfP1HEWMHEo3vc+PyuoxOBz09d7LZnbkka6Mz64ZjRvIml7Wm0pQ08sW8Yu45OYMJLHRX5+vfv7AcAbNk/giNjBZy9olNeH/YIwsucawj8Hb1b7euedfN7X4pPXX2G/Gzd4lY8/ukrcO6qTkRxSm9roLZgPnHWinZccFJXoP8RQcw35sIjWALgYcbYMwAeB3AH5/x3c/Aec8r/eXQ/PvTDJ+XPZ3z2d/izmx/DUMgD6Bst4tsP7sa7v/soRnLlCkMg+vS3pgx0ZoNia1dLEr1tKTywawCvu3GTnAy2tN3dmT/8opu0dc+OY+DcHysJVI53DDeDyygeQUtSD9QArOjMBIwS4IewFhqXn7EEP//gy2X2EEHMR2bdEHDO93DOz/X+nMU5/+Jsv0Mz8McXB/HbbUcDBWCP7hnGUK4EjQFff9d5OGt5O46MFfDoniE43G03vW/ID/kkdQ1rvDTR1rQRKNQCgO5sAgPjfkfSPk9/CFfw7vSMy5qeFqztdUXYsHDblk5g5+evwp++zBXuVY3gY1eehl9+6JLj+w9CEMScQQpXTGyH4zO3b8WegdotmOMylCvDdjgOjRQCYx43PT+I7pYk3njucqzpacGL/ZPYesidGLb18BiOjZewwQux9LQm0a0MfwnvwruyycBO9TFvwMwyZfcuBrx3ZRNYv6wdt99wCe77+KWR75xO6HJecCbph4ZaUoasPyAIYv5BhiAmh0cKuPWxA3jw+YEZuZ8QhfcOTsq2DoC72ItCrOWdafSNFWXh1tZDY+gbK+Cs5e3QvHnA4tzWVKVH0NWSxI/+6iL8wxvOBABZByCKzc5c1o6/fOUpAIDLTl+MpKGhPZ3AKb2tVd9beAFpQ5eZRCnqvEkQ8xoqKIvJmFeRO1ljRGM9+IYgH5jHC/gVuWpV72Wn9+KRPUMomg5WdGawqDWFntYUeltTuP5Vp+DKM5fg51sOVdznzOXtWL+sDTfe+wIe91JKF7en8Mz/ei3aMwYGJkvYdngMn7p6faz3FqJoOqlLo5AkQ0AQ8xr6FxwTaQhitHquuDZvBn52Z/ZWegSislYYgvNWd0JjwA/e91JcdEqPfIdlHRm8feMqXP2SpWCM4dNXr8dpS9oqQkNCPGaMYf1Sf9JYRyaBjmwCjDEsbkvjG396fmwxV/UIUopWQRDE/IX+Bdfg6FhR7tynawieOTiKc//xLvxum98XaDRflkNS9g3mZauIV57qzvMdL7g/b1zTjRe/eDVeffpinK3001/akcbfvO50vPPCYMW1CA2JBbpbMQyi3iCdiD/MJQpxbSbp3yc1g8PgCYKYfcgQ1OCDt27BZ3/pljhM1xDcua0PALDt8Jg8JoxLytCwdzAnw02XrFsU+Bzwm5qdpRiCaq2hhQcgNICuFt8QrPPCT0Vz6pkFtUiTR0AQCw76F1yD/UN52YNH9PGvVyN43kvNVAXVIW+hP3dVJ46MFWTl78aTuvCei1bjX956TsV9OjIJrOnJQmPVc/LPW92Fi0/pwcvXugalWzEEpy2uLgDXg+8R+BpBMzR/Iwhi+tC/4CqULQfDubLMvRcewUQdHoHjcDx5YBSAv/gDkEVjG0/qAufAc5630J5J4AvXnV11rOJ5q7uwvDNTtdPlkvY0fnT9RXjJig4YGkOv0u8nLEhPl7Rc/P30UfIICGJ+Q1lDVRC79LGCiVzJmlbW0B92D8rrVEMwnHPvfYE3o/dZzxCEq3fD/P016zFaMGueAwBv2rAcG1Z1BEJD6vfHg/QIlMriNHkEBDGvIUNQhf7xovy+b6xQoRFMFE384snD6G1L4eqzl0Xe45Y/7sOi1iSWdWQwnCvBdjiOjBakUTh/tWsIth4ag64xWaxVjR4vZXQqErqGdYsrPYDv/fnG4271oC7+foUzicUEMZ8hQ6DwhxcH8dttfTh1cRtu2rRHHj8yWpSTvXKeIfjlU4fxuV8/BwD44ycvx/LODD7y46ewo28c//a2c2HaDu7d2Y8PXbYOO49O4OBwHt9/eC++eOcOrO7OYk1PFl0tSSxqTWJwsoyOTKKufvzT4Yozlxz3PcTYyZakoWQNkUdAEPMZMgQKf/q9xwAAq7ozODzqzwFWPQKhEewe8Pv3P7FvGBev7ZEDYz5z+zbkyxaWd2TwgUvX4gu/2Y6nD47i/l1ut88Dw3l8+DWnAnA7bw5ODjfFIPY4nLOiA19689m4ZN0iHBkruJ5MjWlqBEE0P/Nj9ZkFhib95mzqMHjGgMOjRWkIypaDkmVj31AOZyxtw6GRAh7fO4w9nmF483kr8IunDgMAvvbODWhNGehuSWI4V5bN3QDgT85bAQB42cndsuJ3PqBpTNYvvPm8lThjaRv1GSKIec6C9+k3PT+A2544WHF8vGjinTc9Ige93Lm1L/L65R0Z/OHFQYzkyhD92/YM5LBvMIe1va04/6Qu/PDxA/javS/gknU9+MClawEA7WkDrztrKQA3tm87HMO5Mv7+mvX47/e/DCd7oxZFqqfqgcwXMkkdF5zUPdevQRDEcbLgPYIv3/089g5M4m0bV4IxhnzZgq4xbNk3gkf3DOM933sc775wFb75wG6cs7IDh0YKgYKuj15xKj7xs2cBuIVcfWNFvP5rDwEArjlnGU5d3Ib+8SLe+/I1ePP5K5HQGc5d1YmXr+1B2hN/e5SMndefvSwwqev8k6KHtRAEQcwWC9oQjBdNbD00CocDR8aKWNyWwuX/9iAGJku4cI27kx2cLOHr970IALjhsnX4wR/24ZE9Q/jGu89HNqXjstMXo2DauHv7MZy2pA03P7xX3n9NTwuuO28FrvPCPIJf3RDszS9SN68+e2nFuMaUoePTV58RaA1NEAQxm8yJIWCMXQXgawB0AN/jnH+pEc95Yu8wRKv/HUfGsS+p46iXFvqIN/oRAK49ZxmuPHMJrly/BHsHczBtB9ec46eE/vnFa/DnF6/Bwy8MBgzBqu544xUvWduDL7/tXFx7bnSa6fWvWlvvr0YQBDFjzLohYIzpAL4B4Erg/2/vbmOkuuo4jn9/uywPurQ8LCACW6GsUqoE6ApNRWrBSMGkq0LCWk37woQgpdUXjQHrA8RXEm2N0UhqxFBtAEWtJMVqbdH6SOkDbHlst0p1CxRRKW1qROjfF+fMdhhmZrfL7ty5c/+fZDL3nntn72/PzO6Ze+6959IF7JG0w8wO9ve2/vj8PxlcX8fZ86/z0IETHD5xhqENdcyb2sSvD53k6rdfxor5U7jx3W/rPhVy5fVXsvL64v+Y6+NBgokjh3H7gqndexU9GVRfx9JrJvbPL+Wcc/0siYPFc4DOeMvKs8BWoG0gNrRw2ljWLJ7G4EF1bH+yi/0vnmHBtLHdg7u1jG2kbeaEXo/G+c5xjQxtqGPD0hksf2+z36fWOVcTkugamgDkn8bTBcwdiA1dN7WJ66Y28drZcxw+8Qo3z21m+vjLuk8PLXcnrmJGNw7h8FcWD0RU55xLTBINQbGv0XbRStIKYAVAc3PzRS94M1YvaLlgfvjQBlbfMLX7XH7nnMuyJLqGuoBJefMTgWOFK5nZvWbWamatY8aM6dcA9XXizkXv6vXBXuecq2VJNAR7gBZJkyUNBtqBHQnkcM45RwJdQ2Z2TtJq4JeE00c3mdmBSudwzjkXJHIdgZntBHYmsW3nnHMXqvmxhpxzzpXnDYFzzmWcNwTOOZdx3hA451zGeUPgnHMZJ7OLLuqtOpL+AbzQh5c2Aaf6OU6lpDW75668tGZPa25IT/YrzKzHK3JT0RD0laQnzKw16Rx9kdbsnrvy0po9rbkh3dmL8a4h55zLOG8InHMu42q9Ibg36QCXIK3ZPXflpTV7WnNDurNfpKaPETjnnOtZre8ROOec60HNNgSSbpR0RFKnpDVJ5ylH0lFJz0jaK+mJWDZK0sOSnovPI5POCSBpk6STkvbnlRXNquCb8T3okDS7ynKvk/RirPe9kpbkLVsbcx+RtCiZ1CBpkqRdkg5JOiDpM7G8quu8TO401PlQSY9L2hezr4/lkyXtjnW+LQ6jj6Qhcb4zLn9HUtn7zMxq7kEY3vp5YAowGNgHTE86V5m8R4GmgrINwJo4vQb4atI5Y5b5wGxgf09ZgSXALwh3pbsW2F1ludcBdxZZd3r8zAwBJsfPUn1CuccDs+P0cODZmK+q67xM7jTUuYDGON0A7I51+SOgPZZvBD4dp1cBG+N0O7AtidyX8qjVPYI5QKeZ/cXMzgJbgbaEM71ZbcDmOL0Z+EiCWbqZ2WPAvwqKS2VtA+6z4M/ACEnjK5P0QiVyl9IGbDWz/5rZX4FOwmeq4szsuJk9FadfAQ4R7vtd1XVeJncp1VTnZmavxtmG+DBgAbA9lhfWee692A4slFTslrxVq1YbggnA3/Pmuyj/IUyaAb+S9GS8VzPAODM7DuGPChibWLqelcqahvdhdexC2ZTX/VaVuWOXwyzCN9TU1HlBbkhBnUuql7QXOAk8TNhDOW1m54rk684el78MjK5s4ktTqw1Bsda4mk+Pep+ZzQYWA7dJmp90oH5S7e/Dd4ArgZnAceDrsbzqcktqBH4CfNbMzpRbtUhZYtmL5E5FnZvZeTObSbin+hzgqmKrxeeqyt4XtdoQdAGT8uYnAscSytIjMzsWn08CPyN88F7K7dLH55PJJexRqaxV/T6Y2UvxD/514Lu80RVRVbklNRD+md5vZj+NxVVf58Vyp6XOc8zsNPAbwjGCEZJyd3XMz9edPS6/nN53Q1aFWm0I9gAt8Sj/YMIBnB0JZypK0lslDc9NAx8C9hPy3hpXuxX4eTIJe6VU1h3ALfFMlmuBl3PdGdWgoO/8o4R6h5C7PZ4NMhloAR6vdD4IZwEB3wMOmdndeYuqus5L5U5JnY+RNCJODwM+SDjGsQtYFlcrrPPce7EMeNTikePUSPpo9UA9CGdPPEvo27sr6Txlck4hnC2xDziQy0roY3wEeC4+j0o6a8y1hbBL/z/CN6FPlcpK2GX+dnwPngFaqyz3D2KuDsIf8/i89e+KuY8AixPMPY/QzdAB7I2PJdVe52Vyp6HOZwBPx4z7gS/F8imExqkT+DEwJJYPjfOdcfmUpLL39eFXFjvnXMbVateQc865XvKGwDnnMs4bAuecyzhvCJxzLuO8IXDOuYzzhsDVNEnn80a63KseRqKVtFLSLf2w3aOSmvrwukVxhM6RknZeag7nemNQz6s4l2r/sTBUQK+Y2caBDNML7ydcuDQf+EPCWVxGeEPgMknSUWAbcEMsutnMOiWtA141s69JugNYCZwDDppZu6RRwCbCxUWvASvMrEPSaMJFa2MIFxUpb1ufBO4gDIm+G1hlZucL8iwH1saf2waMA85ImmtmNw1EHTiX411DrtYNK+gaWp637IyZzQG+BXyjyGvXALPMbAahQQBYDzwdyz4P3BfLvwz83sxmEa6YbQaQdBWwnDCw4EzgPPCJwg2Z2TbeuF/CewhXtM7yRsBVgu8RuFpXrmtoS97zPUWWdwD3S3oAeCCWzQOWApjZo5JGS7qc0JXzsVj+oKR/x/UXAtcAe+IQ9cMoPYBgC2GIBYC3WBjH37kB5w2ByzIrMZ3zYcI/+JuAL0q6mvJDDhf7GQI2m9nackEUblHaBAySdBAYH8fDv93Mflf+13Du0njXkMuy5XnPf8pfIKkOmGRmu4DPASOARuAxYteOpA8ApyyMs59fvhjI3XDlEWCZpLFx2ShJVxQGMbNW4EHC8YENhMEHZ3oj4CrB9whcrRsWv1nnPGRmuVNIh0jaTfhC9PGC19UDP4zdPgLuMbPT8WDy9yV1EA4W54YfXg9skfQU8FvgbwBmdlDSFwh3oKsjjH56G/BCkayzCQeVVwF3F1nu3IDw0UddJsWzhlrN7FTSWZxLmncNOedcxvkegXPOZZzvETjnXMZ5Q+CccxnnDYFzzmWcNwTOOZdx3hA451zGeUPgnHMZ93/WBfMv1natSAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f6e73c1f0f0>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(1,1,1)\n",
    "\n",
    "plt.plot(np.arange(1, len(scores)+1), scores)\n",
    "plt.ylabel('Score')\n",
    "plt.xlabel('Episode #')\n",
    "plt.savefig('train_scores_plot.png')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Watch Smart Agent Performance post-training "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total score: 31.389999298378825\n"
     ]
    }
   ],
   "source": [
    "agent.actor_local.load_state_dict(torch.load('checkpoint_actor.pth', map_location='cpu'))\n",
    "agent.critic_local.load_state_dict(torch.load('checkpoint_critic.pth', map_location='cpu'))\n",
    "\n",
    "env_info = env.reset(train_mode=False)[brain_name]     # reset the environment    \n",
    "states = env_info.vector_observations                  # get the current state (for each agent)\n",
    "scores = np.zeros(num_agents)                          # initialize the score (for each agent)\n",
    "\n",
    "while True:\n",
    "    actions = agent.act(states)                        # select actions from loaded model agent\n",
    "    env_info = env.step(actions)[brain_name]           # send all actions to tne environment\n",
    "    next_states = env_info.vector_observations         # get next state (for each agent)\n",
    "    rewards = env_info.rewards                         # get reward (for each agent)\n",
    "    dones = env_info.local_done                        # see if episode finished\n",
    "    scores += env_info.rewards                         # update the score (for each agent)\n",
    "    states = next_states                               # roll over states to next time step\n",
    "    if np.any(dones):                                  # exit loop if episode finished\n",
    "        break\n",
    "print('Total score: {}'.format(np.mean(scores)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When finished, you can close the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "env.close()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}