examples/example_train_jupyter.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ZTuJfCB_CYzh"
   },
   "source": [
    "## OpenAI Gym Available Environment\n",
    "\n",
    "Gym comes with a diverse suite of environments that range from easy to difficult and involve many different kinds of data. View the [full list of environments](https://gym.openai.com/envs) to get the birds-eye view.\n",
    "\n",
    "- [Classic control](https://gym.openai.com/envs#classic_control) and [toy text](https://gym.openai.com/envs#toy_text): complete small-scale tasks, mostly from the RL literature. They’re here to get you started.\n",
    "\n",
    "- [Algorithmic](https://gym.openai.com/envs#algorithmic): perform computations such as adding multi-digit numbers and reversing sequences. One might object that these tasks are easy for a computer. The challenge is to learn these algorithms purely from examples. These tasks have the nice property that it’s easy to vary the difficulty by varying the sequence length.\n",
    "\n",
    "- [Atari](https://gym.openai.com/envs#atari): play classic Atari games. \n",
    "\n",
    "- [2D and 3D robots](https://gym.openai.com/envs#mujoco): control a robot in simulation. These tasks use the MuJoCo physics engine, which was designed for fast and accurate robot simulation. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Don't forget to set matplotlib to inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "NEnwOnr6qHbO"
   },
   "source": [
    "# CartPole-v1 exemple"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Fj9G5c5kqM_y"
   },
   "source": [
    "## Initialize environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "5Q7p1iRwdGpE",
    "outputId": "fa6bfbf8-a868-402e-a1d7-344ba712b002"
   },
   "outputs": [],
   "source": [
    "import gym\n",
    "env = gym.make('CartPole-v1')\n",
    "_ = env.reset()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "observation_space= Box(-3.4028234663852886e+38, 3.4028234663852886e+38, (4,), float32) action_space= Discrete(2)\n"
     ]
    }
   ],
   "source": [
    "print(\"observation_space=\",env.observation_space, \"action_space=\",env.action_space)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mxUtaISFqQ4Q"
   },
   "source": [
    "## Initialize agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "srJt2AZPp58A"
   },
   "outputs": [],
   "source": [
    "from blobrl.agents import DQN\n",
    "from blobrl.explorations import EpsilonGreedy\n",
    "\n",
    "agent = DQN(observation_space=env.observation_space, action_space=env.action_space, greedy_exploration=EpsilonGreedy(0.1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "EX_NctEhqfgu"
   },
   "source": [
    "## Train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fldGbNR2qlDo"
   },
   "source": [
    "Create Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "id": "CY1LF52LqeyH",
    "outputId": "96567768-4a32-4e02-8fc8-c3f7e17897ab"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\site-packages\\blobrl\\trainer.py:29: UserWarning: be sure of your agent need to have good input and output dimension\n",
      "  warnings.warn(\"be sure of your agent need to have good input and output dimension\")\n"
     ]
    }
   ],
   "source": [
    "from blobrl import Trainer\n",
    "trainer = Trainer(environment=env, agent=agent, log_dir=\"./logs\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cGq2ksiRqkRR"
   },
   "source": [
    "Start train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "0BACJeOjqkXO"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "350d1d1695724a95a6974f6d42a2e3e6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/500 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "trainer.train(max_episode=500, nb_evaluation=0, render=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAAGCCAYAAADkJxkCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAI9ElEQVR4nO3dzYpkZx3A4be6q9MzkcxEpUkIQggusnETQcxSBXfjSryBuMgdZJsLcJvkCrJKdoYZSTYBN24UISPBjSgIQabjV8ykZ6a75mQhmo+hq7tCT9X0b54HatHnLfr8aajDr051nTObpmkAAJRtbXoAAID7TfAAAHmCBwDIEzwAQJ7gAQDyBA8AkDc/Yd131gGA82J23IIzPABAnuABAPIEDwCQJ3gAgDzBAwDkCR4AIE/wAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHgAgT/AAAHmCBwDIEzwAQJ7gAQDyBA8AkCd4AIA8wQMA5AkeACBP8AAAeYIHAMgTPABAnuABAPIEDwCQJ3gAgDzBAwDkCR4AIE/wAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHgAgT/AAAHmCBwDIEzwAQJ7gAQDyBA8AkCd4AIA8wQMA5AkeACBP8AAAeYIHAMgTPABAnuABAPIEDwCQJ3gAgDzBAwDkCR4AIE/wAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHgAgT/AAAHmCBwDIEzwAQJ7gAQDyBA8AkCd4AIA8wQMA5AkeACBP8AAAeYIHAMgTPABAnuABAPIEDwCQJ3gAgDzBAwDkCR4AIE/wAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHgAgT/AAAHmCBwDIEzwAQJ7gAQDyBA8AkCd4AIA8wQMA5AkeACBP8AAAeYIHAMgTPABAnuABAPIEDwCQJ3gAgDzBAwDkCR4AIE/wAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHgAgT/AAAHmCBwDIEzwAQJ7gAQDyBA8AkCd4AIA8wQMA5AkeACBP8AAAeYIHAMgTPABAnuABAPIEDwCQJ3gAgDzBAwDkCR4AIE/wAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHgAgT/AAAHmCBwDIEzwAQJ7gAQDyBA8AkCd4AIA8wQMA5AkeACBP8AAAeYIHAMgTPABAnuABAPIEDwCQJ3gAgDzBAwDkCR4AIE/wAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHgAgb77pAYDzbZqmMaa7n9syG7Mt76WAB4vgAU5tcXj7S3Ezxt2jO+O911/6/8+PP/Pd8e0fv7ju0QCWEjzAqf3p7dfGfz7446bHAFiZ884AQJ7gAQDyBA8AkCd4AIA8wQMA5AkeACBP8AAAeYIHAMgTPABAnuABAPIEDwCQJ3gAgDzBAwDkCR4AIE/wAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHuDUnvnRC2P38hNLn/PRX/8wPvjdW2uaCOB0BA9wajuPXh5b2/Olz7l7dGcc3bq5pokATkfwAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHgAgT/AAAHmCBwDIEzwAQJ7gAQDyBA8AkCd4AIA8wQMA5AkeACBP8AAAeYIHAMgTPABAnuABAPIEDwCQJ3gAgDzBAwDkCR4AIE/wAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHgAgT/AAAHmCBwDIEzwAQJ7gAQDyBA8AkCd4AIA8wQMA5AkeACBP8AAAeYIHAMgTPABAnuABAPIEDwCQJ3gAgDzBAwDkCR4AIE/wAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHgAgT/AAAHmCBwDIEzwAQJ7gAQDyBA8AkCd4AIA8wQMA5AkeACBP8AAAeYIHAMgTPABAnuABAPIEDwCQJ3gAgDzBAwDkCR4AIE/wAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHgAgT/DAQ+DatWtjPp+fyeP69esn7u/VV145s/3t7++v4S8E1M03PQBw/03TNBaLxZn9rpPcPcP9AZwFwQN8JbcXF8YYsy9sm80W45GtO5sZCGAJwQOs7OOjS+PX+z8di2nnC9sv7+yP733jnQ1NBXA8/8MDrOw3H/7kntgZY4x/H+6N3//zh+Pm0aUNTAVwPMEDrGT/9rfGYto+dv3vd54af775nTVOBHAywQOs5P2Pnh+H04VNjwGwEsEDAOQJHmAlzz7227Ezu73pMQBWIniAlTx54S9je3Z07PrXH/nbePpr769xIoCTCR5gZd//5q/G9uzwnu2Xdj4czz3+7nhs/q/1DwWwhOvwACu7tPOP8YO9N8cYY9w6XIyfvfzGGGOM7dli7G5/Mm4e3BtDAJskeICV/PwXvxzbs8+usDyNMT4+cHVl4MG2NHhOc88c4MF3lq/lT26t9+zNNE2ORcCpzGaz49eWHUguXrzoKAMBi8ViHB6ez4+Zdnd3lx7EAP7n4ODg2IPF0uAZ/z1bDZxzV69eHVeuXNn0GF/JjRs3xt7e3qbHAM6HY4PHt7QAgDzBAwDkCR4AIE/wAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHgAgb+nd0oGGra2tMZ97uQMPLzcPBQAq3DwUAHh4CR4AIE/wAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHgAgT/AAAHmCBwDIEzwAQJ7gAQDyBA8AkCd4AIA8wQMA5AkeACBP8AAAeYIHAMgTPABAnuABAPIEDwCQJ3gAgDzBAwDkCR4AIE/wAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHgAgT/AAAHmCBwDIEzwAQJ7gAQDyBA8AkCd4AIA8wQMA5AkeACBP8AAAeYIHAMgTPABAnuABAPIEDwCQJ3gAgDzBAwDkCR4AIE/wAAB5ggcAyBM8AECe4AEA8gQPAJAneACAPMEDAOQJHgAgT/AAAHmCBwDIEzwAQJ7gAQDyBA8AkCd4AIA8wQMA5AkeACBP8AAAeYIHAMgTPABAnuABAPIEDwCQJ3gAgDzBAwDkCR4AIE/wAAB5ggcAyBM8AECe4AEA8uYnrM/WMgUAwH3kDA8AkCd4AIA8wQMA5AkeACBP8AAAeYIHAMj7FGWEqW7jw4s7AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "trainer.evaluate()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MountainCar-v0 exemple"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import gym\n",
    "env = gym.make('MountainCar-v0')\n",
    "_ = env.reset()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "observation_space= Box(-1.2000000476837158, 0.6000000238418579, (2,), float32) action_space= Discrete(3)\n"
     ]
    }
   ],
   "source": [
    "print(\"observation_space=\",env.observation_space, \"action_space=\",env.action_space)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from blobrl.agents import DQN\n",
    "from blobrl.memories import ExperienceReplay\n",
    "from blobrl.explorations import AdaptativeEpsilonGreedy\n",
    "import torch.optim as optim\n",
    "\n",
    "memory = ExperienceReplay(max_size=50000, gamma=0.99)  # gamma is discount reward factor (default is 0)\n",
    "greedy_exploration = AdaptativeEpsilonGreedy(epsilon_max=0.8, epsilon_min=0.05, gamma=0.997)\n",
    "\n",
    "agent = DQN(observation_space=env.observation_space, action_space=env.action_space, memory=memory, gamma=0.85, greedy_exploration=greedy_exploration)\n",
    "agent.optimizer = optim.Adam(agent.network.parameters(), lr=0.001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from blobrl import Trainer\n",
    "trainer = Trainer(environment=env, agent=agent, log_dir=\"./logs\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Start train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0e01d138b5d04125906a230ba83adfb2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "trainer.train(max_episode=1000, nb_evaluation=0, render=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAAGCCAYAAADkJxkCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA770lEQVR4nO3deYBN9eP/8de9s9xZjDEMg7GMLSS0SPZMiI+IIgktKCRtpLRYqlGWVMq+K1tpI8petlIIRSrr2Mcy1tmX8/tjfs7XLbs7c2bOfT7+8j7nzr2vlpl5Oed93m+HYRgCAACwM6fVAQAAALIbhQcAANgehQcAANgehQcAANgehQcAANgehQcAANie7xXO88w6AADIKxyXOsEVHgAAYHsUHgAAYHsUHgAAYHsUHgAAYHsUHgAAYHsUHgAAYHsUHgAAYHsUHgAAYHsUHgAAYHsUHgAAYHsUHgAAYHsUHgAAYHsUHgAAYHsUHgAAYHu+VgcAAAD2cOzYMQUEJMjfX/L3LymHw8fqSCaHYRiXO3/ZkwAAAOe1bdtWzZp9qdtukypUWCpf30LmOYfDT4GBt2R3BMelTnCFBwAAeNyOHU3cxr6+4YqKmuF2LCCgolyuqBzJQ+EBAADZLj39uHbubOZ2LCysvUJD3Y8VLNhBDoefxz+fwgMAACxx8uQcnTw5x+1YSspeOZ0uc+zrG67w8Cdv+LMoPAAAINc4fHiQ2zggoDKFBwAA2EtU1DQ5HAHm2Mcnv0fel8IDAAAsUbjw0woLa+d2LF++enI4PF9PKDwAACDb+fpG6Kablrsd8/OLkK9veM58fo58CgAA8CqVK2+Rn1/hC474yM+viGV5KDwAAMBjKlRYrFtvrSOnM0gOR+7ZwYrCAwAAPMbpDJSPTz6rY/xH7qleAAAA2YTCAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbI/CAwAAbM9hGMblzl/2JAAA8G4//fSTZs6cKUlatGiRqlevrmLFit3QezocDn388cdyOBzX/KWXOuF7Q4kAAIBXMAxD7dq1078vlOzdu1cbN240x7t37/bI5x05cuQ/x5o3b64uXbpc1/txhQcAAFxUt27dtH37dnO8Zs0aC9NIxYoVU7ly5czxpEmTVLFixQtfcskrPBQeAABgGjlypMaOHStJio2NVXJyssWJLq106dIKCAgwx3/99ReFBwAA/NeuXbtUp04dc5yQkKCEhISr/vpHHnlEH3zwgUczZWRkKDIy8pq/zjAM5vAAAACZc3CSkpIUHh4uwzCu+irOuXPn/jOR2MfHRy6Xy+MZL1a6xowZo759+17Xe3KFBwAAL5Ceni7DMFSkSBGdOXNGkpSZmfmf1zmdTjmdWavWrFmzRjVq1HA7dx1PTnmMYRhumWvXrq1NmzaZ47S0NG5pAQDgbZKTk5WamipJql+/vn7//ff/vMbpdCo4ONgcv/HGG3r55ZdzLKOHcUsLAABvkZiYqBMnTuidd97RuHHjLvoah8OhyMhIlSlTRqtWrcrhhDmPwgMAgE0kJSXpn3/+0aJFi9SvX7+LvqZKlSry8fFRQECAfvnllxxOaB0KDwAAeVx6erpWrlypPXv26Kmnnrroa+68806FhIRo/vz5brewvAWFBwCAPGzmzJlKSkq6aNG57bbbdPPNN0uSYmJiFBUVlcPpcg8KDwAAedCYMWOUkJCgV1555T/bPVSsWFEtW7ZU8+bNFR0dbVHC3IXCAwBAHjN8+HC9+eab/1mrpmjRonrppZdUsWJFtWjRwqJ0uROFBwCAPGLkyJFat26dvvvuO7ey43K5NHXqVIWGhqp58+YWJsy9KDwAAORyc+bM0cSJE7V161YdPXrU7dySJUvk5+enhg0bWhMuj2DhQQAAcrF58+apR48eOnLkiNvx5cuXKzw8XFWrVrV09eNchoUHAQDIS7Zu3apmzZopMTFRJ0+edDu3YsUKNWjQQD4+Phaly3u4wgMAQC6zb98+Va5cWYmJiW7Hp02bpgceeEDBwcGUnYvjCg8AALldcnKy8uXLJ0nKyMgwj7/88ssaPHiw28aeuDZc4QEAwGKJiYkyDEMhISHmmjpOp1MBAQHq1KmTxo0bxzydq8MVHgAAcqPjx4+rYsWKio+PN485nU5FR0dr2bJlFiazFwoPAAAW2bt3r+6++263slO5cmUVL16csuNhFB4AAHLYjh07dPjwYfXs2VP79u0zj0dHR2vFihUWJrMvCg8AADnozz//1PPPP+92BadFixZyuVz6/PPPLUxmbxQeAAByyLZt2/Tyyy+7lZ1OnTpp7Nix5tNZyB482wYAQA74888/9dJLL+m7774zj/Xo0UMjR46k7OQArvAAAJCNjh49qldeeUX79+/X8uXLJUmPPfaYGjZsqPvuu08FCxa0OKF3YB0eAACyyenTp9WqVSutXLnSPPbII49oyJAhKlWqlIXJbOuS6/BQeAAAyAYpKSm66667tGXLFvNY69atNXr0aBUvXtzCZLZG4QEAIKcYhqEyZcooNjbWPNa4cWPNnj1b4eHhFiazPVZaBgAgJ5QpU0anTp3SqVOnJElRUVH67bff5Ofnx+RkC3GFBwAADzAMQxUrVtSOHTvMY+Hh4Tp48KD8/f0tTOZVLnmFh8fSAQC4Qampqbrjjjvcyk5oaKiOHj1K2ckluKUFAMANOHv2rB544AFt2rRJklSwYEE5nU4dPXqUHc5zEa7wAABwnU6cOKHOnTub6+uUKlVKu3fv1rFjxyg7uQyFBwCA63DkyBH17t1bX375pSSpSpUq+vnnnxUaGmpxMlwMhQcAgGt06NAh9e/fX5988okk6a677tI333zD+jq5GIUHAIBrcPjwYb311luaNGmSJKlBgwaaOHGiypcvb3EyXA6TlgEAuArnzp3TiBEjFBsbq6lTp0qS7rnnHg0fPlxVq1a1OB2uhHV4AAC4gpSUFD355JOaMWOGeSw6OlojRozQbbfdZmEy/AtbSwAAcD0yMzPVqlUrLViwwDxWu3ZtjRs3TtWqVbMwGS6CwgMAwPWoV6+e1q5da46rV6+uzz77TBUrVrQwFS6BwgMAwPXw9fVVRkaGwsPD9dNPPykgIEAlS5a0OhYujsIDAMD18PX1lb+/v/bv369ChQpZHQeXx15aAABcDcMwdOeddyowMFCBgYEyDEPx8fGUnTyOx9IBAPj/0tPT1bRpU23YsEFS1tWdpKQkNgC1Aa7wAAAgKSkpSe3bt9eKFSskSSEhIYqPj6fs2ASFBwDg9U6fPq3nnnvO3BerWLFi+ueffxQSEmJxMngKhQcA4NXi4+M1aNAgc6uI8uXLa+XKlSpatKjFyeBJFB4AgNc6ceKEhg4dqg8//FCSVLVqVX3++eeqUKGCtcHgcRQeAIBXio+P19ChQzVs2DBJ0h133KHx48ezVYRN8ZQWAMDrnDlzRm+99ZZGjhwpSapZs6aGDx+u2rVrW5wM2YWFBwEAXiU5OVnPPPOMpkyZIinrys4HH3yg+vXrW5wMHsBKywAAZGRk6KGHHtLXX38tKWvOzoQJE1SrVi2Lk8FDKDwAAKSlpbmtq9OoUSMtW7bMwkTwMLaWAAB4N8Mw3CYk33TTTZo8ebKFiZCTuMIDAPAKZcuW1Z49eyRlLSy4ZcsWFS5c2OJU8DBuaQEAvNeFZSckJESHDx9WcHCwxamQDS5ZeHgsHQBgWxkZGbrlllvMsuNyuXTy5En5+PhYnAw5jTk8AABbSk5OVv369fXXX39JkoKDg5WYmEjZ8VIUHgCA7Zw+fVpt27bVzz//LEmKiIjQsWPH5HTya89b8V8eAGArx44dU48ePbRw4UJJWZuBbt26VYGBgRYng5UoPAAA2zh8+LD69eunOXPmSJJuvfVWLV++XOHh4RYng9UoPAAAWzh06JAGDRpkbhlRp04dzZ49W6VKlbI4GXIDCg8AwBY2bdqkCRMmmOMXX3xRlSpVsjARchMKDwAgz9u/f7+mTp1qjlu1aqWqVatamAi5DQsPAgDytKNHj+qJJ57Q999/L0m6//77NWzYMFWsWNHiZLAAKy0DAOzn7Nmzat68udasWSNJatq0qUaNGqXy5ctbnAwWofAAAOwlNTVVNWrU0B9//CFJqlevnmbOnMkkZe9G4QEA2IdhGCpdurT2798vSapWrZqWLVvGZqCg8AAA7KNw4cI6fvy4JCkqKkp//PGH8uXLZ3Eq5AIUHgCAPRQtWlRxcXGSpAIFCujo0aPy8/OzOBVyiUsWHh5LBwDkGcnJyUpLSzPHgYGBlB1cFQoPACBPiI+PV6VKlRQfHy9JioyM1KFDhyxOhbyCwgMAyPUOHjyohg0bKjY2VpJUoUIFc8IycDUoPACAXG3nzp1q166d+fj5nXfeqa1bt8rhuOR0DeA/KDwAgFzrzz//VM+ePfXTTz9Jkpo0aaLly5fL39/f4mTIayg8AIBca/bs2Vq6dKk5Hjt2rEJCQixMhLzqsoWnf//+evfdd3MqCwAApl9//VU//vijOe7WrZvCwsKsC4Q87bLr8DgcDsPlcqlPnz4aPHhwDsYCAHizTZs26dlnn9XatWslSb169VL//v1VpEgRi5Mhl7u+hQcdDochSUFBQerRo4dGjBiRDdkAAPg/27dv16OPPqqNGzdKyio7b7zxhiIiIixOhjzg+hYeXL16tSQpMTFR48eP10svveThXAAA/J/Y2Fi1adPGLDvdunVT//79KTu4Yb6XO1mjRg3zzwkJCdqxY0e2BwIAeKfjx4+rfv365vo6HTp00LvvvquCBQtanAx2cNkrPC6XS3///bc5/v777/Xiiy9meygAgPfJyMhwW0wwNDSUsgOPueLmoYZh6K+//tLNN98sSXI6nRo4cKD69+/Pok8AAI9ITExUWFiYUlNTJUmtWrXSF198IV/fy96IAP7txnZLNwxDv/zyi+rXr6/09HRJ0ujRo9WjRw85nSzlAwC4fsnJyQoKCtL530cNGjTQjz/+yF+qcT1urPCcN2/ePLVu3docf/31125jAACuxcmTJ1WkSBGlp6fL4XDojjvu0Pr1662Ohbzr+p7S+regoCAVKlTIHMfFxSk5OfkGcgEAvFlUVJR556Bw4cKUHWSbayo8TZo00cSJE1W8eHFJUo8ePTRr1ixKDwDgmm3ZskUZGRmSJIfDoerVq1ucCHZ2zRNwHnjgAQ0fPlwlS5aUJHXt2lWffvqpOdEMAIArWb16tRo3bqyEhARJWZOUlyxZYnEq2Nl1zTju0KGDYmJiVKpUKUlZC0ONHTvWvCwJAMClLFq0SB07dtTx48clSZ07d9ZXX31lcSrY3TVNWv631q1ba968eeY4MTFRgYGBHooGALCb+fPnq0+fPtq5c6ckqXfv3ho2bJh8fHwsTgab8Myk5X/r1q2bypQpY45ffPFFXaFAAQC82Ny5c82yI0kDBgyg7CBH3FDhad68uSZPnqyiRYtKksaPH6/27dt7JBgAwF4+/fRTrVq1yhyPHTtWQUFBFiaCN7mhW1rnbdiwQY0bN9bp06flcDjUtGlTff/9955JCADI8z7//HP17t1bBw8elCRNnjxZHTt2lMvlsjgZbMYzCw9ezs6dO3XLLbcoJSVFTqdTd999t1asWHEtIQEANrRw4UJ17txZx44dk5RVdjp06KCAgACLk8GGsmcOz4XKly9vLgOemZmp3bt3e+qtAQB52NmzZ82yI0klSpSg7CDHeXQjrBMnTpilZ9++fapduzaTmAHASxmGoVWrVunxxx83j02cOFGNGjWyMBW8lUcLT1BQkJKSkuTj4yPDMLRu3To1a9bMXEkTAOAdDMPQli1b1LBhQ6WmpsrhcGjYsGHq2rUrT2XBEh7f6tzlcpmLSUnSkiVL1LVrV09/DAAglzIMQzt27NBtt90mwzDk6+url19+WX379mUHdFjG44VHytoT5fyj6lLWgoSnTp3Kjo8CAOQyCQkJqlixojlu06aNhgwZYmEiIJsKT2hoqDZu3KgKFSpIylpoqnfv3m6T1gAA9mMYhv744w9zHBwcrNKlS1uYCMiSLYVHkooXL66FCxfq1ltvlSRNnTpVgwYNUlxcXHZ9JADAYsuWLVOdOnUkZZWd7t27a+jQoRanArKx8EhShQoVNH36dNWuXVuSNGbMGMXExFB6AMCG5syZo6ZNm0qSAgMD1atXL40YMcLiVEAWjy08eDnTpk1T586dzfEPP/yghg0beuKtAQC5RL58+ZSQkCBJKlasmA4dOmRxInih7F948HJq1Kih5s2bm+OxY8fqyJEjOfHRAIAc8Oabbyo1NVWS5O/vrwEDBlicCHCXI1d4JOnPP/9Unz59tGjRIklZG4/OmDFDYWFhnvoIAIAF+vbtq1GjRik5OVlOp1Nz5szRQw89ZHUseKfs30vravzzzz/q1q2bVq5cKUmqV6+elixZosDAQE9+DAAgh/Tu3VsTJkwwb2UtXbpUjRs3tjgVvJi1t7TOu+mmmxQREWGO16xZwyrMAJCH/fbbb2bZkaTo6GgL0wCXlqOFR8p6UqtWrVrmuGLFipQeAMiDnn76af3888/m+O+//5bTmeO/VoCrkqO3tM5LTEzUHXfcob/++kuSVKBAAcXHx7PkOADkEa+88oref/99paenS8qaslC+fHl+jsNquWMOz4XS09NVsmRJ82mt0NBQtp8AgDxg2LBh6tevn87//tiyZYuqVq1K2UFukDvm8FzI19dXQUFB5jg9PV2JiYlWxQEAXIX09HSlpKSYZcflcsnPz4+yg1zP0putu3btUpkyZSRlbTZXtmxZnThxwspIAIBLSElJ0ZgxY8w1dkJDQ7VixQpVrlzZ4mTAlVk+u2z37t26+eabJUlxcXGqXbu2YmNjLU4FALhQcnKyPvnkEz3//POSpIiICM2ePdvcNwvI7Sybw3Oh9PR0+fn5meOGDRvqhx9+yImPBgBcQXJysmbOnKknn3xSklSiRAl98MEHatu2rcXJgP/IfXN4LuRwONSqVStzfOzYMf3yyy8WJgIAnHfkyBGz7EhSx44dKTvIc3JF4fHx8dFnn31mbjC6bds29ezZU6tXr7Y4GQB4t5SUFH3yySfmuFKlSqpdu7aFiYDrkysKj5Q103/kyJHm/eHffvtNvXv3NrehAADkrIyMDL366qsaOHCgJKlChQoaPny42xV5IK/INYVHkkJCQtS1a1dzvGHDBm5tAYAFDMPQk08+qQ8++ECSVKpUKX388cdq0aKFxcmA65OrCo8kRUVFaciQIeZ4ypQpWr58uYWJAMA7TZ8+3fxzwYIF1bRpUwvTADcm1xWekJAQ9ezZU4MHD5aUtTfLk08+qXXr1lmcDAC8R926dc3FBYsWLaqZM2danAi4MbnisfSLSUhI0JAhQxQTEyMpa82H5cuXq0qVKlZFAgCvcPvtt2vz5s0yDEMhISHatm2bSpYsaXUs4Gpc8rF035xMcS2Cg4NVsGBBcxwXF6fU1FQLEwGA/dWoUcMsO76+vtq3b58KFChgdSzghuW6W1oXeu6559SnTx9zfNddd2nfvn26wlUpAMB1MAxDp0+fNn/GOhwOyg5sI9fe0jIDGIa6du2q6dOnKzMzU1LW1Z7ChQuzWR0AeEhqaqoaNGhgPhnrcrmUmJgopzNX/70Y+LdLFoNcX3jOu/fee7V06VJznJqa6rYdBQDg+rVu3Vrz5s0zx4mJiQoMDLQwEXBdcvfWElejWLFi8vf3N8e7d+/m1hYAeMCxY8eUkJBgjqOioriCDtvJM4Vn+vTpateunVl6KleurA0bNlicCgDytoMHD6pbt25atmyZJKlatWr67bffFBAQYHEywLPyTOGRpE8//VSdOnWSr6+vDMNQvXr1tGjRIqtjAUCeFBsbq759++qbb76RJNWrV0/ff/+9wsLCrA0GZIM8M4fnQqGhoTpz5owkqUiRIoqLi7M4EQDkPRMmTFD37t3N8U8//cTGoMjr8v4cngu98sor5pMDCQkJGjVqlMWJACBv+fvvv7VgwQJz3K5dO5UoUcLCRED2ypNXeCT3v5mEhobqtdde08svv2xxKgDI/Xbv3q2ePXtq8eLFkqSOHTsqJiZGUVFR1gYDblzefyz93wzD0Ny5c/Xwww9LytrYrm/fvurXr5/FyQAg9zp8+LDat2+vVatWScq6sjN06FDKDuzCXre0pKwVQFu0aGGO4+PjzQWzAAAXl5iYaJYdSapUqRJlB14hzxYeSQoICNDPP/9sjpcuXaqBAwdamAgAcq+TJ0+qUaNG5rhdu3Z67rnnLEwE5Jw8e0vrvMzMTP3666/mkwUBAQGKiYlx24MLALxdcnKyIiMjFR8fL0lq2rSp5s6dq5CQEIuTAR5lvzk8F8rMzNTKlSt1zz33SJJ8fX01fvx4denSxeJkAGA9wzAUFBSk5ORkSdKdd96pNWvWuK1eD9iE/ebwXMjpdMrlcsnHx0eSlJ6erpSUFHOzUQDwZmlpaWbZkSR/f3/KDryOLQqPJNWpU0dff/21uRx6z549NXfuXKWnp1ucDACsc/r0aeXLl09S1sMet956q9asWWNxKiDn2abwSFLLli01YcIEhYaGSpLat2+v7777TmlpaRYnA4Ccd+DAAZUqVcr8GVi9enVt2rTJ4lSANWwxh+ff+vfvr5iYGHO8c+dOlStXzsJEAJDzoqKiFBsbK0lyuVxut7UAm7L3HJ5/K1++vCIjI83x6tWrlZqaamEiAMhZP//8s1vBady4sYVpAOvZsvA8/vjjeuutt8zS07lzZ02cOFEZGRkWJwOA7Lds2TJ16NDB3Fi5U6dO+vbbby1OBVjLloVHkrp06aKYmBgVLVpUktSrVy8NGTJEV7iFBwB52nfffadnnnlGe/fulSQ9//zzmjp1qhyOS17pB7yCLefwXKhmzZpav369Oc7MzOQbH4Btde7cWdOmTTPH8fHxCgsLsy4QkLO8aw7PhQYPHqyIiAhz3LFjRwvTAED2mTVrln788Udz/PHHHys4ONi6QEAuYvsrPJL0yy+/6N5779WZM2fkcDh03333cT8bgK18+eWXevHFF7V//35J0vjx4/XYY4+Za5MBXsLeW0tcjUKFCpl7yISFhZl/BoC8bsmSJXr88cd15MgRSdLYsWP1+OOPKzAw0OJkQI7z3lta5/3555/mUuqnTp1SrVq1LE4EAJ5x5swZs+xIUqlSpSg7wL94TeGJiIjQ8ePHJWVtpPfrr7+qQYMGFqcCgBuzdu1aderUyRyPGTNG9957r4WJgNzJawqPJOXLl0+JiYmSskrP6tWr1bJlS4tTAcC1MwxDW7duVYMGDZSSkiKHw6F33nlH3bt3l6+vr9XxgFzHqwqPw+GQj4+P21MLqampLLcOIM+Ji4tT1apVlZmZKSlrrbF+/frJ6fSqH+vAVfO67wx/f38dOHBARYoUkZQ12a9z5846c+aMxckA4OpkZmaaqyhLUkBAgPLnz88aY8BleF3hkaQCBQpow4YNKlu2rCRpzpw56tu3r06cOGFxMgC4PMMwtHHjRt16662SpKCgIPXo0cNtw2QA/+U1j6VfzPr161WzZk1zPGrUKD3zzDMWJgKAy8vIyHCbo1O/fn2tWrXKwkRArsJj6RdTsGBB1a9f3xxv3LjRXLQLAHKjuXPnmn/Onz+/GjVqZGEaIO/w6sJTrlw5jRs3To0bN5YkTZ06VQMGDNDBgwctTgYA/zV69Ghze5ygoCC9+uqrGjhwoMWpgLzBq29pnff777+rb9++WrJkiSTpkUce0YcffmhObAYAqw0ePFhvvfWWUlNT5efnpw8++IBb8MB/sbXElcTExKh///7mePv27apUqZKFiQDg/5QoUcK8+hwUFKSEhASLEwG5EnN4rqRDhw5uixB27dpVJ0+etDARAGTp3r27uVK8w+HQ/PnzLU4E5D1c4bnA/v371blzZy1fvlySdMstt+jXX39lTxoAlunevbs++eQTc4HU9evXq0aNGhanAnItbmldrePHj6t58+Zav369pKzLyLGxsaxeCiDH9erVS5MmTVJKSoqkrE2QK1WqxAKDwKVxS+tqhYeHy+VymeMDBw5YmAaANztx4oRZdiQpMjKSsgNcJwrPRfzwww+qWLGiOc6XL5+5Xw0AZDfDMNS3b1999tlnkrLm7ezZs0chISEWJwPyLgrPRfj6+mr79u0qVaqUJCkpKUkFChRgk1EA2S49PV3vvvuu3nvvPRmGIX9/f23ZskWlS5fm6g5wA5jDcwVBQUFKSkqSlDWfh5WYAWSX1NRUTZo0yVxfJ3/+/FqwYIHbivAALos5PNerQoUK5p/T09O1d+9e68IAsLVVq1a5LSY4btw4yg7gIRSeK9i8ebPq1KkjSTpy5IgaN26sbdu2WZwKgN0kJCS4/WwpXbq0wsPDLUwE2AuF5wocDodWrlypFi1aSJJ27dqlDh06aMOGDRYnA2AXSUlJmjBhgl544QVJUvny5TVy5Eg1adLE2mCAjTCH5yqdOXNGoaGh5rhjx46aMWOGhYkA2EFqaqree+89vf7665KybqMPGTJEDz74oMXJgDyJOTw3yuVyqU+fPuZ4y5YtWrx4sYWJANhBcnKyWXYkqWbNmpQdIBtQeK6Sy+XSoEGDzA1Gt27dqpdfftncYR0ArlVGRoaee+45c1y5cmV17drVwkSAfXFL6xqdOXNGH330kVl8KleurLFjx+ruu++2OBmAvMQwDLVs2VILFy6UJEVFRWn27NmqVauWxcmAPI29tDzp9OnTGjlypAYOHChJKlu2rObMmaM777zT4mQA8ooGDRpo9erVkqQiRYpo+fLluuWWWyxOBeR5zOHxpNDQUJUtW9Yc7969W2fOnLEwEYC85sInPf39/Sk7QDaj8Fynhx56SIMGDTLHDzzwAOvzALgqlStXNldwz5cvn37//XeLEwH2xy2tG5Cenq6+ffvqww8/lCT5+flp9+7d7GgM4KIMw1C1atW0detWSVk/M86ePSuXy2VxMsA2mMOTXQzDUIcOHTRnzhzz2OnTp5U/f34LUwHIjS6ctyNl/aXJx8fHwkSA7TCHJ7s4HA7ly5dPvr6+5rFTp07pCkUSgJc5d+6c0tLSzHGBAgW4EgzkIAqPB0ycOFEPP/yw/Pz8JGXtgbNr1y6LUwHILY4fP66HHnpI69atkySVKlVKhw4dktPJj2Agp/Dd5iEzZsxQmzZtzNJz0003aePGjRanAmC1Q4cO6emnn9aiRYskSbfccot+++03BQYGWpwM8C7M4fGwEiVK6ODBg5KkoKAgJSQkWJwIgFX279+v119/XZ9++qkkqVatWpo7d65KlChhcTLAtpjDk1Pat29vXuVJT0/XrFmzLE4EwCpLliwxy44kDRs2jLIDWITC42Hndz12OBxKTU1V9+7dNX78eKtjAchhO3fuNLeNkLLW6ipZsqSFiQDvxi2tbDJ69Gj16tVLkhQWFqb+/fvrxRdftDgVgJwQGxurZ555xiw87dq10zvvvKNy5cpZnAywPdbhyWmGYWjWrFnq1KmTJCk8PFyvvPKKXnrpJYuTAchOcXFx6tChg1asWCFJevDBBzVs2DDKDpAzmMOT0xwOh1q3bm2Ojx8/bj6SCsC+EhISzLIjZW0jQdkBrEfhyUaBgYFuq6ouXrxYMTExFiYCkJ1Onz6txo0bm+M2bdro+eeftzARgPO4pZXNMjIy9NNPP6lBgwaSskrQkCFD9Nxzz1mcDIAnpaamqkSJEjp27JgkqVGjRvryyy8VGhpqcTLAqzCHx0qZmZlatmyZmjZtKilrw8BJkybp0UcfZWl5wAYMw1C+fPmUmJgoSbr99tu1du1aBQQEWJwM8DoUHqsZhqEvvvhC7dq1k5Q1x2fhwoX63//+Z3EyADcqMDBQycnJkqQyZcpo586dbBsBWINJy1ZzOBzy9/eXy+WSlFWAEhMTlZGRYXEyADfi7NmzyszMNMf58uWj7AC5EN+VOahVq1YaM2aMQkJCJElt27bV4sWLlZ6ebnEyANfj8OHDKlWqlFJTUyVJVapU0e+//25xKgAXQ+HJYV26dFFMTIzCwsIkSffdd5+WLl1K6QHymB07dqhmzZo6deqUJOmOO+7Q1q1brQ0F4JIoPBZ47rnn9Oqrr6pQoUKSpObNm2vhwoVul8UB5F5btmxRy5YtdeDAAUlSdHS01q9fb3EqAJfDpGULNW/eXN9//705Tk5ONuf4AMi9WrRo4bZPFt+7QK7BpOXcqE2bNoqIiDDHH3zwga5QQAFYbPHixdq9e7c5fuaZZ+Tj42NhIgBXg8Jjoa5du+q9994z5/O8+uqr6tevn8WpAFzKokWL1LdvX23fvl2S1K9fPw0fPly+vr4WJwNwJdzSygXmz5+vRx55RImJiXI6neratasmTJhgdSwAF1i6dKlefPFFbdu2TZL09ttv6/nnnzefugSQK7DwYG63cuVKNW7cWOnp6fLz89PDDz+sTz/91OpYACStWbNGTz75pP7++29JUkxMjJ599lnlz5/f4mQA/oXCkxe4XC5zPY+yZctq165dFicCsHnzZrVu3VqxsbGSpAEDBqhPnz6UHSB3YtJyXnBhwYmNjdX9999vYRoAkpSQkGCWHUmKjIyk7AB5EIUnF4mMjNTRo0clZe2yvnDhQrVv397iVID32rFjh+69915z/Prrr+uJJ56wLhCA68YtrVzGMAwdPXpUxYsXV2ZmphwOh5566imNHTuW/XmAHGIYhuLi4lSiRAlzv7unn35ao0aN4vsQyN24pZVXOBwORUREaMeOHQoICJBhGJowYYJef/11paWlWR0PsD3DMHTixAkVK1ZMGRkZcjqdat++vUaPHk3ZAfIwrvDkYuvWrVPt2rXN8bhx49S9e3cLEwH2d+TIERUrVkyS5OPjo2bNmmnBggUWpwJwlbjCkxcFBgaqRIkS5vjw4cM6e/ashYkA+ytZsqT553LlylF2AJug8ORi1atX1+zZs1WhQgVJ0ptvvqmPPvpIZ86csTgZYE9r1641t3fx8/NTjRo1LE4EwFMoPLlcvXr1NGbMGN18882SpDfeeEOjRo3iSg/gYd99952aNGmijIwM+fj46NFHH9XMmTOtjgXAQyg8eUDjxo313nvvqUqVKpKyHo0dPny4kpKSLE4G2MOcOXPUsWNH83vqueee0+TJky1OBcCTKDx5xP/+9z8NHz5clStXlpS1j0+/fv14cgu4QVOmTNELL7ygU6dOScraNmLEiBHWhgLgcTyllccsX75c3bt3N1dlfuyxxzRt2jQ5HJecmA7gEiZOnKhBgwbp0KFDkqTRo0erR48ePH4O5F3spWUntWvX1rp168zx+bVCAFybRx99VDNmzDDHp06dUmhoqIWJANwgHku3k6lTp7o9OnvhWj0Ars5HH32k7777zhwvWLBA+fLlszARgOzEFZ486sCBA6pWrZpOnjwpSapatap+//13i1MBecPkyZPVu3dvc4mHBQsWqGnTpvL19bU4GYAbxC0tOzp16pSKFi2qlJQUORwO3XLLLZQe4Aq++uorderUyXwi68svv9T9999P2QHsgcJjVykpKQoKClJmZqYk6c4779Qvv/zCJGbgXwzD0MqVK9WkSROlp6dLynpC6/HHH2cOHGAfzOGxK5fLpaSkJLlcLknS+vXrde+99/K4OnABwzD022+/KTo6Wunp6fLx8dF7772nJ554grIDeAm+023A399fcXFxCgsLkyQtW7ZMjz32mBISEixOBljPMAxt27bN3CbC399f/fr1U58+fbgSCngRCo9NhIaG6o8//jA3G50zZ4569+5tLqYGeCPDMPTHH3+oatWqkqSAgAD16NFDMTExFicDkNMoPDYSGRmp+fPnm+MJEyZo+vTpFiYCrLVu3TpVr15dkhQYGKjOnTtr5MiRFqcCYAUKj82EhoaqVq1a5njr1q06ePCghYkA69SrV8/8c7ly5TRmzBgL0wCwEoXHZsqWLatJkybp7rvvliRNmjRJb775pg4fPmxxMiBnTZ06VeefQg0KClKbNm0sTgTAShQeG6pSpYo++OADNWrUSFLWfkGvv/66jh07ZnEyIGe8//776t69uwzDkJ+fnwYNGqRBgwZZHQuAhViHx8Y2b96sPn36aMWKFZKkhx56SBMnTmSvINjaoEGDNGzYMCUlJcnpdGrixInq0qWL1bEA5AzW4fFGt956qz788EPVrVtXkjR37lw9/PDD5gqzVtmwYYNat26tmTNnWpoD9vP666/r/fffN/8f/+KLLyg7ACRxhccr/PPPP3r44Ye1efNmSVLNmjX1008/ycfHJ8ezbN++XQ8++KD++usvRUZGauTIkcytgEe88cYb+vjjj839sZYtW6Z77rmHtXYA78LWEt7uwIEDio6O1s6dOyVJFStW1Pbt23P0l8G+fftUr1497d+/3zxWuHBhhYaGavbs2ebCcMC1iomJ0dChQ3Xu3DlJ0po1a1S7dm1WUQa8D4UHUnx8vCpXrqyjR49KkkqXLq29e/fm2Ofv3r1b5cqVu+i50NBQ+fn5adu2bSpSpEiOZULe99FHH+mVV15RcnKyJGn16tWqU6cOZQfwThQeZElOTlZ4eLi57URUVJR2796d7Vd6Tp48qWLFiiklJeWyr3O5XHI4HDp37pwlt9yQdxiGoRkzZqhLly7mZqCLFy9W48aNKTuA92LSMrIEBATozJkz5maje/fuVbVq1cxfGNnBMAylpaVdsexIWbu/JycnKzAwUGlpaWyCiosyDEPff/+9HnvsMWVmZsrX11efffaZmjRpQtkBcFH8ZPBCTqdT586dU/78+SVlrcYcHR2txMTEbPm81NRURUREXNPXpKWlyd/fX1FRUTp16pQ5NwPIzMzU6tWrdd9990mS+vbtq7S0NLVr144JygAuicLjpXx9fXXo0CFFRkZKyprk+fDDDys+Pt6jn2MYhvbt23fdX3/o0CGFhYWpWbNmOn78uAeTIS/KzMzUzz//bK4kHhwcrAIFClgbCkCeQOHxYsHBwdq0aZNuvfVWlS9fXgsWLNCzzz6ruLg4j33G5s2bddNNN93w+6xdu1aPPfaYNm3a5NF8yDsMw9CaNWvM/bFCQkLUp08f9evXz+JkAPICJi1DkvTDDz/onnvukSR17dpVb731looXL37D7+tyuZSamnrD73Ohp556Sm3atFGNGjVUqFAhj743cqfzc3bO38YKCQlRr1699M4771icDEAuw6RlXF7RokVVv359SdLkyZP15ptv6tChQzf0nnPmzFFmZqYn4rmZOHGimjVrpmHDhmn69OnmE2ewr5kzZ5plJzg4WC+88AJlB8A14QoPTFu2bFHv3r3Nvbe6dOmid95555onHJ9XqFAhj88JuphXX31VAwcONJ88g7189NFH6tOnj9LT0+VyuTRo0CBuYwG4FNbhwdXZtGmTXnrpJbP0PPLIIxo9erTCwsKu6X3efvttvfPOO+ZicNnt6aeflsvl0vvvv8+TOjYSExOjwYMHKzk5WU6nU6NHj1aPHj2sjgUg96Lw4Opt2bJFzz77rFavXi1JatGihT777DMFBQVd9XuUK1dOu3fvlpT1GPyDDz7odn7Pnj3auHGj50L/f+3atVNoaKgmTJjg8fdGznrjjTc0cuRIc0mCzz77TO3atbM4FYBcjsKDa7N9+3Y99thj2rBhgySpbt26+vHHH+Xr63vFr+3Ro4emT5+u5ORkPfroo/Lx8VHp0qXdXnP27FmdOHFC69at099//+3R7H5+fqpTp46qVaumjz76yKPvjZzx+uuva9SoUeZGoIsXL1aTJk24egfgSig8uHaxsbFq1qyZ/vrrL0nSzTffrK1bt172l07Pnj01depUJScnq0uXLipRosRlX3/27FklJSXpm2++0eHDhz2aPygoSE899ZQ+/PBDj74vstegQYP0/vvv6+zZs5LYGwvANaHw4PocP35c1atXN5/YKlOmjHmr6mLi4+PVqFEj3XHHHYqMjLzqv5FPnTr1hhYovJSAgAAVKFBAr732mp599lmPvz88Z8aMGerbt69Onz6tpKQkSex6DuCaUXhw/RISElSsWDHzb9zBwcGqUKGCNm3adNHXJyYmavz48Tp9+vRVf0ZaWprGjRuXbU91+fv7y8/PT1999ZXuvffebPkMXL+FCxeqTZs2bvutLVu2TNHR0ZQdANeCdXhw/YKDg3Xy5EkFBARIyipAmzdvVr169ZSRkfGf18+cOfOayo6UNe8mO3+xpaamKiEhQc2aNZO/v7/27t2bbZ+Fq2cYhn766Se1bNnSLDtOp1NfffWV7rnnHsoOAI/hpwmuio+PjxISEhQcHGweW7t2rR544IH/7IJ+hauGl9SzZ0+3988O53/BRkVFZevn4MoMw9Aff/yhunXryjAM+fr6Kjg4WBMnTtQDDzzABGUAHkXhwVVzOp06duyYihUrpsKFC0uSvv32W3Xr1s283XUjcuIXXMGCBeXv75/tn4PLMwxD27ZtU/Xq1SVlbUHywgsv6Ny5c+rSpYvF6QDYEYUH1yQwMFCHDh3Sr7/+qjJlykiSPvnkE7322ms5sqryjfr0009VrVo1q2N4NcMwtGnTJlWtWlVS1v9TTz31lIYPH25xMgB2duVFVYCLiIqK0vz589W+fXtt27ZNo0aNksvl0n333XfdV3sOHDhw0TlBfn5+F93INC4u7ppWcq5YsSKbjVps9erVSk9PNzeqDQgI0BNPPKGPP/7Y4mQA7I6ntHBDfv31V/Xs2dNt1eTo6GjVqlXrmm8djRo1SidOnDDHTqdTlStXVmhoqJo0afKf169atUpHjx7V33//rfT09Mu+d7Vq1TRy5Eg1bNjwmjLBc7799lu1b99eiYmJkrKenOvRo4dGjhxpcTIANsJTWsgeNWvW1MiRI1WzZk3z2A8//GBuB3C1br75ZvXo0cNt+4ratWurbdu2Fy07ktSgQQO1bdtWdevWveLTPO3bt6fsWGjWrFl6/PHHzbLj4+Ojvn37UnYA5BgKD25Y3bp19d5777mVnrVr1yo1NfWq3+Ouu+5STEyMhg4daj6i3rhx46v62oYNG15224E6deooOjr6qrPAs6ZMmaLevXvr5MmT5rGhQ4cqJibGwlQAvA1zeOAR9evX14cffqidO3dq4MCB+u2335ScnKwHH3xQPj4+l/y66tWrq2zZsubcml69eqlgwYL68ssvr+nza9WqpSVLllz03B133KFatWpd0/vhxn3++ef69ttvtWLFCsXFxUmSxo4dq+DgYHXq1MnidAC8DXN44HG//PKLHnjgAR0+fFhlypSRj4+POnbs6PaacuXKqU6dOgoPD1doaKjbOcMw9Oabb17z5+7atUszZsxwO1avXj1NnDhRlSpVuvZ/EFy3b775Ri+++KLbAo+ffPKJ2rdvLz8/P+uCAbC7S87h4QoPPO6uu+7S0qVLVbduXe3Zs0eSNH78eEVGRurbb7+VlPUocoECBS769ZMnT76uzz3/mPyFIiIiKDs5bNmyZerVq5cOHjxoHpsxY4batm1L2QFgGQoPskWVKlW0bds2lSlTRmlpaTpy5IiOHTumu+66S9HR0Zo+ffolv/b87Y8bVbNmTY0bN84j74Ur27Fjhxo1aqTExES3p+2mT5+uhx56iAUfAViKwoNsExkZqWPHjik1NVVFihRRRkaG9u/fr1mzZikkJESjRo3Kts++6aabtGLFimzfqgJZ4uLidPvtt7s9nTdgwAD17t1bQUFBXNkBYDkKD7JVaGioDMNQXFycihYtKsMwlJ6erjFjxqhgwYIaNGiQHA6Hx7eVOL8vE7JXZmamEhMTVaJECbe1kHr16qUBAwZcdsI6AOQkHktHtnM4HCpcuLB27dqlgIAA+fv7yzAMvf322/Lx8dHs2bPdVlh+7bXXrutzzv/CjYiI0NatWz2SHZeWlJQkf39/hYSEKD09XQ6HQwEBAXrkkUf00UcfUXYA5CoUHuQIh8OhMmXKKCkpSQsXLlRISIh5rmPHjlq4cKHi4uKUmZkpSdd1dWbKlCmSpPDwcHbazkYnTpxQXFycChcubBZVp9OpOnXqKCkpSbNmzeLfP4Bch8KDHNe4cWNNmTLFbV+rVq1aqWjRolq3bp0Mw1Dv3r2v6T0LFiyoAwcOqHr16lzdyUb79+9XnTp1VLRoUSUkJEjKKrO1atXSmjVrLE4HAJfGOjywzLRp0zRx4kT9/fffbk/1LF++XC6XS3/99ZfS09N1+PDhy75P8eLF9cQTT/AUUDbau3evDh48qOeff95t37TatWsrMDBQy5cvtzAdAJgueXmZwgPLDRs2TMOGDXMrPT4+Pvr666/VqFEjzZ8/XwkJCdq/f7/b15UsWVLBwcG6//773fbggufs27dPGzdu1LRp0zR//ny3cy1atNA333zDXB0AuQmFB7nbiBEjtGvXLk2bNk1JSUmSpICAAI0YMULh4eGKjo7WL7/84vY1tWrVUnh4uBVxvcL+/fs1YMAATZs2ze14u3btVKhQIb3//vsKCAiwJhwAXByFB3nD+++/r379+iktLc08VqRIEXXr1k3Vq1dX27ZtLUznHU6fPq333ntPe/bs0cyZM93OPfHEExoyZIgiIiIsSgcAl0XhQd4xZcoUpaam6umnn3Y7Xr58eUVHR6tt27a69957LUpnXxkZGXr66ad17tw5zZ492+1c27Zt1aRJE913332KjIy0KCEAXBGFB3lLZmamvvjiCz388MP/OVe5cmWNHTtWd999twXJ7KlNmzZKTk7Wd99953a8YcOG6tOnjypXrqxy5cpZlA4ArhqFB3lPRkaGNm7cqB07dqhTp05u58qWLatChQpp1qxZKl++vEUJ7aFZs2ZaunSpuQbSeXfeeadmz55N0QGQl1B4kHelpKTowIEDmjdvnvr06eN2rkSJEvL399eWLVuUL18+ixLmTe3bt9f69esVGxvrttJ10aJFtWbNGgUGBqp48eIWJgSAa0bhQd6XkpKic+fOacCAARozZozbubCwMDkcDgUFBf3n8XX8nxEjRuidd96RJJ05c8Zt/ytfX18dOXJETqdTYWFhVkUEgBtB4YF9pKenq2PHjvr8888vet7f31+VKlXS5s2bJcmrtzk4//29aNEitW7dWhkZGW5Xc86Lj49XUFCQXC5XTkcEAE+i8MBezv9/W6NGDW3dulWpqakXfd2DDz6oWbNmScq6guENi+QZhmH++9izZ48qV6580df5+vrK6XQqNjZWERERXl0MAdgGhQf2VqpUKZ07d06GYejUqVMXfc3HH3+sDh06KH/+/PL19c3ZgDnk5MmTyszMvOSCjAEBAQoMDJQkLVy4ULVr187JeACQ3Sg88A6JiYmqUqWK0tLSdPDgwYu+Zt68eapataokKSoqKs9f2Thy5Ii5OvXtt99+0cIXGBioiIgI9erV6z8TvwHARig88C579uxRu3btdPLkSe3ateuSr1u7dq38/PwkSS6XS9WqVcupiNft5MmT2rlzpznu3bv3JXcq9/f3V7Vq1VS3bl19+OGHOZQQACxD4YF3WrNmjd59911J0u+//64DBw5c8rVFixbVpEmTzHGFChV00003ZXvGK8nIyNCiRYvM8c8//6zBgwdf9muaN28uKeufafLkydmaDwByEQoPMGXKFK1du1bz5s1z25n9Ulq2bKlWrVq5HevYsWO2b5j5zTffuOVLTU1Vz549r+prO3bsKJfLpUmTJuX5W3UAcB0oPMB5o0ePNuf3DB069D8rDF/Oa6+9puDg4Euer127tqKjoy/7HmfPntXHH398yfOjRo3S4cOHrzrTQw89ZK42/cYbbygoKOiqvxYAbIbCA1zMuHHjzHVpjh8/rkGDBt3Q+912221XfPIpISFB06dPv+7PqF69up566ilzfN999ykqKuq63w8AbITCA1zJuXPn3ObKzJ0795KLG+Ykf39/zZw50xwXL15cderUsTARAORaFB7gWu3bt0+xsbFux1q2bKnTp09n6+dOnjxZFSpUMMdOp1N169bN1s8EAJug8ACesHPnTrf9p/5t0qRJGjFixGXfo3Dhwlq1atUlz0dFRWX7xGgAsCkKD5ATEhISdO7cucu+xul0qnDhwjmUCAC8CoUHAADY3iULjzMnUwAAAFiBwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGyPwgMAAGzP9wrnHTmSAgAAIBtxhQcAANgehQcAANgehQcAANgehQcAANgehQcAANgehQcAANje/wOWsb8zxMkHNQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "trainer.evaluate()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# FrozenLake-v0 example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "import gym\n",
    "env = gym.make('FrozenLake-v0')\n",
    "_ = env.reset()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "observation_space= Discrete(16) action_space= Discrete(4)\n"
     ]
    }
   ],
   "source": [
    "print(\"observation_space=\",env.observation_space, \"action_space=\",env.action_space)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from blobrl.agents import DQN\n",
    "agent = DQN(observation_space=env.observation_space, action_space=env.action_space)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\site-packages\\blobrl\\trainer.py:29: UserWarning: be sure of your agent need to have good input and output dimension\n",
      "  warnings.warn(\"be sure of your agent need to have good input and output dimension\")\n"
     ]
    }
   ],
   "source": [
    "from blobrl import Trainer\n",
    "trainer = Trainer(environment=env, agent=agent, log_dir=\"./logs\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Start train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d780ac7b1e5049dcb02fbb3a695bc1a8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "trainer.train(max_episode=1000, nb_evaluation=0, render=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (Right)\n",
      "S\u001b[41mF\u001b[0mFF\n",
      "FHFH\n",
      "FFFH\n",
      "HFFG\n",
      "  (Right)\n",
      "S\u001b[41mF\u001b[0mFF\n",
      "FHFH\n",
      "FFFH\n",
      "HFFG\n"
     ]
    }
   ],
   "source": [
    "trainer.evaluate()\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Assault-v0 example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "ename": "FileNotFoundError",
     "evalue": "Could not find module 'C:\\Users\\Nathan\\AppData\\Roaming\\Python\\Python38\\site-packages\\atari_py\\ale_interface\\ale_c.dll' (or one of its dependencies). Try using the full path with constructor syntax.",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mFileNotFoundError\u001b[0m                         Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-22-d9bca508a03d>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[1;32mimport\u001b[0m \u001b[0mgym\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 2\u001b[1;33m \u001b[0menv\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mgym\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmake\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'Assault-v0'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      3\u001b[0m \u001b[0m_\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0menv\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mreset\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\site-packages\\gym\\envs\\registration.py\u001b[0m in \u001b[0;36mmake\u001b[1;34m(id, **kwargs)\u001b[0m\n\u001b[0;32m    143\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    144\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mmake\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mid\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 145\u001b[1;33m     \u001b[1;32mreturn\u001b[0m \u001b[0mregistry\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmake\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mid\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    146\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    147\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mspec\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mid\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\site-packages\\gym\\envs\\registration.py\u001b[0m in \u001b[0;36mmake\u001b[1;34m(self, path, **kwargs)\u001b[0m\n\u001b[0;32m     88\u001b[0m             \u001b[0mlogger\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0minfo\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'Making new env: %s'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mpath\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     89\u001b[0m         \u001b[0mspec\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mspec\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mpath\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 90\u001b[1;33m         \u001b[0menv\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mspec\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmake\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     91\u001b[0m         \u001b[1;31m# We used to have people override _reset/_step rather than\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     92\u001b[0m         \u001b[1;31m# reset/step. Set _gym_disable_underscore_compat = True on\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\site-packages\\gym\\envs\\registration.py\u001b[0m in \u001b[0;36mmake\u001b[1;34m(self, **kwargs)\u001b[0m\n\u001b[0;32m     57\u001b[0m             \u001b[0menv\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mentry_point\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m**\u001b[0m\u001b[0m_kwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     58\u001b[0m         \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 59\u001b[1;33m             \u001b[0mcls\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mload\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mentry_point\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     60\u001b[0m             \u001b[0menv\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mcls\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m**\u001b[0m\u001b[0m_kwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     61\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\site-packages\\gym\\envs\\registration.py\u001b[0m in \u001b[0;36mload\u001b[1;34m(name)\u001b[0m\n\u001b[0;32m     16\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mload\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mname\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     17\u001b[0m     \u001b[0mmod_name\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mattr_name\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mname\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msplit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\":\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 18\u001b[1;33m     \u001b[0mmod\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mimportlib\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mimport_module\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmod_name\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     19\u001b[0m     \u001b[0mfn\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mgetattr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmod\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mattr_name\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     20\u001b[0m     \u001b[1;32mreturn\u001b[0m \u001b[0mfn\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\importlib\\__init__.py\u001b[0m in \u001b[0;36mimport_module\u001b[1;34m(name, package)\u001b[0m\n\u001b[0;32m    125\u001b[0m                 \u001b[1;32mbreak\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    126\u001b[0m             \u001b[0mlevel\u001b[0m \u001b[1;33m+=\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 127\u001b[1;33m     \u001b[1;32mreturn\u001b[0m \u001b[0m_bootstrap\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_gcd_import\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mname\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mlevel\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mpackage\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    128\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    129\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\importlib\\_bootstrap.py\u001b[0m in \u001b[0;36m_gcd_import\u001b[1;34m(name, package, level)\u001b[0m\n",
      "\u001b[1;32mD:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\importlib\\_bootstrap.py\u001b[0m in \u001b[0;36m_find_and_load\u001b[1;34m(name, import_)\u001b[0m\n",
      "\u001b[1;32mD:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\importlib\\_bootstrap.py\u001b[0m in \u001b[0;36m_find_and_load_unlocked\u001b[1;34m(name, import_)\u001b[0m\n",
      "\u001b[1;32mD:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\importlib\\_bootstrap.py\u001b[0m in \u001b[0;36m_load_unlocked\u001b[1;34m(spec)\u001b[0m\n",
      "\u001b[1;32mD:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\importlib\\_bootstrap_external.py\u001b[0m in \u001b[0;36mexec_module\u001b[1;34m(self, module)\u001b[0m\n",
      "\u001b[1;32mD:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\importlib\\_bootstrap.py\u001b[0m in \u001b[0;36m_call_with_frames_removed\u001b[1;34m(f, *args, **kwds)\u001b[0m\n",
      "\u001b[1;32mD:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\site-packages\\gym\\envs\\atari\\__init__.py\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[1;32mfrom\u001b[0m \u001b[0mgym\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0menvs\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0matari\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0matari_env\u001b[0m \u001b[1;32mimport\u001b[0m \u001b[0mAtariEnv\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32mD:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\site-packages\\gym\\envs\\atari\\atari_env.py\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      7\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      8\u001b[0m \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 9\u001b[1;33m     \u001b[1;32mimport\u001b[0m \u001b[0matari_py\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     10\u001b[0m \u001b[1;32mexcept\u001b[0m \u001b[0mImportError\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     11\u001b[0m     raise error.DependencyNotInstalled(\n",
      "\u001b[1;32m~\\AppData\\Roaming\\Python\\Python38\\site-packages\\atari_py\\__init__.py\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[1;32mfrom\u001b[0m \u001b[1;33m.\u001b[0m\u001b[0male_python_interface\u001b[0m \u001b[1;32mimport\u001b[0m \u001b[1;33m*\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      2\u001b[0m \u001b[1;32mimport\u001b[0m \u001b[0mos\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      3\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      4\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0m_game_dir\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      5\u001b[0m     \u001b[1;32mreturn\u001b[0m \u001b[0mos\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpath\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mos\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpath\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mabspath\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mos\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpath\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdirname\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0m__file__\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m\"atari_roms\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Roaming\\Python\\Python38\\site-packages\\atari_py\\ale_python_interface.py\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m     15\u001b[0m                                             'ale_interface/libale_c.so'))\n\u001b[0;32m     16\u001b[0m \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 17\u001b[1;33m     ale_lib = cdll.LoadLibrary(os.path.join(os.path.dirname(__file__),\n\u001b[0m\u001b[0;32m     18\u001b[0m                                             'ale_interface/ale_c.dll'))\n\u001b[0;32m     19\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\ctypes\\__init__.py\u001b[0m in \u001b[0;36mLoadLibrary\u001b[1;34m(self, name)\u001b[0m\n\u001b[0;32m    457\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    458\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mLoadLibrary\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mname\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 459\u001b[1;33m         \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_dlltype\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mname\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    460\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    461\u001b[0m \u001b[0mcdll\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mLibraryLoader\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mCDLL\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Users\\nathan\\Anaconda3\\envs\\RL\\lib\\ctypes\\__init__.py\u001b[0m in \u001b[0;36m__init__\u001b[1;34m(self, name, mode, handle, use_errno, use_last_error, winmode)\u001b[0m\n\u001b[0;32m    379\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    380\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mhandle\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 381\u001b[1;33m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_handle\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0m_dlopen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_name\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmode\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    382\u001b[0m         \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    383\u001b[0m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_handle\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mhandle\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mFileNotFoundError\u001b[0m: Could not find module 'C:\\Users\\Nathan\\AppData\\Roaming\\Python\\Python38\\site-packages\\atari_py\\ale_interface\\ale_c.dll' (or one of its dependencies). Try using the full path with constructor syntax."
     ]
    }
   ],
   "source": [
    "import gym\n",
    "env = gym.make('Assault-v0')\n",
    "_ = env.reset()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainer.evaluate()\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"observation_space=\",env.observation_space, \"action_space=\",env.action_space)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from blobrl.agents import DQN\n",
    "agent = DQN(observation_space=env.observation_space, action_space=env.action_space)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from blobrl import Trainer\n",
    "trainer = Trainer(environment=env, agent=agent, log_dir=\"./logs\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Start train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainer.train(max_episode=200, nb_evaluation=0, render=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainer.evaluate()\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Copie de Gym_Envs_1_preamble_evn_list.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}