examples/qae_two_qubit_demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DEMO: Two-qubit QAE instance\n",
    "\n",
    "In this demo, we compress a two-qubit data set such that we have its (lossy) description using one qubit.\n",
    "\n",
    "We show the state preparation and training circuits below (circuits for both training schemes are shown but in this notebook, we run the \"full with reset\" method):\n",
    "\n",
    "<img src=\"../images/two_qubit_circuits.png\" width=\"500\">\n",
    "\n",
    "We first generate the data set by varying $\\phi$ (40 equally-spaced points from $\\frac{-\\pi}{2}$ to $\\frac{\\pi}{2}$). Then, a single-parameter circuit is used to find the 2-1-2 map. Looking at the circuit, the minimum should be when $\\theta = 0$.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import modules\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import os\n",
    "import scipy.optimize\n",
    "\n",
    "from pyquil.gates import *\n",
    "from pyquil import Program\n",
    "\n",
    "from qcompress.qae_engine import *\n",
    "from qcompress.utils import *\n",
    "\n",
    "global pi\n",
    "pi = np.pi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## QAE Settings\n",
    "\n",
    "In the cell below, we enter the settings for the QAE.\n",
    "\n",
    "__NOTE__: Because QCompress was designed to run on the quantum device (as well as the simulator), we need to anticipate nontrival mappings between abstract qubits and physical qubits. The dictionaries `q_in`,  `q_latent`, and `q_refresh` are abstract-to-physical qubit mappings for the input, latent space, and refresh qubits respectively. A cool plug-in/feature to add would be to have an automated \"qubit mapper\" to determine the optimal or near-optimal abstract-to-physical qubit mappings for a particular QAE instance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "### QAE setup options\n",
    "\n",
    "# Abstract-to-physical qubit mapping\n",
    "q_in = {'q0': 0, 'q1': 1} # Input qubits\n",
    "q_latent = {'q1': 1}      # Latent space qubits\n",
    "q_refresh = {'q0': 0}     # Refresh qubits\n",
    "\n",
    "# Training scheme: Full with reset feature (q_refresh = q_in - q_latent)\n",
    "trash_training = False\n",
    "reset = True\n",
    "\n",
    "# Simulator settings\n",
    "cxn_setting = '2q-qvm'\n",
    "n_shots = 5000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Aside: Running on the QPU\n",
    "\n",
    "To execute the quantum autoencoder on an actual quantum device, the user simply replaces `cxn_setting` to a valid quantum processing unit (QPU) setting. This is also assuming the user has already made reservations on his/her quantum machine image (QMI) to use the QPU. To sign up for an account on Rigetti's Quantum Cloud Services (QCS), click [here](https://www.rigetti.com/qcs)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data preparation circuits\n",
    "\n",
    "To prepare the quantum data, we define the state preparation circuits (and their daggered circuits). In this particular example, we will generate the data by scanning over various values of `phi`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def _state_prep_circuit(phi, qubit_indices):\n",
    "    \"\"\"\n",
    "    Returns parametrized state preparation circuit.\n",
    "    We will vary over phi to generate the data set.\n",
    "    \n",
    "    :param phi: (list or numpy.array, required) List or array of data generation parameters\n",
    "    :param qubit_indices: (list, required) List of qubit indices\n",
    "    :returns: State preparation circuit\n",
    "    :rtype: pyquil.quil.Program\n",
    "    \"\"\"\n",
    "    circuit = Program()\n",
    "    circuit += Program(RY(phi[0], qubit_indices[1]))\n",
    "    circuit += Program(CNOT(qubit_indices[1], qubit_indices[0]))\n",
    "    return circuit\n",
    "\n",
    "def _state_prep_circuit_dag(phi, qubit_indices):\n",
    "    \"\"\"\n",
    "    Returns the daggered version of the state preparation circuit.\n",
    "    \n",
    "    :param phi: (list or numpy.array, required) List or array of data generation parameters\n",
    "    :param qubit_indices: (list, required) List of qubit indices\n",
    "    :returns: State un-preparation circuit\n",
    "    :rtype: pyquil.quil.Program\n",
    "    \"\"\"\n",
    "    circuit = Program()\n",
    "    circuit += Program(CNOT(qubit_indices[1], qubit_indices[0]))\n",
    "    circuit += Program(RY(-phi[0], qubit_indices[1]))\n",
    "    return circuit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Qubit labeling\n",
    "\n",
    "In the cell below, we produce lists of __ordered__ physical qubit indices involved in the compression and recovery maps of the quantum autoencoder. Depending on the training and reset schemes, we may use different qubits for the compression vs. recovery."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Physical qubit indices for compression : [0, 1]\n",
      "Physical qubit indices for recovery    : [0, 1]\n"
     ]
    }
   ],
   "source": [
    "compression_indices = order_qubit_labels(q_in).tolist()\n",
    "\n",
    "q_out = merge_two_dicts(q_latent, q_refresh)\n",
    "recovery_indices = order_qubit_labels(q_out).tolist()\n",
    "\n",
    "if not reset:\n",
    "    recovery_indices = recovery_indices[::-1]\n",
    "\n",
    "print(\"Physical qubit indices for compression : {0}\".format(compression_indices))\n",
    "print(\"Physical qubit indices for recovery    : {0}\".format(recovery_indices))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the full training scheme with no resetting feature, this will require the three total qubits. \n",
    "\n",
    "The first two qubits (`q0`, `q1`) will be used to encode the quantum data. `q1` will then be used as the latent space qubit, meaning our objective will be to reward the training conditions that \"push\" the information to the latent space qubit. Then, a refresh qubit, `q2`, is added to recover the original data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data generation\n",
    "\n",
    "After determining the qubit mapping, we add this physical qubit information to the state preparation circuits and store the \"mapped\" circuits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Lists to store state preparation circuits\n",
    "list_SP_circuits = []\n",
    "list_SP_circuits_dag = []\n",
    "\n",
    "phi_list = np.linspace(-pi/2., pi/2., 40)\n",
    "\n",
    "for angle in phi_list:\n",
    "\n",
    "    # Map state prep circuits\n",
    "    state_prep_circuit = _state_prep_circuit([angle], compression_indices)\n",
    "    \n",
    "    # Map daggered state prep circuits\n",
    "    if reset:\n",
    "        state_prep_circuit_dag = _state_prep_circuit_dag([angle], compression_indices)\n",
    "    else:\n",
    "        state_prep_circuit_dag = _state_prep_circuit_dag([angle], recovery_indices)\n",
    "\n",
    "    # Store mapped circuits\n",
    "    list_SP_circuits.append(state_prep_circuit)\n",
    "    list_SP_circuits_dag.append(state_prep_circuit_dag)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training circuit preparation\n",
    "\n",
    "In this step, we choose a parametrized quantum circuit that will be trained to compress then recover the input data set.\n",
    "\n",
    "__NOTE__: This is a simple one-parameter training circuit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def _training_circuit(theta, qubit_indices):\n",
    "    \"\"\"\n",
    "    Returns parametrized/training circuit.\n",
    "    \n",
    "    :param theta: (list or numpy.array, required) Vector of training parameters\n",
    "    :param qubit_indices: (list, required) List of qubit indices\n",
    "    :returns: Training circuit\n",
    "    :rtype: pyquil.quil.Program\n",
    "    \"\"\"\n",
    "    circuit = Program()\n",
    "    circuit += Program(RY(-theta[0]/2, qubit_indices[0]))\n",
    "    circuit += Program(CNOT(qubit_indices[1], qubit_indices[0]))\n",
    "    return circuit\n",
    "\n",
    "def _training_circuit_dag(theta, qubit_indices):\n",
    "    \"\"\"\n",
    "    Returns the daggered parametrized/training circuit.\n",
    "    \n",
    "    :param theta: (list or numpy.array, required) Vector of training parameters\n",
    "    :param qubit_indices: (list, required) List of qubit indices\n",
    "    :returns: Daggered training circuit\n",
    "    :rtype: pyquil.quil.Program\n",
    "    \"\"\"\n",
    "    circuit = Program()\n",
    "    circuit += Program(CNOT(qubit_indices[1], qubit_indices[0]))\n",
    "    circuit += Program(RY(theta[0]/2, qubit_indices[0]))\n",
    "    return circuit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As was done for the state preparation circuits, we also map the training circuits with physical qubits we want to use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_circuit = lambda param : _training_circuit(param, compression_indices)\n",
    "\n",
    "if reset:\n",
    "    training_circuit_dag = lambda param : _training_circuit_dag(param, compression_indices)\n",
    "else:\n",
    "    training_circuit_dag = lambda param : _training_circuit_dag(param, recovery_indices)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the QAE instance\n",
    "\n",
    "Here, we initialize a QAE instance. This is where the user can decide which optimizer to use, etc. \n",
    "\n",
    "For this demo, we use `scipy`'s POWELL optimizer. Because various optimizers have different output variable names, we allow the user to enter a function that parses the optimization output. This function always returns the optimized parameter then its function value (in this order). We show an example of how to use this feature below (see `opt_result_parse` function). The POWELL optimizer returns a list of output values, in which the first and second elements are the optimized parameters and their corresponding function value, respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "minimizer = scipy.optimize.fmin_powell\n",
    "minimizer_args = []\n",
    "minimizer_kwargs = ({'xtol': 0.0001, 'ftol': 0.0001, 'maxiter': 500,\n",
    "                     'full_output': 1, 'retall': 1})\n",
    "opt_result_parse = lambda opt_res: ([opt_res[0]], opt_res[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "qae = quantum_autoencoder(state_prep_circuits=list_SP_circuits,\n",
    "                          training_circuit=training_circuit,\n",
    "                          q_in=q_in,\n",
    "                          q_latent=q_latent,\n",
    "                          q_refresh=q_refresh,\n",
    "                          state_prep_circuits_dag=list_SP_circuits_dag,\n",
    "                          training_circuit_dag=training_circuit_dag,\n",
    "                          trash_training=trash_training,\n",
    "                          reset=reset,\n",
    "                          minimizer=minimizer,\n",
    "                          minimizer_args=minimizer_args,\n",
    "                          minimizer_kwargs=minimizer_kwargs,\n",
    "                          opt_result_parse=opt_result_parse,\n",
    "                          n_shots=n_shots, \n",
    "                          print_interval=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After defining the instance, we set up the Forest connection (in this case, a simulator) and split the data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "qae.setup_forest_cxn(cxn_setting)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "qae.train_test_split(train_indices=[1, 31, 16, 7, 20, 23, 9, 17])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "QCompress Setting\n",
      "=================\n",
      "QAE type: 2-1-2\n",
      "Data size: 40\n",
      "Training set size: 8\n",
      "Training mode: full cost function\n",
      "  Reset qubits: True\n",
      "Compile program: False\n",
      "Forest connection: 2q-qvm\n",
      "  Connection type: QVM\n"
     ]
    }
   ],
   "source": [
    "print(qae)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training\n",
    "\n",
    "The autoencoder is trained in the cell below. The lowest possible mean loss value is -1.000."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter    0 Mean Loss: -0.5382250\n",
      "Iter    1 Mean Loss: -0.5400500\n",
      "Iter    2 Mean Loss: -0.2839000\n",
      "Iter    3 Mean Loss: -0.9166500\n",
      "Iter    4 Mean Loss: -0.8539250\n",
      "Iter    5 Mean Loss: -0.9167750\n",
      "Iter    6 Mean Loss: -0.2814000\n",
      "Iter    7 Mean Loss: -0.9488000\n",
      "Iter    8 Mean Loss: -0.9997750\n",
      "Iter    9 Mean Loss: -1.0000000\n",
      "Iter   10 Mean Loss: -0.9999000\n",
      "Iter   11 Mean Loss: -0.9999500\n",
      "Iter   12 Mean Loss: -0.5419250\n",
      "Iter   13 Mean Loss: -1.0000000\n",
      "Iter   14 Mean Loss: -0.5503500\n",
      "Iter   15 Mean Loss: -0.1691000\n",
      "Iter   16 Mean Loss: -1.0000000\n",
      "Iter   17 Mean Loss: -0.7972250\n",
      "Iter   18 Mean Loss: -0.9126500\n",
      "Iter   19 Mean Loss: -0.9996750\n",
      "Iter   20 Mean Loss: -0.9999500\n",
      "Iter   21 Mean Loss: -1.0000000\n",
      "Iter   22 Mean Loss: -1.0000000\n",
      "Iter   23 Mean Loss: -1.0000000\n",
      "Iter   24 Mean Loss: -1.0000000\n",
      "Iter   25 Mean Loss: -1.0000000\n",
      "Iter   26 Mean Loss: -1.0000000\n",
      "Iter   27 Mean Loss: -1.0000000\n",
      "Iter   28 Mean Loss: -1.0000000\n",
      "Iter   29 Mean Loss: -1.0000000\n",
      "Iter   30 Mean Loss: -1.0000000\n",
      "Iter   31 Mean Loss: -1.0000000\n",
      "Iter   32 Mean Loss: -1.0000000\n",
      "Iter   33 Mean Loss: -1.0000000\n",
      "Iter   34 Mean Loss: -1.0000000\n",
      "Iter   35 Mean Loss: -1.0000000\n",
      "Iter   36 Mean Loss: -1.0000000\n",
      "Iter   37 Mean Loss: -1.0000000\n",
      "Iter   38 Mean Loss: -1.0000000\n",
      "Iter   39 Mean Loss: -1.0000000\n",
      "Iter   40 Mean Loss: -1.0000000\n",
      "Iter   41 Mean Loss: -1.0000000\n",
      "Iter   42 Mean Loss: -1.0000000\n",
      "Iter   43 Mean Loss: -1.0000000\n",
      "Iter   44 Mean Loss: -1.0000000\n",
      "Iter   45 Mean Loss: -1.0000000\n",
      "Iter   46 Mean Loss: -1.0000000\n",
      "Iter   47 Mean Loss: -1.0000000\n",
      "Iter   48 Mean Loss: -1.0000000\n",
      "Iter   49 Mean Loss: -1.0000000\n",
      "Iter   50 Mean Loss: -1.0000000\n",
      "Iter   51 Mean Loss: -1.0000000\n",
      "Iter   52 Mean Loss: -1.0000000\n",
      "Iter   53 Mean Loss: -1.0000000\n",
      "Iter   54 Mean Loss: -1.0000000\n",
      "Iter   55 Mean Loss: -1.0000000\n",
      "Iter   56 Mean Loss: -1.0000000\n",
      "Iter   57 Mean Loss: -1.0000000\n",
      "Iter   58 Mean Loss: -1.0000000\n",
      "Iter   59 Mean Loss: -1.0000000\n",
      "Iter   60 Mean Loss: -1.0000000\n",
      "Iter   61 Mean Loss: -1.0000000\n",
      "Iter   62 Mean Loss: -1.0000000\n",
      "Iter   63 Mean Loss: -1.0000000\n",
      "Iter   64 Mean Loss: -1.0000000\n",
      "Iter   65 Mean Loss: -1.0000000\n",
      "Iter   66 Mean Loss: -1.0000000\n",
      "Iter   67 Mean Loss: -1.0000000\n",
      "Iter   68 Mean Loss: -1.0000000\n",
      "Iter   69 Mean Loss: -1.0000000\n",
      "Iter   70 Mean Loss: -1.0000000\n",
      "Iter   71 Mean Loss: -1.0000000\n",
      "Iter   72 Mean Loss: -1.0000000\n",
      "Optimization terminated successfully.\n",
      "         Current function value: -1.000000\n",
      "         Iterations: 2\n",
      "         Function evaluations: 73\n",
      "Mean loss for training data: -1.0\n",
      "CPU times: user 6 s, sys: 135 ms, total: 6.14 s\n",
      "Wall time: 57.3 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "initial_guess = [pi/1.2]\n",
    "\n",
    "avg_loss_train = qae.train(initial_guess)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Printing the optimized parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[array(0.00155588)]\n"
     ]
    }
   ],
   "source": [
    "print(qae.optimized_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot training loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaUAAAEmCAYAAADC9o/YAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3XmcXFWZ//HPt9ckbEmGhE0CaYIJaVHRyLggmwjRoUVFQRS3UcF1mJ/roKioOKKi/safK4MzKCggIEqrBCWAGyIEESWQBtIgsiVAJ0A6naTT/fz+OLeam0ot91bd6rrd/bxfr3pV97lLPV3dqSfPOeeeKzPDOeecy4OWZgfgnHPOFXhScs45lxuelJxzzuWGJyXnnHO54UnJOedcbnhScs45lxuelNyUJMkSPO7L6LWmRef7jxqOXRod+8IsYknxuoWYzxjP13WurdkBONckLyr6/grgNuDMWNvmjF5rc/R699dw7B+jY2/PKBbncs2TkpuSzOzG+PeSNgOPFbeXI6nTzBIlLQtXqCc6b4ljn6j1WOcmIu++c64KSRdLukfSoZJulDQEfDba9hZJv5H0qKSnJN0i6Y1Fx2/XfSfpbElbJe0v6WpJg5LulXS6JMX22677LorhGkmvkPQXSRsl/U3Sv5SI/S2S7pK0SdJt0TE3SlqW0XuzWFKvpCckDUn6g6SXldjnyug92iTp75IuiW3fRdK3JP1D0mZJayT9StKCLGJ0E4tXSs4lsytwAfBF4A5gMGqfD1wM3BN9fwRwgaQOMzu/yjkF/AT4HvBl4LXAfwL3ARdVOfYA4EvAF4B1wMeAn0h6ppn9HUDSscD3gcuAfwd2A74NTAP+Uu0HrkbSPsAfgEeB9wAbgNOAqyUdbWbXRgn2KuAB4FTgceAZwLGxU30DOBI4A1hNeK8PBXauN0Y38XhSci6ZXYATzezqeKOZfabwtaQW4Dpgb8KH9PlVztkC/KeZXRQdvxw4CjiJ6klpV+DFsQT0N+AfwPHAV6N9Pgv82cxeH4uxj5BIsvARYAZwlJndH53/KuAu4CzgxcBewDzgXWb2q9ixP4x9/SLg+2b2v7G2n2QUo5tgvPvOuWQ2FickAEkHSPqxpIeArcAwcDKwMOF5f1H4Ihp7Wkn4EK9mZSEhRcc+AKwvHCupE3guoUoitt8NwMMJY6vmUOB3hYQUnX8YuAQ4WNI04BFClXSOpHdI2q/EeW4GTpH0MUnPi5K7m6L8l+9cMo8UN0iaCfwaWESoGg4BXkCoAqYlOOeImT1Z1LY54bEDJdrix+5O6B5cW2K/NQnOn8RsSie4R4BWYBcz20romvsroYvynmh87h2x/U8F/id6vgVYI+nLUVJzU4x33zmXTKl7vLyU0D31ajNbUWiU1D5uUZW3hhDz3BLbdiObxDRASH7FdgdGCJUbZnY3cHJUAT2XML51nqR+M7suSswfBT4qaT5wAvB5YCPw6QzidBOIV0rO1W5G9DxcaJA0F3hlc8J5mpltIkxmeF28XdKLgT0yepnfAC+VtGfs/G2EpPKn4inzZjZqZn8GPhw1PatE3Pea2RcJ41LbbXeTn1dKztXud4RZeN+V9FnCbLFPEaqQZzQzsMingF5JlxK6x3YnVB5rgdGE5+iW9LoS7b8CziGMny2X9BnCe/FvhHGtdwJIOpgwo/DHhJl17dG2LcD10T4rou0ro3McRegS/a9UP62bFDwpOVcjM3tI0vGEqdmXEwb0vwrsQ+iiaioz+7mktxGmWv+UUH28nzC280TC07whehQ70Mxul3QIcDZwLiHh3AosNbNro/0eJIw7fYTQ1TlEGF96pZn9Ldrnt8AbCdPrWwnJ6/1m9t3kP62bLOS3Q3du6pDUBfQBHzezLzc7HueKeVJybpKStAuh62w54aLVBYSLbGcBi83s0SaG51xJ3n3n3OQ1TBjb+ibwT4QVF34DnO4JyeWVV0rOOedyw6eEO+ecyw3vvktp1113tX333bfZYTjn3IRyyy23PGZmc6rt50kppX333ZcVK1ZU39E559wYSX+vvpd33znnnMsRT0rOOedyw5OSc8653PCk5JxzLjc8KTnnnMsNT0rOOedyw5OSa7j+/n66u7tpa2uju7ub/v7+ZofknMspT0qu4Xp6eli1ahUjIyOsWrWKnp6eZofknMspT0qu4fr6+hgdDfeUGx0dpa+vr8kROefyypOSa7iFCxeOfd3S0rLN9845Fzchk5KkFkmnS7pP0iZJt0V3AK123M6SPiXpBkmPS1offf3q8Yh7qurt7aW1tZWWlhYWLVpEb29vs0NyzuXUhExKwOeAM4FvAK8AbgQulfTKKsfNA95LuKfMycCJhFtEXyHpfQ2Ldorr6upi2rRpXHXVVaxcuZKurq5mh+Scy6kJtyCrpLnAh4GzzeycqPk6SQuAs4FfVjj8XqDLzDbG2q6WtDfhjpzfbETMU93o6CiDg4MMDw83OxTnXM5NxErpGKADuLCo/ULgQEnzyx1oZoNFCalgBbBndiG6uI0bw1u+ZcuWJkfinMu7iZiUuoHNwD1F7Suj58U1nPNQYFU9QbnyBgcHAbxScs5VNeG674DZwHrb/j7uA7HtiUk6BXghYYzJNcCGDRsAr5Scc9U1vVKSdJQkS/C4vnAIUJyQCu1pX/tw4OvABWb2wwr7nSJphaQVjz76aNqXmfIKSckrJedcNXmolG4ADkiwX2EsaACYJUlF1dKs2PaqJL0AuBK4FnhHpX3N7FzgXIAlS5aUSoiuAk9Kzrmkmp6UookHacZzVgKdwH5sO65UGEu6o9oJJB0IXA38BTjezPzTsoG8+845l1TTu+9qsAzYArypqP1k4HYzu7fSwZL2B34N9APHmtlQQ6J0Y7xScs4l1fRKKS0zWyvpa8Dpkp4C/ky4CPZI4Lj4vpKWA/uY2YLo+7mEhNQBfBpYLG0zFHWrmW1u/E8xtXil5JxLasIlpcgngA3AacDuQB9wgpkVr1/TyrY/42Jgn+jrn5c473zgvkwjdV4pOecSm5BJycxGgLOiR6X9Di/6/npqmKXn6uOVknMuqYk4puQmmA0bNjBt2jSvlJxzVXlScg23YcMGZs+e7ZWSc64qT0qu4TZs2MCsWbO8UnLOVeVJyTWcV0rOuaQ8KbmG80rJOZeUJyXXcIODg14pOecS8aTkGs4rJedcUp6UXMMVkpJXSs65ajwpuYbzSsk5l5QnJddwXik555LypDSJ9Pf3093dTVtbG93d3fT39zc7JODpKeFeKTnnqvGklBNZJJSenh7uvPNORkZGWLVqFT09PQ2INJ1CdbTDDjt4peScq8qTUk709PRwxx131JVQ+vr6KNyMd3R0lL6+vqzDTG3Dhg3suOOOdHR0eKXknKvKk1JOxBNIrQll4cKFY1+3tLRs832zFJJSe3u7V0rOuao8KeVEFgmlt7eX1tZWWlpaWLRoEb29xbeXGn9eKTnn0vCklBO9vb1IqiuhdHV1MWPGDJYvX87KlSvp6upqQKTpxCslT0rOuWo8KeXEnnvuiZnxgx/8oK6EsmnTJjZvzs8d3eOVknffOeeq8aSUE+vWrQNCUqnV6Ogow8PDuUxKXik555LwpJQTAwMDAHUllEIlksek5JWScy4JT0o5kUVSKhybpw//wcFBr5Scc4l5UsqJQvddFknJKyXn3ETlSSknsqyU8piUvFJyziXhSSknCkmpnokOeU1KO+yww1hSKqw44ZxzpXhSyomBgQF22WWXuhJKIaHlLSntuOOOtLS00NraytatW5sdknMuxzwp5cS6devYY489Jm33HeDjSs65qjwp5cTAwEBmSSlPH/zxpOTjSs65ajwp5USWSckrJefcROVJKSemQlLySsk5V40npZwojClNxtl3Xik555LypJQTWVRKeZ59B14pOeeq86SUAyMjIzz55JPstttudXfftbS05KoaKU5KeYrNOZc/npRy4IknnmCnnXZi+vTpdSelnXfeObeVkt/ozzlXjSelHFi3bh2zZ89m2rRpkyopjY6OsmnTJqZPnw54peScqy5VUlLwHkm/l/S4pE2xbc+V9HVJ+2cf5uQ2MDDA7Nmz6ezsrHuiQ56S0saNG5kxYwYtLeHPzCsl51w1iZOSpHbgauAbwGJgM9Ae2+XvwCnAm7IMcCqIJ6XJVCnFu+7AJzo456pLUyl9GDgKOAuYA5wb32hm64DfAcdkFl0ZkloknS7pPkmbJN0m6fgaztMlaaMkk7SgEbEmMTAwwKxZs+pOSps2bWKnnXbKbVLyKeHOuWrSJKWTgT+a2afNbAQotdxzP7BPJpFV9jngTELV9grgRuBSSa9MeZ5vAU9kG1p6hTGlrCqlvHzwe6XknEsrTVLqAm6oss8A8E+1h1OdpLmEqu1sMzvHzK4zs1OB64CzU5znjcBBwBcbE2lyhe67LCY61LvSeJYKt60o8ErJOVdNmqS0Cdilyj7zgPW1h5PIMUAHcGFR+4XAgZLmVzuBpFnAVwnJrdHxVjVZJzp4peScSytNUvoL8HJJHaU2StoZOBq4KYvAKugmTLK4p6h9ZfS8OME5vgSsMrMLsgysVoUxpcLstNHR0ZrOk/ek5JWSc66aNEnpPMJ40fcl7RjfECWk/wFmA9/NLrySZgPrbftbmA7Etpcl6RDgLcB7k76gpFMkrZC04tFHH00VbBKFSklSXR/ceU9KXik556pJnJTM7IfABcCJwKPAqQCSbgQeBl4LnGtmP08TgKSjotlv1R7XFw6h9CQLJXitDkLS/JqZ3ZE0RjM718yWmNmSOXPmJD0sscJEB6CuyQ6F2Xd5qUa8UnLOpdWWZmcze6uk3wGnEbrRAA4G+oCvmtl/1xDDDcABCfbbGD0PALMkqahamhXbXs6/Eyqpr0uaGbXNiJ53krSTmT2VMO7MFColoK7JDl4pOecmulRJCcDMzgPOi7rwZgNPmFnN06rNbCOwKsUhK4FOYD+2HVcqjCVVqoAWA7sDD5bY9mfgNuC5KWLJRGFMCahrskN8SriZIVUtHhvKKyXnXFqpk1KBmW0ANmQYS1LLgC2ElSM+E2s/GbjdzO6tcOzZwPlFbUuBj0XH92UXZjJmtl1SqqdSmj59Oq2trQwPD9PRUXJOyrjZsGEDu+6669j3Xik556qpOSk1i5mtlfQ14HRJTxEqnBOBI4Hj4vtKWg7sY2YLomNXUVSVSdo3+vJPZlY8o6/hhoaGaG1tHVu0tN6k1NnZOXaOPCQlr5Scc2kkTkqS7kq4q5nZwhrjSeoThCrtNEJ3XB9wgpn1Fu3XSs4Tb3w8CbJNSjvttFNWYdbEx5Scc2ml+cCeQelZb7sAhU+eNcDWeoOqJlrm6KzoUWm/wxOc63y279IbN/GuO6hvosOmTZvo7OzMTUUyODi4XaU0ODjYxIicc3mXOCmZ2TPKbZO0CPgvwqrhr8ggrimjkZVSs3ml5JxLK5Ob/EVjNa8B9gU+mcU5p4r4NUpQ/+y7adOm5TYp5aWCc87lV2Z3no2mdl9NmMXmEvJKyTnnnpb17dCHCRMPXELFY0qTPSl5peScqySzpCRpNqEL74GszjkVFFdK9a7okKeJDqVuXeGVknOukjRTwj9e4Rx7ExLSLOCMDOKaMtatW8c++zx9X8Raq5yRkRFGRkZob2/PRaVkZtslJa+UnHPVpJkSXnH6NeG6obPN7At1xDPllBpTqmWiQ+FiWUm5SEpbtmyhpaVlmwt4vVJyzlWTJim9vEz7KLAOuMPM/L/BKWU1plSYeVfPObJUPJ4EPtHBOVddmuuUljcykKkqq9l3hfGkes6RpVJJKS9jXc65/Mp69p1LqdR1SlkkpWZ/+Hul5JyrRdlKSdKetZ7UzB6q9dipptTsu4GBSreEKi2elDo6OrxScs5NSJW67x6g9Fp31ViV87rI8PAwg4OD2yycWs9Eh7x333ml5JyrplLy+BG1JSWX0Pr165k5cyYtLU/3otaaUAqLsdZzjix5peScq0XZpGRmvlxQgxWPJ8HkmX1XvEI4eKXknKvOJzo0UfF4EvjsO+fc1OZJqYmKr1GC2pcZKp7o0OwPfx9Tcs7VIvWEBEkHAccAewGdJXYxMzu13sCmgnKVUhYTHR577LFMYqyVV0rOuVqkWftOwHnA2wARJkEotovF2j0pJZD1mFLeuu/mzJmzTZtXSs65atJ0370XeDtwEfBCQgL6OnAo8ClgELgYeGbGMU5aWY4p+ew759xkkKb77m3AXYVZeaFwYsDMfg/8XtIy4AbCjf5WZxznpDQwMMD8+fO3aZtMlVJ8hXDwSsk5V12aSmkRULz+3VhSM7MVwM+B92UQ15RQqlKqZ6JDYUp4HioSr5Scc7VIk5QEPBH7fhCYXbTPXcAB9QY1VUy1KeFtbW2MjIwwOjrapKicc3mXJik9RJhxV3Av8LyifRYAG+sNaqooN9Fhsi4zJMm78JxzFaVJSjexbRK6CvhnSadLWijpVOA44E9ZBjgZ9Pf3093dTVtbG93d3fT399Pf388tt9zCoYceOtYGk7tSAh9Xcs5VliYp/QSYJqkwMv8l4B+EO9LeAXwbeAr4j0wjnAR6enpYtWoVIyMj3HnnnRx11FEsXbqULVu2MDIywqpVq+jp6QHyP/uuVIItpVJS8nEl51w5aW7y9xNCYip8/3h0Ie2pwH7AfcD5ZvZg1kFOdH19fWPjKGbGvffeu8320dFR+vr6gGxWdGhkUiok2NHR0bFkunLlyu32K5eU/JbozrlK6lpmyMzWmdnZZvYuM/u8J6TSFi5cOLYSeEtLC4sXL2bx4sXbtC1cuBDIZkHWRs5yiyfYeDItKFRSjz76KIcddth2lZR33znnKqmYlCTdKundknaqtJ+rrLe3l0WLFtHa2sqiRYvo7e0t2QZhhpqZsXXr1lSvMV6V0sKFCwvXqG2TTAsKlRTAXXfdNdYtWeDTwp1zlVTrvnsO8E3gy5IuAc4zsxsbH9bk0tXVVbKLq1QbPJ1U2tqSX9s8Xkmpt7eXJUuWsH79+m2SaUG1SsorJedcJdW67w4BfhB9/a/AHyTdJul9knZpbGhTVy1JZbySUldXF8cddxwnnHACK1eupKura5vtxV2VxZWUV0rOuUoqJiUzu8HM3g7sSVj77lbgQMKadw9JOl/SSxof5tRSy2SH8ZwSvnHjRjZuLH05Wm9vL/vttx9AyUrKKyXnXCWJJjqY2VNm9h0zWwIcBHwH2AK8BfitpNslnSZpVsUTuURqSSrxKeGNrkYqJaWuri6uuOIKFi9eXLKS8krJOVdJ6tl3Znabmb0P2IOwSOsNwGLgq8CDki7INMIpqJZVHcbzduiVklJh+4wZM0pu80rJOVdJzVPCzWyTmf3AzF5KWO/u98A04I1ZBTdV1Tum1N7e3tA15upJSl4pOecqqes6JUmzJf07cDlhUgRsu2irq0G9SUkSHR0dDauWkiSl4ttWFHil5JyrpKakJOlISRcBDwJfIXTf/YkwQ2+vSsdmQVJLtObefZI2RTMCj09x/HRJZ0q6W9JmSWsk/VxSRyPjTqrepFTrOZLySsk51yhpboe+O+HOs+8A5hNuZbEOOBc418xKX3TTGJ8DPgx8ArgFeANwqaRjzeyXlQ6U1E5YTHY+8AXCun1zgJcDrY0MOql6Z98BDa+UhoaGKm73MSXnXC0qJiWFS/f/BXgn8ErCh7YI40f/DfzYzMZ1OWpJcwkJ6WwzOydqvk7SAuBsoGJSAj5EWO2828z+EWu/PPNga1TLRIf47LvCORpVkQwNDXml5JxriGqV0v2Ea5QEPA5cQKiKVjU6sAqOATqAC4vaLwT+R9J8M7t3+8PGvBe4tCgh5cpE6L7bvHkzIyMjtLZuX1wODg56peScq0m1MaW9gN8CbwL2MrMPNjkhAXQDm4F7itoL3YeLyx0oaR6wN9Av6b8lPRmNSS2X9NzGhJterUmpMCW81nMkMTIywpYtW5g+fXrZas4rJedcraolpYVmdoSZXWRmefkkmQ2sNzMrah+IbS9nz+j5Y0AXYSzqJMKY0vVR0tqOpFMkrZC04tFHH6098oTyXCkNDQ0xY8YMdthhh7JdeD6m5JyrVbVlhu5udACSjpJkCR7XFw4BihNSob2aws+7Eegxs1+a2RWEcbPpwPtKHWRm55rZEjNbMmfOnHQ/YA3STnQorCgeX8C1UUmpkHBmzJhRU1LySsk5V0nyZagb5wbCxbfVFD4BB4BZklRULc2KbS/n8ej5D2Y29olqZv+QtIqwhFLTpU0oxVUSNO7Dv5Bwpk+f7pWScy5zTU9KUXJIM061Eugk3O02Pq5UGEu6o8Kx/cAQ5SutxiyBkFLa2XfFM+8K52hkpVQtKZW7eNYrJedcJXWt6NAkywiLwb6pqP1k4PZKM+/MbBj4BfBSSWOfmtFY0kLg5uzDTS+LSimv3XdeKTnnKml6pZSWma2V9DXgdElPAX8GTgSOBI6L7ytpObCPmS2INX8auAn4haSvENbr+zSwHvjGOPwIVXV2dla8OLVY8cy7wjmaWSlVGlNK87M556aWCZeUIp8ANgCnAbsDfcAJZtZbtF8rRT+jmd0h6Ujgi8AlwDBwHfBqM1vT6MCTmDZtGuvXr0+8fzMqpXrGlJ588snM43LOTQ6ZJKXoPkpbzGwwi/NVY2YjwFnRo9J+h5dpvwk4IvvIspHVRIdGJaXp06dX7L7zi2edc7VKPKYk6WWSvhS/kZ+kuZJ+AzwGDEj6aiOCnGrSTnQoVynlcfadT3RwzlWSZqLDB4DXmtm6WNs5wEsJs+AeB06TdEKG8U1JaSulZsy+84kOzrlGSJOUnkNYiBUIt38AXgf82swWEmav/QN4d6YRTkGTefadV0rOuUrSJKW5wEOx7/+ZMHPtfAAzewr4OSE5uTqkXdFhPGffFZYZ8krJOdcIaZLSZsJSPAUvJVyE+ttY25NUXnvOJZD3iQ6VkpKZsWnTJqZPn17iaK+UnHOVpUlK9xKuBSo4HrjbzB6Mte1NmPTg6jARJjqUS0qF8a2WltJ/Wl4pOecqSZOUvg8cKOlPkn4HHAj8qGif5xGuGXJ1mChjSqUugq3UdQdeKTnnKktzndK3gRcSVk8Q0Eu4ABUASQcTFla9KMsAp6KJMPuu3JTwStcogVdKzrnKEielaN24N0p6d/jWnirapZ+wyvZ92YU3NdUy0SEvSckrJedcPVKv6GBmJdeIMbPH8PGkTNTSfZeXte+qJSWvlJxzlaRZ0WGWpMWSOova3y7pZ5J+FHXhuTplMdGhWbPvvFJyztUjTaX0n4TbQ8wtNEj6APB/efqur6+WtMTMKt3TyFVRS6VUPAW7WcsMeaXknKtHmtl3LwGWm1l8ytWHgQeBQ4HC8kIfzCi2KWuizL4rl5TK3eAPvFJyzlWWplLaC1he+EbSYsJ1SR8zs99Hba8nJChXh7xPdKi0SrhXSs65eqSplKYD8YGOlxBWdLgm1raakLxcHQrVhFmpu7ZvL08LsvqYknOuHmmS0oPAotj3xxCWFbot1jYL8NuK1qmlpYW2trbEH94TaaKDV0rOuUrSdN9dB7xV0vsJFdOrgMvNbDS2zwLCSuGuToVKpzjZlFJuSngjJzq0t7czMjLC8PAw7e3tY9urXTzrlZJzrpI0ldIXCLcg/y/gXEJiOrOwUdJc4DDghgzjm7LSdL+N15jS8PAwZkZ7ezuSSi415JWSc64eaVZ0uFdSN+EeSgBXmtn9sV32Ab7J9uvhuRrkMSkVblshhSsACl14O++889g+GzduZObMmWXP0dHR4UnJOVdWqhUdzOwR4Btltt0M3JxFUC7dDLzxSkrFVVCpcaUklZJ33znnykm9zBCApHbCpIeZwBPAndHaeC4jaZJKqdl3jZjokEVSam1tZXR0lJGREVpbWzONzzk38aUZU0LSzpK+A6wH/gJcD9wKrJf0HUnl+21cKmmWGspbpVTp4llJ3oXnnCsrcaUkaWfgD0A38BTwO+BhYA/gucApwCGSXlxu0VaXXNoxpXKz78xsbAyoXqWSUtqJDvD0ZIfimJ1zLk2ldDohIX0b2MfMDjezk8zscJ6e5LA42s/Vqd6JDq2trbS0tLB169bMYsqi+w58Wrhzrrw0Sem1wI1m9j4zWx/fYGZPmNkHgD8SbpPu6lTvRAfIvgsvq6Tk08Kdc+WkSUrzCGNIlfyGsB6eq1O9lVLacySRJClVu3gWvFJyzpWXJiltJHbbijLmRPu5OqWZ6FBq9h1kPwPPKyXnXKOlSUo3A6+XtH+pjZL2I9y+wq9VykBWlVKWFUlhhfACH1NyzmUtzXVKXwZ+Bdws6f8R1sJ7GNgdOBz4ALAjcE7GMU5JSZOSmbFly5bcdN95peScq0eaZYaWS3ovYe27j0ePAgHDwPvN7JpSx7t0kk50GB4eprW1teSFqOOdlMzMKyXnXF3SLjP0XUlXAW8GDgJ2IazocCtwoZn9PfsQp6akCaXSSuKNSErxC2NnzJjBY489NvZ9IUG2tVX+s/JKyTlXTuplhqJFWD9fapukaUCHXzxbv6QTHSolpawnOgwNDTFnzpyx74srpSRVUiEur5Scc6WkWmYogW8DAxmfc0pKWuWUm3mX5hxJFSed6dOn15SUvFJyzpWTdVKCML7k6pRV913Ws+8qjSkluUYJvFJyzpXXiKTUcJJaJJ0u6T5JmyTdJinRShKSWiX9H0m3SxqU9LCkKyQ9u9Fxp5F0osN4jylVSkpeKTnn6jUhkxLwOcJdb78BvAK4EbhU0isTHnsO8FOgBzgN2A+4TtIzGhJtDdJUSuUWNs1rUvJKyTlXTk33U2qm6LbrHwbONrPCNVHXSVoAnA38ssop3gZcYmZnxM75V+BO4F+A72YedA3yONHBKyXnXKNNxErpGKADuLCo/ULgQEnzqxzfARTPDiwsMJub9yOvU8IbUSn19/fT3d1NW1sb3d3d9Pf3Zxazc25iyc2HcArdwGbgnqL2ldHz4irHfws4WdJx0U0Lu6K2B4BLMo20DhNlokP8fkrVbvBXUFwp9fT0cOeddzIyMsKqVavo6enJLGbn3MRSsftO0sh4BZLCbGC9mVlR+0Bse1lm9ilJm4Gf8HRSvgs43Mzq4CWLAAAa5UlEQVRKTmeXdArhJobMmzev1rhTSTrRoZlTwuuplOJJqa+vj8Kvc3R0lL6+vsxids5NLNUqJdXwSEXSUZIsweP6WEzFCanQnuT13gOcAZwFHAG8nnAn3V9J2rPUMWZ2rpktMbMl8YtHG2kyd9+1t7dvU8EtXLhw7O64LS0tLFy4MLOYnXMTS8VKyczGo3vvBuCABPsVPv0GgFmSVFQtzYptL0nSbOBrwJfN7NOx9muB+4CPAP8neeiNk9fZd/FVwgsXzxZuuV7rRIfe3l6WLFnCunXrWLBgAb29vZnF7JybWJo++87MNgKrUhyyEugkTOOOjysVxpLuqHDsM6Njt7m9hpkNSFpNsuQ4LrKafbdhw4ZM4iksthpPSu3t7bS0tDA8PExHR0fNF892dXVx+OGHc8UVV/Dd736Xrq6uTGJ2zk08E3GiwzJgC/CmovaTgdvN7N4Kxz4SPR8cb4wqqAXAg1kFWa+8dd9t2bKF1tZW2tvbt2mPd+HVMyV8zZo1LFy4kFWr0vz/xDk32TS9UkrLzNZK+hpwuqSngD8DJwJHAsfF95W0HNjHzBZEx94n6efARySNEm7f/k/ARwkV1LfH7yepLKsVHbKafVcu4RSS0syZM1NNdBgcHNymbe3atRxxxBE+ycG5KW7CJaXIJ4ANhNUYdgf6gBPMrHgwopXtf8YTgQ8BJ0XPTxIS2yFmtqKRQaeRtwVZh4aGKiYlqL9SOvTQQ/nhD3+YSbzOuYlpInbfYWYjZnaWme1jZp1m9mwzu6zEfoeb2b5FbRvN7HNmttjMdjCzPczsX8zspnH7ARLIW/ddtUqp0j7FiseUhoaG2LJlCwcffLBXSs5NcRMyKU0FaSY6lJt9l+UyQ0mTUi0Xz65du5a5c+cyf/58HnrooUQ/t3NucvKklFMTpVKK31Op1kqpkJTa29uZP38+d999dyYxO+cmHk9KOZW3ZYay7L4rrpTWrFnDbrvtBoQLab0Lz7mpy5NSTrW3tzMyMsLo6ChQftHSZldKWYwpFSolgEWLFnlScm4K86SUU5K2SSo9PT2sWrVqu0VLx2v2XZKklPTi2XJjSoBfq+TcFOdJKcfikx36+vrGqqb4oqWToVKKd995peTc1OZJKcfiSSW+SGl80dLxuslfpaRUuH1FrWNKpSql7ReBd85NBZ6UciyelC677DJaWsKva/fddx9btHS8FmQdr0pp9uzZdHZ28sgjj5Q73Dk3iXlSyrH4UkMPPvggBx98MP/7v//LS17ykrFFS8dz9l18MdaCQlLaunUrW7dupaOjo+q5KlVK4F14zk1lnpRyLF7pLFu2jFe84hUcc8wxXHPNNWzduhXIz5jS0NAQO+yww9h9kSqpVCmBT3ZwbirzpJRj8YkOV111FUuXLmWPPfZgn3324aabwqpIeZl9l7TrDratlEZGRhgYGGDXXXcd2+6VknNTlyelHCsklfvvv5+1a9eyZMkSAJYuXcpVV10F5GOiQ9qkFL8d+sDAADNnzqSt7el1c71Scm7q8qSUY4WktGzZMo4++uixiQ5Lly5l2bJlwPh131VbJTzpNUqw7e3Q16xZs814EviqDs5NZZ6Ucqww0aEwnlTw4he/mLvvvpu1a9dWXZB169atY9c31aNRlVLxJAfAF2Z1bgrzpJRjnZ2dbNiwgWuvvZajjz56rL29vZ0jjzySX/3qVxUrJUnbTSqoVdZjSvFKKT7JobDdF2Z1bmrypJRjnZ2dXHvttey///7bVROFLrxKSalwjiy68Bo10aFUpQQ+2cG5qcqTUo51dnZy5ZVXsnTp0u22LV26lKuvvrri7DvIbrJD1t13lSol8HEl56YqT0o51tnZycMPP7zNeFLBvHnzmDt3LsPDwxUvWG10pVS4n1KWlVJ/fz8XXHABn/zkJ7dZEd05N/l5Usqp/v5+Lr/8cgDe+c53bvfB3N/fP7YUz4EHHlj2g3s8u++S3HUWtq2U1q5du12l1NPTw8MPP4yZbbMiunNu8vOklFM9PT2sW7cOCCuEF38wx7dX+uDOaqmhRowpmVnJKeF9fX1jC7LGV0R3zk1+npRyqtoHc9IP7vHovhsaGkp1nVJrayuSGBkZKVkpLVy4cOyarPiK6M65yc+TUk5V+2BO+sGdRVIyM4aGhkouyNra2kpHRwcDAwOJkxI8PS28VKXU29vLokWLkLTNiujOucnPk1JOFT6YW1tbWbRo0XYfzNW2F2Qx+27Tpk10dHSMJcFiM2bM4LHHHkuVlDo6Oli3bh2SthuL6urqYuXKlXzlK1/hNa95zdiK6M65ya+t+i6uGQofzLVuL8iiUqo2XlRLUmpvb+eBBx4oOR28YMmSJVxyySWpYnXOTWxeKU1yWUx0aERS6ujo4IEHHih54WzBQQcdxN/+9rfM7gnlnMs/T0qTXJ4rpQcffLBipbTjjjsyf/78RBWhc25y8KQ0ifX397N8+XJOOOGEui5CLbdCeEEhKSW9TgmSVUoQuvBWrFiR+LzOuYnNk9Ik1tPTw5NPPsno6GhdF6EmqZQ2bNiQ+ZgShKR08803Jz6vc25i86Q0icWvXarnItQkSSn+nIRXSs65UjwpTWJZXYTaiKSUtFJ6znOew6pVq/zeSs5NEZ6UJrGsLkJtVKX04IMPVq2Upk+fzsKFC/nrX/+a+NzOuYnLk9IkVriW6eKLL6a7u7vmi1AbVSlt2bKlalICH1dybirxpDQFHHvssdx0002sWbOmpuMbVSkBVbvvAF7wghf4uJJzU4QnpSlgxowZHHvssVx66aU1HV8tKRXWxJs2bVric7a3t9Pa2srs2bOr7uuTHZybOjwpTRFvfOMb+dGPflTTsUkqpenTp5ddG6+Ujo4O5syZk+iYZz3rWaxevZrBwcHE53fOTUwTMilJ+qCkXkkPSzJJZ6Y8/hBJN0gakvSIpK9K2n4J7Enk5S9/OXfffTf33Xdf6mOTJKU0F85CqJSSjCdBSGDPetazuPXWW1O9hnNu4pmQSQl4FzAX+GnaAyU9G/g1sBY4FjgDeDtwfobx5U57ezvHH388F198cepjN27cWPK2FQUzZsxINZ5UiCfJeFKBjys5NzVM1KTUbWb/DHyghmM/AzwAvN7MlpvZecBpwAmSnpdlkHlz0kkncdFFF6U+LkmllDYpdXR0JK6UwMeVnJsyzGzCPgi33jDgzIT7twNDwOeL2qcBm4HPVDvH85//fJuo7r77bmtra7PW1lZbvHixXXfddbZ48eKx71evXm2rV6/epu26666znXfe2VpaWsb2iVu9erXtvffeBpTcXsrq1att5syZJinxMb/85S+to6OjbOzVvi/3s9XzfRbnnMxxTeafLa9xjdfPluTfbDFghSX5nE6yU14fNSSlRdH+J5XYdgdwabVzTOSktHjxYpNk0XtgLS0tY19LspkzZ9rMmTPH2or3KSSmcucstb1aHEmPOeCAA8rGJclaWlrGzlnq+1mzZtmsWbMq7pP2+yzOOZnjmsw/W17jGo+fLem/2WJJk5LCvhOTpDZgmFDhnJlg/xcDfwBeYWbLirb9HthsZi8rcdwpwCkA8+bNe/7f//73DKIff21tbYyMjJTdXpgJNzo6Wnaf1tZWtm7dWvacxduTxFHLMWkl+dmacc7JHNdk/tkacc68nCPJOZP8my0m6RYzW1L19VLGlzlJR0Uz6Ko9rs/i5aLnUplYJdrCzmbnmtkSM1syZ86cDMJojuK18OK3OG9paWHRokUsWrSo4j7F6+fVsr5eFscUx1Xt+yQ/WzPOOZnjmsw/W17jGq+frdZ1NBNJUk418gHMIHSrVXvMK3Fs2u67A5jC3XfV+pqT9EeXGlNK29ecxTF57Wv3uKbGz5bXuCbDmNJU677rAJ4EzjGzM2Lt04D1wBfN7NOVzrFkyRLzWWDOOZfOhOm+G09mtgVYRpj+3Rbb9DqgE7iyKYE555wDQvfXhCNpCbAvTyfVxZJeF339SzPbGO33PeCtZhb/Oc8E/gj8WNI3o/N8GbjMzG5pfPTOOefKmZBJCXg/8NbY96+PHgDzgfuir1ujxxgz+4ukY4AvAr8AngB+AHy8gfE655xLYEKPKTWDjyk551x6PqbknHNuwvGk5JxzLje8+y4lSY8CtS7psCvwWIbhNNJEidXjzNZEiRMmTqweZ7CPmVVdfcCT0jiStCJJn2oeTJRYPc5sTZQ4YeLE6nGm4913zjnncsOTknPOudzwpDS+zm12AClMlFg9zmxNlDhh4sTqcabgY0rOOedywysl55xzueFJyTnnXG54UmowSXtLukzSE5KelPQTSfOaHNMzJP0/SX+UtDG6ieK+JfabJunLkh6WNBTtf+g4xvk6SZdL+nv0+n2SviBpp6L9Zkk6T9JjkgYlXSPpwHGM8xhJ10p6RNJmSQ9I+rGkxUX75fFvYVn0+z+rqL3Z7+nhZW72uT5PccbieKWk30raEP1uV0g6Mi9xSrq+wg1Ul8X2a/r76UmpgSTNAK4l3KTwrcCbgf2B6yTt0MTQFgAnAOuA31XY73vAu4BPAccCDwNXS3puwyMMPgyMEBbLXQp8G3gP8GtJLQCSRLjlyFLgA8DxQDvhPX7GOMU5G7iFsFDw0cDpQDdwo6R9ojhz97cg6STgOSXa8/CeFvwb8KLY46i8xSnpVOBnhL+B1xAWh76UcAPTvMT5XrZ9H18EfDDadmWO4mz+nWcn8wM4jfChuiDWNh/YCnywiXG1xL5+J+FuvPsW7fOcqP3tsbY2oA+4cpzinFOi7S1RXEdG3x8XfX9EbJ9dgAHg6018jxdGcX0oj38LwEzgEeCkKM6zYtua/p4Ch0cxHFVhnzzEuS8wBPx7nuMsE9f3gM3A7DzF6ZVSY70KuNHM7ik0mNm9wB8IfwBNYWajCXZ7FeGuvpfEjtsKXAwcI6mzQeGNMbNHSzTfHD3vFT2/CnjIzK6LHfcE0EsT32Pg8eh5OHrO29/Cl4CVZnZRiW15fU+L5SHOfwVGge9U2CcPcW5D0nRCRddrZgNRcy7i9KTUWN3A7SXaVwKLS7TnSTdwr0U3TIxZCXQQugCb4bDo+c7oudJ7PE/SjuMSFSCpVVKHpP2B7xIqkYujzbn5W5B0CKHifG+ZXXLzngI/lDQi6XFJPyoag8tDnIcAq4A3SFotaaukeyS9L2dxFnstsBPw/VhbLuL0pNRYswnjNsUGgFnjHEtalWIvbB9XkvYCPgtcY2aFm1pVi3M83+c/EbpD7gKeTehiXBtty8XfgqR2QsI8x8z6yuyWh/f0CeArhO7lI4HPEcaT/ihpbrRPHuLckzA2+GXgbMKY4q+Bb0g6LdonD3EWewuwFrgq1paLOCfqnWcnklJXJ2vco0hP5Cj26H9pPyOMwby9KJ68xPlmYGegizBJ49eSDjGz+6LteYjzY8B04PMV9mn6e2pmtwK3xpp+I+m3wE2EyQ9nkIM4Cf+x3wl4m5n9JGq7VmE26+mSvk4+4nz6RaU9CQn+v6Iu+Xg8TY/TK6XGWkfpimIWpf9HkicDlI+9sH1cSJpGmBXUBRxjZg/ENleLc9zeZzO708z+FI3TvAzYEfiPWBxN/VuIur4+AXwS6JQ0U9LMaHPh+1Zy9J7GmdmfCVXoC6KmPMRZGDv8dVH7r4DdgD3IR5xxJxM++79f1J6LOD0pNdZKQj9tscXAHeMcS1orgfnRVOa4xcAW4J7tD8le1N10OXAw8Eoz+1vRLpXe4/vNbEODQyzJzNYT3qPC2Fse/ha6gGnAhYQPmMIDQmW3DjiQnL6nkfj/5vMQ58oy7YXqYpR8xBn3FuA2M7utqD0XcXpSaqwrgRdK6io0RGX9S6JteXYl4RqF1xcaJLUBJwK/MrPNjQ4guhbph4Sq4zgzu7FMnHtJOix23M5AD018jyXtRrgmaXXUlIe/hb8AR5R4QEhURxASaV7f0yXAMwljd5CPOK+Ino8paj8GeMDMHiEfcRZedwkh8RRXSZCXOJs1R34qPIAdCP/I/0aYUvkq4DagH9ixybG9Lnp8m/A/z/dE3x8W2+diwv+e30lIDJcBm4DnjVOMhdjOAl5Y9HhGtE8LcAPwD+ANhA+D6wldEXuPU5xXELrEjiN8sJ9KmJG1HnjmBPhbKL5OKQ/v6Q+j3/trCRMdPkS4K+r9wK45ilOEi6IfB95NmOhwbvSevi0vccbi/TrhMoXdSmzLRZzj9mZM1Qcwj9D99CTwFPBTii5UbVJcVuZxfWyf6cBXCVObNxH+h3r4OMZ4X4U4z4ztNxv4n+gfz0ZgOfCccYzzY4Sr+ddHr99HmOG2b9F+ef5bOKuordnv6enAXwmz8IajD8pzgT3yFGcUw87AN4E1hK7tvwJvzGGc7cCjhGuTyu3T9Dj91hXOOedyw8eUnHPO5YYnJeecc7nhSck551xueFJyzjmXG56UnHPO5YYnJeecc7nhScm5KiQdFd02+oxmx5Jnks6K3qdDmh1LnKS2KK5rmh2Lq86TkstE9I++0uNtzY6xHEkLohjPa3YsScUSZaXH1upnmhwkPSBpXNZjdI3lt65wWftMmfa/jGsU2boBOIBwNXze3Av8oMy2JHcYnvTMbKukA4DBZsfiqvOk5DJlZmc2O4asWbj77qpmx1FG/2R8z7NmZnn9/bki3n3nxp2k35frWpL0zqjr6eSi9gei20zvIOkrku6XtFnS3ZI+LKnkjcgkvVDSjyU9JGlL9Hy1pNdF288C7o52f0dR99fJ0T5lx5QkLZR0QdH5vy9pvxL7jo25SDpR0s2SNsZu9b1Huncyueh92iSp5N1DJZ0RxXZqrO1lks6TdKekJ6NYb5f0SUmdCV+3Ytdoqb8FSZ2SPiDpKkl/j37PA5J+LemYon2PkmTAXsB+Rb+/86J9yo4pKdxD6ouS7orenwFJyyQdWWLfsb8DSQdF8T0haVDS9ZL+Ocl74irzSslNJB3ANcBc4JfACPAawq2oOym6m6qkdxMWyhwmLL1/T3TsCwgrOl9GWOF5Z+ADhDudxpfo/2ulYCS9kHAzt8JdcVcRblfxZuBVkl5m4cZ0xf6Np28HcD3wIuAk4DmSDjKzLVXfifR+QLiV/BsIq68XezPhVu6XxNpOJ9yD6UagF5hBuNXGZ4HDJB1jZiMNiHUO8H8J3aa/JnSb7kFYWf0qSf9qZudH+/YTuow/SLgr8ddj5yn13o+RNBv4A+F3dhPwk+i1TwCukXSKmZVKpgcTbpb4e+C/gX2A4wl3nH2umd1d4hiX1Hiu/uqPyfsgtnp3icfbivb9PbC1zHneGZ3n5KL2B6L2XmB6rH13wqrbA0BrrP3ZhA+px4EDSrzO3rGvF0TnPq9MTEdF28+ItbUQ7oJqwIlF+78par8dwqLHUftZUfsTQHesXcCPo22vTfh+F2LqL/OenwmcENt/HmGM6cYS53pRdK5Litq74vHH2r8Q7X98UXvh5zskxXu73d8C4UaEe5XYdyZwJyFJdZb4+7inzGu0RTFcU9T+vaj9W0XtiwiruG8q+jspvOel/j7fF7V/vdn/Fif6o+kB+GNyPCh9e4ntbocR7VtPUtq3xDE/jLYtirUV7sX0gQSx15KUDovaflvmmD9G218cayt8aJ9ZYv+XR9vOTvh+xz8gyz0uKzrm2qh9YVH7d6L2VyZ87bnR/ucWtWeSlKq89keL39fY30fipESorIcI/0GYWeKYQuL9eIn3/PoS+3cSKvftkr4/0j18TMllysxU4nF4Rqd/3MzuK9H+j+g5Pl7ywuj5qoxeu9jzoudry2wvtB9UYtuKEm2lfoYklpd5z2Vmryva9/zo+a2Fhmhs6ETCPbOuju8sacdo/GRFNKY0Go3frIl22StlrIlJOlDSDyT1R2M9Fr32FzN67cWEiuxWC7euL5bq92fhTsyPkv7354r4mJKbSEp9eEDopgNojbXNjJ4fbFAsu0TPD5fZXmifWWJbqZ+j1M+QtcsJY2xvlnSGmY0S7oI7EzjHYuNDkjoI413PJ9wt92LCh+4woevyk4TqIHOSXkIYO2wh3GTuZ4TutFHCfwZ6MnjtrH9/EH6Hjfz9TQmelFwzjAKS1BJ9MMaV+hCoReGDYy/CBIesPRE9715m+x5F+zWdmQ1Kugx4G+EW49fwdNX0/aLdX0tISN8zs3fGN0jam5CUkij8fst91pT6fX+SUMW81Mx+X/TanyQkpXpNuN/fVOHdd64Z1hH+9kp1wSzJ6DVujJ5fkWDfQoWQ5n+5t0bPh5fZXmivOAOsCc6Pnt8qaTfgaODPZnZ70X4LoufLS5zjsBSvty563rt4g6RdYq9T/NprixNSldceId3v7w7CRIaDJO1cYvsR0XPefn+Tnicl1ww3Rc/vijdKOhp4fUav8S3CB9WZkhYVb5QUT4gD0fO8FOf/LaECO1zSq4vO/QbgxYSZYn9ME/Q4+C1hFYjXEqbFt/F0ooq7L3o+PN4YXX/1haQvZmbrCO/ToZIWxs7TRpj2Xaob7j5gjqTuotc+FXhZmZd6HJib9PqpaAzoIkI33meLXmd/4P3AFuDCJOdz2fHuO9cM3wM+BHxS0kGED+9FwFLgCsI1H3Uxs79J+gDwDeAvkn4GrAb+iXCd0gBhNhVm9oSkFcARki4kTPUeBX5aooIonH9U0lsJ1yldLumnQF/0cxxHmKb+FoumZjVQl6QzK2z/qpk9WfjGzEzSD4BPE661GSZ8OBf7GSF5fVTSc4DbCNfjHAv8nDA5IqkvA98F/ijpUsKH/RGE/xT/jTDpIO5rhORzg6QfE97LgwlT1y+n9N/HcsKkhGWSfhe9xq1m9osKcX0EOAQ4TdLBwG94+jqlHYH3mNn9KX5Ol4VmT//zx+R4EE1DTrH/gYSZcU8BG4DrgJdSeUp4uSm/201Fjm17CSHRrSV8UD0Uve5rivZ7JuHDdoCQkMZioMSU8NhxBxCmpD9M+IB/GLgA2D9lnBWnTpfYP8mUcAOeUeLY+bGf8YoKrzEP+FH0ng0BK4EPE6qbUtf9VPr5TiF0mW0mzPT7DjCbMlPCCRfK/in6+1hHmBl4SIW/jx0Jie9BwoSDsfeSMtcpRdtmEZLmPVFshdc6qsJ7vt3fQbW/UX8kfyh6M51zzrmm8zEl55xzueFJyTnnXG54UnLOOZcbnpScc87lhicl55xzueFJyTnnXG54UnLOOZcbnpScc87lhicl55xzufH/Aa5KMK3on2WtAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualize loss across function evaluations\n",
    "fig = plt.figure(figsize=(6, 4))\n",
    "plt.plot(qae.train_history, 'ko-', markersize=4, linewidth=1)\n",
    "plt.title(\"Training Loss\", fontsize=16)\n",
    "plt.xlabel(\"Function Evaluation\",fontsize=20)\n",
    "plt.ylabel(\"Loss Value\", fontsize=20)\n",
    "plt.xticks(fontsize=16)\n",
    "plt.yticks(fontsize=16)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Testing\n",
    "\n",
    "Now test the optimized network against the rest of the data set (i.e. use the optimized parameters to try to compress then recover each test data point)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter   73 Mean Loss: -1.0000000\n",
      "Mean loss for test data: -1.0\n"
     ]
    }
   ],
   "source": [
    "avg_loss_test = qae.predict()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}