Solution/期末專題實作(三)/電商網頁銷售改進效果之AB Test分析_Solution.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "ABTestExercise_作業解答.ipynb",
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CUUk4M9ESPiL"
      },
      "source": [
        "# **[作業]電商如何以A/B Test 驗證新網頁設計有效**\n",
        "## **Can eCommerce UX change boost the conversion rate from 0.13 to 0.15?**\n",
        "知識點:\n",
        "\n",
        "*   effect size\n",
        "*   sample size for A/B test \n",
        "*   type I error = 0.05 and Power= 0.8\n",
        "*   z-score, confidence interval\n",
        "\n",
        "參考：A/B testing: A step-by-step guide in Python by Renato Fillinich @ medium.com\n",
        "\n",
        "數據 : ab_data.csv from Kaggle"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_CFV1buAeAMz"
      },
      "source": [
        "# **[作業目標]**\n",
        "\n",
        "1.   了解Binomial分布，以及用常態分布求統計解的方法\n",
        "2.   判讀A/B Test 結果\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "19cuAxsOeI3k"
      },
      "source": [
        "# **[作業重點]**\n",
        "\n",
        "\n",
        "\n",
        "1.   如何決定最小樣本數\n",
        "2.   如何以Z值，p-Value和信賴區間(Confidence Interval)判斷A/B結果是否顯著\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "b5pbmX1PCiUD",
        "outputId": "d05c1562-4a67-4966-e806-8c38dd234599"
      },
      "source": [
        "# Packages imports\n",
        "# \n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import scipy.stats as stats\n",
        "import statsmodels.stats.api as sms\n",
        "import matplotlib as mpl\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "from math import ceil\n",
        "\n",
        "%matplotlib inline\n",
        "\n",
        "# Some plot styling preferences\n",
        "plt.style.use('seaborn-whitegrid')\n",
        "font = {'family' : 'Helvetica',\n",
        "        'weight' : 'bold',\n",
        "        'size'   : 14}\n",
        "\n",
        "mpl.rc('font', **font)\n",
        "\n",
        "#求樣本大小\n",
        "effect_size = sms.proportion_effectsize(0.12, 0.11)    # Calculating effect size based on our expected rates\n",
        "\n",
        "required_n = sms.NormalIndPower().solve_power(\n",
        "    effect_size, \n",
        "    power=0.8, \n",
        "    alpha=0.05, \n",
        "    ratio=1\n",
        "    )                                                  # Calculating sample size needed\n",
        "\n",
        "required_n = ceil(required_n)                          # Rounding up to next whole number                          \n",
        "\n",
        "print(required_n)"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "15970\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 198
        },
        "id": "NzCIP30MEIWw",
        "outputId": "b15e4328-457c-48db-c36d-ae4a180cd359"
      },
      "source": [
        "#展示實驗資料\n",
        "df = pd.read_csv('./sample_data/ab_data.csv')\n",
        "\n",
        "df.head()"
      ],
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>user_id</th>\n",
              "      <th>timestamp</th>\n",
              "      <th>group</th>\n",
              "      <th>landing_page</th>\n",
              "      <th>converted</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>851104</td>\n",
              "      <td>2017-01-21 22:11:48.556739</td>\n",
              "      <td>control</td>\n",
              "      <td>old_page</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>804228</td>\n",
              "      <td>2017-01-12 08:01:45.159739</td>\n",
              "      <td>control</td>\n",
              "      <td>old_page</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>661590</td>\n",
              "      <td>2017-01-11 16:55:06.154213</td>\n",
              "      <td>treatment</td>\n",
              "      <td>new_page</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>853541</td>\n",
              "      <td>2017-01-08 18:28:03.143765</td>\n",
              "      <td>treatment</td>\n",
              "      <td>new_page</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>864975</td>\n",
              "      <td>2017-01-21 01:52:26.210827</td>\n",
              "      <td>control</td>\n",
              "      <td>old_page</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   user_id                   timestamp      group landing_page  converted\n",
              "0   851104  2017-01-21 22:11:48.556739    control     old_page          0\n",
              "1   804228  2017-01-12 08:01:45.159739    control     old_page          0\n",
              "2   661590  2017-01-11 16:55:06.154213  treatment     new_page          0\n",
              "3   853541  2017-01-08 18:28:03.143765  treatment     new_page          0\n",
              "4   864975  2017-01-21 01:52:26.210827    control     old_page          1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 345
        },
        "id": "i1FXBm0sEMz4",
        "outputId": "32ce4c49-e608-40de-e479-24001f52d08f"
      },
      "source": [
        "df.info()\n",
        "\n",
        "# To make sure all the control group are seeing the old page and viceversa\n",
        "# 用 crosstab 將 landing_page 當作 column，group 當作 row\n",
        "pd.crosstab(df['group'], df['landing_page'])"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 294478 entries, 0 to 294477\n",
            "Data columns (total 5 columns):\n",
            " #   Column        Non-Null Count   Dtype \n",
            "---  ------        --------------   ----- \n",
            " 0   user_id       294478 non-null  int64 \n",
            " 1   timestamp     294478 non-null  object\n",
            " 2   group         294478 non-null  object\n",
            " 3   landing_page  294478 non-null  object\n",
            " 4   converted     294478 non-null  int64 \n",
            "dtypes: int64(2), object(3)\n",
            "memory usage: 11.2+ MB\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>landing_page</th>\n",
              "      <th>new_page</th>\n",
              "      <th>old_page</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>group</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>control</th>\n",
              "      <td>1928</td>\n",
              "      <td>145274</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>treatment</th>\n",
              "      <td>145311</td>\n",
              "      <td>1965</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "landing_page  new_page  old_page\n",
              "group                           \n",
              "control           1928    145274\n",
              "treatment       145311      1965"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "smK1d9OoEQ4c",
        "outputId": "de2d3035-056c-4011-fe8e-e03c6f20cc08"
      },
      "source": [
        "#偵測重複出現使用者\n",
        "session_counts = df['user_id'].value_counts(ascending=False)\n",
        "multi_users = session_counts[session_counts > 1].count()\n",
        "\n",
        "print(f'There are {multi_users} users that appear multiple times in the dataset')"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "There are 3894 users that appear multiple times in the dataset\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1Iu-jUPxEUFX",
        "outputId": "4bf86fb4-f8d2-4a80-a057-f06d4be478ef"
      },
      "source": [
        "#除去重複出現使用者\n",
        "users_to_drop = session_counts[session_counts > 1].index\n",
        "\n",
        "df = df[~df['user_id'].isin(users_to_drop)]\n",
        "print(f'The updated dataset now has {df.shape[0]} entries')"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "The updated dataset now has 286690 entries\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 408
        },
        "id": "gOh_R80CEYVs",
        "outputId": "3e440572-63b5-4e15-9ca5-155083bf7f6c"
      },
      "source": [
        "#選取 控制組和實驗組各半 4720 * 2 = 9440\n",
        "control_sample = df[df['group'] == 'control'].sample(n=required_n, random_state=22)\n",
        "treatment_sample = df[df['group'] == 'treatment'].sample(n=required_n, random_state=22)\n",
        "\n",
        "ab_test = pd.concat([control_sample, treatment_sample], axis=0)\n",
        "ab_test.reset_index(drop=True, inplace=True)\n",
        "ab_test"
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>user_id</th>\n",
              "      <th>timestamp</th>\n",
              "      <th>group</th>\n",
              "      <th>landing_page</th>\n",
              "      <th>converted</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>763854</td>\n",
              "      <td>2017-01-21 03:43:17.188315</td>\n",
              "      <td>control</td>\n",
              "      <td>old_page</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>690555</td>\n",
              "      <td>2017-01-18 06:38:13.079449</td>\n",
              "      <td>control</td>\n",
              "      <td>old_page</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>861520</td>\n",
              "      <td>2017-01-06 21:13:40.044766</td>\n",
              "      <td>control</td>\n",
              "      <td>old_page</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>630778</td>\n",
              "      <td>2017-01-05 16:42:36.995204</td>\n",
              "      <td>control</td>\n",
              "      <td>old_page</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>656634</td>\n",
              "      <td>2017-01-04 15:31:21.676130</td>\n",
              "      <td>control</td>\n",
              "      <td>old_page</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>31935</th>\n",
              "      <td>780954</td>\n",
              "      <td>2017-01-19 07:49:58.295232</td>\n",
              "      <td>treatment</td>\n",
              "      <td>new_page</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>31936</th>\n",
              "      <td>700881</td>\n",
              "      <td>2017-01-13 12:31:37.243352</td>\n",
              "      <td>treatment</td>\n",
              "      <td>new_page</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>31937</th>\n",
              "      <td>829626</td>\n",
              "      <td>2017-01-14 09:07:57.784950</td>\n",
              "      <td>treatment</td>\n",
              "      <td>new_page</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>31938</th>\n",
              "      <td>773197</td>\n",
              "      <td>2017-01-05 00:23:56.177295</td>\n",
              "      <td>treatment</td>\n",
              "      <td>new_page</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>31939</th>\n",
              "      <td>720502</td>\n",
              "      <td>2017-01-15 15:54:12.049241</td>\n",
              "      <td>treatment</td>\n",
              "      <td>new_page</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>31940 rows × 5 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "       user_id                   timestamp      group landing_page  converted\n",
              "0       763854  2017-01-21 03:43:17.188315    control     old_page          0\n",
              "1       690555  2017-01-18 06:38:13.079449    control     old_page          0\n",
              "2       861520  2017-01-06 21:13:40.044766    control     old_page          0\n",
              "3       630778  2017-01-05 16:42:36.995204    control     old_page          0\n",
              "4       656634  2017-01-04 15:31:21.676130    control     old_page          0\n",
              "...        ...                         ...        ...          ...        ...\n",
              "31935   780954  2017-01-19 07:49:58.295232  treatment     new_page          0\n",
              "31936   700881  2017-01-13 12:31:37.243352  treatment     new_page          0\n",
              "31937   829626  2017-01-14 09:07:57.784950  treatment     new_page          0\n",
              "31938   773197  2017-01-05 00:23:56.177295  treatment     new_page          0\n",
              "31939   720502  2017-01-15 15:54:12.049241  treatment     new_page          0\n",
              "\n",
              "[31940 rows x 5 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "I0DU6C8uEbbZ",
        "outputId": "fac2e170-6d00-4555-db36-84bb69dde08e"
      },
      "source": [
        "ab_test.info()"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 31940 entries, 0 to 31939\n",
            "Data columns (total 5 columns):\n",
            " #   Column        Non-Null Count  Dtype \n",
            "---  ------        --------------  ----- \n",
            " 0   user_id       31940 non-null  int64 \n",
            " 1   timestamp     31940 non-null  object\n",
            " 2   group         31940 non-null  object\n",
            " 3   landing_page  31940 non-null  object\n",
            " 4   converted     31940 non-null  int64 \n",
            "dtypes: int64(2), object(3)\n",
            "memory usage: 1.2+ MB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "viU4ZRx6W3Yh",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "616f13cb-60f2-41fe-9d85-c3e73152b9f9"
      },
      "source": [
        "#確認 ab_test 控制組實驗組各半\n",
        "ab_test['group'].value_counts()"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "treatment    15970\n",
              "control      15970\n",
              "Name: group, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 101
        },
        "id": "ewFAGIwMEfJJ",
        "outputId": "933e9583-ead7-4eec-bfe7-3c3856c00e81"
      },
      "source": [
        "#計算conversion rate 平均值，標準差，標準誤\n",
        "conversion_rates = ab_test.groupby('group')['converted']\n",
        "\n",
        "std_p = lambda x: np.std(x, ddof=0)              # Std. deviation of the proportion\n",
        "se_p = lambda x: stats.sem(x, ddof=0)            # Std. error of the proportion (std / sqrt(n))\n",
        "\n",
        "conversion_rates = conversion_rates.agg([np.mean, std_p, se_p])\n",
        "conversion_rates.columns = ['conversion_rate', 'std_deviation', 'std_error']\n",
        "\n",
        "\n",
        "conversion_rates.style.format('{:.3f}')"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<style  type=\"text/css\" >\n",
              "</style><table id=\"T_e466920c_352b_11eb_b371_0242ac1c0002\" ><thead>    <tr>        <th class=\"blank level0\" ></th>        <th class=\"col_heading level0 col0\" >conversion_rate</th>        <th class=\"col_heading level0 col1\" >std_deviation</th>        <th class=\"col_heading level0 col2\" >std_error</th>    </tr>    <tr>        <th class=\"index_name level0\" >group</th>        <th class=\"blank\" ></th>        <th class=\"blank\" ></th>        <th class=\"blank\" ></th>    </tr></thead><tbody>\n",
              "                <tr>\n",
              "                        <th id=\"T_e466920c_352b_11eb_b371_0242ac1c0002level0_row0\" class=\"row_heading level0 row0\" >control</th>\n",
              "                        <td id=\"T_e466920c_352b_11eb_b371_0242ac1c0002row0_col0\" class=\"data row0 col0\" >0.121</td>\n",
              "                        <td id=\"T_e466920c_352b_11eb_b371_0242ac1c0002row0_col1\" class=\"data row0 col1\" >0.326</td>\n",
              "                        <td id=\"T_e466920c_352b_11eb_b371_0242ac1c0002row0_col2\" class=\"data row0 col2\" >0.003</td>\n",
              "            </tr>\n",
              "            <tr>\n",
              "                        <th id=\"T_e466920c_352b_11eb_b371_0242ac1c0002level0_row1\" class=\"row_heading level0 row1\" >treatment</th>\n",
              "                        <td id=\"T_e466920c_352b_11eb_b371_0242ac1c0002row1_col0\" class=\"data row1 col0\" >0.121</td>\n",
              "                        <td id=\"T_e466920c_352b_11eb_b371_0242ac1c0002row1_col1\" class=\"data row1 col1\" >0.326</td>\n",
              "                        <td id=\"T_e466920c_352b_11eb_b371_0242ac1c0002row1_col2\" class=\"data row1 col2\" >0.003</td>\n",
              "            </tr>\n",
              "    </tbody></table>"
            ],
            "text/plain": [
              "<pandas.io.formats.style.Styler at 0x7f49b1452320>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 438
        },
        "id": "IVM21KkUEitI",
        "outputId": "b1a78e86-86db-4509-e351-9be987d47a21"
      },
      "source": [
        "#繪出 conversion rate 棒狀圖\n",
        "plt.figure(figsize=(8,6))\n",
        "\n",
        "sns.barplot(x=ab_test['group'], y=ab_test['converted'], ci=False)\n",
        "\n",
        "plt.ylim(0, 0.17)\n",
        "plt.title('Conversion rate by group', pad=20)\n",
        "plt.xlabel('Group', labelpad=15)\n",
        "plt.ylabel('Converted (proportion)', labelpad=15);"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAg8AAAGlCAYAAAB9ShW3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeVwV9f7H8RcuLKIJoqKQu4EmuGFuXdxBJHPpZi5huVzTcrtCZqXRzcxubpnLLdMolxIlTXMF0zS3CjG95pYRioKggmCuEJzfHz44v3ti8ZzkyBHfz8fDR8PMfL/z+SIn3s58Z8bOYDAYEBERETFTmZIuQERERO4vCg8iIiJiEYUHERERsYjCg4iIiFhE4UFEREQsovAgIiIiFlF4EBGjc+fO4e3tzdq1a0u6FJvj7e1NeHh4SZchYhPKlXQBIrYgPT2diIgIvv32W5KSkjAYDNSuXZuOHTsyePBg3N3dS7pEMUNqaiqrV6+mW7duNG7cuKTLESm1FB7kgXf06FFGjBjB77//Ts+ePXn22WcpU6YMJ0+eJCoqim3bthEdHV3SZd4Tnp6e/Pe//6Vcufvzfw0XLlxgwYIFeHp6KjyIWNH9+X8IkWLy+++/M3r0aADWrl3LI488YrI9NDSUxYsXl0RpxcZgMHDr1i0cHR3vuK+dnR0ODg73oCrzXL9+nQoVKpR0GfcNfb/kXtGcB3mgRUZGcv78eSZNmpQvOABUqlSJ0NBQk3XR0dE89dRTNG3alDZt2hAaGkpycrLJPq+++iq+vr6kpqby0ksv0aJFC9q2bct7771HTk4OANnZ2bRu3ZqJEyfmO+7Nmzdp2bIlr776qnFdVlYWCxYsIDAwEB8fH/z9/Zk+fTo3btwwaZt3bX7z5s08+eST+Pr6snnzZgD279/Ps88+y2OPPUazZs3o1q0bU6dONbYtbM7DiRMnGDFiBC1btqR58+YMHjyYAwcOmOyzdu1avL29+fHHH3n33Xdp27YtzZs3Z/To0aSnpxf6d5Bn/vz5eHt788svvzBx4kRat25Nz549AUhKSuKtt96iR48eNGvWjFatWjFy5EhOnjxpbP/DDz/w9NNPA/Daa6/h7e2Nt7c38+fPN+6TkJDA+PHjadOmDb6+vvTp04etW7fesbb/tXnzZoKDg/H19aVXr1589913xm2nT5/G29ubTz/9NF+7EydO4O3tzRdffFFk/5cvX2bixIm0bNmSVq1aMWnSJI4fP57v7yXvZ+zcuXOMGjWKli1bMnLkSABycnL48MMPCQgIwMfHh06dOjFjxgxu3rxpcqw/f3/ydOnSxeRnL+/v9vvvv2fq1Km0adOGFi1aMH78eC5dumTeN05KFZ15kAfajh07cHBwoEePHmbtv379el555RWaNGlCaGgoly9fZtmyZcTFxfHVV19RpUoV474Gg4F//OMf+Pr68sorr7B//34iIiKoVasWgwYNonz58gQEBLBlyxZu3bpl8i/+Xbt2ce3aNYKDg419jR49mtjYWPr160fDhg2Jj4/niy++4Ndff+WTTz7Bzs7O2P7AgQNER0cTEhJC1apVqV+/Pr/++isvvPACXl5ejBkzBicnJxITE9mzZ0+RY46Pj2fQoEFUqFCB4cOH4+DgQFRUFEOGDOHTTz/lscceM9l/+vTpVK5cmTFjxpCUlMTSpUuZOnUqc+fONet7PGHCBB5++GHGjx9PdnY2AEeOHOHAgQN0794dDw8PLly4QGRkJIMHD2bjxo1Ur16dBg0aMG7cOObNm0f//v3x8/MDbv+CzBvHgAEDqFq1KsOHD8fZ2Zlt27Yxfvx4ZsyYQe/eve9Y208//cTWrVsZPHgwzs7OrFq1ihdffJGlS5fSqlUr6tatS4sWLfj6668ZOnSoSduvv/6a8uXLG/9OC5Kbm8uLL77I4cOHGThwIA0aNGDHjh1MmjSpwP0NBgPDhw83/oyVLVsWgPDwcL788ksCAwMZMmQIP//8M5988gmnTp3i448/NvlZscQ777xDhQoVGD16NMnJyaxYsYKEhAS+/PJL7O3t/1Kfcp8yiDzAHnvsMUOvXr3M2jcrK8vQvn17Q48ePQw3btwwrv/+++8NXl5ehn//+9/GdZMmTTJ4eXkZ5s+fb9JHnz59DH379jV+vXfvXoOXl5chJibGZL9x48YZ2rRpY8jOzjYYDAbD119/bfD29jb88MMPJvutX7/e4OXlZdi9e7dxnZeXl8Hb29tw7Ngxk30/++wzg5eXlyEtLa3QMZ49e9bg5eVlWLNmjXHd6NGjDU2aNDEkJCQY16WlpRlat25tMpY1a9YYvLy8DM8//7whNzfXuH769OmGxo0bG65cuVLocQ0Gg2HevHkGLy8vw5gxY/Jt+9/vd57ExESDj4+PYeHChcZ1//3vf/PVn2fo0KGG4ODgfH0NHTrU4O/vb1JzQby8vAxeXl6GgwcPGtelp6cbHnvsMcOAAQOM67744guDl5eX4dSpU8Z1OTk5hg4dOhhGjx5d5DGio6MNXl5ehk8++cSk7XPPPZdvXHk/Y9OnTzfp4/jx4wYvLy/Dq6++arI+7/u7Y8cOkzHNmzcvXx2dO3c2TJo0yfh13t9tz549Dbdu3TKuj4qKMnh5eRlWr15d5Lik9NFlC3mgXb16FWdnZ7P2/fnnn7l06RIDBw40mT/Qpk0bmjRpws6dO/O1eeaZZ0y+9vPz49y5cyZtq1atarysALevW+/atYvAwEDjxMUtW7ZQt25dGjZsSHp6uvFP69atsbOz44cffjA5TosWLfJNGKxUqRIA27dvJzc316wx5+TksGfPHjp37kzdunWN66tUqcJTTz3F0aNH8522fvrpp03+ZduqVStycnJISkoy65gDBw7Mt+5/v983btzg8uXLODs7U69ePY4ePXrHPjMyMti3bx89evTg+vXrJt9Df39/UlNTSUhIuGM/Pj4+tGjRwvi1q6srPXv25ODBg2RmZgIQHByMvb0969evN+73ww8/kJKScsezG7t376Zs2bL079/fuK5MmTI8++yzhbYZNGiQyde7du0CYMiQISbrhwwZQtmyZQv8OTXXgAEDTM4w9OnTh4ceeohvv/32L/cp9yddtpAHWsWKFbl27ZpZ++bNa6hXr16+bQ0aNMh3R0b58uWpXr26ybrKlSsbf8kAlC1blu7du7N27VrjZLcdO3Zw48YNnnjiCeN+p0+fJiEhgXbt2hVYW1pamsnXtWvXzrdPcHAwX375JVOmTGHWrFm0bduWbt260aNHj0LvrkhPT+fGjRsFjrl+/frA7fkIVatWNa6vWbOmyX4PPfQQAFeuXCnwGH9Wq1atfOtu3brFBx98wNdff83FixdNtrm4uNyxz8TERAwGA/Pnzy/wGj/cHmvemArzvwHqz+uSk5OpXLkylStXpkuXLmzcuJHQ0FDs7Oz4+uuvcXFxoWPHjkX2n5ycjJubW75AW9DfJ9wOFp6enibrkpKSsLOzy/d3VqlSJapVq2Z2iCtInTp1TL4uV64cDz/88F31KfcnhQd5oNWvX59jx46RlZVV7Ndszb2uHBwczOeff87OnTsJDg5m8+bNVK9e3WQuQW5uLg0bNmTy5MkF9vHnkFLQHROOjo6sWLGC2NhYdu3axZ49e3j55Zf59NNP+eKLL8y6G8Mcedfd/8xgMJjVvqA63n77bdasWUNISAgtWrTgoYceokyZMkyfPt2sfvPOtAwZMqTQX+AFTZj9q/ImYsbGxtKsWTNiYmLo2bNnsf+MlStXziq31eZN6hUpjMKDPNC6dOlinATXq1evIvf18PAAbs/Y/9vf/may7bfffsv3L0Bz+fn5UbNmTTZv3kyHDh3YvXs3AwYMoEyZ/7+qWLt2bY4ePUq7du3+8mQ3uP0v1TZt2tCmTRteeeUVvvjiC9566y1iYmIKHH+VKlVwcnIq8JT+b7/9BvCXx22JrVu30qdPn3zhKTMzE1dXV+PXhX1v8s5mlC1blvbt2//lOk6fPl3ouryfDwB/f3/c3NxYv349ly5d4urVq2ZNyPTw8GD//v1cu3bN5OxDYmKi2TV6enpiMBhISEgwThaF25foLl68SKdOnYzrKleunO+MUFZWVr6zO3nOnDlj8rP/xx9/cO7cuXyTZqX005wHeaANGDAAd3d33nvvPeLj4/Ntv3r1Ku+//z5w+3p31apVWbVqFbdu3TLuc+DAAX7++WeT/ylbws7Ojh49evDdd9+xbt06srKyTC5ZwO2zE5cuXSrwNr+srCyuXr16x+Ncvnw537omTZoAt593UZCyZcvi7+/Pt99+a/ILLCMjg3Xr1hm/J9ZWtmzZfGcYNm7cyIULF0zWOTk5Afkvkbi5udGmTRuioqJITU3N1785t5LC7XkvP/30k/Hry5cvs3HjRlq0aEHlypWN68uVK8eTTz5JdHQ0a9asoU6dOrRs2fKO/f/tb38jJyeHVatWGdfl5uby+eefm1UfYDyzsnTpUpP1S5cuJScnh86dOxvX1apVK98tt6tXry70zENkZCRZWVnGr9etW8eVK1f+8s++3L905kEeaA899BALFy7khRdeoG/fvvTs2RNfX1/KlCnDqVOn2LhxI5UrV2bChAmUL1+eiRMnMmnSJAYNGkSvXr1IT09n+fLluLu7M2LEiL9cR3BwMBEREcyZMwdPT0+aN29usr1Xr15s3bqVqVOnEhsbi5+fn/Ffl1u2bOGDDz6gTZs2RR7jP//5Dz/++COdOnXC09OTzMxMIiMjqVChQpH/8//nP//Jnj17GDRoEIMGDcLBwYHVq1fz+++/mzwLwJq6dOnCunXrqFixIo888gjHjx9ny5Yt+eZH1K5dm8qVK7Ny5UoqVKiAs7MzjzzyCF5eXvzrX/9i4MCB9OrVi379+lG7dm3S0tI4fPgw8fHxbNu27Y51eHl5MWrUKEJCQoy3al67do2wsLB8+/bp04fPPvuMPXv2MHbsWLPG2a1bN5o2bcrMmTM5d+6c8VbNvHky5px1atSoEU8//TRffvklV69epU2bNhw7dow1a9bg7+9vctmmX79+vPnmm4wdO5b27dtz4sQJ9uzZY3I258+ee+45nnjiCZKSklixYgVeXl706dPHrPFJ6aHwIA88X19fNm7caHy3xaZNmzAYDNSpU4f+/fszePBg4759+vTBycmJRYsWMWvWLJycnOjYsSMvv/yyyTMe/koNderU4cyZM/lmz8Ptyw0LFixg6dKlrFu3ju3bt+Po6Gh8ZsT/np4uTNeuXTl//jxfffUV6enpuLi40KJFC0aPHl3kpYcGDRqwcuVKZs+ezeLFizEYDPj4+DBt2rR7drp68uTJlCtXjs2bN3P9+nV8fHxYvHgxM2fONNmvfPnyzJgxg9mzZzN16lSys7MZM2YMXl5e1K9fnzVr1rBw4ULWrVtHRkYGrq6uNGrUiPHjx5tVR97DvubPn8/Zs2epV68eCxcuLPD70LhxY7y9vTl58uQdL4nlKVu2LIsWLeKdd95h3bp1lClThoCAAF566SVjcDPH1KlTefjhh1mzZg07duzAzc2NYcOGMW7cOJMA8swzz3Du3Dm+/PJLdu/ejZ+fH59++mm+OzXyTJ48mejoaBYsWEBWVhZdunRhypQpesbDA8jOYO4sJhERscjf//53ypcvT2Rk5F3188033zB69Gi++OIL48Ov7qW1a9fy2muvsWrVqnxnxeTBpDkPIiJWcPz4cX7++Wf69u1rUbs/P0I6JyeH5cuXU7FiReMcFZGSpssWIiLF6JdffuHo0aN89tlnuLm5mX3JIs/bb7/NzZs3adGiBVlZWcTExPDTTz8RGhpabLfTitwthQcRkWIUHR3NwoULqVOnDnPmzDHeAWKutm3b8umnn7Jz505u3bpFnTp1eOONNwgJCbFSxSKW05wHERERsYjmPIiIiIhFFB5ERETEIgoPIiIiYhGFBxEREbGIwoOIiIhYROFBRERELKLwICIiIhZReBARERGLKDyIiIiIRRQeRERExCIKDyIiImIRhQcRERGxiMKDiIiIWEThQURERCyi8CAiIiIWKVfSBdwv4uLiSroEERGRe8rPz6/A9QoPFijsmygiIlLaFPWPZl22EBEREYsoPIiIiIhFFB5ERETEIgoPIiIiYhGFBxEREbGIwoOIiIhYROFBRERELKLwICIiIhZReBARERGLKDyIiIiIRRQeRERExCIKDyIiImIRhQcRERGxiE2Hh02bNtG3b1+aNm1K69atGTduHGfOnCmyTUxMDEOHDqV169Z4e3vj7e3N2rVrC9w3Li6OF154gdatW+Pj40PHjh2ZOHGiNYYiIiJSathseIiKiiI0NJRjx45RrVo1cnJyiI6OZuDAgVy8eLHQdrGxscTGxuLq6lpk/9HR0QwePJhdu3aRm5tLw4YNsbe3Z8eOHcU9FBERkVLFJsNDVlYWs2fPBqB79+5s376dzZs34+zsTFpaGosWLSq07ciRIzl48CDz588vdJ8bN24QHh5OTk4Offr0Ye/evaxbt45t27axa9euYh+PiIhIaWKT4eHIkSNcvnwZgMDAQADc3d1p3rw5ALt37y60bdWqVbG3ty+y/71795KRkQHAH3/8QWBgIK1atWLo0KGcO3euOIYgIiJSapUr6QIKkpKSYlx2c3PLt5ycnHxX/SckJBiXN27cSP369bly5Qr79u1j8ODBbNy4EXd393ztjh8/flfHFRERKQ1sMjwUxmAwFEs/OTk5xuVnnnmGt99+m/j4eIKDg7ly5Qrr16/nhRdeyNeucePGxXJ8ERERWxcXF1foNpu8bFGjRg3jclpamnE5PT0dAA8Pj7vq/3/PKvj4+ADQoEEDKlSoAKBLFyIiIkWwyfDg6+uLi4sLcPvWS4DU1FQOHToEgL+/PwBBQUEEBQWxYsUKi/pv164dZcuWBeDo0aMA/Pbbb1y/fh2AevXq3f0gRERESimbDA/29vaEhoYCt2+p7Nq1K8HBwVy7dg1XV1dGjBgB3J67kJCQYJxcCbBs2TICAgJMLjvMmjWLgIAAZs6cCdw+s/Hcc88BsGrVKoKDg3n66aeB22c1/v73v9+TcYqIiNyPbHbOQ//+/XFyciIiIoL4+HgcHBwICAggLCyswMmMeTIzM0lMTDRZl5aWZvyTZ9KkSVSvXp3Vq1eTmJhI9erVCQ4OZsKECTz00ENWG5eIiMj9zs5QXLMQS7m4uDj8/PxKugwREZF7oqjfezZ52UJERERsl8KDiIiIWEThQURERCyi8CAiIiIWUXgQERERiyg8iIiIiEUUHkRERMQiCg8iIiJiEYUHERERsYjCg4iIiFhE4UFEREQsovAgIiIiFlF4EBEREYsoPIiIiIhFFB5ERETEIgoPIiIiYhGFBxEREbGIwoOIiIhYROFBRERELKLwICIiIhZReBARERGLKDyIiIiIRRQeRERExCIKDyIiImIRhQcRERGxiMKDiIiIWEThQURERCxi0+Fh06ZN9O3bl6ZNm9K6dWvGjRvHmTNnimwTExPD0KFDad26Nd7e3nh7e7N27dpC909JSTHZ99tvvy3uYYiIiJQqNhseoqKiCA0N5dixY1SrVo2cnByio6MZOHAgFy9eLLRdbGwssbGxuLq63vEYubm5vPLKK2RmZhZn6SIiIqWaTYaHrKwsZs+eDUD37t3Zvn07mzdvxtnZmbS0NBYtWlRo25EjR3Lw4EHmz59/x+MsWbKEH374gR49ehRb7SIiIqWdTYaHI0eOcPnyZQACAwMBcHd3p3nz5gDs3r270LZVq1bF3t7+jsc4evQo8+bNo3PnzgwcOLAYqhYREXkw2GR4SElJMS67ubnlW05OTr6r/m/cuEFYWBguLi5Mnz79rvoSERF50JQr6QIsYTAYiqWf2bNnc/r0aT755BOqVKlidrvjx48Xy/FFRETuZzYZHmrUqGFcTktLMy6np6cD4OHhcVf9nzx5EoAxY8YAkJOTY9w2fvx4unXrxpw5c/K1a9y48V0dV0RE5H4RFxdX6DabvGzh6+uLi4sLcPvWS4DU1FQOHToEgL+/PwBBQUEEBQWxYsUKi49hMBi4fv06169f59atW8b1t27d4ubNm3c7BBERkVLLJs882NvbExoaSnh4ONHR0XTt2pWMjAyuXbuGq6srI0aMACAhIQHAOLkSYNmyZSxfvpzs7GzjulmzZvHhhx8SGBjIxIkTWb58ucnxfvjhB5577jkAPvroIzp37mztIYqIiNy3bDI8APTv3x8nJyciIiKIj4/HwcGBgIAAwsLCcHd3L7RdZmYmiYmJJuvS0tKMf0REROTu2BmKaxZiKRcXF4efn19JlyEiInJPFPV7zybnPIiIiIjtUngQERERiyg8iIiIiEUUHkRERMQiCg8iIiJiEYUHERERsYjCg4iIiFhE4UFEREQsovAgIiIiFlF4EBEREYsoPIiIiIhFFB5ERETEIgoPIiIiYhGFBxEREbGIwoOIiIhYROFBRERELKLwICIiIhZReBARERGLKDyIiIiIRRQeRERExCIKDyIiImKRcnfa4cqVK8TExBAbG0tSUhI3b96kSpUqPProo3To0IGWLVveizpFRETERhQaHlJTU5k3bx4bNmygevXqNG3aFB8fHxwcHMjMzCQ2NpbPPvsMDw8PxowZQ3Bw8L2sW0REREpIoeGhT58+9OnThzVr1vDII48UuM/Nmzf55ptv+PTTTzl//jzDhw+3WqEiIiJiGwoND5s2baJKlSpFNnZ0dKRnz5707NmT9PT0Yi9OREREbE+hEybvFBzudn8RERG5P91xwmSelJQUYmNjSUtLw2AwmGwbOnRosRcmIiIitsms8PD111/z+uuvU65cuXxnGOzs7BQeREREHiBmhYd58+YxbNgwxo8fT9myZa1dk9GmTZtYsmQJ8fHxODo60rZtW8LCwqhTp06hbWJiYli5ciVHjx4lMzMTgHfffZennnrKuE9qair/+c9/OHjwIKmpqdy6dYuaNWvyxBNPMGzYMJydna0+NhERkfuVWQ+JSktLo1+/fvc0OERFRREaGsqxY8eoVq0aOTk5REdHM3DgQC5evFhou9jYWGJjY3F1dS10n9OnTxMZGUliYiIeHh44OjqSkJDAggULCA0NtcZwRERESg2zzjx06NCBw4cPU6tWLWvXA0BWVhazZ88GoHv37sybN4/U1FR69OhBWloaixYtYsqUKQW2HTlyJBMnTuT06dM8+eSTBe5TuXJl3n77bXr37o2DgwM3b97kueee4/Dhw+zcuZPMzEwqV65stfH9L7+Jy+7JcUSsLW7mcyVdgojcI2aFh8cff5xZs2Zx6tQpvL29KVfOtFlgYGCxFnXkyBEuX75s0re7uzvNmzdn79697N69u9C2VatWvWP/jRo1olGjRsavHR0dadmyJYcPH6ZMmTL5xicipVPiVN+SLkHkrtUOP3LPj2nWb8nw8HAAFi1alG+bnZ0dx48fL9aiUlJSjMtubm75lpOTk4v1eBcuXGDjxo0A9OrVq9A5D8U9TpHS5H78fGh2k5QGJfHZMys8nDhxwtp1mOXPt4gWh19//ZWRI0dy8eJFWrVqxZtvvlnovo0bNy7240OsFfoUufes8/mwrsSSLkCkGFjrsxcXF1foNpt8q2aNGjWMy2lpacblvKdYenh4FMtxdu/ezYABAzh37hwBAQF88sknVKhQoVj6FhERKa3MDg87d+7k2WefpU2bNrRt25aQkBB27dpllaJ8fX1xcXEBbt96Cbdvrzx06BAA/v7+AAQFBREUFMSKFSssPsbnn3/OyJEj+f333xk+fDjz5s3D0dGxmEYgIiJSepkVHqKiohgzZgy1a9fm5ZdfJiwsjIcffpjRo0fz5ZdfFntR9vb2xlsmo6Oj6dq1K8HBwVy7dg1XV1dGjBgBQEJCAgkJCcbJlQDLli0jICCAF154wbhu1qxZBAQEMHPmTAAOHTrE1KlTycnJwdHRkQMHDjBgwACeeeYZnnnmGS5cuFDsYxIRESktzJrzsHjxYl599VVCQkKM6/r160eTJk1YvHgxTz/9dLEX1r9/f5ycnIiIiCA+Ph4HBwcCAgIICwvD3d290HaZmZkkJppeyUxLSzP+Abh165Zx282bNzl8+LDJ/llZWcU4EhERkdLFrPCQnJxsvFTwvzp06MB7771X7EXl6dWrF7169Sp0+8mTJ/OtGzt2LGPHji2y3zZt2hTYVkRERO7MrMsWHh4e7N27N9/6PXv24OnpWexFiYiIiO0y68zDsGHDmDZtGseOHaNFixYAHDx4kPXr1/PGG29YtUARERGxLWaFhwEDBuDm5kZERATbtm0DoH79+sydO5du3bpZtUARERGxLWY/hzkgIICAgABr1iIiIiL3AZt8SJSIiIjYrkLPPLRs2ZJvvvmGKlWq0KJFC+zs7Art5ODBg1YpTkRERGxPoeHhjTfeoGLFisblosKDiIiIPDgKDQ99+/Y1Lj/11FP3pBgRERGxfWbNeejatavJI6DzXLlyha5duxZ7USIiImK7zAoPSUlJ5Obm5luflZVFampqsRclIiIitqvIWzXz3mgJt9+qWalSJePXOTk57N+/X0+YFBERecAUGR7GjRtnXJ48ebJpw3Ll8PT05NVXX7VOZSIiImKTigwPJ06cAKBLly6sWbMGV1fXe1KUiIiI2K47znnIzs7Gzc2twAmTIiIi8uC5Y3goX748586d03MeREREBDDzbos+ffqwevVqa9ciIiIi9wGzXox148YNNmzYwL59+2jSpAkVKlQw2T5lyhSrFCciIiK2x6zwEB8fz6OPPgrA2bNnTbbpcoaIiMiDxazwsHz5cmvXISIiIvcJs8JDnlu3bnHmzBns7OyoXbs2Dg4O1qpLREREbJRZ4SE7O5s5c+bw+eefk52djcFgwN7enpCQECZMmED58uWtXaeIiIjYCLPCw6xZs9i0aRNvvfUWfn5+ABw4cIA5c+ZgMBiYNGmSVYsUERER22FWeNi4cSPTp0+nY8eOxnW1a9emSpUqTJkyReFBRETkAWLWcx5+//13atWqlW99rVq1uHLlSrEXJSIiIrbLrPDQqFGjAu+4WLZsGY0bNy72okRERG9njakAACAASURBVMR2mXXZYuLEibzwwgvs27eP5s2bA3Do0CEuXLjA4sWLrVqgiIiI2Bazzjw89thjbN26laCgIK5fv87169cJCgpi69attGrVyto1ioiIiA0x+zkP7u7uTJgwwZq15LNp0yaWLFlCfHw8jo6OtG3blrCwMOrUqVNom5iYGFauXMnRo0fJzMwE4N133+Wpp54y2e/q1avMmzePrVu3kp6ejru7O3369GHUqFG69VRERKQIZoeHCxcusHLlSuLj4wFo0KABAwYMwN3d3SqFRUVFGd+Z8fDDD5ORkUF0dDQHDhxg/fr1VKtWrcB2sbGxxMbG4unpaQwPf5abm8uLL77Ijz/+SPny5Xn44Yc5c+YMCxYs4OzZs8yYMcMqYxIRESkNzLpssXfvXgICAti8eTOOjo44OjqyZcsWAgMD2bNnT7EXlZWVxezZswHo3r0727dvZ/PmzTg7O5OWlsaiRYsKbTty5EgOHjzI/PnzC93nm2++4ccffwRg/vz5bN26lddffx2A9evXc/To0WIcjYiISOliVniYNm0a/fr1Y+vWrcyYMYMZM2awdetW+vXrxzvvvFPsRR05coTLly8DEBgYCNy+bJI3WXP37t2Ftq1atSr29vZF9v/dd98B4OjoaHx2Rd5x7tS/iIjIg86syxZJSUk8++yz+d6g+eyzzxIVFVXsRaWkpBiX3dzc8i0nJyffVf/nz58HwMXFhTJlbuenqlWrGrcX1v/x48fv6rgipdn9+PlwLukCRIpBSXz2zAoPPj4+/PLLL9SrV89k/S+//HJPn/NgMBhKtG/rjDXWCn2K3Hv34zNfEku6AJFiYK3PXlxcXKHbzAoPgwYN4t133+XMmTM0a9YMgMOHD7Ny5UrCwsJM5gg0adLkLsuFGjVqGJfT0tKMy+np6QB4eHjcVf81a9YE4PLly+Tm5lKmTBmT49xt/yIiIqWZWeHh5ZdfBmDOnDmFbgOws7MrltMnvr6+uLi4kJGRQUxMDD179iQ1NZVDhw4B4O/vD0BQUBAAISEhhISEmN2/v78/UVFR3Lp1i127dtG5c2diYmJMtouIiEjBzAoP27dvt3YdJuzt7QkNDSU8PJzo6Gi6du1KRkYG165dw9XVlREjRgCQkJAAYJxcCbcfmb18+XKys7ON62bNmsWHH35IYGAgEydOpFu3bvj5+REXF8fYsWOpVasWp0+fBqBnz57FcvZERESktDIrPHh6elq7jnz69++Pk5MTERERxMfH4+DgQEBAAGFhYUU+WyIzM5PERNMrmWlpacY/AGXLluXjjz9m7ty5REdHc/bsWWrWrEnv3r156aWXrDouERGR+52dwcxZiCdOnCAiIoJff/0VOzs7GjZsyPDhw/Hy8rJ2jTYhLi4OPz+/Yu/Xb+KyYu9TpCTEzXyupEuwWOJU35IuQeSu1Q4/YpV+i/q9Z9ZzHrZv385TTz3F+fPn6dChA/7+/iQnJ9O3b1927NhRrMWKiIiIbTPrssXcuXMZNWoU48aNM1n/wQcfMHfuXLp06WKV4kRERMT2mHXm4fTp0/Tu3Tvf+t69exsnLYqIiMiDwazw4ObmVuD7Ho4ePWryZEYREREp/cy6bNGvXz/Cw8M5c+YMLVq0AODgwYNEREQwfPhwqxYoIiIitsWs8PDSSy/h7OxMREQEH3zwAQDVq1dn7NixPPfc/TfDWkRERP66O4aHP/74g1WrVtGjRw+GDBnC1atXAahYsaLVixMRERHbc8c5D+XKlWPmzJn88ccfwO3QoOAgIiLy4DJrwmSzZs0KnDApIiIiDx6z5jw888wzvPfeeyQnJ+Pj44OTk5PJdr0LQkRE5MFhVngICwsD4N///ne+bcX1Jk0RERG5P9jkWzVFRETEdtnsWzVFRETENpkVHuD20ySXLl3Kr7/+CkCDBg0YMmSI5juIiIg8YMy62+Lrr7/m6aef5uLFi3Ts2JGOHTuSlpZGv379WL9+vbVrFBERERti9ls1x48fz6hRo0zWL1q0iA8++KDAl2aJiIhI6WTWmYf09HR69OiRb31QUBBpaWnFXpSIiIjYLrPCQ5s2bfjxxx/zrf/xxx957LHHir0oERERsV1mXbbo0KEDs2fP5siRIzRv3hyAQ4cOsW3bNsaMGUNMTIxx38DAQOtUKiIiIjbBrPDw9ttvA7B69WpWr15d4DbQA6NEREQeBGaFhxMnTli7DhEREblPmDXnQURERCRPoeFh/fr1GAwGszpJSkriwIEDxVaUiIiI2K5Cw8OaNWsICgrio48+4uTJk/mCREZGBtu3b2f8+PE8/fTT/P7771YvVkREREpeoXMeli1bxs6dO1m+fDkffPABDg4OVKlSBQcHB65cuUJ6ejpubm489dRTvPnmm1SpUuVe1i0iIiIlpMgJk506daJTp06kp6dz8OBBkpKSuHXrFq6urjRu3JhHH32UMmU0bUJERORBYtbdFlWqVKFbt27WrkVERETuA2a/VbMkbNq0iSVLlhAfH4+joyNt27YlLCyMOnXqFNlu+fLlrFy5ksTERCpVqkSnTp0ICwujatWqxn3279/Pxx9/zMmTJ7ly5QouLi40a9aMF198ER8fH2sPTURE5L5ls9ccoqKiCA0N5dixY1SrVo2cnByio6MZOHAgFy9eLLTd3LlzmTZtGvHx8Xh6enL9+nXWrl1LSEgIN27cACA+Pp4RI0awb98+/vjjDx555BEyMjL45ptveP7557l169a9GqaIiMh9xybDQ1ZWFrNnzwage/fubN++nc2bN+Ps7ExaWhqLFi0qsN2lS5dYsmQJAMOGDSM6OppVq1ZhZ2dHQkICkZGRABw+fJjs7Gzg9ptBv/rqK8aOHQvAtWvXyMzMtPYQRURE7ls2GR6OHDnC5cuXgf9/V4a7u7vxvRq7d+8usN2+ffuMoSCvXaNGjYyXOfLaNWvWjPLlywMwcuRI+vbty/z583F2dmbixIlUr17dSiMTERG5/9lkeEhJSTEuu7m55VtOTk6+q3YNGjQgIiKCKlWqkJmZybFjx8jOzsbd3Z1HHnmk+AYiIiJSChU6YXLBggVmdzJmzJhiKeZOzH3i5Z3apaamMnnyZNLT05kxYwYBAQH85z//YfHixbz00kts2bKFWrVq5etHL/0SKdz9+PlwLukCRIpBSXz2Cg0P0dHRJl8nJSVx8+ZN4yn9Cxcu4OjoiKenZ7GHhxo1ahiX09LSjMvp6ekAeHh4mNWudu3aBbb74osvSExMxMnJid69ewPQt29fFi9eTHZ2NrGxsQWGh8aNG9/NsAoRa4U+Re4963w+rCuxpAsQKQbW+uzFxcUVuq3QyxYbNmww/hkyZAhNmjThm2++YefOnezcuZNvvvkGX19fnn/++WIv2NfXFxcXFwBiYmKA22cLDh06BIC/vz8AQUFBBAUFsWLFCgDatWtHuXLlTNqdOHGCM2fOmLTLe5T2jRs3OHXqFHB7nkUeJyenYh+TiIhIaWHWnIeFCxcyefJkk3/xe3h48Oqrr7Jw4cJiL8re3p7Q0FDg9hmQrl27EhwczLVr13B1dWXEiBEAJCQkkJCQYJxcWa1aNYYPHw5AREQE3bt3p3///hgMBurWrUv//v0BCAgIwM7ODoB+/frx5JNP8vrrrxv7yAsZIiIikp9Z4SEtLY2bN2/mW3/r1i3jL+7i1r9/f2bOnEnjxo25cOECdnZ2BAQEsHLlStzd3QttN2HCBF5//XXq169PUlISTk5O9OnTh+XLl1OhQgXg9hmKjz/+mPbt2+Ps7Mzp06epUaMGffv2ZeXKlVSsWNEqYxIRESkNzHrCZPv27XnjjTd4++238fX1xc7Ojv/+97+Eh4fTvn17qxXXq1cvevXqVej2kydP5ltnZ2fH888/f8fLKR06dKBDhw53XaOIiMiDxqzw8M477zBp0iQGDBhA2bJlAcjNzeVvf/sb06ZNs2qBIiIiYlvMfjHW4sWLSUhI4LfffgOgfv361KtXz6rFiYiIiO2x6MVY9erVo1KlSlSpUkWv4hYREXlAmZUAsrOzmTFjBi1atKBDhw4kJSUBMHPmTD7//HOrFigiIiK2xazwsGDBAr799ltmzpyJvb29cX3Tpk356quvrFaciIiI2B6zLlts2rSJ6dOn07p1a+PzEQAeeeQRTp8+ba3aRERExAaZdebhwoULBT4SOicnh5ycnGIvSkRERGyXWeGhYcOGHDhwIN/6LVu20KRJk2IvSkRERGyXWZctxowZw8SJEzl//jy5ubls2bKFhIQENmzYwMcff2ztGkVERMSGmHXmoUuXLsydO5e9e/dSpkwZFi5cyOnTp/noo4+s+oRJERERsT1mP+fB399fL4wSERER8848dO3atcAXYF25coWuXbsWe1EiIiJiu8wKD0lJSeTm5uZbn5WVRWpqarEXJSIiIraryMsWMTExxuWdO3dSqVIl49c5OTns378fT09P61UnIiIiNqfI8DBu3Djj8uTJk00bliuHp6cnr776qnUqExEREZtUZHg4ceIEcPtuizVr1uDq6npPihIRERHbdcc5D9nZ2bi5uRU4YVJEREQePHcMD+XLl+fcuXMm77QQERGRB5dZd1v06dOH1atXW7sWERERuQ+Y9ZCoGzdusGHDBvbt20eTJk2oUKGCyfYpU6ZYpTgRERGxPWaFh/j4eB599FEAzp49a7JNlzNEREQeLGaFh+XLl1u7DhEREblPmDXnIU96ejqHDx8mKyvLWvWIiIiIjTMrPFy9epVx48bRvn17BgwYYHwkdXh4OPPnz7dqgSIiImJbzAoPs2bN4sKFC3z11Vc4Ojoa13fu3Jlt27ZZrTgRERGxPWbNedixYwcLFiygcePGJusbNGiQbwKliIiIlG5mnXm4cuVKgY+mvnbtGmXLli32okRERMR2mRUefH192b59e771kZGRtGjRotiLEhEREdtl1mWLCRMmMHz4cH799VdycnL47LPPOHXqFEeOHGHFihVWK27Tpk0sWbKE+Ph4HB0dadu2LWFhYdSpU6fIdsuXL2flypUkJiZSqVIlOnXqRFhYGFWrVjXZ79tvvyUiIoKjR4+SnZ1N9erV6d69O6+88orVxiQiInK/M+vMQ8uWLYmMjCQ7O5vatWuzf/9+qlevTmRkJE2aNLFKYVFRUYSGhnLs2DGqVatGTk4O0dHRDBw4kIsXLxbabu7cuUybNo34+Hg8PT25fv06a9euJSQkhBs3bhj3++yzzxg1ahQ//vgj5cuXp2HDhhgMBnbv3m2V8YiIiJQWZp15APD29ua9996zZi1GWVlZzJ49G4Du3bszb948UlNT6dGjB2lpaSxatKjAR2JfunSJJUuWADBs2DAmTZrEiRMn6NOnDwkJCURGRjJ06FBSUlKYNWsWACNHjmT8+PHGuRtXr169J2MUERG5X5l15qF3795ERERw4cIFa9cDwJEjR4yvAA8MDATA3d2d5s2bAxR6dmDfvn1kZ2ebtGvUqJHxMkdeu+joaLKzs7Gzs+PChQt06NCBNm3aMGbMGL16XERE5A7MOvPQsWNHVqxYwezZs3nsscfo3bs3gYGBODs7W6WolJQU47Kbm1u+5eTkZIvbnT592tguISEBAIPBwLp162jYsCGJiYls27aNY8eOsWHDhgLHdvz48bsYlUjpdj9+PqzzfzCRe6skPntmhYfQ0FBCQ0M5cOAAGzdu5L333uOtt96ic+fO9O7dm06dOlm5zNsMBkOxtMvJyTEu//Of/2TUqFF89913jBgxgqSkJHbu3MkTTzyRr58/P+eieMRaoU+Re886nw/rSizpAkSKgbU+e3FxcYVus+jdFq1ateJf//oXe/bs4f333ychIYEXX3zxrgv8sxo1ahiX09LSjMvp6ekAeHh43FU7d3d34zYfHx/g9u2oeZKSkv5y7SIiIqWdReEB4Pz580RERDBnzhxOnjyJn59fsRfl6+uLi4sLADExMQCkpqZy6NAhAPz9/QEICgoiKCjIeLtou3btKFeunEm7EydOcObMGZN2jz/+uPFYx44dA+Dnn382rqtbt26xj0lERKS0MOuyRWZmJlu3bmXDhg0cPHiQevXq0atXL5588slCzwLcDXt7e0JDQwkPDyc6OpquXbuSkZHBtWvXcHV1ZcSIEcD/z13Im+RYrVo1hg8fzqJFi4iIiGDHjh2kpKRgMBioW7cu/fv3B6BFixYEBgYSExPD+++/z4YNG4wBo0mTJnTp0qXYxyQiIlJamBUeHn/8capUqUJwcDCvv/46jz76qLXron///jg5OREREUF8fDwODg4EBAQQFhZmctnhzyZMmICbmxuRkZGcPXuWihUrEhQURFhYGBUqVDDuN3v2bBYuXMiGDRtISEjAw8ODrl27Mnr0aOPZCxEREcnPzmDGLMS9e/fSrl07ypSx+CpHqREXF2eVSzR+E5cVe58iJSFu5nMlXYLFEqf63nknERtXO/yIVfot6vee2WceRERERMDM8JCRkcH777/P999/T1paGrm5uSbbDx48aJXiRERExPaYFR4mT57M8ePHeeaZZ6hevTp2dnbWrktERERslFnhYf/+/Xz66ac0a9bM2vWIiIiIjTNrBqSbm5vJnQoiIiLy4DIrPEyYMIF58+Zx7do1a9cjIiIiNs6syxYffvgh586do3379nh4eOR7DsKGDRusUpyIiIjYHrPCQ/fu3a1dh4iIiNwnzAoPY8aMsXYdIiIicp+w6DnM+/fvJz4+Hjs7Oxo2bEibNm2sVZeIiIjYKLPCQ2pqKqNHj+bo0aNUr14dgAsXLuDj48OCBQuKfNeEiIiIlC5m3W0xbdo0ypYtS0xMDLt27WLXrl3ExMRQtmxZ3nnnHWvXKCIiIjbErPCwd+9ewsPDqVWrlnFdrVq1mDx5Mnv37rVacSIiImJ7zH5NZkGPpNZjqkVERB48ZoWHdu3a8fbbb3P+/HnjuuTkZKZPn067du2sVpyIiIjYHrMmTE6ZMoUXX3yRbt26mUyY9PLyYsqUKVYtUERERGyLWeGhZs2afPXVV+zbt4/ffvsNgAYNGtC+fXurFiciIiK2x+znPNjZ2fH444/z+OOPW7MeERERsXFFznnYtWsXXbp04erVq/m2/f7773Tp0kV3W4iIiDxgigwPn3/+OcOHD6dixYr5tlWqVIl//OMfLF261GrFiYiIiO0pMjycPHmyyLsp2rZty4kTJ4q9KBEREbFdRYaH9PR0ypQpfBc7OzsyMjKKvSgRERGxXUWGhxo1anDy5MlCt588eVLvtRAREXnAFBkeOnbsyAcffMDNmzfzbbtx4wbz5s2jY8eOVitOREREbE+Rt2q++OKLREdH0717d5599lnq168PwG+//caKFSswGAyMGjXqnhQqIiIitqHI8ODm5kZkZCT/+te/eP/99zEYDMDtuQ5/+9vfCA8Pp2rVqvekUBEREbENd3xIlKenJ4sXLyYzM5MzZ84AUKdOHSpXrmz14kRERMT2mP1WzcqVK9O0aVOaNm16z4LDpk2b6Nu3L02bNqV169aMGzfOGGCKsnz5coKDg/Hx8aFdu3a89tprXLp0qcB9jx07ho+PD97e3nh7e/PLL78U9zBERERKFbPDw70WFRVFaGgox44do1q1auTk5BAdHc3AgQO5ePFioe3mzp3LtGnTiI+Px9PTk+vXr7N27VpCQkK4ceOGyb43b94kLCyM7Oxsaw9HRESk1LDJ8JCVlcXs2bMB6N69O9u3b2fz5s04OzuTlpbGokWLCmx36dIllixZAsCwYcOIjo5m1apV2NnZkZCQQGRkpMn+7777Lr/99htBQUHWHZCIiEgpYpPh4ciRI1y+fBmAwMBAANzd3WnevDkAu3fvLrDdvn37jGcR8to1atSIOnXq5Gu3Y8cOIiMjGTx4sG43FRERsYBNhoeUlBTjspubW77l5OTku2p38eJFJk+ejJeXFxMnTiy+wkVERB4AZr+S2xbk3Sp6t+3Cw8O5du0aS5cuxcHBwex+jh8//peOL/IguB8/H84lXYBIMSiJz55NhocaNWoYl9PS0ozL6enpAHh4eJjVrnbt2gW2O3nyJNnZ2fTv3x+AP/74w9iuX79+hISEFHhGonHjxn9pPEWLtUKfIveedT4f1pVY0gWIFANrffbi4uIK3WaTly18fX1xcXEBICYmBoDU1FQOHToEgL+/PwBBQUEEBQWxYsUKANq1a0e5cuVM2p04ccJ4e2deO4Dc3FyuX7/O9evXycrKMq6/efOmydciIiJiyibPPNjb2xMaGkp4eDjR0dF07dqVjIwMrl27hqurKyNGjAAgISEBwDi5slq1agwfPpxFixYRERHBjh07SElJwWAwULduXeOZhh07dpgcb+3atbz22msAbNiwAS8vr3s1VBERkfuOTYYHgP79++Pk5ERERATx8fE4ODgQEBBAWFhYkW/ynDBhgvGx2mfPnqVixYoEBQURFhZGhQoV7uEIRERESic7w1+dhfiAiYuLw8/Pr9j79Zu4rNj7FCkJcTOfK+kSLJY41bekSxC5a7XDj1il36J+79nknAcRERGxXQoPIiIiYhGFBxEREbGIwoOIiIhYROFBRERELKLwICIiIhZReBARERGLKDyIiIiIRRQeRERExCIKDyIiImIRhQcRERGxiMKDiIiIWEThQURERCyi8CAiIiIWUXgQERERiyg8iIiIiEUUHkRERMQiCg8iIiJiEYUHERERsYjCg4iIiFhE4UFEREQsovAgIiIiFlF4EBEREYsoPIiIiIhFFB5ERETEIgoPIiIiYhGFBxEREbFIuZIuoCibNm1iyZIlxMfH4+joSNu2bQkLC6NOnTpFtlu+fDkrV64kMTGRSpUq0alTJ8LCwqhatSoAp06dIiIigkOHDnHhwgVyc3N5+OGH+fvf/05ISAjlytn0t0VERKRE2eyZh6ioKEJDQzl27BjVqlUjJyeH6OhoBg4cyMWLFwttN3fuXKZNm0Z8fDyenp5cv36dtWvXEhISwo0bNwA4cuQIa9eu5eLFi9SqVQs7Ozt++eUX3n33XaZPn36vhigiInJfssnwkJWVxezZswHo3r0727dvZ/PmzTg7O5OWlsaiRYsKbHfp0iWWLFkCwLBhw4iOjmbVqlXY2dmRkJBAZGQkADVr1uT999/n+++/Z926dezYsQNPT08A1q9ffw9GKCIicv+yyfBw5MgRLl++DEBgYCAA7u7uNG/eHIDdu3cX2G7fvn1kZ2ebtGvUqJHxMkdeu3bt2hEcHGy8POHi4kKTJk0AsLe3t8aQRERESg2bDA8pKSnGZTc3t3zLycnJxdru1KlTxmAxcODAv1i1iIjIg+G+mhloMBiKvd2BAwcYPXo0N27cICgoiNGjRxe67/Hjx//S8UUeBPfj58O5pAsQKQYl8dmzyfBQo0YN43JaWppxOT09HQAPDw+z2tWuXbvIduvWrWPKlClkZ2czaNAg3njjDcqUKfxkTOPGjS0ciTlirdCnyL1nnc+HdSWWdAEixcBan724uLhCt9nkZQtfX19cXFwAiImJASA1NZVDhw4B4O/vD0BQUBBBQUGsWLECuD2XIW8eQ167EydOcObMGZN2BoOB999/n0mTJpGbm8vrr7/Om2++WWRwEBERkdts8syDvb09oaGhhIeHEx0dTdeuXcnIyODatWu4uroyYsQIABISEgCMkyurVavG8OHDWbRoEREREezYsYOUlBQMBgN169alf//+AGzevJmPPvoIAGdnZzZt2sSmTZuMx1+9evW9HK6IiMh9xSbDA0D//v1xcnIiIiKC+Ph4HBwcCAgIICwsDHd390LbTZgwATc3NyIjIzl79iwVK1YkKCiIsLAwKlSoAMCtW7eM+1+5coXDhw9bfTwiIiKlhZ3hr85CfMDExcXh5+dX7P36TVxW7H2KlIS4mc+VdAkWS5zqW9IliNy12uFHrNJvUb/3dJFfRERELKLwICIiIhZReBARERGLKDyIiIiIRRQeRERExCIKDyIiImIRhQcRERGxiMKDiIiIWEThQURERCyi8CAiIiIWUXgQERERiyg8iIiIiEUUHkRERMQiCg8iIiJiEYUHERERsYjCg4iIiFhE4UFEREQsovAgIiIiFlF4EBEREYsoPIiIiIhFFB5ERETEIgoPIiIiYhGFBxEREbGIwoOIiIhYROFBRERELKLwICIiIhZReBARERGLKDyIiIiIRWw6PGzatIm+ffvStGlTWrduzbhx4zhz5swd2y1fvpzg4GB8fHxo164dr732GpcuXTLZ59KlS7z22mu0a9cOHx8fgoODWbZsmbWGIiIiUmqUK+kCChMVFcWUKVMAePjhh8nIyCA6OpoDBw6wfv16qlWrVmC7uXPn8uGHHwJQt25dUlJSWLt2LT/99BNfffUVTk5OXL9+nZCQEBISEnB0dMTT05P4+HjeeecdLl++zPjx4+/ZOEVERO43NnnmISsri9mzZwPQvXt3tm/fzubNm3F2diYtLY1FixYV2O7SpUssWbIEgGHDhhEdHc2qVauws7MjISGByMhIAFatWkVCQgJ2dnasWrWK6Ohohg4dCsDixYvznaUQERGR/2eT4eHIkSNcvnwZgMDAQADc3d1p3rw5ALt37y6w3b59+8jOzjZp16hRI+rUqWPS7rvvvgOgTp06NGrUyGT/7Oxs9u/fX+xjEhERKS1s8rJFSkqKcdnNzS3fcnJyssXtTp8+bWx3/vz5fPtUrVrVuJy3/c/i4uLMHoO5Ph7QpNj7FCkJ1vh8WN0Tn5V0BSJ37WIJfPZsMjwUxmAwWK3dnfbx8/P7S8cWEREpbWzyskWNGjWMy2lpacbl9PR0ADw8PO6qXc2aNfPt87/LedtFREQkP5sMD76+vri4uAAQExMDQGpqKocOHQLA398fgKCgIIKCglixYgUA7dq1o1y5cibtTpw4Yby9M69db8DDwQAADjVJREFU3n/PnDnDiRMnTPYvX7487dq1s+4ARURE7mN2hr96LcDKVq1aRXh4OPD/t2pevXoVV1dX1q9fj7u7O97e3gCMGTOGsWPHAjBnzhzj3Rh5t2revHnz/9q715gorj4M4M/iKwsLAm4brRSqLDqrAkUgQeuldreipqtYtxEt12hNtVEj0Chei1pibdC2FmO1FhOCaIvBS9VGRRt7L9baItYr4nJZNhRTUEQQkPN+IDtxi7SOL4ivPr9PM2fOnDlnmSH/OWfmDAYMGIC9e/dCo9Ggvr4eZrMZFosFLi4ueOaZZ2CxWAAAc+fORVJS0sNvMP3fMhqNsFqtmDp1KtatW9fd1SEi6nKPZM8DAEyfPh3p6ekYMmQI/vzzT6hUKkRERGDXrl3o27dvh/slJSVh2bJl0Ol0sFqtcHV1xauvvors7GxoNBoAgJubG7KzszF16lS4urrCarXCz88PS5cuRWJi4sNqIj1EcXFx0Ov1iIuL6+6qEHW6JUuWQK/Xw2g0PvRjV1RUQK/XQ6/XY8+ePQ/9+PcjIyNDriN1jkf6gcnIyEhERkZ2uP3ixYvt0lQqFRISEpCQkPCPZffp04d3idShpqYmODs7d3c1iDodz23qDI9szwM9/oQQ2LVrF8xmM4KDgxESEgKz2YzTp08DAI4fP47o6GiEhIQgKCgIkZGR2Llzp8ObMfa7ifT0dKxZswbDhw/HCy+8gLS0NLS0tMh5Tp48CQA4efKkvE9FRQX27Nkjr3/11Vcwm80IDAyU5wI5deoU3njjDYSFhSEwMBATJkzAJ598Is8nQtTdjEYj9u7dCwCwWq3y+VxQUCAv5+bmIi4uDkFBQdi5cycA4OrVq0hKSpKn6B8/fjw+++wztLa2ymVnZmZiypQpCA8PR0BAAEaMGIH58+fj6tWrAIA9e/bg5ZdflvMvXbrUoYfv7h6/7du3Y9SoUQgPD0dGRgYaGhqwcuVKhISEwGAwYPfu3Q7tqq6uxvLlyzFmzBgEBgbCYDBg/fr1aGpqkvPcXX5OTg6MRiNCQkIwZ84cVFdXy3k2bdok7/Oo95L8v3ikex7o8ZaWliY/7Orp6Ym+ffuiuLgYFosF5eXlWLx4MYC2+ThcXFxw8eJFrF69GjabDW+//bZDWVlZWXBzc4NarUZVVRWys7MhSRKioqIQHByM4uJi1NfXw83NDQMHDgSAdndfixcvRu/eveW3cgoKCjBr1iy0tLTAw8MDzz77LCwWCz766CNcvnwZH3zwQVf/RET/asiQIbh16xZqamrQs2dPDB06FABw7tw5Oc+aNWvg7u4OX19fODk5obS0FFFRUbhx4wY8PDyg0+lw+fJlpKenw2azYeXKlQDagu2ysjL069cPffv2xZUrV5Cfn4+ioiIcPXoUWq0WQ4YMwfnz5wEAvr6+0Gq18Pf3d6jj77//jnPnzsHDwwPXrl3Dpk2bcPjwYfz111/QaDSorKzEO++8g9DQUPj7+6OmpgbTp0+H1WqFRqOBTqdDSUkJtm3bhuLiYmzZssWh/N9++w2FhYXo168fbt26hRMnTmDdunXYsGED/P39UVpaiqqqKgBAcHAwAECr1XbNH+RJIYi6QXl5udDr9UKSJDF37lzR2NgohBCitrZWlJWViZdeeklIkiTMZrNobGwUra2tIikpSUiSJAICAkRNTY0QQghJkoQkScJoNIobN26IxsZGMXr0aCFJkkhMTJSPFxsbKyRJErGxsQ71yMvLk8tITk4Wd+7cEUII0dLSImJiYoQkSWLs2LGitrZWCCFEenq6nP/ChQtCCCEMBoOQJEmkpKR0+e9GdC8pKSlCkiRhMBjktJ9//lk+V2NjY+VrrKWlRSxZskRIkiTGjx8v6urqhBBC7N+/X0iSJAYPHiwqKyuFEEJcunRJNDU1yWX+8MMPcpk//vijEKLtWran5eXlOdTLft0FBASI8vJyUV9fLwICAoQkSWLEiBHi+vXrorS0VN5/586dQgghMjIyhCRJIjw8XFRVVQkhhDh16pSc79SpUw7lDx48WJw/f14IIcS8efOEJEli5MiRcj0+/vhjeV/qHBy2oG5RVFQkDz/MnDkTarUaQFsPhP1OBAAiIiKgVquhUqlgMpkAtE0hbn/F1s5oNKJXr15Qq9Xw8fEBAMXfKImLi4OTU9sl0aNHDxQVFQFoe7XX09MTADBp0iQ5/9mzZxWVT9RdZsyYIV9jPXr0wJkzZwAAFosFYWFh0Ov1WLRoEQCgtbVV3l5ZWYn4+HiEhoZi8ODB8jeAAMh38vdj0KBB8PHxgUajke/4w8LC4OHhAV9fXzmffb4d+/Fra2sxZswY6PV6REdHy/kKCwsdypckSf7UgL3Xg98o6loctqDHgoeHh7xsn+tDqbunKyd6nHR0bnt5ecnf/rmbi4sLysvLMW/ePDQ3N8PNzQ0BAQG4c+eOPERx97MR/8bd3V1etl+f9jSVSiVvE3+bOUCj0WDQoEHtyuvVq5fDemdc/6QMf2XqFkFBQVCpVBBCICsrC8OGDYOzszPq6urQ0NAAb29vVFZWIj8/HzNnzoSzszMOHToEoG0iL/tdxv1ycXEBANy6davDPHf/E7PX8ZdffsF3332H69evw9PTEwcPHpS3BwYGKqoDUVexn98NDQ0QQrQ7l+91bhcXF0Oj0WDLli1yb8DNmzeRn5+PsWPH4siRI/KDwZmZmQgJCcGhQ4eQnJzsUJarq6u83NDQ0CntCQoKwjfffAOVSoX09HQ5wGlsbMSJEycwcuRIReXZfx+g7X+A/bV9enAMHqhb+Pj4ICYmBjt27MCxY8cwZswY9OnTB2VlZUhNTUViYiIWL16Ms2fPwmg0Qq1Ww2q1Amgb5rDPQHq/dDodvv32W5w9exaTJ09Gnz59kJmZ+Y/7LFiwALNmzYLNZsO4ceOg1WrlycRMJhPfGadHhk6nA9A2Ff/EiRPh6emJefPmdZh/zpw5OHbsGCorK2EwGDBgwADU1dWhqqoKLS0tmDp1KgYOHIgePXrgzp07mD17Nry9veU3GO6m1Wrh5eWF2tparF+/Hvv27UNkZOT/NKdKTEwM8vLyYLPZYDKZ4Ofnh9u3b8Nms6GpqQnHjx936G34N/bfB2gbenz66aexYcMGhyETUobPPFC3WbFiBVJTUzF06FA0NDSgoqIC/v7+6N+/P6ZMmYLNmzcjNDQU9fX1qK6uhl6vR2pqars7n/sxa9YsjBw5EhqNBpcuXbqv5xWGDx+OrKwsjB49Gq2trbBarRgwYAAWLlyI999//0GaTNQlXnvtNUyYMAG9evWCxWJBYWHhPw4r+Pn5ITc3FyaTCW5ubrhy5Qqam5sRHh6OZcuWAWh7dmDt2rXw8fFBc3MzvLy87vmGkUqlQlpaGvr374/bt2/jzJkzHX75+H5ptVp88cUXmDZtGrRaLUpKSnDz5k0EBQUhOTnZ4SvI98NgMCAqKgpeXl6wWq0oLCzstF6SJ9UjOz01ERERPZrY80BERESKMHggIiIiRRg8EBERkSIMHoiIiEgRBg9ERESkCIMHIiIiUoTBAxERESnC4IGIiIgUYfBAREREijB4ICIiIkUYPBAREZEiDB6IiIhIEQYPREREpAiDByIiIlKEwQMREREpwuCBiIiIFGHwQERERIoweCAiIiJF/tPdFSCix9O1a9fw6aef4sSJE7DZbHB3d0f//v1hMplgNpvh5ubW3VUkogfE4IGIOl1FRQVef/11uLu7Y+HChdDr9VCr1SguLsbu3bvh5eWFyZMnt9uvqakJzs7O3VBjIlKCwxZE1OlWrVoFJycn5OXlwWQyYeDAgfD19YXBYMDmzZsxadIkAIBer0dOTg7mz5+PYcOG4cMPPwQAfP7554iIiEBgYCAiIiKQm5vrUL5er8fhw4cd0oxGIzIzMx3y7NixA2+++SaCg4NhMBiwf//+Lm450ZOBwQMRdaqamhp8//33iImJgUajuWcelUolL2/atAljx47FgQMHEB0djfz8fLz77rtISEjAgQMHEB8fj9WrV+Prr79WXJeMjAwYjUbs27cPUVFRSElJQVFR0QO3jYjacNiCiDpVWVkZhBDw8/NzSH/xxRdRV1cHAJg8eTLWrFkDAHjllVcwbdo0Od+iRYsQGRmJ2NhYAICfnx/++OMPbNu2DUajUVFdIiIiMGPGDADAW2+9hYKCAmRlZWH9+vUP3D4iYs8DET0kOTk52LdvH4KCgtDU1CSnBwYGOuQrKSlBaGioQ1pYWBiuXLmi+JjDhg1rt/4g5RCRI/Y8EFGneu6556BSqVBSUuKQ7uvrCwBwdXV1SP/7ekfuHupQqVQQQjhsb25ufpDqEtEDYM8DEXWq3r17Y9SoUdixYwfq6+sV76/T6XD69GmHtF9//RX+/v7yularRXV1tbx+7do1h3W7wsLCdus6nU5xnYjIEYMHIup0q1atghACZrMZBw8eRHFxMa5evYqDBw/iwoULcHLq+F/P7Nmz8eWXXyInJwcWiwXZ2dk4cOAAZs+eLecZMWIEcnJyUFRUhHPnzmHJkiVQq9Xtyjp69Chyc3NhsViwdetW/PTTT0hISOiSNhM9SThsQUSdztfXF3v37sXWrVuxceNG2Gw29OzZEzqdDtHR0YiJielw33HjxmHFihXYvn071q5dC29vb6Smpjo8LJmSkoLly5cjPj4eTz31FBYtWtRumAQAFixYgCNHjiAtLQ1arRbvvfcenn/++S5pM9GTRCX+PnBIRPQY0Ov12LhxIyZOnNjdVSF67HDYgoiIiBRh8EBERESKcNiCiIiIFGHPAxERESnC4IGIiIgUYfBAREREijB4ICIiIkUYPBAREZEiDB6IiIhIkf8Ctx8+9Dsgf8MAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 576x432 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3SJ2HsmYEm4M",
        "outputId": "b8dd342b-e929-4668-fd19-9890d47cbf34"
      },
      "source": [
        "#以函數計算z_stat, pval, confidence interval\n",
        "from statsmodels.stats.proportion import proportions_ztest, proportion_confint\n",
        "control_results = ab_test[ab_test['group'] == 'control']['converted']\n",
        "treatment_results = ab_test[ab_test['group'] == 'treatment']['converted']\n",
        "n_con = control_results.count()\n",
        "n_treat = treatment_results.count()\n",
        "successes = [control_results.sum(), treatment_results.sum()]\n",
        "nobs = [n_con, n_treat]\n",
        "\n",
        "z_stat, pval = proportions_ztest(successes, nobs=nobs)\n",
        "(lower_con, lower_treat), (upper_con, upper_treat) = proportion_confint(successes, nobs=nobs, alpha=0.05)\n",
        "\n",
        "print(f'z statistic: {z_stat:.2f}')\n",
        "print(f'p-value: {pval:.3f}')\n",
        "print(f'ci 95% for control group: [{lower_con:.3f}, {upper_con:.3f}]')\n",
        "print(f'ci 95% for treatment group: [{lower_treat:.3f}, {upper_treat:.3f}]')\n",
        "#判讀統計結果"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "z statistic: 0.07\n",
            "p-value: 0.945\n",
            "ci 95% for control group: [0.116, 0.126]\n",
            "ci 95% for treatment group: [0.116, 0.126]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ckQFAPewwRJL"
      },
      "source": [
        "# 作業：判讀程式最後統計結果，A/B test 是否顯著\n",
        "\n",
        "(0.13, 0.15)\n",
        "\n",
        "在Alpha=0.05的情況下，Z>1.96 或 <-1.96 方代表顯著\n",
        "\n",
        "同時 p 也應小於 0.05\n",
        "\n",
        "因此判斷不顯著"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "25S0aik5wjew"
      },
      "source": [
        "# 作業：試以(0.12, 0.11)計算結果是否顯著\n",
        "z statistic: 0.07\n",
        "\n",
        "p-value: 0.945\n",
        "\n",
        "ci 95% for control group: [0.116, 0.126]\n",
        "\n",
        "ci 95% for treatment group: [0.116, 0.126]\n",
        "\n",
        "z 沒有超過正負1.96\n",
        "\n",
        "p 大於 aplha 0.05\n",
        "\n",
        "信賴區間說明 control group 在區間內 0.12 可信\n",
        "\n",
        "treatment group在區間外，0.11 數字不可信"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Sjbijdxtw_3N"
      },
      "source": [
        "# 作業：樣本數是以那些模組/函數算的"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dEVzDdX6_OCJ"
      },
      "source": [
        "#求樣本大小\n",
        "effect_size = sms.proportion_effectsize(0.13, 0.15)    # Calculating effect size based on our expected rates\n",
        "\n",
        "required_n = sms.NormalIndPower().solve_power(\n",
        "    effect_size, \n",
        "    power=0.8, \n",
        "    alpha=0.05, \n",
        "    ratio=1\n",
        "    )                                                  # Calculating sample size needed\n",
        "\n",
        "required_n = ceil(required_n)                          # Rounding up to next whole number                          \n",
        "\n",
        "print(required_n)"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}