Handling_Categorical_Variables.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "c9f2UmyLpn7f"
   },
   "source": [
    "# **Handling Categorical Variables**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "dwnArd3oopc3"
   },
   "source": [
    "Handling Categorical/Qualitative variables is an important step in data preprocessing.Many Machine learning algorithms can not handle categorical variables by themself unless we convert them to numerical values.<br>\n",
    "And performance of ML algorithms is based on how Categorical variables are encoded.\n",
    "The results produced by the model varies from different encoding techniques used."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Categorical variables can be divided into two categories:<br>\n",
    "1. Nominal (No particular order) \n",
    "2. Ordinal (some ordered)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T10:28:59.919116Z",
     "start_time": "2020-08-24T10:28:59.913130Z"
    }
   },
   "source": [
    "<img src=\"Screenshots/Categorical_variables.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T10:36:01.423728Z",
     "start_time": "2020-08-24T10:36:01.414746Z"
    }
   },
   "source": [
    "<div class=\"alert alert-block alert-warning\">  \n",
    "<b>You can use the below cheat-sheet as a guiding tool. </b> \n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "H6oK8r3EtUCv"
   },
   "source": [
    "<img src=\"Screenshots/Categorical_Encoding.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "z3eVd-X_p0Rn"
   },
   "source": [
    "There are many ways we can encode these categorical variables.\n",
    "\n",
    "\n",
    "1. One Hot Encoding\n",
    "2. Label Encoding\n",
    "3. Ordinal Encoding\n",
    "4. Frequency or Count Encoding\n",
    "5. Binary Encoding\n",
    "6. Base-N Encoding\n",
    "7. Helmert Encoding \n",
    "8. Mean Encoding or Target Encoding\n",
    "9. Weight of Evidence Encoding\n",
    "10. Sum Encoder (Deviation Encoding or Effect Encoding)\n",
    "11. Leave One Out Encoding \n",
    "12. CatBoost Encoding\n",
    "13. James-Stein Encoding\n",
    "14. M-estimator Encoding\n",
    "15. Hashing Encoding\n",
    "16. Backward Difference Encoding\n",
    "17. Polynomial Encoding\n",
    "18. MultiLabelBinarizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T06:10:46.458364Z",
     "start_time": "2020-08-24T06:10:26.294327Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 360
    },
    "colab_type": "code",
    "id": "anTu1nZaYNS0",
    "outputId": "027d88f4-34f3-4ba4-b68e-8e24f2401131"
   },
   "outputs": [],
   "source": [
    "# We are gonna use following libraries to perform encoding.\n",
    "!pip install scikit-learn\n",
    "!pip install category-encoders"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-09T17:45:52.766543Z",
     "start_time": "2020-09-09T17:45:49.762833Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 71
    },
    "colab_type": "code",
    "id": "lQ6mfV-vpnYe",
    "outputId": "d103a648-46c0-44e5-d543-f04664e267a8"
   },
   "outputs": [],
   "source": [
    "import pandas as pd , numpy as np\n",
    "import category_encoders as ce"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T10:03:59.781388Z",
     "start_time": "2020-08-24T10:03:59.757724Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "colab_type": "code",
    "id": "yXsGmEXsvL10",
    "outputId": "54b16d67-9639-4c06-d37c-b015b588bf44"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target\n",
       "0         Hot     Red       1\n",
       "1        Cold  Yellow       1\n",
       "2    Very Hot    Blue       1\n",
       "3        Warm    Blue       0\n",
       "4         Hot     Red       1\n",
       "5        Warm  Yellow       0\n",
       "6        Warm     Red       1\n",
       "7         Hot  Yellow       0\n",
       "8         Hot  Yellow       1\n",
       "9        Cold  Yellow       1"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = {'Temperature':['Hot','Cold','Very Hot','Warm','Hot','Warm','Warm','Hot','Hot','Cold'],\n",
    "        'Color':['Red','Yellow','Blue','Blue','Red','Yellow','Red','Yellow','Yellow','Yellow'],\n",
    "        'Target':[1,1,1,0,1,0,1,0,1,1]}\n",
    "df = pd.DataFrame(data)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "TI9yRFgZu5Ee"
   },
   "source": [
    "# 1. One Hot Encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "m_X_wTW8wwEO"
   },
   "source": [
    "In this technique, it creates a new column/feature for each category in the Categorical Variable and replaces with either 1 (presence of the feature) or 0 (absence of the feature). The number of column/feature depends on the number of categories in the Categorical Variable.This method slows down the learning process significantly if the number of the categories are very high."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T06:11:49.189042Z",
     "start_time": "2020-08-24T06:11:49.147151Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "colab_type": "code",
    "id": "Sy6fRuB4ooZx",
    "outputId": "d42cbc71-deb9-4d94-fd4b-a33a71253fc2"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Temp_Cold</th>\n",
       "      <th>Temp_Hot</th>\n",
       "      <th>Temp_Very Hot</th>\n",
       "      <th>Temp_Warm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>Hot</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>Cold</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>Very Hot</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>Warm</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>Hot</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>Warm</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>Warm</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>Hot</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>Hot</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>Cold</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Color  Target Temperature  Temp_Cold  Temp_Hot  Temp_Very Hot  Temp_Warm\n",
       "0     Red       1         Hot          0         1              0          0\n",
       "1  Yellow       1        Cold          1         0              0          0\n",
       "2    Blue       1    Very Hot          0         0              1          0\n",
       "3    Blue       0        Warm          0         0              0          1\n",
       "4     Red       1         Hot          0         1              0          0\n",
       "5  Yellow       0        Warm          0         0              0          1\n",
       "6     Red       1        Warm          0         0              0          1\n",
       "7  Yellow       0         Hot          0         1              0          0\n",
       "8  Yellow       1         Hot          0         1              0          0\n",
       "9  Yellow       1        Cold          1         0              0          0"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using get_dummies method in pandas\n",
    "df_ohe = df.copy()\n",
    "one_hot_1 = pd.get_dummies(df_ohe,prefix = 'Temp' ,columns=['Temperature'],drop_first=False)\n",
    "one_hot_1.insert(loc=2, column='Temperature', value=df.Temperature.values)\n",
    "one_hot_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T06:11:55.289725Z",
     "start_time": "2020-08-24T06:11:54.111873Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "colab_type": "code",
    "id": "w0a1n_X01PDK",
    "outputId": "4fc43cf7-f270-4877-df74-43958638ca5b"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temp_Cold</th>\n",
       "      <th>Temp_Hot</th>\n",
       "      <th>Temp_Very_Hot</th>\n",
       "      <th>Temp_Warm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temp_Cold  Temp_Hot  Temp_Very_Hot  Temp_Warm\n",
       "0         Hot     Red       1        0.0       1.0            0.0        0.0\n",
       "1        Cold  Yellow       1        1.0       0.0            0.0        0.0\n",
       "2    Very Hot    Blue       1        0.0       0.0            1.0        0.0\n",
       "3        Warm    Blue       0        0.0       0.0            0.0        1.0\n",
       "4         Hot     Red       1        0.0       1.0            0.0        0.0\n",
       "5        Warm  Yellow       0        0.0       0.0            0.0        1.0\n",
       "6        Warm     Red       1        0.0       0.0            0.0        1.0\n",
       "7         Hot  Yellow       0        0.0       1.0            0.0        0.0\n",
       "8         Hot  Yellow       1        0.0       1.0            0.0        0.0\n",
       "9        Cold  Yellow       1        1.0       0.0            0.0        0.0"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using OneHotEncoder in sklearn\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "# ohe = OneHotEncoder(drop='first')\n",
    "ohe = OneHotEncoder()\n",
    "oh_array = ohe.fit_transform(df['Temperature'].values.reshape(-1, 1)).toarray()\n",
    "oh_df = pd.DataFrame(oh_array,columns=['Temp_Cold','Temp_Hot','Temp_Very_Hot','Temp_Warm'])\n",
    "pd.concat([df,oh_df],axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T06:12:01.271756Z",
     "start_time": "2020-08-24T06:12:01.215909Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "colab_type": "code",
    "id": "7hMLCmJectB9",
    "outputId": "e169a52c-3d39-40a4-ccc3-fad35c0ad34c"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temp_Hot</th>\n",
       "      <th>Temp_Cold</th>\n",
       "      <th>Temp_Very_Hot</th>\n",
       "      <th>Temp_Warm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temp_Hot  Temp_Cold  Temp_Very_Hot  Temp_Warm\n",
       "0         Hot     Red       1         1          0              0          0\n",
       "1        Cold  Yellow       1         0          1              0          0\n",
       "2    Very Hot    Blue       1         0          0              1          0\n",
       "3        Warm    Blue       0         0          0              0          1\n",
       "4         Hot     Red       1         1          0              0          0\n",
       "5        Warm  Yellow       0         0          0              0          1\n",
       "6        Warm     Red       1         0          0              0          1\n",
       "7         Hot  Yellow       0         1          0              0          0\n",
       "8         Hot  Yellow       1         1          0              0          0\n",
       "9        Cold  Yellow       1         0          1              0          0"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using category_encoders OneHotEncoder\n",
    "import category_encoders as ce\n",
    "ohe = ce.OneHotEncoder(cols=['Temperature'])\n",
    "ce_ohe = ohe.fit_transform(df.iloc[:,0], df.iloc[:,-1])\n",
    "ce_ohe.columns = ['Temp_Hot','Temp_Cold','Temp_Very_Hot','Temp_Warm']\n",
    "pd.concat([df,ce_ohe],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FhCurUQT49JV"
   },
   "source": [
    "1. For Regression, we can use N-1 (drop first or last column of One Hot Coded new feature ), \n",
    "2. For classification, the recommendation is to use all N columns as most of the tree-based algorithm builds a tree based on all available variables. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "BRa_OKPJ0kvW"
   },
   "source": [
    "**Disadvantages:** \n",
    "1. Tree algorithms cannot be applied to one-hot encoded data since it creates a sparse matrix.\n",
    "2. When the feature contains too many unique values, that many features are created which may result in overfitting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ejGLTAH61Cea"
   },
   "source": [
    "# 2. Label Encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "XrWPT35t6Cdq"
   },
   "source": [
    "1. In this encoding, a unique value is assigned for different labels/categories.<br>\n",
    "2. One major issue with sklearn.LabelEncoder is it assigns the values to the labels based on the Alphabetical order of the lables.<br>\n",
    "Ex : Cold<Hot<Very Hot<Warm….0 < 1 < 2 < 3 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T06:13:02.035281Z",
     "start_time": "2020-08-24T06:13:02.022317Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "colab_type": "code",
    "id": "3FTzzSeQ0-s-",
    "outputId": "efbe7d24-f453-41ec-f15f-9fef292b27fd"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temperature_encoded</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temperature_encoded\n",
       "0         Hot     Red       1                    1\n",
       "1        Cold  Yellow       1                    0\n",
       "2    Very Hot    Blue       1                    2\n",
       "3        Warm    Blue       0                    3\n",
       "4         Hot     Red       1                    1\n",
       "5        Warm  Yellow       0                    3\n",
       "6        Warm     Red       1                    3\n",
       "7         Hot  Yellow       0                    1\n",
       "8         Hot  Yellow       1                    1\n",
       "9        Cold  Yellow       1                    0"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using sklearn LabelEncoder()\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "le = LabelEncoder()\n",
    "df_ohe['Temperature_encoded'] = le.fit_transform(df.Temperature)\n",
    "df_ohe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T06:13:06.658917Z",
     "start_time": "2020-08-24T06:13:06.643956Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "colab_type": "code",
    "id": "Zn9Lo-TrD_wI",
    "outputId": "3ed565fd-337b-4bea-838d-d0f2ca5f7385"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temperature_factor</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temperature_factor\n",
       "0         Hot     Red       1                   0\n",
       "1        Cold  Yellow       1                   1\n",
       "2    Very Hot    Blue       1                   2\n",
       "3        Warm    Blue       0                   3\n",
       "4         Hot     Red       1                   0\n",
       "5        Warm  Yellow       0                   3\n",
       "6        Warm     Red       1                   3\n",
       "7         Hot  Yellow       0                   0\n",
       "8         Hot  Yellow       1                   0\n",
       "9        Cold  Yellow       1                   1"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using Pandas factorize()\n",
    "fact = df.copy()\n",
    "fact['Temperature_factor'] = pd.factorize(df.Temperature)[0]\n",
    "fact"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wRWiy0zeIoZc"
   },
   "source": [
    "**Disadvantages:** \n",
    "1. It mis-leads the information by assigning values based on Alphabetical order instead of actual label order."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DIWKGq5kHeGo"
   },
   "source": [
    "# 3. Ordinal Encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T06:13:25.544443Z",
     "start_time": "2020-08-24T06:13:25.528451Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "colab_type": "code",
    "id": "t6NPUUhg0-ij",
    "outputId": "45899d56-c97d-4d63-9922-ec0c32205919"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temperature_Oe</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temperature_Oe\n",
       "0         Hot     Red       1             1.0\n",
       "1        Cold  Yellow       1             0.0\n",
       "2    Very Hot    Blue       1             2.0\n",
       "3        Warm    Blue       0             3.0\n",
       "4         Hot     Red       1             1.0\n",
       "5        Warm  Yellow       0             3.0\n",
       "6        Warm     Red       1             3.0\n",
       "7         Hot  Yellow       0             1.0\n",
       "8         Hot  Yellow       1             1.0\n",
       "9        Cold  Yellow       1             0.0"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import OrdinalEncoder\n",
    "oe = OrdinalEncoder()\n",
    "oe_val = oe.fit_transform(df['Temperature'].values.reshape(-1, 1))\n",
    "pd.concat([df,pd.DataFrame(oe_val,columns=['Temperature_Oe'])],axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T06:13:30.790375Z",
     "start_time": "2020-08-24T06:13:30.761452Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "colab_type": "code",
    "id": "OQebcz38gciN",
    "outputId": "f0b1347f-a50e-43a0-c18a-5de0573f62ca"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temp_ce_oe</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temp_ce_oe\n",
       "0         Hot     Red       1           1\n",
       "1        Cold  Yellow       1           2\n",
       "2    Very Hot    Blue       1           3\n",
       "3        Warm    Blue       0           4\n",
       "4         Hot     Red       1           1\n",
       "5        Warm  Yellow       0           4\n",
       "6        Warm     Red       1           4\n",
       "7         Hot  Yellow       0           1\n",
       "8         Hot  Yellow       1           1\n",
       "9        Cold  Yellow       1           2"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using category_encoders OrdinalEncoder\n",
    "import category_encoders as ce\n",
    "ohe = ce.OrdinalEncoder(cols=['Temperature'])\n",
    "df['Temp_ce_oe'] = ohe.fit_transform(df.iloc[:,0], df.iloc[:,-1])\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T06:13:42.412502Z",
     "start_time": "2020-08-24T06:13:42.395545Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "colab_type": "code",
    "id": "iOvyQ3uf0-xA",
    "outputId": "d6c0b725-a2cf-4011-be4b-dba946054cd7"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temperature_Order</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temperature_Order\n",
       "0         Hot     Red       1                  3\n",
       "1        Cold  Yellow       1                  1\n",
       "2    Very Hot    Blue       1                  4\n",
       "3        Warm    Blue       0                  2\n",
       "4         Hot     Red       1                  3\n",
       "5        Warm  Yellow       0                  2\n",
       "6        Warm     Red       1                  2\n",
       "7         Hot  Yellow       0                  3\n",
       "8         Hot  Yellow       1                  3\n",
       "9        Cold  Yellow       1                  1"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Best way is mapping based on their actual label order\n",
    "# Ex : Cold < Warm <Hot < Very Hot = 1 < 2 < 3 < 4\n",
    "Temp_order = {'Cold' : 1 , 'Warm' : 2 , 'Hot' : 3 , 'Very Hot' : 4}\n",
    "df['Temperature_Order'] = df.Temperature.map(Temp_order)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "IqQ2zggJJth8"
   },
   "source": [
    "# 4. Frequency or Count Encoder"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Pmotgtk9TsM4"
   },
   "source": [
    "In frequency encoding, each of the categories in the feature is replaced with the frequencies of categories.<br>\n",
    "Here frequency of the categories is related somewhat with the target variable, it helps the model to understand and assign the weight in direct and inverse proportion, depending on the nature of the data.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "aLDaCvAIW4FF"
   },
   "source": [
    "<img src=\"Screenshots/frequency_encoding.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-3WYMk6WUCd0"
   },
   "source": [
    "Category refers to each of the unique values in a feature.\n",
    "1. **Frequency(category)** = Number of values in that category\n",
    "2. **Size(data)** = Size of the entire dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T06:14:09.614432Z",
     "start_time": "2020-08-24T06:14:09.046950Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "colab_type": "code",
    "id": "_V3rHQMAIdYw",
    "outputId": "30e4a729-1c03-4c30-ff4a-dff6bdf1cc68"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temp_Freq_Enc</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>0.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temp_Freq_Enc\n",
       "0         Hot     Red       1            0.4\n",
       "1        Cold  Yellow       1            0.2\n",
       "2    Very Hot    Blue       1            0.1\n",
       "3        Warm    Blue       0            0.3\n",
       "4         Hot     Red       1            0.4\n",
       "5        Warm  Yellow       0            0.3\n",
       "6        Warm     Red       1            0.3\n",
       "7         Hot  Yellow       0            0.4\n",
       "8         Hot  Yellow       1            0.4\n",
       "9        Cold  Yellow       1            0.2"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using Pandas groupby()\n",
    "cat_freq = df.groupby('Temperature').size() / len(df)\n",
    "df['Temp_Freq_Enc'] = df.Temperature.map(cat_freq)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T06:14:16.813161Z",
     "start_time": "2020-08-24T06:14:16.788225Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "colab_type": "code",
    "id": "UBWgekiVZFXY",
    "outputId": "028b49af-37eb-434e-f286-0d7fb50c5a9c"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temp_Count_Enc</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temp_Count_Enc\n",
       "0         Hot     Red       1               4\n",
       "1        Cold  Yellow       1               2\n",
       "2    Very Hot    Blue       1               1\n",
       "3        Warm    Blue       0               3\n",
       "4         Hot     Red       1               4\n",
       "5        Warm  Yellow       0               3\n",
       "6        Warm     Red       1               3\n",
       "7         Hot  Yellow       0               4\n",
       "8         Hot  Yellow       1               4\n",
       "9        Cold  Yellow       1               2"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using category_encoders CountEncoder\n",
    "import category_encoders as ce\n",
    "ce = ce.CountEncoder(cols=['Temperature'])\n",
    "df['Temp_Count_Enc'] = ce.fit_transform(df.iloc[:,0], df.iloc[:,-1])\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fqneYsnIUgOB"
   },
   "source": [
    "**Disadvantage**:\n",
    "1. If two categories have the same frequency then it is hard to distinguish between them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "KC_X6344holD"
   },
   "source": [
    "# 5. Binary Encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9-nJaVG-ixCL"
   },
   "source": [
    "1. It similar to onehot, but stores categories as binary bitstrings i.e., each binary bitstring creates one feature column.\n",
    "2. Compared to One Hot Encoding, this will require fewer feature columns (for 100 categories, One Hot Encoding will have 100 features while for Binary encoding, we will need just seven features).<br>\n",
    "\n",
    "**Feature -> ordinal encoding -> binary code -> digits of the binary code to separate columns.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "lE1gXYKQjfsl"
   },
   "source": [
    "<img src=\"Screenshots/Binary_encoding.png\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T09:49:04.340319Z",
     "start_time": "2020-08-24T09:49:04.286682Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "colab_type": "code",
    "id": "JidiXY8zJ1ac",
    "outputId": "8adcbee7-9c43-4331-c397-ac66bdf325bc"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temperature_0</th>\n",
       "      <th>Temperature_1</th>\n",
       "      <th>Temperature_2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temperature_0  Temperature_1  Temperature_2\n",
       "0         Hot     Red       1              0              0              1\n",
       "1        Cold  Yellow       1              0              1              0\n",
       "2    Very Hot    Blue       1              0              1              1\n",
       "3        Warm    Blue       0              1              0              0\n",
       "4         Hot     Red       1              0              0              1\n",
       "5        Warm  Yellow       0              1              0              0\n",
       "6        Warm     Red       1              1              0              0\n",
       "7         Hot  Yellow       0              0              0              1\n",
       "8         Hot  Yellow       1              0              0              1\n",
       "9        Cold  Yellow       1              0              1              0"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import category_encoders as ce\n",
    "be = ce.BinaryEncoder(cols=['Temperature'])\n",
    "be_df = be.fit_transform(df['Temperature'])\n",
    "pd.concat([df,be_df],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. Base-N encoder"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Base-N encoder encodes the categories into arrays of their base-N representation. \n",
    "2. A base of 1 is equivalent to one-hot encoding (not really base-1, but useful), a base of 2 is equivalent to binary encoding. N=number of actual categories is equivalent to vanilla ordinal encoding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T09:47:45.021458Z",
     "start_time": "2020-08-24T09:47:44.966696Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temperature_0</th>\n",
       "      <th>Temperature_1</th>\n",
       "      <th>Temperature_2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temperature_0  Temperature_1  Temperature_2\n",
       "0         Hot     Red       1              0              0              1\n",
       "1        Cold  Yellow       1              0              1              0\n",
       "2    Very Hot    Blue       1              0              1              1\n",
       "3        Warm    Blue       0              1              0              0\n",
       "4         Hot     Red       1              0              0              1\n",
       "5        Warm  Yellow       0              1              0              0\n",
       "6        Warm     Red       1              1              0              0\n",
       "7         Hot  Yellow       0              0              0              1\n",
       "8         Hot  Yellow       1              0              0              1\n",
       "9        Cold  Yellow       1              0              1              0"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bne = ce.BaseNEncoder(cols=['Temperature'],base=2)\n",
    "bne_df = bne.fit_transform(df['Temperature'],df.Target)\n",
    "pd.concat([df,bne_df],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GagsVrCkpR0k"
   },
   "source": [
    "# 7. Helmert Encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "HxGKMMOa4PQp"
   },
   "source": [
    "1. Helmert coding is a third commonly used type of categorical encoding for regression along with OHE and Sum Encoding.\n",
    "2. It compares each level of a categorical variable to the mean of the subsequent levels.\n",
    "3. The version in category_encoders is sometimes referred to as **Reverse Helmert Coding.**\n",
    "4. It is useful in certain situations where levels of the categorical variable are ordered, say, from lowest to highest, or from smallest to largest."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T06:15:25.281806Z",
     "start_time": "2020-08-24T06:15:25.232970Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "colab_type": "code",
    "id": "KfDhMhw7J1qX",
    "outputId": "2ae1476a-0359-4648-c6a8-c10f31c0e157"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>intercept</th>\n",
       "      <th>Temperature_0</th>\n",
       "      <th>Temperature_1</th>\n",
       "      <th>Temperature_2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>-1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  intercept  Temperature_0  Temperature_1  \\\n",
       "0         Hot     Red       1          1           -1.0           -1.0   \n",
       "1        Cold  Yellow       1          1            1.0           -1.0   \n",
       "2    Very Hot    Blue       1          1            0.0            2.0   \n",
       "3        Warm    Blue       0          1            0.0            0.0   \n",
       "4         Hot     Red       1          1           -1.0           -1.0   \n",
       "5        Warm  Yellow       0          1            0.0            0.0   \n",
       "6        Warm     Red       1          1            0.0            0.0   \n",
       "7         Hot  Yellow       0          1           -1.0           -1.0   \n",
       "8         Hot  Yellow       1          1           -1.0           -1.0   \n",
       "9        Cold  Yellow       1          1            1.0           -1.0   \n",
       "\n",
       "   Temperature_2  \n",
       "0           -1.0  \n",
       "1           -1.0  \n",
       "2           -1.0  \n",
       "3            3.0  \n",
       "4           -1.0  \n",
       "5            3.0  \n",
       "6            3.0  \n",
       "7           -1.0  \n",
       "8           -1.0  \n",
       "9           -1.0  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import category_encoders as ce\n",
    "he = ce.HelmertEncoder(cols=['Temperature'])\n",
    "he_df = he.fit_transform(df['Temperature'])\n",
    "pd.concat([df,he_df],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "7XhAXfXc7vWP"
   },
   "source": [
    "# 8. Mean Encoding or Target Encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "KvTnsnFZ74EE"
   },
   "source": [
    "1. It has become the most popular encoding type because of Kaggle competitions.\n",
    "2. It takes information about the target to encode categories, which makes it extremely powerful.\n",
    "3. In Target Encoding, labels are correlated directly with the target.i.e., for each category in the feature label is decided with the mean value of the target variable on a training data.<br>\n",
    "<img src=\"Screenshots/Target_Encoding.png\"><br>\n",
    "Here, mdl — min data (samples) in leaf,<br> \n",
    "a — smoothing parameter, representing the power of regularization.<br>\n",
    "<img src=\"Screenshots/Mean_Encoding.png\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T06:24:25.799885Z",
     "start_time": "2020-08-24T06:24:25.746064Z"
    },
    "colab": {},
    "colab_type": "code",
    "id": "cHbmYxTNAXJ1"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temperature_ce_TarEnc</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.747629</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.919318</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>0.700000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>0.377041</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.747629</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.377041</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.377041</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.747629</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.747629</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.919318</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temperature_ce_TarEnc\n",
       "0         Hot     Red       1               0.747629\n",
       "1        Cold  Yellow       1               0.919318\n",
       "2    Very Hot    Blue       1               0.700000\n",
       "3        Warm    Blue       0               0.377041\n",
       "4         Hot     Red       1               0.747629\n",
       "5        Warm  Yellow       0               0.377041\n",
       "6        Warm     Red       1               0.377041\n",
       "7         Hot  Yellow       0               0.747629\n",
       "8         Hot  Yellow       1               0.747629\n",
       "9        Cold  Yellow       1               0.919318"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using category_encoders TargetEncoder\n",
    "import category_encoders as ce\n",
    "te = ce.TargetEncoder(cols=['Temperature'])\n",
    "df['Temperature_ce_TarEnc'] = te.fit_transform(df['Temperature'],df.Target)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T06:25:05.146285Z",
     "start_time": "2020-08-24T06:25:05.127335Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temperature_tar_enc</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>0.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temperature_tar_enc\n",
       "0         Hot     Red       1             0.750000\n",
       "1        Cold  Yellow       1             1.000000\n",
       "2    Very Hot    Blue       1             1.000000\n",
       "3        Warm    Blue       0             0.333333\n",
       "4         Hot     Red       1             0.750000\n",
       "5        Warm  Yellow       0             0.333333\n",
       "6        Warm     Red       1             0.333333\n",
       "7         Hot  Yellow       0             0.750000\n",
       "8         Hot  Yellow       1             0.750000\n",
       "9        Cold  Yellow       1             1.000000"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using Pandas groupby()\n",
    "tar_enc = df.groupby('Temperature')['Target'].mean()\n",
    "# print(tar_enc)\n",
    "df['Temperature_tar_enc'] = df['Temperature'].map(tar_enc)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jLvWqPLu_A0D"
   },
   "source": [
    "Advantage :\n",
    "1. It does not affect the volume of the data and helps in faster learning. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PAukimxV_dj7"
   },
   "source": [
    "Disadvantage :\n",
    "1.   Target leakage: it uses information about the target. Because of the target leakage, model overfits the training data which results in unreliable validation and lower test scores.<br>\n",
    "**To reduce the effect of target leakage.**\n",
    "1. Increase regularization\n",
    "2. Add random noise to the representation of the category in train dataset (some sort of augmentation)\n",
    "3. Use Double Validation (using other validation)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "9__o2pCgJ1um"
   },
   "source": [
    "# 9. Weight of Evidence Encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "AsAkPucuJ1kK"
   },
   "source": [
    "1. This method was developed primarily to build a predictive model to evaluate the risk of loan default in the credit and financial industry.\n",
    "2. It is a measure of the “strength” of a grouping for separating good and bad risk (default).\n",
    "3. Weight of evidence (WOE) is a measure of how much the evidence supports or undermines a hypothesis.<br>\n",
    "<img src=\"Screenshots/WoE.PNG\">\n",
    "Distr Goods -> Distribution of Good Credit Outcomes<br>\n",
    "Distr bads -> Distribution of Bad Credit Outcomes<br>\n",
    "However, above formulas might lead to target leakage and overfit. <br>\n",
    "To avoid that, regularization parameter a is induced and WoE is calculated in the following way:<br>\n",
    "<img src=\"Screenshots/WoE1.png\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T07:21:32.611324Z",
     "start_time": "2020-08-24T07:21:32.564450Z"
    },
    "colab": {},
    "colab_type": "code",
    "id": "TreAc_R8J1X7"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temperature_ce_WOE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.105361</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.510826</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.993252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.105361</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.993252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.993252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.105361</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.105361</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.510826</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temperature_ce_WOE\n",
       "0         Hot     Red       1            0.105361\n",
       "1        Cold  Yellow       1            0.510826\n",
       "2    Very Hot    Blue       1            0.000000\n",
       "3        Warm    Blue       0           -0.993252\n",
       "4         Hot     Red       1            0.105361\n",
       "5        Warm  Yellow       0           -0.993252\n",
       "6        Warm     Red       1           -0.993252\n",
       "7         Hot  Yellow       0            0.105361\n",
       "8         Hot  Yellow       1            0.105361\n",
       "9        Cold  Yellow       1            0.510826"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "woe = ce.WOEEncoder(cols=['Temperature'])\n",
    "df['Temperature_ce_WOE'] = woe.fit_transform(df['Temperature'],df.Target)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "QXCQtESQZaBs"
   },
   "source": [
    "# 10. Sum Encoder (Deviation Encoding or Effect Encoding)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Compares the mean of the dependent variable (target) for a given level of a categorical column to the overall mean of the target. \n",
    "2. Sum Encoding is very similar to OHE and both of them are commonly used in Linear Regression (LR) types of models.\n",
    "3. However, the difference between them is the interpretation of LR coefficients: in OHE model the intercept represents the mean for the baseline condition and coefficients represents simple effects (the difference between one particular condition and the baseline), whereas in Sum Encoder model the intercept represents the grand mean (across all conditions) and the coefficients can be interpreted directly as the main effects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T07:39:59.636120Z",
     "start_time": "2020-08-24T07:39:59.576314Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>intercept</th>\n",
       "      <th>Temperature_0</th>\n",
       "      <th>Temperature_1</th>\n",
       "      <th>Temperature_2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  intercept  Temperature_0  Temperature_1  \\\n",
       "0         Hot     Red       1          1            1.0            0.0   \n",
       "1        Cold  Yellow       1          1            0.0            1.0   \n",
       "2    Very Hot    Blue       1          1            0.0            0.0   \n",
       "3        Warm    Blue       0          1           -1.0           -1.0   \n",
       "4         Hot     Red       1          1            1.0            0.0   \n",
       "5        Warm  Yellow       0          1           -1.0           -1.0   \n",
       "6        Warm     Red       1          1           -1.0           -1.0   \n",
       "7         Hot  Yellow       0          1            1.0            0.0   \n",
       "8         Hot  Yellow       1          1            1.0            0.0   \n",
       "9        Cold  Yellow       1          1            0.0            1.0   \n",
       "\n",
       "   Temperature_2  \n",
       "0            0.0  \n",
       "1            0.0  \n",
       "2            1.0  \n",
       "3           -1.0  \n",
       "4            0.0  \n",
       "5           -1.0  \n",
       "6           -1.0  \n",
       "7            0.0  \n",
       "8            0.0  \n",
       "9            0.0  "
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "se = ce.SumEncoder(cols=['Temperature'])\n",
    "se_df = se.fit_transform(df['Temperature'],df.Target)\n",
    "pd.concat([df,se_df],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 11. Leave-one-out Encoder (LOO or LOOE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. It is another example of target-based encoders.\n",
    "2. This encoder calculate mean target of category k for observation j if observation j is removed from the dataset:<br>\n",
    "<img src=\"Screenshots/LOO.png\"><br>\n",
    "While encoding the test dataset, a category is replaced with the mean target of the category k in the train dataset:\n",
    "<img src=\"Screenshots/LOO1.png\"><br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T08:05:47.913638Z",
     "start_time": "2020-08-24T08:05:47.866713Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temperature_ce_CBE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>0.700000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temperature_ce_CBE\n",
       "0         Hot     Red       1            0.666667\n",
       "1        Cold  Yellow       1            1.000000\n",
       "2    Very Hot    Blue       1            0.700000\n",
       "3        Warm    Blue       0            0.500000\n",
       "4         Hot     Red       1            0.666667\n",
       "5        Warm  Yellow       0            0.500000\n",
       "6        Warm     Red       1            0.000000\n",
       "7         Hot  Yellow       0            1.000000\n",
       "8         Hot  Yellow       1            0.666667\n",
       "9        Cold  Yellow       1            1.000000"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loue = ce.LeaveOneOutEncoder(cols=['Temperature'])\n",
    "df['Temperature_ce_CBE'] = loue.fit_transform(df['Temperature'],df.Target)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Disadvantage :\n",
    "1. Just like with all other target-based encoders, the problems with LOO is target leakage."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 12. CatBoost Encoder"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Catboost is a recently created target-based categorical encoder. \n",
    "2. It is intended to overcome target leakage problems inherent in LOO.\n",
    "3. To prevent overfitting, the process of target encoding for train dataset is repeated several times on shuffled versions of the dataset and results are averaged."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T07:50:40.893179Z",
     "start_time": "2020-08-24T07:50:40.818529Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temperature_ce_CBE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.700000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.700000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>0.700000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>0.700000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.850000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.350000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.233333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.900000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.675000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.850000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temperature_ce_CBE\n",
       "0         Hot     Red       1            0.700000\n",
       "1        Cold  Yellow       1            0.700000\n",
       "2    Very Hot    Blue       1            0.700000\n",
       "3        Warm    Blue       0            0.700000\n",
       "4         Hot     Red       1            0.850000\n",
       "5        Warm  Yellow       0            0.350000\n",
       "6        Warm     Red       1            0.233333\n",
       "7         Hot  Yellow       0            0.900000\n",
       "8         Hot  Yellow       1            0.675000\n",
       "9        Cold  Yellow       1            0.850000"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cbe = ce.CatBoostEncoder(cols=['Temperature'])\n",
    "df['Temperature_ce_CBE'] = cbe.fit_transform(df['Temperature'],df.Target)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 13. James-Stein Encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. James-Stein Encoder is a target-based encoder.\n",
    "2. The idea behind James-Stein Encoder is simple. Estimation of the mean target for category k could be calculated according to the following formula:\n",
    "<img src=\"Screenshots/James_Stein_Encoder.png\"><br>\n",
    "One way to select B is to tune it like a hyperparameter via cross-validation, but Charles Stein came up with another solution to the problem:<br>\n",
    "<img src=\"Screenshots/James_Stein_Encoder.png\"><br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T08:40:27.122335Z",
     "start_time": "2020-08-24T08:40:27.071317Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temperature_ce_JSE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.741379</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>0.405229</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.741379</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.405229</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.405229</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.741379</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.741379</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temperature_ce_JSE\n",
       "0         Hot     Red       1            0.741379\n",
       "1        Cold  Yellow       1            1.000000\n",
       "2    Very Hot    Blue       1            1.000000\n",
       "3        Warm    Blue       0            0.405229\n",
       "4         Hot     Red       1            0.741379\n",
       "5        Warm  Yellow       0            0.405229\n",
       "6        Warm     Red       1            0.405229\n",
       "7         Hot  Yellow       0            0.741379\n",
       "8         Hot  Yellow       1            0.741379\n",
       "9        Cold  Yellow       1            1.000000"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "jse = ce.JamesSteinEncoder(cols=['Temperature'])\n",
    "df['Temperature_ce_JSE'] = jse.fit_transform(df['Temperature'],df.Target)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Disadvantage :\n",
    "1. It is defined only for normal distribution (which is not the case in real time).\n",
    "2. **To avoid that**, we can either convert binary targets with a log-odds ratio as it was done in WoE Encoder (which is used by default because it is simple) or use beta distribution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 14. M-estimator Encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. M-Estimate Encoder is a simplified version of Target Encoder. \n",
    "2. It has only one hyperparameter — m, which represents the power of regularization. \n",
    "3. The higher value of m results into stronger shrinking. \n",
    "4. Recommended values for m is in the range of 1 to 100. <br>\n",
    "<img src=\"Screenshots/M-Estimate_Encoder.png\"><br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T08:52:54.431801Z",
     "start_time": "2020-08-24T08:52:54.385456Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>Temperature_ce_JSE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.740</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>0.850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>0.425</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.740</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.425</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0.425</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0.740</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.740</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0.900</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  Temperature_ce_JSE\n",
       "0         Hot     Red       1               0.740\n",
       "1        Cold  Yellow       1               0.900\n",
       "2    Very Hot    Blue       1               0.850\n",
       "3        Warm    Blue       0               0.425\n",
       "4         Hot     Red       1               0.740\n",
       "5        Warm  Yellow       0               0.425\n",
       "6        Warm     Red       1               0.425\n",
       "7         Hot  Yellow       0               0.740\n",
       "8         Hot  Yellow       1               0.740\n",
       "9        Cold  Yellow       1               0.900"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mee = ce.MEstimateEncoder(cols=['Temperature'],m=1.0)\n",
    "df['Temperature_ce_JSE'] = mee.fit_transform(df['Temperature'],df.Target)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 15. Hashing Encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Hashing converts categorical variables to a higher dimensional space of integers, where the distance between two vectors of categorical variables in approximately maintained the transformed numerical dimensional space. \n",
    "2. With Hashing, the number of dimensions will be far less than the number of dimensions with encoding like One Hot Encoding. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T09:22:45.995351Z",
     "start_time": "2020-08-24T09:22:36.299386Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>col_0</th>\n",
       "      <th>col_1</th>\n",
       "      <th>col_2</th>\n",
       "      <th>col_3</th>\n",
       "      <th>col_4</th>\n",
       "      <th>col_5</th>\n",
       "      <th>col_6</th>\n",
       "      <th>col_7</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  col_0  col_1  col_2  col_3  col_4  col_5  \\\n",
       "0         Hot     Red       1      1      0      0      0      0      0   \n",
       "1        Cold  Yellow       1      0      0      1      0      0      0   \n",
       "2    Very Hot    Blue       1      0      1      0      0      0      0   \n",
       "3        Warm    Blue       0      0      1      0      0      0      0   \n",
       "4         Hot     Red       1      1      0      0      0      0      0   \n",
       "5        Warm  Yellow       0      0      1      0      0      0      0   \n",
       "6        Warm     Red       1      0      1      0      0      0      0   \n",
       "7         Hot  Yellow       0      1      0      0      0      0      0   \n",
       "8         Hot  Yellow       1      1      0      0      0      0      0   \n",
       "9        Cold  Yellow       1      0      0      1      0      0      0   \n",
       "\n",
       "   col_6  col_7  \n",
       "0      0      0  \n",
       "1      0      0  \n",
       "2      0      0  \n",
       "3      0      0  \n",
       "4      0      0  \n",
       "5      0      0  \n",
       "6      0      0  \n",
       "7      0      0  \n",
       "8      0      0  \n",
       "9      0      0  "
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hash_df = ce.HashingEncoder(cols=['Temperature'],n_components=8)\n",
    "hash_df = hash_df.fit_transform(df['Temperature'],df.Target)\n",
    "pd.concat([df,hash_df],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Advantage :\n",
    "1. This method is advantageous when the cardinality of categorical is very high with parameter **n_components**.<br>\n",
    "\n",
    "Disadvantage :\n",
    "1. It is slow comparing to other encoder's."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 16. Backward Difference Encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. In backward difference coding, the mean of the dependent variable for a level is compared with the mean of the dependent variable for the prior level. \n",
    "2. This technique falls under the contrast coding system for categorical features. A feature of K categories, or levels, usually enters a regression as a sequence of K-1 dummy variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T09:32:56.077938Z",
     "start_time": "2020-08-24T09:32:56.016090Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>intercept</th>\n",
       "      <th>Temperature_0</th>\n",
       "      <th>Temperature_1</th>\n",
       "      <th>Temperature_2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.75</td>\n",
       "      <td>-0.5</td>\n",
       "      <td>-0.25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.25</td>\n",
       "      <td>-0.5</td>\n",
       "      <td>-0.25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.25</td>\n",
       "      <td>0.5</td>\n",
       "      <td>-0.25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.25</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.75</td>\n",
       "      <td>-0.5</td>\n",
       "      <td>-0.25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.25</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.25</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.75</td>\n",
       "      <td>-0.5</td>\n",
       "      <td>-0.25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.75</td>\n",
       "      <td>-0.5</td>\n",
       "      <td>-0.25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.25</td>\n",
       "      <td>-0.5</td>\n",
       "      <td>-0.25</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  intercept  Temperature_0  Temperature_1  \\\n",
       "0         Hot     Red       1          1          -0.75           -0.5   \n",
       "1        Cold  Yellow       1          1           0.25           -0.5   \n",
       "2    Very Hot    Blue       1          1           0.25            0.5   \n",
       "3        Warm    Blue       0          1           0.25            0.5   \n",
       "4         Hot     Red       1          1          -0.75           -0.5   \n",
       "5        Warm  Yellow       0          1           0.25            0.5   \n",
       "6        Warm     Red       1          1           0.25            0.5   \n",
       "7         Hot  Yellow       0          1          -0.75           -0.5   \n",
       "8         Hot  Yellow       1          1          -0.75           -0.5   \n",
       "9        Cold  Yellow       1          1           0.25           -0.5   \n",
       "\n",
       "   Temperature_2  \n",
       "0          -0.25  \n",
       "1          -0.25  \n",
       "2          -0.25  \n",
       "3           0.75  \n",
       "4          -0.25  \n",
       "5           0.75  \n",
       "6           0.75  \n",
       "7          -0.25  \n",
       "8          -0.25  \n",
       "9          -0.25  "
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bde = ce.BackwardDifferenceEncoder(cols=['Temperature'])\n",
    "bde_df = bde.fit_transform(df['Temperature'],df.Target)\n",
    "pd.concat([df,bde_df],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T10:03:28.678422Z",
     "start_time": "2020-08-24T10:03:28.659784Z"
    }
   },
   "source": [
    "# 17. Polynomial Encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-24T10:04:05.781467Z",
     "start_time": "2020-08-24T10:04:05.726124Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Color</th>\n",
       "      <th>Target</th>\n",
       "      <th>intercept</th>\n",
       "      <th>Temperature_0</th>\n",
       "      <th>Temperature_1</th>\n",
       "      <th>Temperature_2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.670820</td>\n",
       "      <td>0.5</td>\n",
       "      <td>-0.223607</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.223607</td>\n",
       "      <td>-0.5</td>\n",
       "      <td>0.670820</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Very Hot</td>\n",
       "      <td>Blue</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.223607</td>\n",
       "      <td>-0.5</td>\n",
       "      <td>-0.670820</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Blue</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.670820</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.223607</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.670820</td>\n",
       "      <td>0.5</td>\n",
       "      <td>-0.223607</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.670820</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.223607</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Warm</td>\n",
       "      <td>Red</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.670820</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.223607</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.670820</td>\n",
       "      <td>0.5</td>\n",
       "      <td>-0.223607</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hot</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.670820</td>\n",
       "      <td>0.5</td>\n",
       "      <td>-0.223607</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Cold</td>\n",
       "      <td>Yellow</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.223607</td>\n",
       "      <td>-0.5</td>\n",
       "      <td>0.670820</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Temperature   Color  Target  intercept  Temperature_0  Temperature_1  \\\n",
       "0         Hot     Red       1          1      -0.670820            0.5   \n",
       "1        Cold  Yellow       1          1      -0.223607           -0.5   \n",
       "2    Very Hot    Blue       1          1       0.223607           -0.5   \n",
       "3        Warm    Blue       0          1       0.670820            0.5   \n",
       "4         Hot     Red       1          1      -0.670820            0.5   \n",
       "5        Warm  Yellow       0          1       0.670820            0.5   \n",
       "6        Warm     Red       1          1       0.670820            0.5   \n",
       "7         Hot  Yellow       0          1      -0.670820            0.5   \n",
       "8         Hot  Yellow       1          1      -0.670820            0.5   \n",
       "9        Cold  Yellow       1          1      -0.223607           -0.5   \n",
       "\n",
       "   Temperature_2  \n",
       "0      -0.223607  \n",
       "1       0.670820  \n",
       "2      -0.670820  \n",
       "3       0.223607  \n",
       "4      -0.223607  \n",
       "5       0.223607  \n",
       "6       0.223607  \n",
       "7      -0.223607  \n",
       "8      -0.223607  \n",
       "9       0.670820  "
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pe = ce.PolynomialEncoder(cols=['Temperature'])\n",
    "pe_df = pe.fit_transform(df['Temperature'],df.Target)\n",
    "pd.concat([df,pe_df],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 18. MultiLabelBinarizer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " MultiLabel Binarizer is used when any column has multiple labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-09T17:56:29.090636Z",
     "start_time": "2020-09-09T17:56:29.071685Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[fruits, vegitables]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[animals, vegitables]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[animals, fruits]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[vehicals, fruits]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    Type\n",
       "0   [fruits, vegitables]\n",
       "1  [animals, vegitables]\n",
       "2      [animals, fruits]\n",
       "3     [vehicals, fruits]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = {'Type':[['fruits','vegitables'],['animals','vegitables'],['animals','fruits'],['vehicals','fruits']]}\n",
    "df = pd.DataFrame(data)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-09T17:56:30.531106Z",
     "start_time": "2020-09-09T17:56:30.513158Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>animals</th>\n",
       "      <th>fruits</th>\n",
       "      <th>vegitables</th>\n",
       "      <th>vehicals</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   animals  fruits  vegitables  vehicals\n",
       "0        0       1           1         0\n",
       "1        1       0           1         0\n",
       "2        1       1           0         0\n",
       "3        0       1           0         1"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# importing MultiLabelBinarizer\n",
    "from sklearn.preprocessing import MultiLabelBinarizer\n",
    "\n",
    "# instantiating MultiLabelBinarizer\n",
    "mlb = MultiLabelBinarizer()\n",
    "types_encoded = pd.DataFrame(mlb.fit_transform(df['Type']),columns=mlb.classes_)\n",
    "types_encoded.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-danger\"> \n",
    "<b>It is essential to understand, for all machine learning models, all these encodings do not work well in all situations or for every dataset. <br>\n",
    "Data Scientists still need to experiment and find out which works best for their specific case.  </b>\n",
    "</div>\n"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [
    "TI9yRFgZu5Ee",
    "ejGLTAH61Cea",
    "DIWKGq5kHeGo",
    "IqQ2zggJJth8",
    "KC_X6344holD"
   ],
   "name": "Handling Categorical Variables.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}