答案代码-练习10-租金预测实战(xgboost 0.82or0.9版本）.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 运用公开住宅出租挂牌数据进行住宅租金预测\n",
    "\n",
    "此脚本适用于使用xgboost0.82和0.90版本的同学：由于xgboost模型0.81版本和0.82及以上版本差距较大，训练出来的模型相对重要性会有非常大的差距，因此，此版本代码根据0.82版本的模型的训练结果，更新了模型优化策略，修改了样例结果。\n",
    "\n",
    "### 数据来源：\n",
    "\n",
    "* 上海在租房源信息\n",
    "\n",
    "* 时间节点：2019年7月17日\n",
    "\n",
    "* 数据集包括在线租房源信息及其基本情况\n",
    "\n",
    "* 属性信息：楼盘名称；租赁方式；面积；朝向；房租；城市；区域；街道或片区；地址；公交站等\n",
    "\n",
    "### 学习目标：\n",
    "\n",
    "#### 1）探索查看真实房源数据，找一找数据存在哪些问题？\n",
    "\n",
    "* 要求：\n",
    "\n",
    "  ① 读取一个真实房源的数据表\n",
    "  \n",
    "  ② 查看已有的数据存在哪些问题，确认数据应该如何清洗\n",
    "  \n",
    "#### 2）针对数据存在的问题，应该采取怎么样的数据清洗策略？\n",
    "\n",
    "* 要求：\n",
    "\n",
    "  ① 根据上一节发现的数据问题，编写数据清洗函数，并测试清洗函数\n",
    "  \n",
    "  ② 使用数据清洗函数对所有真实房源数据表进行处理并合并\n",
    "  \n",
    "  ③ 输出清洗合并后的数据表\n",
    "\n",
    "#### 3）如何根据房源地理位置信息关联更多空间特征来提高租金模型的泛化能力？\n",
    "\n",
    "* 要求：\n",
    "\n",
    "  ① 利用已有的数据获取有效的样本地址信息\n",
    "  \n",
    "  ② 根据地址信息关联更多的空间特征\n",
    "  \n",
    "  ③ 输出关联了地理特征的完整模型数据表\n",
    "\n",
    "\n",
    "#### 4）如何进行模型数据预处理？\n",
    "\n",
    "* 要求：\n",
    "\n",
    "  ① 以平米租金（单位：元/月/每平米)为应变量\n",
    "  \n",
    "  ② 处理模型数据中存在的缺失值、异常值问题\n",
    "  \n",
    "  ③ 输出最终的建模数据表\n",
    "\n",
    "#### 5）选取不同的模型进行模型训练并对比模型效果、优化模型效果，确定最终预测模型\n",
    "\n",
    "* 要求：\n",
    "\n",
    "  ① 使用留一法进行数据集划分，验证集比例：30%\n",
    "  \n",
    "  ② 训练至少两个不同类型的模型进行租金模型训练，并对比模型效果\n",
    "  \n",
    "  ③ 优化模型效果，确定最佳模型，并保存模型文件\n",
    "\n",
    "\n",
    "### 建议工作流程：\n",
    "\n",
    "#### 1. 数据探索\n",
    "\n",
    "#### 2. 数据清洗\n",
    "\n",
    "#### 3. 特征补充\n",
    "\n",
    "#### 4. 数据预处理\n",
    "\n",
    "#### 5. 训练并优化模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# **加载必要的模块**\n",
    "\n",
    "# 基本模块：时间与路径\n",
    "import os\n",
    "import time\n",
    "import string\n",
    "# 数据读取与计算\n",
    "import math\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "# 绘图\n",
    "import matplotlib.pyplot as plt\n",
    "# 模型数据集划分\n",
    "from sklearn.model_selection import train_test_split, KFold\n",
    "# 模型流水线整合器\n",
    "from sklearn.pipeline import Pipeline\n",
    "# 模型数据预处理\n",
    "from sklearn.preprocessing import PowerTransformer, StandardScaler, PolynomialFeatures\n",
    "# 模型\n",
    "from sklearn.linear_model import LassoCV\n",
    "import xgboost as xgb\n",
    "# 加载回归模型的评价指标\n",
    "from sklearn.metrics import mean_absolute_error\n",
    "# 保存模型模块\n",
    "from sklearn.externals import joblib \n",
    "# 屏蔽warnings\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据初探\n",
    "\n",
    "查看数据的基本情况，发现数据存在的问题，针对问题制定下一步的数据处理计划"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C:\\Users\\wcd\\租金实战forxgboost082andabove\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>楼盘名称</th>\n",
       "      <th>租赁方式</th>\n",
       "      <th>户型</th>\n",
       "      <th>面积</th>\n",
       "      <th>朝向</th>\n",
       "      <th>房租</th>\n",
       "      <th>更新时间</th>\n",
       "      <th>采集时间</th>\n",
       "      <th>城市</th>\n",
       "      <th>区域</th>\n",
       "      <th>街道或片区</th>\n",
       "      <th>地段</th>\n",
       "      <th>地址</th>\n",
       "      <th>公交站</th>\n",
       "      <th>主图链接</th>\n",
       "      <th>楼盘详情链接</th>\n",
       "      <th>页面网址</th>\n",
       "      <th>标签</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>鹏海小区,13号,线陈春路站,交通方便,商业地段,随时看房。</td>\n",
       "      <td>整租</td>\n",
       "      <td>2室1厅</td>\n",
       "      <td>65㎡</td>\n",
       "      <td>朝南</td>\n",
       "      <td>4500元/月</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2019-07-17 19:34:54</td>\n",
       "      <td>上海</td>\n",
       "      <td>浦东</td>\n",
       "      <td>北蔡</td>\n",
       "      <td>鹏海东苑</td>\n",
       "      <td>浦东-北蔡-鹏海东苑</td>\n",
       "      <td>距13号线陈春路站约641米。</td>\n",
       "      <td>https://img.soufunimg.com/secondhouse/image/lo...</td>\n",
       "      <td>https://sh.zu.fang.com/chuzu/3_356163941_1.htm</td>\n",
       "      <td>https://sh.zu.fang.com/house-a025-b01646/</td>\n",
       "      <td>繁华地段/家电齐全/拎包入住</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>十六街区 欧式风格 南北通 拎包入住</td>\n",
       "      <td>整租</td>\n",
       "      <td>3室2厅</td>\n",
       "      <td>148㎡</td>\n",
       "      <td>朝南</td>\n",
       "      <td>14000元/月</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2019-07-17 19:34:54</td>\n",
       "      <td>上海</td>\n",
       "      <td>浦东</td>\n",
       "      <td>北蔡</td>\n",
       "      <td>大华锦绣华城第19街区</td>\n",
       "      <td>浦东-北蔡-大华锦绣华城第19街区</td>\n",
       "      <td>距13号线下南路站约423米。</td>\n",
       "      <td>https://img.soufunimg.com/secondhouse/image/lo...</td>\n",
       "      <td>https://sh.zu.fang.com/chuzu/3_356381296_1.htm</td>\n",
       "      <td>https://sh.zu.fang.com/house-a025-b01646/</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>莲宏苑,精装全配复式,三房两厅两卫,随时看房,期待来电</td>\n",
       "      <td>整租</td>\n",
       "      <td>3室1厅</td>\n",
       "      <td>124㎡</td>\n",
       "      <td>朝南</td>\n",
       "      <td>6800元/月</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2019-07-17 19:34:54</td>\n",
       "      <td>上海</td>\n",
       "      <td>浦东</td>\n",
       "      <td>北蔡</td>\n",
       "      <td>莲宏苑</td>\n",
       "      <td>浦东-北蔡-莲宏苑</td>\n",
       "      <td>距13号线北蔡站约765米。</td>\n",
       "      <td>https://img.soufunimg.com/secondhouse/image/lo...</td>\n",
       "      <td>https://sh.zu.fang.com/chuzu/3_356095628_1.htm</td>\n",
       "      <td>https://sh.zu.fang.com/house-a025-b01646/</td>\n",
       "      <td>繁华地段/家电齐全/拎包入住</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>万科海上传奇新出两房,一家人住优先!惜缺精装修全配,出租!</td>\n",
       "      <td>整租</td>\n",
       "      <td>2室2厅</td>\n",
       "      <td>92㎡</td>\n",
       "      <td>朝南</td>\n",
       "      <td>6500元/月</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2019-07-17 19:34:54</td>\n",
       "      <td>上海</td>\n",
       "      <td>浦东</td>\n",
       "      <td>北蔡</td>\n",
       "      <td>万科海上传奇</td>\n",
       "      <td>浦东-北蔡-万科海上传奇</td>\n",
       "      <td>距11号线御桥站约376米。</td>\n",
       "      <td>https://img.soufunimg.com/secondhouse/image/lo...</td>\n",
       "      <td>https://sh.zu.fang.com/chuzu/3_325998408_1.htm</td>\n",
       "      <td>https://sh.zu.fang.com/house-a025-b01646/</td>\n",
       "      <td>全装全配/紧邻地铁/家电齐全</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>13号。线。北蔡加东花园,全配,随时看房,期待来电</td>\n",
       "      <td>整租</td>\n",
       "      <td>2室2厅</td>\n",
       "      <td>89㎡</td>\n",
       "      <td>朝西</td>\n",
       "      <td>4800元/月</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2019-07-17 19:34:54</td>\n",
       "      <td>上海</td>\n",
       "      <td>浦东</td>\n",
       "      <td>北蔡</td>\n",
       "      <td>通宇加东花园</td>\n",
       "      <td>浦东-北蔡-通宇加东花园</td>\n",
       "      <td>距13号线北蔡站约193米。</td>\n",
       "      <td>https://img.soufunimg.com/secondhouse/image/lo...</td>\n",
       "      <td>https://sh.zu.fang.com/chuzu/3_356116300_1.htm</td>\n",
       "      <td>https://sh.zu.fang.com/house-a025-b01646/</td>\n",
       "      <td>采光好/紧邻地铁/首次出租</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             楼盘名称 租赁方式    户型    面积  朝向        房租 更新时间  \\\n",
       "0  鹏海小区,13号,线陈春路站,交通方便,商业地段,随时看房。   整租  2室1厅   65㎡  朝南   4500元/月  NaN   \n",
       "1              十六街区 欧式风格 南北通 拎包入住   整租  3室2厅  148㎡  朝南  14000元/月  NaN   \n",
       "2     莲宏苑,精装全配复式,三房两厅两卫,随时看房,期待来电   整租  3室1厅  124㎡  朝南   6800元/月  NaN   \n",
       "3   万科海上传奇新出两房,一家人住优先!惜缺精装修全配,出租!   整租  2室2厅   92㎡  朝南   6500元/月  NaN   \n",
       "4       13号。线。北蔡加东花园,全配,随时看房,期待来电   整租  2室2厅   89㎡  朝西   4800元/月  NaN   \n",
       "\n",
       "                  采集时间  城市  区域 街道或片区           地段                 地址  \\\n",
       "0  2019-07-17 19:34:54  上海  浦东    北蔡         鹏海东苑         浦东-北蔡-鹏海东苑   \n",
       "1  2019-07-17 19:34:54  上海  浦东    北蔡  大华锦绣华城第19街区  浦东-北蔡-大华锦绣华城第19街区   \n",
       "2  2019-07-17 19:34:54  上海  浦东    北蔡          莲宏苑          浦东-北蔡-莲宏苑   \n",
       "3  2019-07-17 19:34:54  上海  浦东    北蔡       万科海上传奇       浦东-北蔡-万科海上传奇   \n",
       "4  2019-07-17 19:34:54  上海  浦东    北蔡       通宇加东花园       浦东-北蔡-通宇加东花园   \n",
       "\n",
       "               公交站                                               主图链接  \\\n",
       "0  距13号线陈春路站约641米。  https://img.soufunimg.com/secondhouse/image/lo...   \n",
       "1  距13号线下南路站约423米。  https://img.soufunimg.com/secondhouse/image/lo...   \n",
       "2   距13号线北蔡站约765米。  https://img.soufunimg.com/secondhouse/image/lo...   \n",
       "3   距11号线御桥站约376米。  https://img.soufunimg.com/secondhouse/image/lo...   \n",
       "4   距13号线北蔡站约193米。  https://img.soufunimg.com/secondhouse/image/lo...   \n",
       "\n",
       "                                           楼盘详情链接  \\\n",
       "0  https://sh.zu.fang.com/chuzu/3_356163941_1.htm   \n",
       "1  https://sh.zu.fang.com/chuzu/3_356381296_1.htm   \n",
       "2  https://sh.zu.fang.com/chuzu/3_356095628_1.htm   \n",
       "3  https://sh.zu.fang.com/chuzu/3_325998408_1.htm   \n",
       "4  https://sh.zu.fang.com/chuzu/3_356116300_1.htm   \n",
       "\n",
       "                                        页面网址              标签  \n",
       "0  https://sh.zu.fang.com/house-a025-b01646/  繁华地段/家电齐全/拎包入住  \n",
       "1  https://sh.zu.fang.com/house-a025-b01646/             NaN  \n",
       "2  https://sh.zu.fang.com/house-a025-b01646/  繁华地段/家电齐全/拎包入住  \n",
       "3  https://sh.zu.fang.com/house-a025-b01646/  全装全配/紧邻地铁/家电齐全  \n",
       "4  https://sh.zu.fang.com/house-a025-b01646/   采光好/紧邻地铁/首次出租  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设置读取文件路径为脚本所在位置\n",
    "# 读取文件\n",
    "current_path = os.getcwd()\n",
    "print(current_path)\n",
    "data_path = os.path.join(current_path,'data', '租房信息a.csv')\n",
    "df = pd.read_csv(open(data_path,encoding='utf-8'))\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "数据维度：(20000, 18)\n",
      "\n",
      "特征类型：\n",
      "楼盘名称      object\n",
      "租赁方式      object\n",
      "户型        object\n",
      "面积        object\n",
      "朝向        object\n",
      "房租        object\n",
      "更新时间      object\n",
      "采集时间      object\n",
      "城市        object\n",
      "区域        object\n",
      "街道或片区     object\n",
      "地段        object\n",
      "地址        object\n",
      "公交站       object\n",
      "主图链接      object\n",
      "楼盘详情链接    object\n",
      "页面网址      object\n",
      "标签        object\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "print(f'数据维度：{df.shape}\\n')\n",
    "print(f'特征类型：\\n{df.dtypes}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<租赁方式>的分类别频数统计：\n",
      "整租       19066\n",
      "合租主卧       520\n",
      "合租次卧       231\n",
      "整租㎡        161\n",
      "合租单间        16\n",
      "合租主卧㎡        5\n",
      "合租次卧㎡        1\n",
      "Name: 租赁方式, dtype: int64\n",
      "\n",
      "<户型>的分类别频数统计：\n",
      "2室2厅    4848\n",
      "3室2厅    3907\n",
      "2室1厅    3589\n",
      "1室1厅    3026\n",
      "3室1厅    1177\n",
      "4室2厅    1137\n",
      "1室0厅     603\n",
      "2户合租     398\n",
      "1室2厅     273\n",
      "3户合租     262\n",
      "4室3厅     166\n",
      "5室3厅     124\n",
      "5室2厅     121\n",
      "4室1厅      67\n",
      "4户合租      53\n",
      "2室0厅      39\n",
      "3室3厅      37\n",
      "5户合租      35\n",
      "7室1厅      19\n",
      "1户合租      19\n",
      "6室2厅      19\n",
      "6室3厅      16\n",
      "8室2厅      10\n",
      "4室4厅      10\n",
      "6室4厅       8\n",
      "7室3厅       8\n",
      "5室4厅       7\n",
      "6户合租       6\n",
      "7室2厅       5\n",
      "2室3厅       5\n",
      "0室1厅       3\n",
      "1室3厅       1\n",
      "8室3厅       1\n",
      "3室0厅       1\n",
      "Name: 户型, dtype: int64\n",
      "\n",
      "<朝向>的分类别频数统计：\n",
      "朝南    18740\n",
      "朝东      611\n",
      "朝北      315\n",
      "朝西      167\n",
      "Name: 朝向, dtype: int64\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for var in ['租赁方式','户型','朝向']:\n",
    "    print(f'<{var}>的分类别频数统计：\\n{df[var].value_counts()}\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据初探：结论\n",
    "\n",
    "#### 一、数据需要清洗\n",
    "\n",
    "* 连续数值型变量租金、面积包含单位字符，导致数据类型为object\n",
    "* 类别变量租赁方式的类别记录标准不一，部分类别中包含特殊字符\n",
    "* 类别变量户型中的种类众多；记录方式不统一；部分记录中包含整租合租信息\n",
    "* 类别变量朝向中朝南的样本比例非常高，可能属于方差近零特征\n",
    "* 公交站字段中信息为文字描述，站点、距离信息需要提取\n",
    "\n",
    "#### 二、数据包含信息较少\n",
    "\n",
    "* 数据中不包含房屋装修情况、房龄等区分房屋品质的变量\n",
    "* 数据中相关区位的特征只有公交站、所在片区两个字段\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据清洗与信息提取"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义数据处理函数\n",
    "def data_extract(df):\n",
    "    # 面积，租金，租赁方式特殊符号去除\n",
    "    df['area'] = (df['面积'].str.replace('㎡','')).astype(float)\n",
    "    df['total_price'] = (df['房租'].str.replace('元/月','')).astype(float)\n",
    "    df['rental_type'] = df['租赁方式'].str.replace('㎡','')\n",
    "    # 户型，公交站信息提取\n",
    "    room_type = df['户型'].str.extract('(?P<room>\\d+)室(?P<hall>\\d+)厅')\n",
    "    subway = df['公交站'].str.extract('距(?P<sub_line>\\d+)号线(?P<sub_station>\\w+)站约(?P<sub_dist>\\d+)米。')\n",
    "    # 朝向制作哑变量：\n",
    "    df['is_nan'] = df['朝向'].map(lambda x: 1 if x=='朝南' else 0)\n",
    "    # 地址\n",
    "    df['address'] = df['城市'] + '市' + df['区域'] + '区' + df['地段']\n",
    "    # 整合\n",
    "    result = pd.concat([df[['id','total_price','area', 'is_nan', 'rental_type', 'address','区域']], room_type, subway], axis=1)\n",
    "    for var in [ 'room', 'hall','sub_dist']:\n",
    "        result[var] = result[var].astype(float)\n",
    "    return result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用加载的第一份表格数据测试定义好的数据处理函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['id', 'total_price', 'area', 'is_nan', 'rental_type', 'address', '区域',\n",
      "       'room', 'hall', 'sub_line', 'sub_station', 'sub_dist'],\n",
      "      dtype='object')\n",
      "id              object\n",
      "total_price    float64\n",
      "area           float64\n",
      "is_nan           int64\n",
      "rental_type     object\n",
      "address         object\n",
      "区域              object\n",
      "room           float64\n",
      "hall           float64\n",
      "sub_line        object\n",
      "sub_station     object\n",
      "sub_dist       float64\n",
      "dtype: object\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>total_price</th>\n",
       "      <th>area</th>\n",
       "      <th>is_nan</th>\n",
       "      <th>rental_type</th>\n",
       "      <th>address</th>\n",
       "      <th>区域</th>\n",
       "      <th>room</th>\n",
       "      <th>hall</th>\n",
       "      <th>sub_line</th>\n",
       "      <th>sub_station</th>\n",
       "      <th>sub_dist</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>a_0</td>\n",
       "      <td>4500.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>1</td>\n",
       "      <td>整租</td>\n",
       "      <td>上海市浦东区鹏海东苑</td>\n",
       "      <td>浦东</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13</td>\n",
       "      <td>陈春路</td>\n",
       "      <td>641.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>a_1</td>\n",
       "      <td>14000.0</td>\n",
       "      <td>148.0</td>\n",
       "      <td>1</td>\n",
       "      <td>整租</td>\n",
       "      <td>上海市浦东区大华锦绣华城第19街区</td>\n",
       "      <td>浦东</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>13</td>\n",
       "      <td>下南路</td>\n",
       "      <td>423.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>a_2</td>\n",
       "      <td>6800.0</td>\n",
       "      <td>124.0</td>\n",
       "      <td>1</td>\n",
       "      <td>整租</td>\n",
       "      <td>上海市浦东区莲宏苑</td>\n",
       "      <td>浦东</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13</td>\n",
       "      <td>北蔡</td>\n",
       "      <td>765.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>a_3</td>\n",
       "      <td>6500.0</td>\n",
       "      <td>92.0</td>\n",
       "      <td>1</td>\n",
       "      <td>整租</td>\n",
       "      <td>上海市浦东区万科海上传奇</td>\n",
       "      <td>浦东</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>11</td>\n",
       "      <td>御桥</td>\n",
       "      <td>376.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>a_4</td>\n",
       "      <td>4800.0</td>\n",
       "      <td>89.0</td>\n",
       "      <td>0</td>\n",
       "      <td>整租</td>\n",
       "      <td>上海市浦东区通宇加东花园</td>\n",
       "      <td>浦东</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>13</td>\n",
       "      <td>北蔡</td>\n",
       "      <td>193.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id  total_price   area  is_nan rental_type            address  区域  room  \\\n",
       "0  a_0       4500.0   65.0       1          整租         上海市浦东区鹏海东苑  浦东   2.0   \n",
       "1  a_1      14000.0  148.0       1          整租  上海市浦东区大华锦绣华城第19街区  浦东   3.0   \n",
       "2  a_2       6800.0  124.0       1          整租          上海市浦东区莲宏苑  浦东   3.0   \n",
       "3  a_3       6500.0   92.0       1          整租       上海市浦东区万科海上传奇  浦东   2.0   \n",
       "4  a_4       4800.0   89.0       0          整租       上海市浦东区通宇加东花园  浦东   2.0   \n",
       "\n",
       "   hall sub_line sub_station  sub_dist  \n",
       "0   1.0       13         陈春路     641.0  \n",
       "1   2.0       13         下南路     423.0  \n",
       "2   1.0       13          北蔡     765.0  \n",
       "3   2.0       11          御桥     376.0  \n",
       "4   2.0       13          北蔡     193.0  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 测试处理函数\n",
    "df.insert(0,'id', [f'a_{j}' for j in range(0,len(df))])\n",
    "test_df = data_extract(df)\n",
    "print(test_df.columns)\n",
    "print(test_df.dtypes)\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "处理函数的测试通过\n",
    "\n",
    "运用循环读取每个文件并进行处理，处理之后将处理结果保留"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(222243, 12)\n",
      "Index(['id', 'total_price', 'area', 'is_nan', 'rental_type', 'address', '区域',\n",
      "       'room', 'hall', 'sub_line', 'sub_station', 'sub_dist'],\n",
      "      dtype='object')\n",
      "id              object\n",
      "total_price    float64\n",
      "area           float64\n",
      "is_nan           int64\n",
      "rental_type     object\n",
      "address         object\n",
      "区域              object\n",
      "room           float64\n",
      "hall           float64\n",
      "sub_line        object\n",
      "sub_station     object\n",
      "sub_dist       float64\n",
      "dtype: object\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>total_price</th>\n",
       "      <th>area</th>\n",
       "      <th>is_nan</th>\n",
       "      <th>rental_type</th>\n",
       "      <th>address</th>\n",
       "      <th>区域</th>\n",
       "      <th>room</th>\n",
       "      <th>hall</th>\n",
       "      <th>sub_line</th>\n",
       "      <th>sub_station</th>\n",
       "      <th>sub_dist</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>a_0</td>\n",
       "      <td>4500.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>1</td>\n",
       "      <td>整租</td>\n",
       "      <td>上海市浦东区鹏海东苑</td>\n",
       "      <td>浦东</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13</td>\n",
       "      <td>陈春路</td>\n",
       "      <td>641.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>a_1</td>\n",
       "      <td>14000.0</td>\n",
       "      <td>148.0</td>\n",
       "      <td>1</td>\n",
       "      <td>整租</td>\n",
       "      <td>上海市浦东区大华锦绣华城第19街区</td>\n",
       "      <td>浦东</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>13</td>\n",
       "      <td>下南路</td>\n",
       "      <td>423.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>a_2</td>\n",
       "      <td>6800.0</td>\n",
       "      <td>124.0</td>\n",
       "      <td>1</td>\n",
       "      <td>整租</td>\n",
       "      <td>上海市浦东区莲宏苑</td>\n",
       "      <td>浦东</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13</td>\n",
       "      <td>北蔡</td>\n",
       "      <td>765.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>a_3</td>\n",
       "      <td>6500.0</td>\n",
       "      <td>92.0</td>\n",
       "      <td>1</td>\n",
       "      <td>整租</td>\n",
       "      <td>上海市浦东区万科海上传奇</td>\n",
       "      <td>浦东</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>11</td>\n",
       "      <td>御桥</td>\n",
       "      <td>376.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>a_4</td>\n",
       "      <td>4800.0</td>\n",
       "      <td>89.0</td>\n",
       "      <td>0</td>\n",
       "      <td>整租</td>\n",
       "      <td>上海市浦东区通宇加东花园</td>\n",
       "      <td>浦东</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>13</td>\n",
       "      <td>北蔡</td>\n",
       "      <td>193.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id  total_price   area  is_nan rental_type            address  区域  room  \\\n",
       "0  a_0       4500.0   65.0       1          整租         上海市浦东区鹏海东苑  浦东   2.0   \n",
       "1  a_1      14000.0  148.0       1          整租  上海市浦东区大华锦绣华城第19街区  浦东   3.0   \n",
       "2  a_2       6800.0  124.0       1          整租          上海市浦东区莲宏苑  浦东   3.0   \n",
       "3  a_3       6500.0   92.0       1          整租       上海市浦东区万科海上传奇  浦东   2.0   \n",
       "4  a_4       4800.0   89.0       0          整租       上海市浦东区通宇加东花园  浦东   2.0   \n",
       "\n",
       "   hall sub_line sub_station  sub_dist  \n",
       "0   1.0       13         陈春路     641.0  \n",
       "1   2.0       13         下南路     423.0  \n",
       "2   1.0       13          北蔡     765.0  \n",
       "3   2.0       11          御桥     376.0  \n",
       "4   2.0       13          北蔡     193.0  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 数据的读取\n",
    "# 获取当前路径\n",
    "current_path = os.getcwd()\n",
    "# 定义空dataframe用于接收数据\n",
    "data_raw = pd.DataFrame()\n",
    "# 循环读取12个子文件并进行处理\n",
    "for i in range(0,12):\n",
    "    # 获取12个小写字母\n",
    "    name_index = string.ascii_lowercase[i]\n",
    "    # 组合数据路径\n",
    "    data_path = os.path.join(current_path,'data', f'租房信息{name_index}.csv')\n",
    "    # 依次读取数据\n",
    "    df = pd.read_csv(open(data_path,encoding='utf-8'))\n",
    "    # 按文件字母+index的形式构建数据唯一id\n",
    "    df.insert(0,'id', [f'{name_index}_{j}' for j in range(0,len(df))])\n",
    "    # 读取表格并进行处理，处理之后将结果加入到data_raw中\n",
    "    data_raw = data_raw.append(data_extract(df))\n",
    "# 重新设置合并后的数据index\n",
    "data_raw = data_raw.reset_index(drop=True)\n",
    "# 设置输出路径\n",
    "output_path = os.path.join(current_path,'processed_data', f'data_extracted.csv')\n",
    "data_raw.to_csv(output_path,index=False)\n",
    "# 查看数据基本情况\n",
    "print(data_raw.shape)\n",
    "print(data_raw.columns)\n",
    "print(data_raw.dtypes)\n",
    "data_raw.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，所有表格都已经按照我们的要求进行了相应的处理\n",
    "\n",
    "下面就开始对建模数据进行预处理\n",
    "\n",
    "首先是处理缺失样本"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id                 0\n",
       "total_price        0\n",
       "area             210\n",
       "is_nan             0\n",
       "rental_type        0\n",
       "address            0\n",
       "区域                 0\n",
       "room            6488\n",
       "hall            6488\n",
       "sub_line       58844\n",
       "sub_station    58844\n",
       "sub_dist       58844\n",
       "dtype: int64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 查看数据缺失情况：\n",
    "data_raw.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由于户型及房屋面积对于房租影响非常显著，因此删除缺失面积和户型的样本"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id                 0\n",
       "total_price        0\n",
       "area               0\n",
       "is_nan             0\n",
       "rental_type        0\n",
       "address            0\n",
       "区域                 0\n",
       "room               0\n",
       "hall               0\n",
       "sub_line       56774\n",
       "sub_station    56774\n",
       "sub_dist       56774\n",
       "selected           0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 数据缺失值处理：面积、户型\n",
    "data = data_raw\n",
    "data['selected'] = 1 # 设置筛选标签，记录被删除样本\n",
    "for var in ['area','room','hall']:\n",
    "    data.loc[data[var].isnull(), ['selected']] = 0\n",
    "data.loc[data['selected']==1,:].isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "样本中公交站相关信息缺失量非常大，因此不能直接删除样本\n",
    "\n",
    "根据样本中距离地铁站最远的距离是1千米左右可以推测出，样本中没有公交站信息的样本基本上都是距离地铁站非常远的样本\n",
    "\n",
    "因此选择一个非常大的数值对缺失距离地铁站距离的样本进行填充\n",
    "\n",
    "以none对地铁站及线路进行填充"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1199.0\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "id             0\n",
       "total_price    0\n",
       "area           0\n",
       "is_nan         0\n",
       "rental_type    0\n",
       "address        0\n",
       "区域             0\n",
       "room           0\n",
       "hall           0\n",
       "sub_line       0\n",
       "sub_station    0\n",
       "sub_dist       0\n",
       "selected       0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 数据缺失值处理：公交站\n",
    "for var in ['sub_line','sub_station']:\n",
    "    data.loc[data[var].isnull(), [var]] = 'none'\n",
    "# 根据距离地铁站最远距离定义缺失值填充值\n",
    "longest_sub_dist = data.sub_dist.max()\n",
    "print(longest_sub_dist)\n",
    "data.loc[data['sub_dist'].isnull(), ['sub_dist']] = longest_sub_dist*100\n",
    "data.loc[data['selected']==1,:].isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下一步我们要对类别变量进行处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<is_nan>的分类别频数统计：\n",
      "1    206475\n",
      "0     15768\n",
      "Name: is_nan, dtype: int64\n",
      "\n",
      "<区域>的分类别频数统计：\n",
      "浦东      55473\n",
      "闵行      31851\n",
      "黄浦      17998\n",
      "静安      17422\n",
      "徐汇      17105\n",
      "长宁      14734\n",
      "松江      14702\n",
      "普陀      11002\n",
      "宝山      10606\n",
      "杨浦       9267\n",
      "嘉定       9198\n",
      "虹口       6370\n",
      "青浦       4077\n",
      "奉贤       1729\n",
      "金山        413\n",
      "上海周边      236\n",
      "崇明         60\n",
      "Name: 区域, dtype: int64\n",
      "\n",
      "<rental_type>的分类别频数统计：\n",
      "整租      215755\n",
      "合租主卧      4559\n",
      "合租次卧      1674\n",
      "合租单间       180\n",
      "合租床位        75\n",
      "Name: rental_type, dtype: int64\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 类别变量处理\n",
    "for var in ['is_nan','区域','rental_type']:\n",
    "    print(f'<{var}>的分类别频数统计：\\n{data[var].value_counts()}\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "根据三个类别变量的类别频数分布：\n",
    "\n",
    "* 数据中包含上海以外的样本，应该被清除\n",
    "* 合租样本的信息是按照实际合租面积进行统计的，与整租样本记录标准不一致\n",
    "* 合租样本量非常小，可以删除"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1    215483\n",
      "0      6760\n",
      "Name: selected, dtype: int64\n",
      "保留样本比例：96.958%\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "id             0\n",
       "total_price    0\n",
       "area           0\n",
       "is_nan         0\n",
       "rental_type    0\n",
       "address        0\n",
       "区域             0\n",
       "room           0\n",
       "hall           0\n",
       "sub_line       0\n",
       "sub_station    0\n",
       "sub_dist       0\n",
       "selected       0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 去除上海市以外的样本\n",
    "data.loc[data['区域'] == '上海周边', ['selected']] = 0\n",
    "# 去除非整租的样本\n",
    "data.loc[data['rental_type'] != '整租', ['selected']] = 0\n",
    "print(data.selected.value_counts())\n",
    "percent = 100 * data.loc[data.selected==1,:].shape[0]/data.shape[0]\n",
    "print('保留样本比例：{0:.3f}%'.format(percent))\n",
    "data.loc[data['selected']==1,:].isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 输出处理数据结果\n",
    "clean_data = (data\n",
    "              .drop(columns='区域')\n",
    "              .loc[data.selected==1,:]\n",
    "              .drop(columns='selected')\n",
    "              .reset_index(drop=True))\n",
    "output_path = os.path.join(current_path,'processed_data', f'data_clean.csv')\n",
    "clean_data.to_csv(output_path,index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 地址解析与空间数据关联\n",
    "\n",
    "#### 地址解析\n",
    "\n",
    "* 通过关联房源所在城市、所在行政区以及小区名称，构成小区地址\n",
    "* 通过geocoding解析地址，获取房源所在小区经纬度\n",
    "\n",
    "#### 空间数据关联\n",
    "\n",
    "* 获取城市各类设施的POI点位\n",
    "* 根据房源所在位置（经纬度）匹配和计算房源周围的设施指标，构建新的特征\n",
    "\n",
    "* 本案例中一共获取了如下几类空间特征：\n",
    "\n",
    "* * 1）500、1000、2000米范围内的地铁站数量\n",
    "\n",
    "* * 2）500、1000、2000米范围内的公交车站数量\n",
    "\n",
    "* * 3）500、1000、2000米范围内的三甲医院数量\n",
    "\n",
    "* * 4）500、1000、2000米范围内的重点学校数量\n",
    "\n",
    "* * 5）500、1000、2000米范围内的中学的数量\n",
    "\n",
    "* * 6）500、1000、2000米范围内的幼儿园数量\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 地理信息数据读取\n",
    "current_path = os.getcwd()\n",
    "locdata_path = os.path.join(current_path,'processed_data', f'loc_data.xlsx')\n",
    "# 读取地理特征数据\n",
    "df_loc = pd.read_excel(locdata_path,encoding='utf-8')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id                      object\n",
       "total_price            float64\n",
       "area                   float64\n",
       "is_nan                   int64\n",
       "address                 object\n",
       "room                   float64\n",
       "hall                   float64\n",
       "sub_line                object\n",
       "sub_station             object\n",
       "sub_dist               float64\n",
       "subway_500             float64\n",
       "subway_1000            float64\n",
       "subway_2000            float64\n",
       "bus_500                float64\n",
       "bus_1000               float64\n",
       "bus_2000               float64\n",
       "sanjia_500             float64\n",
       "sanjia_1000            float64\n",
       "sanjia_2000            float64\n",
       "primary_school_500     float64\n",
       "primary_school_1000    float64\n",
       "primary_school_2000    float64\n",
       "middle_school_500      float64\n",
       "middle_school_1000     float64\n",
       "middle_school_2000     float64\n",
       "kindergarten_500       float64\n",
       "kindergarten_1000      float64\n",
       "kindergarten_2000      float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 合并地理特征数据\n",
    "df_model = (clean_data\n",
    "            .merge(df_loc, how='left', on='address')\n",
    "            .drop(columns=['lng','lat','rental_type'])\n",
    "            .fillna(0))\n",
    "# 查看合并之后的数据情况\n",
    "df_model.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 模型数据预处理\n",
    "\n",
    "* 将类别变量制作成哑变量\n",
    "\n",
    "\n",
    "* 着重处理应变量\n",
    "\n",
    "\n",
    "* 删除不需要的特征\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['id', 'total_price', 'area', 'is_nan', 'address', 'room', 'hall',\n",
       "       'sub_station', 'sub_dist', 'subway_500', 'subway_1000', 'subway_2000',\n",
       "       'bus_500', 'bus_1000', 'bus_2000', 'sanjia_500', 'sanjia_1000',\n",
       "       'sanjia_2000', 'primary_school_500', 'primary_school_1000',\n",
       "       'primary_school_2000', 'middle_school_500', 'middle_school_1000',\n",
       "       'middle_school_2000', 'kindergarten_500', 'kindergarten_1000',\n",
       "       'kindergarten_2000', 'sub_line_1', 'sub_line_10', 'sub_line_11',\n",
       "       'sub_line_12', 'sub_line_13', 'sub_line_16', 'sub_line_17',\n",
       "       'sub_line_2', 'sub_line_3', 'sub_line_4', 'sub_line_5', 'sub_line_6',\n",
       "       'sub_line_7', 'sub_line_8', 'sub_line_9', 'sub_line_none'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 制作哑变量\n",
    "df_model = pd.get_dummies(df_model,columns=['sub_line'])\n",
    "df_model.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 以租金每平米为因变量\n",
    "df_model['y'] = df_model.apply(lambda x: x['total_price']/x['area'] if x['area']>0 else 0, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show(*args, **kw)>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAD8CAYAAACLrvgBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAEvZJREFUeJzt3XGsnfVdx/H3VzpYZWMtQ24qJRZiY6wjMrhhnTPmOrQUXFaWQFJCbMcwNciSTUlccX+gwyVDZTPgZFaplKWDIdtsg8XadJwsJhujOKQw6HrHKty10rEyRiG6Vb/+cX4XHy7n3vvrubd97hnvV3JynvN9fs/zfM9zeu/nnud5zmlkJpIk1fipthuQJA0OQ0OSVM3QkCRVMzQkSdUMDUlSNUNDklTN0JAkVTM0JEnVDA1JUrV5bTcw20477bRcsmRJX8u+9NJLnHzyybPb0HE0yP0Pcu9g/20a5N5h7vT/8MMPP5eZPzPduJ+40FiyZAm7du3qa9lOp8PIyMjsNnQcDXL/g9w72H+bBrl3mDv9R8R/1Izz8JQkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSp2k/cJ8JnYvd3X+D96/+plW3v+8RvtbJdSToavtOQJFUzNCRJ1QwNSVI1Q0OSVM3QkCRVMzQkSdUMDUlSNUNDklTN0JAkVTM0JEnVpg2NiDgzIh6IiCci4vGI+FCpnxoROyJib7lfWOoREbdExGhEPBoR5zXWtbaM3xsRaxv18yNid1nmloiIqbYhSWpHzTuNI8B1mfmLwHLg2ohYBqwHdmbmUmBneQxwMbC03NYBt0E3AIAbgHcAFwA3NELgtjJ2fLmVpT7ZNiRJLZg2NDLzQGb+W5l+EXgCOANYBWwqwzYBl5bpVcCd2fU1YEFELAIuAnZk5qHMfB7YAaws807JzK9mZgJ3TlhXr21IklpwVOc0ImIJ8HbgQWAoMw9AN1iA08uwM4BnGouNldpU9bEedabYhiSpBdVfjR4RbwK+AHw4M39YTjv0HNqjln3Uq0XEOrqHtxgaGqLT6RzN4q8Ymg/XnXOkr2Vnqt+emw4fPjwr62nDIPcO9t+mQe4dBq//qtCIiDfQDYzNmfnFUn42IhZl5oFyiOlgqY8BZzYWXwzsL/WRCfVOqS/uMX6qbbxKZm4ANgAMDw/nyMhIr2HTunXzFm7e3c5/MbLvypEZr6PT6dDvc2/bIPcO9t+mQe4dBq//mqunArgdeCIzP9mYtRUYvwJqLbClUV9TrqJaDrxQDi1tB1ZExMJyAnwFsL3MezEilpdtrZmwrl7bkCS1oObP6ncBvw3sjohHSu2PgE8A90TE1cDTwOVl3jbgEmAUeBm4CiAzD0XEjcBDZdzHMvNQmb4GuAOYD9xfbkyxDUlSC6YNjcz8V3qfdwC4sMf4BK6dZF0bgY096ruAt/Wof7/XNiRJ7fAT4ZKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqjZtaETExog4GBGPNWp/HBHfjYhHyu2SxrzrI2I0IvZExEWN+spSG42I9Y36WRHxYETsjYjPR8SJpX5SeTxa5i+ZrSctSepPzTuNO4CVPeqfysxzy20bQEQsA1YDv1SW+euIOCEiTgA+DVwMLAOuKGMBbirrWgo8D1xd6lcDz2fmzwOfKuMkSS2aNjQy8yvAocr1rQLuzsz/zszvAKPABeU2mplPZeaPgLuBVRERwLuBe8vym4BLG+vaVKbvBS4s4yVJLZnJOY0PRsSj5fDVwlI7A3imMWas1CarvxX4QWYemVB/1brK/BfKeElSS+b1udxtwI1AlvubgQ8Avd4JJL3DKacYzzTzXiUi1gHrAIaGhuh0OlO0Prmh+XDdOUemH3gM9Ntz0+HDh2dlPW0Y5N7B/ts0yL3D4PXfV2hk5rPj0xHxt8B95eEYcGZj6GJgf5nuVX8OWBAR88q7ieb48XWNRcQ84C1McpgsMzcAGwCGh4dzZGSkn6fFrZu3cPPufnN0ZvZdOTLjdXQ6Hfp97m0b5N7B/ts0yL3D4PXf1+GpiFjUePg+YPzKqq3A6nLl01nAUuDrwEPA0nKl1Il0T5ZvzcwEHgAuK8uvBbY01rW2TF8GfLmMlyS1ZNo/qyPiLmAEOC0ixoAbgJGIOJfu4aJ9wO8CZObjEXEP8E3gCHBtZv5PWc8Hge3ACcDGzHy8bOIjwN0R8afAN4DbS/124LMRMUr3HcbqGT9bSdKMTBsamXlFj/LtPWrj4z8OfLxHfRuwrUf9KbpXV02s/xdw+XT9SZKOHz8RLkmqZmhIkqoZGpKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqoZGpKkatOGRkRsjIiDEfFYo3ZqROyIiL3lfmGpR0TcEhGjEfFoRJzXWGZtGb83ItY26udHxO6yzC0REVNtQ5LUnpp3GncAKyfU1gM7M3MpsLM8BrgYWFpu64DboBsAwA3AO4ALgBsaIXBbGTu+3MpptiFJasm0oZGZXwEOTSivAjaV6U3ApY36ndn1NWBBRCwCLgJ2ZOahzHwe2AGsLPNOycyvZmYCd05YV69tSJJa0u85jaHMPABQ7k8v9TOAZxrjxkptqvpYj/pU25AktWTeLK8vetSyj/rRbTRiHd1DXAwNDdHpdI52FQAMzYfrzjnS17Iz1W/PTYcPH56V9bRhkHsH+2/TIPcOg9d/v6HxbEQsyswD5RDTwVIfA85sjFsM7C/1kQn1Tqkv7jF+qm28RmZuADYADA8P58jIyGRDp3Tr5i3cvHu2c7TOvitHZryOTqdDv8+9bYPcO9h/mwa5dxi8/vs9PLUVGL8Cai2wpVFfU66iWg68UA4tbQdWRMTCcgJ8BbC9zHsxIpaXq6bWTFhXr21Ikloy7Z/VEXEX3XcJp0XEGN2roD4B3BMRVwNPA5eX4duAS4BR4GXgKoDMPBQRNwIPlXEfy8zxk+vX0L1Caz5wf7kxxTYkSS2ZNjQy84pJZl3YY2wC106yno3Axh71XcDbetS/32sbkqT2+IlwSVI1Q0OSVM3QkCRVMzQkSdUMDUlSNUNDklTN0JAkVTM0JEnVDA1JUjVDQ5JUzdCQJFUzNCRJ1QwNSVI1Q0OSVM3QkCRVMzQkSdUMDUlSNUNDklTN0JAkVTM0JEnVDA1JUjVDQ5JUzdCQJFUzNCRJ1QwNSVI1Q0OSVM3QkCRVMzQkSdUMDUlSNUNDklTN0JAkVTM0JEnVDA1JUjVDQ5JUzdCQJFUzNCRJ1QwNSVI1Q0OSVM3QkCRVm1FoRMS+iNgdEY9ExK5SOzUidkTE3nK/sNQjIm6JiNGIeDQizmusZ20Zvzci1jbq55f1j5ZlYyb9SpJmZjbeafx6Zp6bmcPl8XpgZ2YuBXaWxwAXA0vLbR1wG3RDBrgBeAdwAXDDeNCUMesay62chX4lSX06FoenVgGbyvQm4NJG/c7s+hqwICIWARcBOzLzUGY+D+wAVpZ5p2TmVzMzgTsb65IktWCmoZHAv0TEwxGxrtSGMvMAQLk/vdTPAJ5pLDtWalPVx3rUJUktmTfD5d+Vmfsj4nRgR0Q8OcXYXucjso/6a1fcDax1AENDQ3Q6nSmbnszQfLjunCN9LTtT/fbcdPjw4VlZTxsGuXew/zYNcu8weP3PKDQyc3+5PxgRX6J7TuLZiFiUmQfKIaaDZfgYcGZj8cXA/lIfmVDvlPriHuN79bEB2AAwPDycIyMjvYZN69bNW7h590xztD/7rhyZ8To6nQ79Pve2DXLvYP9tGuTeYfD67/vwVEScHBFvHp8GVgCPAVuB8Sug1gJbyvRWYE25imo58EI5fLUdWBERC8sJ8BXA9jLvxYhYXq6aWtNYlySpBTP5s3oI+FK5CnYe8LnM/OeIeAi4JyKuBp4GLi/jtwGXAKPAy8BVAJl5KCJuBB4q4z6WmYfK9DXAHcB84P5ykyS1pO/QyMyngF/uUf8+cGGPegLXTrKujcDGHvVdwNv67VGSNLv8RLgkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqoZGpKkaoaGJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRqhoYkqZqhIUmqZmhIkqrN+dCIiJURsSciRiNifdv9SNLr2ZwOjYg4Afg0cDGwDLgiIpa125UkvX7N6dAALgBGM/OpzPwRcDewquWeJOl1a66HxhnAM43HY6UmSWrBvLYbmEb0qOVrBkWsA9aVh4cjYk+f2zsNeK7PZWckbpqV1bTW/ywY5N7B/ts0yL3D3On/52oGzfXQGAPObDxeDOyfOCgzNwAbZrqxiNiVmcMzXU9bBrn/Qe4d7L9Ng9w7DF7/c/3w1EPA0og4KyJOBFYDW1vuSZJet+b0O43MPBIRHwS2AycAGzPz8ZbbkqTXrTkdGgCZuQ3Ydpw2N+NDXC0b5P4HuXew/zYNcu8wYP1H5mvOK0uS1NNcP6chSZpDDI1iLn5dSUScGREPRMQTEfF4RHyo1E+NiB0RsbfcLyz1iIhbynN4NCLOa6xrbRm/NyLWHsfncEJEfCMi7iuPz4qIB0sfny8XOBARJ5XHo2X+ksY6ri/1PRFx0XHsfUFE3BsRT5bX4J0Dtu9/v/y7eSwi7oqIN87l/R8RGyPiYEQ81qjN2v6OiPMjYndZ5paI6HVJ/2z2/ufl386jEfGliFjQmNdzn072e2iy160Vmfm6v9E9yf5t4GzgRODfgWVzoK9FwHll+s3At+h+ncqfAetLfT1wU5m+BLif7udblgMPlvqpwFPlfmGZXnicnsMfAJ8D7iuP7wFWl+nPANeU6d8DPlOmVwOfL9PLyutxEnBWeZ1OOE69bwJ+p0yfCCwYlH1P90Ow3wHmN/b7++fy/gd+DTgPeKxRm7X9DXwdeGdZ5n7g4mPc+wpgXpm+qdF7z33KFL+HJnvd2ri1stG5div/kLY3Hl8PXN92Xz363AL8JrAHWFRqi4A9ZfpvgCsa4/eU+VcAf9Oov2rcMex3MbATeDdwX/lhfa7xg/TKfqd7hdw7y/S8Mi4mvhbNcce491Po/tKNCfVB2ffj36Zwatmf9wEXzfX9DyyZ8It3VvZ3mfdko/6qccei9wnz3gdsLtM99ymT/B6a6uemjZuHp7rm/NeVlMMFbwceBIYy8wBAuT+9DJvsebT1/P4S+EPgf8vjtwI/yMwjPfp4pccy/4Uyvq3ezwa+B/x9Obz2dxFxMgOy7zPzu8BfAE8DB+juz4cZnP0/brb29xllemL9ePkA3Xc3cPS9T/Vzc9wZGl1VX1fSloh4E/AF4MOZ+cOphvao5RT1YyYi3gMczMyHm+Up+pgzvRfz6B5uuC0z3w68RPfwyGTmVP/l2P8quoc/fhY4me63RU/Wy5zqv8LR9tva84iIjwJHgM3jpUl6mXO992JodFV9XUkbIuINdANjc2Z+sZSfjYhFZf4i4GCpT/Y82nh+7wLeGxH76H478bvpvvNYEBHjnw9q9vFKj2X+W4BDLfU+3s9YZj5YHt9LN0QGYd8D/Abwncz8Xmb+GPgi8CsMzv4fN1v7e6xMT6wfU+VE/HuAK7McW5qmx17155j8dTvuDI2uOfl1JeXqjtuBJzLzk41ZW4Hxq0LW0j3XMV5fU64sWQ68UN7SbwdWRMTC8hfoilI7ZjLz+sxcnJlL6O7PL2fmlcADwGWT9D7+nC4r47PUV5ere84CltI9oXlMZeZ/As9ExC+U0oXANxmAfV88DSyPiJ8u/47G+x+I/d8wK/u7zHsxIpaX/bGmsa5jIiJWAh8B3puZL094Tr32ac/fQ+V1mOx1O/7aOpky1250r8b4Ft2rFz7adj+lp1+l+zb0UeCRcruE7jHOncDecn9qGR90/9OqbwO7geHGuj4AjJbbVcf5eYzw/1dPnU33B2QU+AfgpFJ/Y3k8Wuaf3Vj+o+U57WEWr3ip6PtcYFfZ//9I92qcgdn3wJ8ATwKPAZ+le7XOnN3/wF10z7/8mO5f3VfP5v4Ghsu++DbwV0y4yOEY9D5K9xzF+M/uZ6bbp0zye2iy162Nm58IlyRV8/CUJKmaoSFJqmZoSJKqGRqSpGqGhiSpmqEhSapmaEiSqhkakqRq/wfZME2G4P5BMQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 查看y的分布情况\n",
    "df_model['y'].hist()\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD8CAYAAACcjGjIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAFCdJREFUeJzt3XGMXeV55/HvU08cUmgChGbWsp0Mq1rdcUZblY6A3VqrGbwLJhvV/iNINsnGmJEsITp1u5U2sPMH2tCRGu1q2aY0WVkZL6RKhrBsFkzLllpkripLDcEs2QSYph6RgGdhQ6iBrZ0FM86zf8w7zrXfMcb3XvvOwPcjje49z3nPnedKF35+z3nPnchMJElq9gvdbkCStPQYDpKkiuEgSaoYDpKkiuEgSaoYDpKkiuEgSaoYDpKkiuEgSar0dLuBVl122WXZ19fX7TakytGjR7nwwgu73Ya0qCeffPKVzPzlM41btuHQ19fHgQMHut2GVGk0GgwNDXW7DWlREfH8OxnnaSVJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkDpkcnKSgYEBNm7cyMDAAJOTk91uSWrZsl3KKi0lk5OTjI2NMTExwfHjx1mxYgUjIyMAbNu2rcvdSWfPmYPUAePj40xMTDA8PExPTw/Dw8NMTEwwPj7e7daklhgOUgdMT0+zYcOGk2obNmxgenq6Sx1J7TEcpA7o7+9n//79J9X2799Pf39/lzqS2mM4SB0wNjbGyMgIU1NTzM3NMTU1xcjICGNjY91uTWqJF6SlDli46Dw6Osr09DT9/f2Mj497MVrLVmRmt3toyeDgYPrFe1qK/OI9LWUR8WRmDp5pnKeVJEkVw0GSVDEcJEkVw0GSVDljOETEnoh4OSKebqr9+4j4m4j4XkT894i4uGnf7RExExE/iIjrmuqbSm0mIm5rql8eEY9HxMGI+EZErOzkG5Qknb13MnO4B9h0Sm0fMJCZ/xj4W+B2gIhYD2wFPl6O+VJErIiIFcCfANcD64FtZSzAF4C7MnMd8Cow0tY7kiS17YzhkJl/BRw+pfaXmTlXNr8NrCnPNwP3ZeabmflDYAa4svzMZOZzmXkMuA/YHBEBXAM8UI6/F9jS5nuSJLWpEzfB3Qx8ozxfzXxYLJgtNYBDp9SvAj4MvNYUNM3jKxGxE9gJ0NvbS6PRaLd3qeOOHDniZ1PLXlvhEBFjwBzwtYXSIsOSxWco+TbjF5WZu4HdMH8TnDcaaSnyJji9G7QcDhGxHfgksDF/fpv1LLC2adga4MXyfLH6K8DFEdFTZg/N4yVJXdLSUtaI2AR8DvitzPxp0669wNaIeH9EXA6sA74DPAGsKyuTVjJ/0XpvCZUp4FPl+O3AQ629FUlSp7yTpayTwF8DvxoRsxExAtwN/BKwLyK+GxH/GSAznwHuB54F/gK4NTOPl1nBbwOPAtPA/WUszIfMv46IGeavQUx09B1Kks7aGU8rZeZiXyt52v+BZ+Y4UP35q8x8BHhkkfpzzK9mkiQtEd4hLUmqGA6SpIrhIEmqGA6SpIrhIEmqGA6SpIrhIEmqGA6SpIrhIEmqGA6SpIrhIEmqGA6SpIrhIEmqGA6SpIrhIEmqGA6SpIrhIEmqGA6SpIrhIEmqGA6SpIrhIEmqGA6SpMoZwyEi9kTEyxHxdFPt0ojYFxEHy+MlpR4R8cWImImI70XEFU3HbC/jD0bE9qb6b0TE98sxX4yI6PSblCSdnXcyc7gH2HRK7TbgscxcBzxWtgGuB9aVn53Al2E+TIA7gKuAK4E7FgKljNnZdNypv0uSdJ6dMRwy86+Aw6eUNwP3luf3Alua6l/Ned8GLo6IVcB1wL7MPJyZrwL7gE1l3wcz868zM4GvNr2WJKlLelo8rjczXwLIzJci4iOlvho41DRuttTerj67SH1REbGT+VkGvb29NBqNFtuXzp0jR4742dSy12o4nM5i1wuyhfqiMnM3sBtgcHAwh4aGWmhROrcajQZ+NrXctbpa6cfllBDl8eVSnwXWNo1bA7x4hvqaReqSpC5qNRz2AgsrjrYDDzXVP1tWLV0NvF5OPz0KXBsRl5QL0dcCj5Z9fx8RV5dVSp9tei1JUpec8bRSREwCQ8BlETHL/KqjPwTuj4gR4AXghjL8EeATwAzwU2AHQGYejog7gSfKuM9n5sJF7luYXxH1AeB/lB9JUhedMRwyc9tpdm1cZGwCt57mdfYAexapHwAGztSHJOn88Q5pSVLFcJAkVQwHSVLFcJAkVQwHSVLFcJAkVQwHSVLFcJAkVQwHSVLFcJAkVQwHSVLFcJAkVQwHSVLFcJAkVQwHSVLFcJAkVQwHSVLFcJAkVQwHSVLFcJAkVQwHSVKlrXCIiN+LiGci4umImIyICyLi8oh4PCIORsQ3ImJlGfv+sj1T9vc1vc7tpf6DiLiuvbckSWpXy+EQEauB3wEGM3MAWAFsBb4A3JWZ64BXgZFyyAjwamb+CnBXGUdErC/HfRzYBHwpIla02pckqX3tnlbqAT4QET3ALwIvAdcAD5T99wJbyvPNZZuyf2NERKnfl5lvZuYPgRngyjb7kiS1oeVwyMz/DfwH4AXmQ+F14EngtcycK8NmgdXl+WrgUDl2roz/cHN9kWMkSV3Q0+qBEXEJ8//qvxx4DfivwPWLDM2FQ06z73T1xX7nTmAnQG9vL41G4+yals6DI0eO+NnUstdyOAD/HPhhZv4EICK+CfxT4OKI6CmzgzXAi2X8LLAWmC2noT4EHG6qL2g+5iSZuRvYDTA4OJhDQ0NttC+dG41GAz+bWu7auebwAnB1RPxiuXawEXgWmAI+VcZsBx4qz/eWbcr+b2VmlvrWsprpcmAd8J02+pIktanlmUNmPh4RDwD/E5gDnmL+X/V/DtwXEX9QahPlkAngTyNihvkZw9byOs9ExP3MB8sccGtmHm+1L0lS+9o5rURm3gHccUr5ORZZbZSZbwA3nOZ1xoHxdnqRJHWOd0hLkiqGgySpYjhIkiqGgySpYjhIkiqGgySpYjhIkiqGgySpYjhIkiqGgySpYjhIkiqGgySpYjhIkiqGgySpYjhIkiqGgySpYjhIkiqGgySpYjhIkiqGgySpYjhIkiqGgySp0lY4RMTFEfFARPxNRExHxD+JiEsjYl9EHCyPl5SxERFfjIiZiPheRFzR9Drby/iDEbG93TclSWpPuzOHPwL+IjP/EfBrwDRwG/BYZq4DHivbANcD68rPTuDLABFxKXAHcBVwJXDHQqBIkrqj5XCIiA8C/wyYAMjMY5n5GrAZuLcMuxfYUp5vBr6a874NXBwRq4DrgH2ZeTgzXwX2AZta7UuS1L52Zg7/EPgJ8F8i4qmI+EpEXAj0ZuZLAOXxI2X8auBQ0/GzpXa6uiSpS3raPPYKYDQzH4+IP+Lnp5AWE4vU8m3q9QtE7GT+lBS9vb00Go2zalg6H44cOeJnU8teO+EwC8xm5uNl+wHmw+HHEbEqM18qp41ebhq/tun4NcCLpT50Sr2x2C/MzN3AboDBwcEcGhpabJjUVY1GAz+bWu5aPq2Umf8HOBQRv1pKG4Fngb3Awoqj7cBD5fle4LNl1dLVwOvltNOjwLURcUm5EH1tqUmSuqSdmQPAKPC1iFgJPAfsYD5w7o+IEeAF4IYy9hHgE8AM8NMylsw8HBF3Ak+UcZ/PzMNt9iVJakNb4ZCZ3wUGF9m1cZGxCdx6mtfZA+xppxdJUud4h7QkqWI4SJIqhoMkqWI4SJIqhoMkqWI4SJIqhoMkqWI4SJIqhoMkqWI4SJIqhoMkqWI4SJIqhoMkqWI4SB0yOTnJwMAAGzduZGBggMnJyW63JLWs3b/nIIn5YBgbG2NiYoLjx4+zYsUKRkZGANi2bVuXu5POnjMHqQPGx8eZmJhgeHiYnp4ehoeHmZiYYHx8vNutSS0xHKQOmJ6eZsOGDSfVNmzYwPT0dJc6ktpjOEgd0N/fz/79+0+q7d+/n/7+/i51JLXHcJA6YGxsjJGREaamppibm2NqaoqRkRHGxsa63ZrUEi9ISx2wcNF5dHSU6elp+vv7GR8f92K0lq3IzG730JLBwcE8cOBAt9uQKo1Gg6GhoW63IS0qIp7MzMEzjfO0kiSp0nY4RMSKiHgqIv6sbF8eEY9HxMGI+EZErCz195ftmbK/r+k1bi/1H0TEde32JElqTydmDruA5vV6XwDuysx1wKvASKmPAK9m5q8Ad5VxRMR6YCvwcWAT8KWIWNGBviRJLWorHCJiDfAvga+U7QCuAR4oQ+4FtpTnm8s2Zf/GMn4zcF9mvpmZPwRmgCvb6UuS1J52Zw7/Cfg3wM/K9oeB1zJzrmzPAqvL89XAIYCy//Uy/kR9kWMkSV3Q8lLWiPgk8HJmPhkRQwvlRYbmGfa93TGn/s6dwE6A3t5eGo3G2bQsnRdHjhzxs6llr537HH4T+K2I+ARwAfBB5mcSF0dET5kdrAFeLONngbXAbET0AB8CDjfVFzQfc5LM3A3shvmlrC4X1FLkUla9G7R8Wikzb8/MNZnZx/wF5W9l5qeBKeBTZdh24KHyfG/Zpuz/Vs7fZLEX2FpWM10OrAO+02pfkqT2nYs7pD8H3BcRfwA8BUyU+gTwpxExw/yMYStAZj4TEfcDzwJzwK2Zefwc9CVJeoc6Eg6Z2QAa5flzLLLaKDPfAG44zfHjgN9tLElLhHdIS5IqhoMkqWI4SJIqhoMkqWI4SJIqhoMkqWI4SJIqhoMkqWI4SJIqhoMkqWI4SJIqhoMkqWI4SJIqhoMkqWI4SJIqhoMkqWI4SJIqhoMkqWI4SB0yOTnJwMAAGzduZGBggMnJyW63JLWsI39DWnqvm5ycZGxsjImJCY4fP86KFSsYGRkBYNu2bV3uTjp7kZnd7qElg4ODeeDAgW63IQEwMDDAli1bePDBB5menqa/v//E9tNPP93t9qQTIuLJzBw80zhnDlIHPPvssxw9epQ9e/acmDncfPPNPP/8891uTWpJy9ccImJtRExFxHREPBMRu0r90ojYFxEHy+MlpR4R8cWImImI70XEFU2vtb2MPxgR29t/W9L5tXLlSkZHRxkeHqanp4fh4WFGR0dZuXJlt1uTWtLOBek54Pczsx+4Grg1ItYDtwGPZeY64LGyDXA9sK787AS+DPNhAtwBXAVcCdyxECjScnHs2DHuvvtupqammJubY2pqirvvvptjx451uzWpJS2fVsrMl4CXyvO/j4hpYDWwGRgqw+4FGsDnSv2rOX+R49sRcXFErCpj92XmYYCI2AdsAlzqoWVj/fr1bNmyhdHR0RPXHG688UYefPDBbrcmtaQj1xwiog/4deBxoLcEB5n5UkR8pAxbDRxqOmy21E5Xl5aNsbGxRVcrjY+Pd7s1qSVth0NEXAT8N+B3M/P/RsRphy5Sy7epL/a7djJ/Sore3l4ajcZZ9yudC6tWreLTn/40N998My+88AIf/ehH+cxnPsOqVav8nGpZaiscIuJ9zAfD1zLzm6X844hYVWYNq4CXS30WWNt0+BrgxVIfOqXeWOz3ZeZuYDfML2UdGhpabJjUFUNDQ9x55500Gg38bGq5a2e1UgATwHRm/semXXuBhRVH24GHmuqfLauWrgZeL6efHgWujYhLyoXoa0tNWla8Q1rvJu3MHH4T+FfA9yPiu6X2b4E/BO6PiBHgBeCGsu8R4BPADPBTYAdAZh6OiDuBJ8q4zy9cnJaWi8nJSXbt2sWFF15IZnL06FF27doFeIe0lifvkJY6YO3atczNzfH1r3/9xAXpG2+8kZ6eHg4dOnTmF5DOk3d6h7RfvCd1wOzsLDfddBOjo6Ncd911jI6OctNNNzE7O9vt1qSW+PUZUofcc8891cxBWq4MB6kDenp6eOWVV7jmmmtOqvX0+J+YlidPK0kdMDc3x9zcHBdddBEAF1100YmatBwZDlKH9PX18dZbbwHw1ltv0dfX192GpDY455U65Ec/+tGJ52+++eZJ29Jy48xB6qALLrjgpEdpuTIcpA7asWMHDz/8MDt27Oh2K1JbvAlO6oCI4NJLL+Xw4Z/f3L+wvVz/G9O7kzfBSefZ4cOHueWWW3j44Ye55ZZbTgoKablx5iB1wMJX1V9wwQW88cYbJx4BZw5aUpw5SOdZT0/PiUB44403vAFOy5rhIHXIqTe8eQOcljPDQZJUMRwkSRXDQZJUMRwkSRXDQZJUMRwkSRXDQZJUMRwkSRXDQZJUWTLhEBGbIuIHETETEbd1ux9Jei9bEl/+EhErgD8B/gUwCzwREXsz89nudib9/Ev1zvXxfkGflpIlEQ7AlcBMZj4HEBH3AZsBw0Ed9Wv/7i95/f+9dVbHfOxzf3bGMc9/4ZNtHQ/Qd9ufv+OePvSB9/G/7rj2HY+XztZSCYfVwKGm7Vngqi71onexn/X9Pr90Dl534J6Bt9nb+bOkPwPg+x1/XWnBUgmHxebd1Rw7InYCOwF6e3tpNBrnuC292/zxx/74rI8ZHh4+B53Upqamzmq8n3+dS0slHGaBtU3ba4AXTx2UmbuB3TD/x36GhobOS3N6bzvbawGNRgM/m1rulspqpSeAdRFxeUSsBLYCe7vckyS9Zy2JmUNmzkXEbwOPAiuAPZn5TJfbkqT3rCURDgCZ+QjwSLf7kCQtndNKkqQlxHCQJFUMB0lSxXCQJFUMB0lSJZbrl31FxE+A57vdh7SIy4BXut2EdBofy8xfPtOgZRsO0lIVEQcyc7DbfUjt8LSSJKliOEiSKoaD1Hm7u92A1C6vOUiSKs4cJEkVw0GSVDEcJEkVw0HqgIi4MyJ2NW2PR8TvdLMnqR1ekJY6ICL6gG9m5hUR8QvAQeDKzPy7rjYmtWjJ/LEfaTnLzB9FxN9FxK8DvcBTBoOWM8NB6pyvADcB/wDY091WpPZ4WknqkIhYCXwfeB+wLjOPd7klqWXOHKQOycxjETEFvGYwaLkzHKQOKReirwZu6HYvUrtcyip1QESsB2aAxzLzYLf7kdrlNQdJUsWZgySpYjhIkiqGgySpYjhIkiqGgySpYjhIkir/H2342IRur4ncAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 查看y的箱线图\n",
    "df_model[['y']].boxplot()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD8CAYAAACcjGjIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAEuVJREFUeJzt3X+s3XV9x/Hn21awwx+A4E3TMm+dnRFtJtBgE6e5EQcFnGWbLCVEimNpZmDRrMssMxlOJYEtyMKCmm40FsMszB+hkRpskDuzRH7/KqWyXrCTKx2NFpHqxF1974/zueZwP+f03p7e3u85uc9HcnK/3/f38z3nfb/3nvs63+/5nu+NzESSpHavaLoBSVL/MRwkSRXDQZJUMRwkSRXDQZJUMRwkSRXDQZJUMRwkSRXDQZJUWdh0A7066aSTcnh4uKd1f/azn3HcccfNbkOzbBB6hMHocxB6hMHocxB6hMHos4keH3zwwR9l5skzGpyZA3k744wzsld33313z+vOlUHoMXMw+hyEHjMHo89B6DFzMPpsokfggZzh31gPK0mSKoaDJKliOEiSKoaDJKliOEiSKoaDJKliOEiSKoaDJKliOEiSKgN7+QxpOsMb7wBgw4oJLi3Tc2HvNefP2WNJR4t7DpKkiuEgSaoYDpKkiuEgSar4hrSOquE5fCNY0uxxz0GSVDEcJEkVw0GSVDEcJEkVw0GSVJlxOETEgoh4OCK+UeaXRcS9EbEnIm6NiGNK/dgyP1aWD7fdx5Wl/mREnNNWX11qYxGxcfa+PUlSLw5nz+GjwO62+WuB6zNzOfA8cFmpXwY8n5lvBq4v44iIU4G1wNuA1cDnSuAsAG4EzgVOBS4qYyVJDZlROETEUuB84F/LfADvBb5ShmwBLijTa8o8ZflZZfwaYGtmvpSZ3wfGgDPLbSwzn87MXwJby1hJUkNmuufwT8DfAL8u868HfpKZE2V+HFhSppcAzwCU5S+U8b+pT1mnW12S1JBpPyEdEe8H9mfmgxExMlnuMDSnWdat3imgskONiFgPrAcYGhpidHS0e+OHcPDgwZ7XnSuD0CNM3+eGFRNdl82VoUVz24e/l80bhD77vceZXD7jXcAHIuI84FXAa2ntSRwfEQvL3sFS4Nkyfhw4BRiPiIXA64ADbfVJ7et0q79MZm4CNgGsXLkyR0ZGZtB+bXR0lF7XnSuD0CNM3+dc/h+FbjasmOC6nXN3pZi9F4/0tN4g/MwHoUcYjD77vcdpDytl5pWZuTQzh2m9ofztzLwYuBv4YBm2Dri9TG8r85Tl387MLPW15WymZcBy4D7gfmB5OfvpmPIY22blu5Mk9eRIXk59HNgaEZ8BHgZuKvWbgC9FxBitPYa1AJm5KyJuA54AJoDLM/NXABFxBXAnsADYnJm7jqAvSdIROqxwyMxRYLRMP03rTKOpY34BXNhl/auBqzvUtwPbD6cXSdLR4yekJUkVw0GSVDEcJEkVw0GSVDEcJEkVw0GSVDEcJEkVw0GSVDEcJEkVw0GSVDEcJEkVw0GSVDEcJEkVw0GSVDEcJEkVw0GSVDEcJEkVw0GSVDEcJEmVw/of0pKmN7zxjp7W27Bigkt7XBdg7zXn97yuNJV7DpKkiuEgSaoYDpKkiuEgSaoYDpKkiuEgSaoYDpKkiuEgSaoYDpKkiuEgSaoYDpKkiuEgSaoYDpKkildlnSd6vVLodI70SqKS+pN7DpKkiuEgSaoYDpKkiuEgSaoYDpKkyrThEBGvioj7IuLRiNgVEX9f6ssi4t6I2BMRt0bEMaV+bJkfK8uH2+7rylJ/MiLOaauvLrWxiNg4+9+mJOlwzGTP4SXgvZn5e8A7gNURsQq4Frg+M5cDzwOXlfGXAc9n5puB68s4IuJUYC3wNmA18LmIWBARC4AbgXOBU4GLylhJUkOmDYdsOVhmX1luCbwX+EqpbwEuKNNryjxl+VkREaW+NTNfyszvA2PAmeU2lplPZ+Yvga1lrCSpITN6z6G8wn8E2A/sAJ4CfpKZE2XIOLCkTC8BngEoy18AXt9en7JOt7okqSEz+oR0Zv4KeEdEHA98HXhrp2Hla3RZ1q3eKaCyQ42IWA+sBxgaGmJ0dPTQjXdx8ODBntedK7Pd44YVE9MP6sHQoqN337NlEHqEI+9zLn6nB+G5A4PRZ7/3eFiXz8jMn0TEKLAKOD4iFpa9g6XAs2XYOHAKMB4RC4HXAQfa6pPa1+lWn/r4m4BNACtXrsyRkZHDaf83RkdH6XXduTLbPR6tS1xsWDHBdTv7+yosg9AjHHmfey8emb1muhiE5w4MRp/93uNMzlY6uewxEBGLgPcBu4G7gQ+WYeuA28v0tjJPWf7tzMxSX1vOZloGLAfuA+4Hlpezn46h9ab1ttn45iRJvZnJy5TFwJZyVtErgNsy8xsR8QSwNSI+AzwM3FTG3wR8KSLGaO0xrAXIzF0RcRvwBDABXF4OVxERVwB3AguAzZm5a9a+Q0nSYZs2HDLzMeC0DvWnaZ1pNLX+C+DCLvd1NXB1h/p2YPsM+pUkzQE/IS1JqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqixsugFJs2N44x1H/TE2rJjg0g6Ps/ea84/6Y2tuuecgSapMGw4RcUpE3B0RuyNiV0R8tNRPjIgdEbGnfD2h1CMiboiIsYh4LCJOb7uvdWX8nohY11Y/IyJ2lnVuiIg4Gt+sJGlmZrLnMAFsyMy3AquAyyPiVGAjcFdmLgfuKvMA5wLLy2098HlohQlwFfBO4EzgqslAKWPWt623+si/NUlSr6YNh8zcl5kPlekXgd3AEmANsKUM2wJcUKbXADdnyz3A8RGxGDgH2JGZBzLzeWAHsLose21mfjczE7i57b4kSQ04rPccImIYOA24FxjKzH3QChDgDWXYEuCZttXGS+1Q9fEOdUlSQ2Z8tlJEvBr4KvCxzPzpId4W6LQge6h36mE9rcNPDA0NMTo6Ok3XnR08eLDndefKbPe4YcXErN1Xu6FFR+++Z8sg9AiD0We3Hvvt+TQfn+OzbUbhEBGvpBUMt2Tm10r5uYhYnJn7yqGh/aU+DpzStvpS4NlSH5lSHy31pR3GVzJzE7AJYOXKlTkyMtJp2LRGR0fpdd25Mts9djr9cDZsWDHBdTv7+4zoQegRBqPPbj3uvXhk7ps5hPn4HJ9tMzlbKYCbgN2Z+dm2RduAyTOO1gG3t9UvKWctrQJeKIed7gTOjogTyhvRZwN3lmUvRsSq8liXtN2XJKkBM3mZ8i7gQ8DOiHik1P4WuAa4LSIuA34AXFiWbQfOA8aAnwMfBsjMAxHxaeD+Mu5TmXmgTH8E+CKwCPhmuUmSGjJtOGTmf9L5fQGAszqMT+DyLve1Gdjcof4A8PbpepEkzQ0/IS1JqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqhgOkqSK4SBJqixsugFJg2944x2NPO7ea85v5HHnA/ccJEkVw0GSVDEcJEkVw0GSVDEcJEkVw0GSVDEcJEkVw0GSVDEcJEmVacMhIjZHxP6IeLytdmJE7IiIPeXrCaUeEXFDRIxFxGMRcXrbOuvK+D0Rsa6tfkZE7Czr3BARMdvfpCTp8Mxkz+GLwOoptY3AXZm5HLirzAOcCywvt/XA56EVJsBVwDuBM4GrJgOljFnftt7Ux5IkzbFpwyEzvwMcmFJeA2wp01uAC9rqN2fLPcDxEbEYOAfYkZkHMvN5YAewuix7bWZ+NzMTuLntviRJDen1PYehzNwHUL6+odSXAM+0jRsvtUPVxzvUJUkNmu2rsnZ6vyB7qHe+84j1tA5BMTQ0xOjoaA8twsGDB3te90js/OELMx47tAj++ZbbZ+2xN6yYtbt6maFFsGHFxNG581kyCD3CYPTZbz12ex439Rw/HP3eY6/h8FxELM7MfeXQ0P5SHwdOaRu3FHi21Eem1EdLfWmH8R1l5iZgE8DKlStzZGSk29BDGh0dpdd1j8Slh3FZ4w0rJrhuZ/9fUX0Q+hyEHmEw+uy3HvdePNKx3tRz/HD0e4+9HlbaBkyecbQOuL2tfkk5a2kV8EI57HQncHZEnFDeiD4buLMsezEiVpWzlC5puy9JUkOmfQkQEV+m9ar/pIgYp3XW0TXAbRFxGfAD4MIyfDtwHjAG/Bz4MEBmHoiITwP3l3GfyszJN7k/QuuMqEXAN8tNktSgacMhMy/qsuisDmMTuLzL/WwGNneoPwC8fbo+JElzx09IS5IqhoMkqWI4SJIqhoMkqWI4SJIqhoMkqdI/H3WUpMM03OWqAxtWTBzWFQl6sfea84/q/TfNPQdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsX/5yBJPej2vyRmqtf/OTFX/0fCPQdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRVDAdJUsVwkCRV+iYcImJ1RDwZEWMRsbHpfiRpPuuL/wQXEQuAG4E/AMaB+yNiW2Y+cTQeb+cPX+jpPzBJ0nzRL3sOZwJjmfl0Zv4S2AqsabgnSZq3+iUclgDPtM2Pl5okqQGRmU33QERcCJyTmX9e5j8EnJmZfzll3HpgfZl9C/Bkjw95EvCjHtedK4PQIwxGn4PQIwxGn4PQIwxGn030+MbMPHkmA/viPQdaewqntM0vBZ6dOigzNwGbjvTBIuKBzFx5pPdzNA1CjzAYfQ5CjzAYfQ5CjzAYffZ7j/1yWOl+YHlELIuIY4C1wLaGe5Kkeasv9hwycyIirgDuBBYAmzNzV8NtSdK81RfhAJCZ24Htc/RwR3xoag4MQo8wGH0OQo8wGH0OQo8wGH32dY998Ya0JKm/9Mt7DpKkPjKvwqFfL9EREadExN0RsTsidkXER0v9kxHxw4h4pNzOa7jPvRGxs/TyQKmdGBE7ImJP+XpCwz2+pW17PRIRP42Ij/XDtoyIzRGxPyIeb6t13H7RckP5XX0sIk5vsMd/jIjvlT6+HhHHl/pwRPxv2zb9wlz0eIg+u/6MI+LKsi2fjIhzGuzx1rb+9kbEI6Xe2LbsKjPnxY3WG91PAW8CjgEeBU5tuq/S22Lg9DL9GuC/gFOBTwJ/3XR/bX3uBU6aUvsHYGOZ3ghc23SfU37m/wO8sR+2JfAe4HTg8em2H3Ae8E0ggFXAvQ32eDawsExf29bjcPu4PtiWHX/G5bn0KHAssKz8HVjQRI9Tll8H/F3T27LbbT7tOfTtJToyc19mPlSmXwR2MzifEF8DbCnTW4ALGuxlqrOApzLzv5tuBCAzvwMcmFLutv3WADdnyz3A8RGxuIkeM/NbmTlRZu+h9TmkRnXZlt2sAbZm5kuZ+X1gjNbfg6PqUD1GRAB/Cnz5aPfRq/kUDgNxiY6IGAZOA+4tpSvK7vzmpg/ZAAl8KyIeLJ9WBxjKzH3QCjngDY11V1vLy598/bQtJ3Xbfv36+/pntPZoJi2LiIcj4j8i4t1NNdWm08+4H7flu4HnMnNPW62vtuV8CofoUOurU7Ui4tXAV4GPZeZPgc8DvwO8A9hHaze0Se/KzNOBc4HLI+I9DffTVfkw5QeAfy+lftuW0+m739eI+AQwAdxSSvuA387M04C/Av4tIl7bVH90/xn33bYELuLlL1z6bVvOq3CY0SU6mhIRr6QVDLdk5tcAMvO5zPxVZv4a+BfmYFf4UDLz2fJ1P/D10s9zk4c7ytf9zXX4MucCD2Xmc9B/27JNt+3XV7+vEbEOeD9wcZaD5OUwzY/L9IO0juX/blM9HuJn3G/bciHwx8Ctk7V+25Ywv8Khby/RUY4/3gTszszPttXbjzH/EfD41HXnSkQcFxGvmZym9Sbl47S24boybB1wezMdVl72yqyftuUU3bbfNuCSctbSKuCFycNPcy0iVgMfBz6QmT9vq58crf/FQkS8CVgOPN1Ej6WHbj/jbcDaiDg2IpbR6vO+ue6vzfuA72Xm+GSh37YlMH/OViovds6jdSbQU8Anmu6nra/fp7Wb+xjwSLmdB3wJ2Fnq24DFDfb4JlpnfDwK7JrcfsDrgbuAPeXriX2wPX8L+DHwurZa49uSVljtA/6P1qvZy7ptP1qHQm4sv6s7gZUN9jhG65j95O/mF8rYPym/C48CDwF/2PC27PozBj5RtuWTwLlN9VjqXwT+YsrYxrZlt5ufkJYkVebTYSVJ0gwZDpKkiuEgSaoYDpKkiuEgSaoYDpKkiuEgSaoYDpKkyv8Dybk8nIwyB34AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 根据箱线图的上下限进行异常值的删除\n",
    "def boxplot_drop(df,colname):\n",
    "    # 计算iqr：数据四分之三分位值与四分之一分位值的差\n",
    "    col = df[colname]\n",
    "    iqr = col.quantile(0.75)-col.quantile(0.25)\n",
    "    # 根据iqr计算异常值判断阈值\n",
    "    u_th = col.quantile(0.75) + 1.5*iqr # 上界\n",
    "    l_th = col.quantile(0.25) - 1.5*iqr # 下界\n",
    "    # 删除界限外的异常值\n",
    "    return (df\n",
    "            .loc[(df[colname]<u_th)&(df[colname]>l_th),:]\n",
    "            .reset_index(drop=True))\n",
    "# 查看异常值删除效果\n",
    "boxplot_drop(df_model,'y').y.hist()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 删除异常样本\n",
    "df_model = boxplot_drop(df_model,'y')\n",
    "# 删除不需要的特征\n",
    "df_model = df_model.drop(columns=['total_price','address','sub_station']).fillna(0)\n",
    "# 输出模型数据\n",
    "output_path = os.path.join(current_path,'processed_data', f'data_model.csv')\n",
    "df_model.to_csv(output_path,index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 使用留一法进行模型训练集和验证集的划分\n",
    "\n",
    "* 训练集：70%\n",
    "\n",
    "* 验证集：30%\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 划分数据集\n",
    "training, testing = train_test_split(df_model, test_size=0.30, random_state=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 划分出自变量与因变量\n",
    "x_train = training.copy().drop(columns=['id', 'y']).fillna(0)\n",
    "y_train = training.copy()['y']\n",
    "x_test = testing.copy().drop(columns=['id', 'y']).fillna(0)\n",
    "y_test = testing.copy()['y']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['area', 'is_nan', 'room', 'hall', 'sub_dist', 'subway_500',\n",
       "       'subway_1000', 'subway_2000', 'bus_500', 'bus_1000', 'bus_2000',\n",
       "       'sanjia_500', 'sanjia_1000', 'sanjia_2000', 'primary_school_500',\n",
       "       'primary_school_1000', 'primary_school_2000', 'middle_school_500',\n",
       "       'middle_school_1000', 'middle_school_2000', 'kindergarten_500',\n",
       "       'kindergarten_1000', 'kindergarten_2000', 'sub_line_1', 'sub_line_10',\n",
       "       'sub_line_11', 'sub_line_12', 'sub_line_13', 'sub_line_16',\n",
       "       'sub_line_17', 'sub_line_2', 'sub_line_3', 'sub_line_4', 'sub_line_5',\n",
       "       'sub_line_6', 'sub_line_7', 'sub_line_8', 'sub_line_9',\n",
       "       'sub_line_none'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 保存训练集自变量名称\n",
    "features = x_train.columns.tolist()\n",
    "x_train.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 利用参数搜索的Lasso回归模型训练租金预测模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "done\n"
     ]
    }
   ],
   "source": [
    "# 定义lasso回归模型\n",
    "pipe_lasso = Pipeline([\n",
    "    ('poly', PolynomialFeatures(degree=2)),\n",
    "    ('sc', StandardScaler()),\n",
    "    ('pwt', PowerTransformer('yeo-johnson')),\n",
    "    ('lasso_regr', LassoCV(\n",
    "        alphas=(list(np.arange(1, 11) * 0.1) +\n",
    "                list(np.arange(1, 20)) +\n",
    "                list(np.arange(1, 10) * 10)),\n",
    "        cv=KFold(n_splits=3, shuffle=True),\n",
    "        n_jobs=-1))\n",
    "])\n",
    "# 训练模型\n",
    "pipe_lasso.fit(x_train, y_train)\n",
    "print('done')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['C:\\\\Users\\\\wcd\\\\租金实战forxgboost082andabove\\\\model\\\\lasso_model.pkl']"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 保存lasso模型\n",
    "lasso_path = os.path.join(current_path,'model', f'lasso_model.pkl')\n",
    "# 保存模型的名称、使用的特征名称、模型、训练数据、预测数据\n",
    "joblib.dump({'model_name':'lasso_model',\n",
    "             'feature_list':features,\n",
    "             'model':pipe_lasso,\n",
    "             'data':[x_train,y_train,x_test,y_test]}, lasso_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.795793490636854\n",
      "24.22480279960347\n"
     ]
    }
   ],
   "source": [
    "# 定义一个计算mape的函数\n",
    "def mape(y_true, y_pred): \n",
    "    y_true, y_pred = np.array(y_true), np.array(y_pred)\n",
    "    return np.mean(np.abs((y_true - y_pred) / y_true)) * 100\n",
    "# 检查模型效果\n",
    "y_test_predict = pipe_lasso.predict(x_test)\n",
    "print(mean_absolute_error(y_test, y_test_predict))\n",
    "def mape(y_true, y_pred): \n",
    "    y_true, y_pred = np.array(y_true), np.array(y_pred)\n",
    "    return np.mean(np.abs((y_true - y_pred) / y_true)) * 100\n",
    "mape(y_test.astype(float), y_test_predict)\n",
    "print(mape(y_test.astype(float), y_test_predict))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 训练XGBOOST模型训练租金预测模型\n",
    "\n",
    "此脚本适用于使用xgboost0.82和0.90版本的同学：由于xgboost模型0.81版本和0.82及以上版本差距较大，训练出来的模型相对重要性会有非常大的差距，因此，此版本代码根据0.82版本的模型的训练结果，更新了模型优化策略，修改了样例结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "done\n"
     ]
    }
   ],
   "source": [
    "# 训练xgboost模型\n",
    "xgb_model = xgb.XGBRegressor(objective='reg:linear',\n",
    "                             nthread=8, n_jobs=-1,\n",
    "                             booster='gbtree', n_estimators=2000,\n",
    "                             learning_rate=0.05, max_depth=9,\n",
    "                             colsample_bylevel=1, colsample_bynode=1,\n",
    "                             colsample_bytree=0.8, base_score=0.5,\n",
    "                             eval_metric='mae', gamma=0, missing=-999,\n",
    "                             min_child_weight=11, max_delta_step=0,\n",
    "                             random_state=0, reg_alpha=0, reg_lambda=1,\n",
    "                             scale_pos_weight=1, seed=1337, silent=1,\n",
    "                             subsample=1, verbosity=1)\n",
    "\n",
    "pipe_xgb = Pipeline([\n",
    "        ('sc',StandardScaler()),\n",
    "        ('xgb_rg', xgb_model)\n",
    "        ])\n",
    "pipe_xgb.fit(x_train, y_train)\n",
    "print('done')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['C:\\\\Users\\\\wcd\\\\租金实战forxgboost082andabove\\\\model\\\\xgb_model.pkl']"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 保存xgboost模型\n",
    "xgb_path = os.path.join(current_path,'model', f'xgb_model.pkl')\n",
    "joblib.dump({'model_name':'xgb_model',\n",
    "             'feature_list':features,\n",
    "             'model':pipe_xgb,\n",
    "             'data':[x_train,y_train,x_test,y_test]}, xgb_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7.294062864575143\n",
      "9.961152469903965\n"
     ]
    }
   ],
   "source": [
    "y_test_predict = pipe_xgb.predict(x_test)\n",
    "print(mean_absolute_error(y_test, y_test_predict))\n",
    "def mape(y_true, y_pred): \n",
    "    y_true, y_pred = np.array(y_true), np.array(y_pred)\n",
    "    return np.mean(np.abs((y_true - y_pred) / y_true)) * 100\n",
    "\n",
    "mape(y_test.astype(float), y_test_predict)\n",
    "print(mape(y_test.astype(float), y_test_predict))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 根据XGBOOST模型的优化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用之前的lasso模型进行简单的集成学习\n",
    "training['ensemble_lasso'] = pipe_lasso.predict(training.fillna(0).loc[:,features])\n",
    "testing['ensemble_lasso'] = pipe_lasso.predict(testing.fillna(0).loc[:,features])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 提取新的训练集和验证集\n",
    "x_train = training.copy().fillna(0).drop(columns=['id', 'y'])\n",
    "y_train = training.copy()['y']\n",
    "x_test = testing.copy().fillna(0).drop(columns=['id', 'y'])\n",
    "y_test = testing.copy()['y']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "集成了lasso模型结果的变量结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['area', 'is_nan', 'room', 'hall', 'sub_dist', 'subway_500',\n",
       "       'subway_1000', 'subway_2000', 'bus_500', 'bus_1000', 'bus_2000',\n",
       "       'sanjia_500', 'sanjia_1000', 'sanjia_2000', 'primary_school_500',\n",
       "       'primary_school_1000', 'primary_school_2000', 'middle_school_500',\n",
       "       'middle_school_1000', 'middle_school_2000', 'kindergarten_500',\n",
       "       'kindergarten_1000', 'kindergarten_2000', 'sub_line_1', 'sub_line_10',\n",
       "       'sub_line_11', 'sub_line_12', 'sub_line_13', 'sub_line_16',\n",
       "       'sub_line_17', 'sub_line_2', 'sub_line_3', 'sub_line_4', 'sub_line_5',\n",
       "       'sub_line_6', 'sub_line_7', 'sub_line_8', 'sub_line_9', 'sub_line_none',\n",
       "       'ensemble_lasso'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "done\n"
     ]
    }
   ],
   "source": [
    "# 训练新的XGBOOST模型\n",
    "pipe_xgb.fit(x_train, y_train)\n",
    "print('done')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7.047857318849756\n",
      "9.569755681903105\n"
     ]
    }
   ],
   "source": [
    "# 查看模型的表现\n",
    "y_test_predict = pipe_xgb.predict(x_test)\n",
    "print(mean_absolute_error(y_test, y_test_predict))\n",
    "print(mape(y_test.astype(float), y_test_predict))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['C:\\\\Users\\\\wcd\\\\租金实战forxgboost082andabove\\\\model\\\\xgb_ensemble_model.pkl']"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 保存最终模型\n",
    "from sklearn.externals import joblib \n",
    "xgb_ensemble_path = os.path.join(current_path,'model', f'xgb_ensemble_model.pkl')\n",
    "joblib.dump({'model_name':'xgb_ensemble_model',\n",
    "             'feature_list':x_train.columns.tolist(),\n",
    "             'model':pipe_xgb,\n",
    "             'data':[x_train,y_train,x_test,y_test]}, xgb_ensemble_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}