exercise-baseline-model.ipynb

{"cells":[{"metadata":{},"cell_type":"markdown","source":"**[Feature Engineering Home Page](https://www.kaggle.com/learn/feature-engineering)**\n\n---\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Introduction\n\nIn the exercise, you will work with data from the TalkingData AdTracking competition.  The goal of the competition is to predict if a user will download an app after clicking through an ad. \n\n<center><a href=\"https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection\"><img src=\"https://i.imgur.com/srKxEkD.png\" width=600px></a></center>\n\nFor this course you will use a small sample of the data, dropping 99% of negative records (where the app wasn't downloaded) to make the target more balanced.\n\nAfter building a baseline model, you'll be able to see how your feature engineering and selection efforts improve the model's performance.\n\n## Setup\n\nBegin by running the code cell below to set up the exercise.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Set up code checking\nfrom learntools.core import binder\nbinder.bind(globals())\nfrom learntools.feature_engineering.ex1 import *","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Baseline Model\n\nThe first thing you'll do is construct a baseline model. We'll begin by looking at the data.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import pandas as pd\n\nclick_data = pd.read_csv('../input/feature-engineering-data/train_sample.csv',\n                         parse_dates=['click_time'])\nclick_data.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 1) Construct features from timestamps\n\nNotice that the `click_data` DataFrame has a `'click_time'` column with timestamp data.\n\nUse this column to create features for the coresponding day, hour, minute and second. \n\nStore these as new integer columns `day`, `hour`, `minute`, and `second` in a new DataFrame `clicks`.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Add new columns for timestamp features day, hour, minute, and second\nclicks = click_data.copy()\nclicks['day'] = clicks['click_time'].dt.day.astype('uint8')\n# Fill in the rest\nclicks['hour'] = clicks['click_time'].dt.hour.astype('uint8')\nclicks['minute'] = clicks['click_time'].dt.minute.astype('uint8')\nclicks['second'] = clicks['click_time'].dt.second.astype('uint8')\n\n# Check your answer\nq_1.check()","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"# Uncomment these if you need guidance\nq_1.hint()\nq_1.solution()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 2) Label Encoding\nFor each of the categorical features `['ip', 'app', 'device', 'os', 'channel']`, use scikit-learn's `LabelEncoder` to create new features in the `clicks` DataFrame. The new column names should be the original column name with `'_labels'` appended, like `ip_labels`.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn import preprocessing\n\ncat_features = ['ip', 'app', 'device', 'os', 'channel']\nlabel_encoder = preprocessing.LabelEncoder()\nfor feature in cat_features:\n    encoded = label_encoder.fit_transform(clicks[feature])\n    clicks[feature + '_labels'] = encoded\n\n# Check your answer\nq_2.check()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Uncomment these if you need guidance\nq_2.hint()\nq_2.solution()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Run the next code cell to view your new DataFrame.","execution_count":null},{"metadata":{"trusted":false},"cell_type":"code","source":"clicks.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3) One-hot Encoding\n\nIn the code cell above, you used label encoded features.  Would it have also made sense to instead use one-hot encoding for the categorical variables `'ip'`, `'app'`, `'device'`, `'os'`, or `'channel'`?\n\n**Note**: If you're not familiar with one-hot encoding, please check out **[this lesson](https://www.kaggle.com/alexisbcook/categorical-variables)** from the Intermediate Machine Learning course.\n\nRun the following line after you've decided your answer.","execution_count":null},{"metadata":{"trusted":false},"cell_type":"code","source":"# Check your answer (Run this code cell to receive credit!)\nq_3.solution()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Train, validation, and test sets\nWith our baseline features ready, we need to split our data into training and validation sets. We should also hold out a test set to measure the final accuracy of the model.\n\n### 4) Train/test splits with time series data\nThis is time series data. Are there any special considerations when creating train/test splits for time series? If so, what are they?\n\nUncomment the following line after you've decided your answer.","execution_count":null},{"metadata":{"trusted":false},"cell_type":"code","source":"# Check your answer (Run this code cell to receive credit!)\nq_4.solution()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Create train/validation/test splits\n\nHere we'll create training, validation, and test splits. First, `clicks` DataFrame is sorted in order of increasing time. The first 80% of the rows are the train set, the next 10% are the validation set, and the last 10% are the test set.","execution_count":null},{"metadata":{"trusted":false},"cell_type":"code","source":"feature_cols = ['day', 'hour', 'minute', 'second', \n                'ip_labels', 'app_labels', 'device_labels',\n                'os_labels', 'channel_labels']\n\nvalid_fraction = 0.1\nclicks_srt = clicks.sort_values('click_time')\nvalid_rows = int(len(clicks_srt) * valid_fraction)\ntrain = clicks_srt[:-valid_rows * 2]\n# valid size == test size, last two sections of the data\nvalid = clicks_srt[-valid_rows * 2:-valid_rows]\ntest = clicks_srt[-valid_rows:]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Train with LightGBM\n\nNow we can create LightGBM dataset objects for each of the smaller datasets and train the baseline model.","execution_count":null},{"metadata":{"trusted":false},"cell_type":"code","source":"import lightgbm as lgb\n\ndtrain = lgb.Dataset(train[feature_cols], label=train['is_attributed'])\ndvalid = lgb.Dataset(valid[feature_cols], label=valid['is_attributed'])\ndtest = lgb.Dataset(test[feature_cols], label=test['is_attributed'])\n\nparam = {'num_leaves': 64, 'objective': 'binary'}\nparam['metric'] = 'auc'\nnum_round = 1000\nbst = lgb.train(param, dtrain, num_round, valid_sets=[dvalid], early_stopping_rounds=10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Evaluate the model\nFinally, with the model trained, we evaluate its performance on the test set. ","execution_count":null},{"metadata":{"trusted":false},"cell_type":"code","source":"from sklearn import metrics\n\nypred = bst.predict(test[feature_cols])\nscore = metrics.roc_auc_score(test['is_attributed'], ypred)\nprint(f\"Test score: {score}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This will be our baseline score for the model. When we transform features, add new ones, or perform feature selection, we should be improving on this score. However, since this is the test set, we only want to look at it at the end of all our manipulations. At the very end of this course you'll look at the test score again to see if you improved on the baseline model.\n\n# Keep Going\nNow that you have a baseline model, you are ready to **[use categorical encoding techniques](https://www.kaggle.com/matleonard/categorical-encodings)** to improve it.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"---\n**[Feature Engineering Home Page](https://www.kaggle.com/learn/feature-engineering)**\n\n\n\n\n\n*Have questions or comments? Visit the [Learn Discussion forum](https://www.kaggle.com/learn-forum/161443) to chat with other Learners.*","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}