Recommendation System .ipynb

{
    "cells": [
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "# Recommendation System content based filtering\n- Create a recommendation system using collaborative filtering"
        },
        {
            "metadata": {
                "collapsed": true
            },
            "cell_type": "markdown",
            "source": "### Table of contents\n\n<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n    <ol>\n        <li><a href=\"#ref1\">Acquiring the Data</a></li>\n        <li><a href=\"#ref2\">Preprocessing</a></li>\n        <li><a href=\"#ref3\">Content-Based Filtering</a></li>\n        <li><a href=\"#ref4\">Collaborative Filtering</a></li>\n    </ol>\n</div>\n<br>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "### Advantages and Disadvantages of Content-Based Filtering\n\n##### Advantages\n\n-   Learns user's preferences\n-   Highly personalized for the user\n\n##### Disadvantages\n\n-   Doesn't take into account what others think of the item, so low quality item recommendations might happen\n-   Extracting data is not always intuitive\n-   Determining what characteristics of the item the user dislikes or likes is not always obvious"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<a id=\"ref1\"></a>\n\n# Acquiring the Data\n\n- Acquire and extract the dataset"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "!wget -O moviedataset.zip https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/moviedataset.zip\nprint('unziping files...')\n!unzip -o -j moviedataset.zip ",
            "execution_count": 1,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "--2020-10-16 03:16:06--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/moviedataset.zip\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 160301210 (153M) [application/zip]\nSaving to: \u2018moviedataset.zip\u2019\n\nmoviedataset.zip    100%[===================>] 152.88M  42.8MB/s    in 3.6s    \n\n2020-10-16 03:16:10 (42.5 MB/s) - \u2018moviedataset.zip\u2019 saved [160301210/160301210]\n\nunziping files...\nArchive:  moviedataset.zip\n  inflating: links.csv               \n  inflating: movies.csv              \n  inflating: ratings.csv             \n  inflating: README.txt              \n  inflating: tags.csv                \n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "# Preprocessing\n"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# Import librtries\nimport pandas as pd\nfrom math import sqrt\nimport numpy as np\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\nprint('imported')",
            "execution_count": 2,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "imported\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "## read file into dataframes\nmovies_df = pd.read_csv('movies.csv')\nratings_df = pd.read_csv('ratings.csv')\nmovies_df.head()",
            "execution_count": 3,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 3,
                    "data": {
                        "text/plain": "   movieId                               title  \\\n0        1                    Toy Story (1995)   \n1        2                      Jumanji (1995)   \n2        3             Grumpier Old Men (1995)   \n3        4            Waiting to Exhale (1995)   \n4        5  Father of the Bride Part II (1995)   \n\n                                        genres  \n0  Adventure|Animation|Children|Comedy|Fantasy  \n1                   Adventure|Children|Fantasy  \n2                               Comedy|Romance  \n3                         Comedy|Drama|Romance  \n4                                       Comedy  ",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>movieId</th>\n      <th>title</th>\n      <th>genres</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>Toy Story (1995)</td>\n      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>Jumanji (1995)</td>\n      <td>Adventure|Children|Fantasy</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>Grumpier Old Men (1995)</td>\n      <td>Comedy|Romance</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>Waiting to Exhale (1995)</td>\n      <td>Comedy|Drama|Romance</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>5</td>\n      <td>Father of the Bride Part II (1995)</td>\n      <td>Comedy</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# split the values in the Genres column into a list of Genres\n# Every genre is separated by a | so we simply have to call the split function on |\nmovies_df['genres'] = movies_df.genres.str.split('|')\nmovies_df.head()",
            "execution_count": 4,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 4,
                    "data": {
                        "text/plain": "   movieId                               title  \\\n0        1                    Toy Story (1995)   \n1        2                      Jumanji (1995)   \n2        3             Grumpier Old Men (1995)   \n3        4            Waiting to Exhale (1995)   \n4        5  Father of the Bride Part II (1995)   \n\n                                              genres  \n0  [Adventure, Animation, Children, Comedy, Fantasy]  \n1                     [Adventure, Children, Fantasy]  \n2                                  [Comedy, Romance]  \n3                           [Comedy, Drama, Romance]  \n4                                           [Comedy]  ",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>movieId</th>\n      <th>title</th>\n      <th>genres</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>Toy Story (1995)</td>\n      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>Jumanji (1995)</td>\n      <td>[Adventure, Children, Fantasy]</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>Grumpier Old Men (1995)</td>\n      <td>[Comedy, Romance]</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>Waiting to Exhale (1995)</td>\n      <td>[Comedy, Drama, Romance]</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>5</td>\n      <td>Father of the Bride Part II (1995)</td>\n      <td>[Comedy]</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# remove the year from the title column\n\n#We specify the parantheses so we don't conflict with movies that have years in their titles\nmovies_df['year'] = movies_df.title.str.extract('(\\(\\d\\d\\d\\d\\))',expand=False)\n#Removing the parentheses\nmovies_df['year'] = movies_df.year.str.extract('(\\d\\d\\d\\d)',expand=False)\n\n#Removing the years from the 'title' column\nmovies_df['title'] = movies_df.title.str.replace('(\\(\\d\\d\\d\\d\\))', '')\n#Applying the strip function to get rid of any ending whitespace characters that may have appeared\nmovies_df['title'] = movies_df['title'].apply(lambda x: x.strip())\n\nmovies_df.head()",
            "execution_count": 5,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 5,
                    "data": {
                        "text/plain": "   movieId                        title  \\\n0        1                    Toy Story   \n1        2                      Jumanji   \n2        3             Grumpier Old Men   \n3        4            Waiting to Exhale   \n4        5  Father of the Bride Part II   \n\n                                              genres  year  \n0  [Adventure, Animation, Children, Comedy, Fantasy]  1995  \n1                     [Adventure, Children, Fantasy]  1995  \n2                                  [Comedy, Romance]  1995  \n3                           [Comedy, Drama, Romance]  1995  \n4                                           [Comedy]  1995  ",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>movieId</th>\n      <th>title</th>\n      <th>genres</th>\n      <th>year</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>Toy Story</td>\n      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n      <td>1995</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>Jumanji</td>\n      <td>[Adventure, Children, Fantasy]</td>\n      <td>1995</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>Grumpier Old Men</td>\n      <td>[Comedy, Romance]</td>\n      <td>1995</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>Waiting to Exhale</td>\n      <td>[Comedy, Drama, Romance]</td>\n      <td>1995</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>5</td>\n      <td>Father of the Bride Part II</td>\n      <td>[Comedy]</td>\n      <td>1995</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Now, since keeping genres in a list format isn't optimal for the content-based recommendation system technique, I will convert the list of genres to a vector where each column corresponds to binary value of the feature.\n\n1 shows that a movie has that genre and\n0 shows that it doesn't.\n\nBut first I will copy the original dataframe into another variable since genres won't be important for our first recommendation system."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# copy dataframe into a new variable\ngenresMovies_df = movies_df.copy()\n\n#For every row in the dataframe, iterate through the list of genres and place a 1 into the corresponding column\nfor index, row in movies_df.iterrows():\n    for genre in row['genres']:\n        genresMovies_df.at[index, genre] = 1\n         \n        \n#Filling in the NaN values with 0 to show that a movie doesn't have that column's genre\ngenresMovies_df = genresMovies_df.fillna(0)\n#genresMovies_df.astype(int)\n\ngenresMovies_df.head()",
            "execution_count": 6,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 6,
                    "data": {
                        "text/plain": "   movieId                        title  \\\n0        1                    Toy Story   \n1        2                      Jumanji   \n2        3             Grumpier Old Men   \n3        4            Waiting to Exhale   \n4        5  Father of the Bride Part II   \n\n                                              genres  year  Adventure  \\\n0  [Adventure, Animation, Children, Comedy, Fantasy]  1995        1.0   \n1                     [Adventure, Children, Fantasy]  1995        1.0   \n2                                  [Comedy, Romance]  1995        0.0   \n3                           [Comedy, Drama, Romance]  1995        0.0   \n4                                           [Comedy]  1995        0.0   \n\n   Animation  Children  Comedy  Fantasy  Romance  ...  Horror  Mystery  \\\n0        1.0       1.0     1.0      1.0      0.0  ...     0.0      0.0   \n1        0.0       1.0     0.0      1.0      0.0  ...     0.0      0.0   \n2        0.0       0.0     1.0      0.0      1.0  ...     0.0      0.0   \n3        0.0       0.0     1.0      0.0      1.0  ...     0.0      0.0   \n4        0.0       0.0     1.0      0.0      0.0  ...     0.0      0.0   \n\n   Sci-Fi  IMAX  Documentary  War  Musical  Western  Film-Noir  \\\n0     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n1     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n2     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n3     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n4     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n\n   (no genres listed)  \n0                 0.0  \n1                 0.0  \n2                 0.0  \n3                 0.0  \n4                 0.0  \n\n[5 rows x 24 columns]",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>movieId</th>\n      <th>title</th>\n      <th>genres</th>\n      <th>year</th>\n      <th>Adventure</th>\n      <th>Animation</th>\n      <th>Children</th>\n      <th>Comedy</th>\n      <th>Fantasy</th>\n      <th>Romance</th>\n      <th>...</th>\n      <th>Horror</th>\n      <th>Mystery</th>\n      <th>Sci-Fi</th>\n      <th>IMAX</th>\n      <th>Documentary</th>\n      <th>War</th>\n      <th>Musical</th>\n      <th>Western</th>\n      <th>Film-Noir</th>\n      <th>(no genres listed)</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>Toy Story</td>\n      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n      <td>1995</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>Jumanji</td>\n      <td>[Adventure, Children, Fantasy]</td>\n      <td>1995</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>Grumpier Old Men</td>\n      <td>[Comedy, Romance]</td>\n      <td>1995</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>Waiting to Exhale</td>\n      <td>[Comedy, Drama, Romance]</td>\n      <td>1995</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>5</td>\n      <td>Father of the Bride Part II</td>\n      <td>[Comedy]</td>\n      <td>1995</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows \u00d7 24 columns</p>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Check the user ratings dataframe"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "ratings_df.head()",
            "execution_count": 7,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 7,
                    "data": {
                        "text/plain": "   userId  movieId  rating   timestamp\n0       1      169     2.5  1204927694\n1       1     2471     3.0  1204927438\n2       1    48516     5.0  1204927435\n3       2     2571     3.5  1436165433\n4       2   109487     4.0  1436165496",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>userId</th>\n      <th>movieId</th>\n      <th>rating</th>\n      <th>timestamp</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>169</td>\n      <td>2.5</td>\n      <td>1204927694</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1</td>\n      <td>2471</td>\n      <td>3.0</td>\n      <td>1204927438</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1</td>\n      <td>48516</td>\n      <td>5.0</td>\n      <td>1204927435</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>2</td>\n      <td>2571</td>\n      <td>3.5</td>\n      <td>1436165433</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>2</td>\n      <td>109487</td>\n      <td>4.0</td>\n      <td>1436165496</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "I will not need timestamp column"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# remove the timestamp column from dataframe\nratings_df = ratings_df.drop('timestamp', 1)\nratings_df.head()",
            "execution_count": 8,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 8,
                    "data": {
                        "text/plain": "   userId  movieId  rating\n0       1      169     2.5\n1       1     2471     3.0\n2       1    48516     5.0\n3       2     2571     3.5\n4       2   109487     4.0",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>userId</th>\n      <th>movieId</th>\n      <th>rating</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>169</td>\n      <td>2.5</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1</td>\n      <td>2471</td>\n      <td>3.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1</td>\n      <td>48516</td>\n      <td>5.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>2</td>\n      <td>2571</td>\n      <td>3.5</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>2</td>\n      <td>109487</td>\n      <td>4.0</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<a id=\"ref3\"></a>\n\n# Content-Based recommendation system"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Now, what a user's favourite aspects of an item is, I'm going to figure out the input's favorite genres from the movies and ratings given.\n\nFirst, creating an input user to recommend movies to:\n\nNotice: To add more movies, simply increase the amount of elements in the **userInput**. To add more in! write it in with capital letters and if a movie starts with a \"The\", like \"The Matrix\" then write it in like this: 'Matrix, The' ."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "userInput = [\n            {'title':'Breakfast Club, The', 'rating':5},\n            {'title':'Toy Story', 'rating':3.5},\n            {'title':'Jumanji', 'rating':2},\n            {'title':\"Pulp Fiction\", 'rating':5},\n            {'title':'Akira', 'rating':4.5}\n         ] \ninputMovies = pd.DataFrame(userInput)\ninputMovies",
            "execution_count": 9,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 9,
                    "data": {
                        "text/plain": "                 title  rating\n0  Breakfast Club, The     5.0\n1            Toy Story     3.5\n2              Jumanji     2.0\n3         Pulp Fiction     5.0\n4                Akira     4.5",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>title</th>\n      <th>rating</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>Breakfast Club, The</td>\n      <td>5.0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>Toy Story</td>\n      <td>3.5</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>Jumanji</td>\n      <td>2.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>Pulp Fiction</td>\n      <td>5.0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>Akira</td>\n      <td>4.5</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "#### Add movieId to input user\n\nNow extract the input movie's ID's from the movies dataframe and add them into it.\n\nWe can achieve this by first filtering out the rows that contain the input movie's title and then merging this subset with the input dataframe. We also drop unnecessary columns for the input to save memory space."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Filtering out the movies by title\ninputId = movies_df[movies_df['title'].isin(inputMovies['title'].tolist())]\ninputId\n",
            "execution_count": 10,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 10,
                    "data": {
                        "text/plain": "      movieId                title  \\\n0           1            Toy Story   \n1           2              Jumanji   \n293       296         Pulp Fiction   \n1246     1274                Akira   \n1885     1968  Breakfast Club, The   \n\n                                                 genres  year  \n0     [Adventure, Animation, Children, Comedy, Fantasy]  1995  \n1                        [Adventure, Children, Fantasy]  1995  \n293                    [Comedy, Crime, Drama, Thriller]  1994  \n1246             [Action, Adventure, Animation, Sci-Fi]  1988  \n1885                                    [Comedy, Drama]  1985  ",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>movieId</th>\n      <th>title</th>\n      <th>genres</th>\n      <th>year</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>Toy Story</td>\n      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n      <td>1995</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>Jumanji</td>\n      <td>[Adventure, Children, Fantasy]</td>\n      <td>1995</td>\n    </tr>\n    <tr>\n      <th>293</th>\n      <td>296</td>\n      <td>Pulp Fiction</td>\n      <td>[Comedy, Crime, Drama, Thriller]</td>\n      <td>1994</td>\n    </tr>\n    <tr>\n      <th>1246</th>\n      <td>1274</td>\n      <td>Akira</td>\n      <td>[Action, Adventure, Animation, Sci-Fi]</td>\n      <td>1988</td>\n    </tr>\n    <tr>\n      <th>1885</th>\n      <td>1968</td>\n      <td>Breakfast Club, The</td>\n      <td>[Comedy, Drama]</td>\n      <td>1985</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Then merging it so we can get the movieId. It's implicitly merging it by title.\ninputMovies = pd.merge(inputId, inputMovies)\ninputMovies",
            "execution_count": 11,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 11,
                    "data": {
                        "text/plain": "   movieId                title  \\\n0        1            Toy Story   \n1        2              Jumanji   \n2      296         Pulp Fiction   \n3     1274                Akira   \n4     1968  Breakfast Club, The   \n\n                                              genres  year  rating  \n0  [Adventure, Animation, Children, Comedy, Fantasy]  1995     3.5  \n1                     [Adventure, Children, Fantasy]  1995     2.0  \n2                   [Comedy, Crime, Drama, Thriller]  1994     5.0  \n3             [Action, Adventure, Animation, Sci-Fi]  1988     4.5  \n4                                    [Comedy, Drama]  1985     5.0  ",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>movieId</th>\n      <th>title</th>\n      <th>genres</th>\n      <th>year</th>\n      <th>rating</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>Toy Story</td>\n      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n      <td>1995</td>\n      <td>3.5</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>Jumanji</td>\n      <td>[Adventure, Children, Fantasy]</td>\n      <td>1995</td>\n      <td>2.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>296</td>\n      <td>Pulp Fiction</td>\n      <td>[Comedy, Crime, Drama, Thriller]</td>\n      <td>1994</td>\n      <td>5.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1274</td>\n      <td>Akira</td>\n      <td>[Action, Adventure, Animation, Sci-Fi]</td>\n      <td>1988</td>\n      <td>4.5</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1968</td>\n      <td>Breakfast Club, The</td>\n      <td>[Comedy, Drama]</td>\n      <td>1985</td>\n      <td>5.0</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Dropping information we won't use from the input dataframe\ninputMovies = inputMovies.drop('genres', 1).drop('year', 1)\n\n#Final input dataframe\ninputMovies",
            "execution_count": 12,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 12,
                    "data": {
                        "text/plain": "   movieId                title  rating\n0        1            Toy Story     3.5\n1        2              Jumanji     2.0\n2      296         Pulp Fiction     5.0\n3     1274                Akira     4.5\n4     1968  Breakfast Club, The     5.0",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>movieId</th>\n      <th>title</th>\n      <th>rating</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>Toy Story</td>\n      <td>3.5</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>Jumanji</td>\n      <td>2.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>296</td>\n      <td>Pulp Fiction</td>\n      <td>5.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1274</td>\n      <td>Akira</td>\n      <td>4.5</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1968</td>\n      <td>Breakfast Club, The</td>\n      <td>5.0</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Filtering out the movies from the input\nuserMovies = genresMovies_df[genresMovies_df['movieId'].isin(inputMovies['movieId'].tolist())]\n\n#Resetting the index to avoid future issues\nuserMovies = userMovies.reset_index(drop=True)\nuserMovies",
            "execution_count": 13,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 13,
                    "data": {
                        "text/plain": "   movieId                title  \\\n0        1            Toy Story   \n1        2              Jumanji   \n2      296         Pulp Fiction   \n3     1274                Akira   \n4     1968  Breakfast Club, The   \n\n                                              genres  year  Adventure  \\\n0  [Adventure, Animation, Children, Comedy, Fantasy]  1995        1.0   \n1                     [Adventure, Children, Fantasy]  1995        1.0   \n2                   [Comedy, Crime, Drama, Thriller]  1994        0.0   \n3             [Action, Adventure, Animation, Sci-Fi]  1988        1.0   \n4                                    [Comedy, Drama]  1985        0.0   \n\n   Animation  Children  Comedy  Fantasy  Romance  ...  Horror  Mystery  \\\n0        1.0       1.0     1.0      1.0      0.0  ...     0.0      0.0   \n1        0.0       1.0     0.0      1.0      0.0  ...     0.0      0.0   \n2        0.0       0.0     1.0      0.0      0.0  ...     0.0      0.0   \n3        1.0       0.0     0.0      0.0      0.0  ...     0.0      0.0   \n4        0.0       0.0     1.0      0.0      0.0  ...     0.0      0.0   \n\n   Sci-Fi  IMAX  Documentary  War  Musical  Western  Film-Noir  \\\n0     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n1     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n2     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n3     1.0   0.0          0.0  0.0      0.0      0.0        0.0   \n4     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n\n   (no genres listed)  \n0                 0.0  \n1                 0.0  \n2                 0.0  \n3                 0.0  \n4                 0.0  \n\n[5 rows x 24 columns]",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>movieId</th>\n      <th>title</th>\n      <th>genres</th>\n      <th>year</th>\n      <th>Adventure</th>\n      <th>Animation</th>\n      <th>Children</th>\n      <th>Comedy</th>\n      <th>Fantasy</th>\n      <th>Romance</th>\n      <th>...</th>\n      <th>Horror</th>\n      <th>Mystery</th>\n      <th>Sci-Fi</th>\n      <th>IMAX</th>\n      <th>Documentary</th>\n      <th>War</th>\n      <th>Musical</th>\n      <th>Western</th>\n      <th>Film-Noir</th>\n      <th>(no genres listed)</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>Toy Story</td>\n      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n      <td>1995</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>Jumanji</td>\n      <td>[Adventure, Children, Fantasy]</td>\n      <td>1995</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>296</td>\n      <td>Pulp Fiction</td>\n      <td>[Comedy, Crime, Drama, Thriller]</td>\n      <td>1994</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1274</td>\n      <td>Akira</td>\n      <td>[Action, Adventure, Animation, Sci-Fi]</td>\n      <td>1988</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1968</td>\n      <td>Breakfast Club, The</td>\n      <td>[Comedy, Drama]</td>\n      <td>1985</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows \u00d7 24 columns</p>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "We'll only need the actual genre table, I'll clean this up by resetting the index and dropping the movieId, title, genres and year columns."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Resetting the index to avoid future issues\nuserMovies = userMovies.reset_index(drop=True)\nuserMovies",
            "execution_count": 14,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 14,
                    "data": {
                        "text/plain": "   movieId                title  \\\n0        1            Toy Story   \n1        2              Jumanji   \n2      296         Pulp Fiction   \n3     1274                Akira   \n4     1968  Breakfast Club, The   \n\n                                              genres  year  Adventure  \\\n0  [Adventure, Animation, Children, Comedy, Fantasy]  1995        1.0   \n1                     [Adventure, Children, Fantasy]  1995        1.0   \n2                   [Comedy, Crime, Drama, Thriller]  1994        0.0   \n3             [Action, Adventure, Animation, Sci-Fi]  1988        1.0   \n4                                    [Comedy, Drama]  1985        0.0   \n\n   Animation  Children  Comedy  Fantasy  Romance  ...  Horror  Mystery  \\\n0        1.0       1.0     1.0      1.0      0.0  ...     0.0      0.0   \n1        0.0       1.0     0.0      1.0      0.0  ...     0.0      0.0   \n2        0.0       0.0     1.0      0.0      0.0  ...     0.0      0.0   \n3        1.0       0.0     0.0      0.0      0.0  ...     0.0      0.0   \n4        0.0       0.0     1.0      0.0      0.0  ...     0.0      0.0   \n\n   Sci-Fi  IMAX  Documentary  War  Musical  Western  Film-Noir  \\\n0     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n1     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n2     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n3     1.0   0.0          0.0  0.0      0.0      0.0        0.0   \n4     0.0   0.0          0.0  0.0      0.0      0.0        0.0   \n\n   (no genres listed)  \n0                 0.0  \n1                 0.0  \n2                 0.0  \n3                 0.0  \n4                 0.0  \n\n[5 rows x 24 columns]",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>movieId</th>\n      <th>title</th>\n      <th>genres</th>\n      <th>year</th>\n      <th>Adventure</th>\n      <th>Animation</th>\n      <th>Children</th>\n      <th>Comedy</th>\n      <th>Fantasy</th>\n      <th>Romance</th>\n      <th>...</th>\n      <th>Horror</th>\n      <th>Mystery</th>\n      <th>Sci-Fi</th>\n      <th>IMAX</th>\n      <th>Documentary</th>\n      <th>War</th>\n      <th>Musical</th>\n      <th>Western</th>\n      <th>Film-Noir</th>\n      <th>(no genres listed)</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>Toy Story</td>\n      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n      <td>1995</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>Jumanji</td>\n      <td>[Adventure, Children, Fantasy]</td>\n      <td>1995</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>296</td>\n      <td>Pulp Fiction</td>\n      <td>[Comedy, Crime, Drama, Thriller]</td>\n      <td>1994</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1274</td>\n      <td>Akira</td>\n      <td>[Action, Adventure, Animation, Sci-Fi]</td>\n      <td>1988</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1968</td>\n      <td>Breakfast Club, The</td>\n      <td>[Comedy, Drama]</td>\n      <td>1985</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows \u00d7 24 columns</p>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Dropping unnecessary issues due to save memory and to avoid issues\nuserGenreTable = userMovies.drop('movieId', 1).drop('title', 1).drop('genres', 1).drop('year', 1)\nuserGenreTable",
            "execution_count": 15,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 15,
                    "data": {
                        "text/plain": "   Adventure  Animation  Children  Comedy  Fantasy  Romance  Drama  Action  \\\n0        1.0        1.0       1.0     1.0      1.0      0.0    0.0     0.0   \n1        1.0        0.0       1.0     0.0      1.0      0.0    0.0     0.0   \n2        0.0        0.0       0.0     1.0      0.0      0.0    1.0     0.0   \n3        1.0        1.0       0.0     0.0      0.0      0.0    0.0     1.0   \n4        0.0        0.0       0.0     1.0      0.0      0.0    1.0     0.0   \n\n   Crime  Thriller  Horror  Mystery  Sci-Fi  IMAX  Documentary  War  Musical  \\\n0    0.0       0.0     0.0      0.0     0.0   0.0          0.0  0.0      0.0   \n1    0.0       0.0     0.0      0.0     0.0   0.0          0.0  0.0      0.0   \n2    1.0       1.0     0.0      0.0     0.0   0.0          0.0  0.0      0.0   \n3    0.0       0.0     0.0      0.0     1.0   0.0          0.0  0.0      0.0   \n4    0.0       0.0     0.0      0.0     0.0   0.0          0.0  0.0      0.0   \n\n   Western  Film-Noir  (no genres listed)  \n0      0.0        0.0                 0.0  \n1      0.0        0.0                 0.0  \n2      0.0        0.0                 0.0  \n3      0.0        0.0                 0.0  \n4      0.0        0.0                 0.0  ",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Adventure</th>\n      <th>Animation</th>\n      <th>Children</th>\n      <th>Comedy</th>\n      <th>Fantasy</th>\n      <th>Romance</th>\n      <th>Drama</th>\n      <th>Action</th>\n      <th>Crime</th>\n      <th>Thriller</th>\n      <th>Horror</th>\n      <th>Mystery</th>\n      <th>Sci-Fi</th>\n      <th>IMAX</th>\n      <th>Documentary</th>\n      <th>War</th>\n      <th>Musical</th>\n      <th>Western</th>\n      <th>Film-Noir</th>\n      <th>(no genres listed)</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Now it's ready to start learning the input's preferences!\n\nFirst, turn each genre into weights using the input's reviews and multiplying them into the input's genre table. Than summing up the resulting table by column. "
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# rating preferences\ninputMovies['rating']",
            "execution_count": 16,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 16,
                    "data": {
                        "text/plain": "0    3.5\n1    2.0\n2    5.0\n3    4.5\n4    5.0\nName: rating, dtype: float64"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# the weights for every of the user's prefereces\nuserProfile = userGenreTable.transpose().dot(inputMovies['rating'])\nuserProfile",
            "execution_count": 17,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 17,
                    "data": {
                        "text/plain": "Adventure             10.0\nAnimation              8.0\nChildren               5.5\nComedy                13.5\nFantasy                5.5\nRomance                0.0\nDrama                 10.0\nAction                 4.5\nCrime                  5.0\nThriller               5.0\nHorror                 0.0\nMystery                0.0\nSci-Fi                 4.5\nIMAX                   0.0\nDocumentary            0.0\nWar                    0.0\nMusical                0.0\nWestern                0.0\nFilm-Noir              0.0\n(no genres listed)     0.0\ndtype: float64"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# drop the unnecessary information from original dataframe\ngenreTable = genresMovies_df.set_index(genresMovies_df['movieId'])\ngenreTable = genreTable.drop('movieId', 1).drop('title', 1).drop('genres', 1).drop('year', 1)\ngenreTable.head()",
            "execution_count": 18,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 18,
                    "data": {
                        "text/plain": "         Adventure  Animation  Children  Comedy  Fantasy  Romance  Drama  \\\nmovieId                                                                    \n1              1.0        1.0       1.0     1.0      1.0      0.0    0.0   \n2              1.0        0.0       1.0     0.0      1.0      0.0    0.0   \n3              0.0        0.0       0.0     1.0      0.0      1.0    0.0   \n4              0.0        0.0       0.0     1.0      0.0      1.0    1.0   \n5              0.0        0.0       0.0     1.0      0.0      0.0    0.0   \n\n         Action  Crime  Thriller  Horror  Mystery  Sci-Fi  IMAX  Documentary  \\\nmovieId                                                                        \n1           0.0    0.0       0.0     0.0      0.0     0.0   0.0          0.0   \n2           0.0    0.0       0.0     0.0      0.0     0.0   0.0          0.0   \n3           0.0    0.0       0.0     0.0      0.0     0.0   0.0          0.0   \n4           0.0    0.0       0.0     0.0      0.0     0.0   0.0          0.0   \n5           0.0    0.0       0.0     0.0      0.0     0.0   0.0          0.0   \n\n         War  Musical  Western  Film-Noir  (no genres listed)  \nmovieId                                                        \n1        0.0      0.0      0.0        0.0                 0.0  \n2        0.0      0.0      0.0        0.0                 0.0  \n3        0.0      0.0      0.0        0.0                 0.0  \n4        0.0      0.0      0.0        0.0                 0.0  \n5        0.0      0.0      0.0        0.0                 0.0  ",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Adventure</th>\n      <th>Animation</th>\n      <th>Children</th>\n      <th>Comedy</th>\n      <th>Fantasy</th>\n      <th>Romance</th>\n      <th>Drama</th>\n      <th>Action</th>\n      <th>Crime</th>\n      <th>Thriller</th>\n      <th>Horror</th>\n      <th>Mystery</th>\n      <th>Sci-Fi</th>\n      <th>IMAX</th>\n      <th>Documentary</th>\n      <th>War</th>\n      <th>Musical</th>\n      <th>Western</th>\n      <th>Film-Noir</th>\n      <th>(no genres listed)</th>\n    </tr>\n    <tr>\n      <th>movieId</th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>1</th>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "print(genreTable.shape)",
            "execution_count": 19,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "(34208, 20)\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "With the input's profile and the complete list of movies and their genres , I'm going to take the weighted average of every movie based on the input profile and recommend the top twenty movies that most satisfy it."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Multiply the genres by the weights and then take the weighted average\nrecommendationTable_df = ((genreTable*userProfile).sum(axis=1))/(userProfile.sum())\nrecommendationTable_df.head()",
            "execution_count": 20,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 20,
                    "data": {
                        "text/plain": "movieId\n1    0.594406\n2    0.293706\n3    0.188811\n4    0.328671\n5    0.188811\ndtype: float64"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Sort recommendations in descending order\nrecommendationTable_df = recommendationTable_df.sort_values(ascending=False)\n\nrecommendationTable_df.head()",
            "execution_count": 21,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 21,
                    "data": {
                        "text/plain": "movieId\n5018      0.748252\n26093     0.734266\n27344     0.720280\n148775    0.685315\n6902      0.678322\ndtype: float64"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "### Recommendation table"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#The final recommendation table\nmovies_df.loc[movies_df['movieId'].isin(recommendationTable_df.head(20).keys())]",
            "execution_count": 22,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 22,
                    "data": {
                        "text/plain": "       movieId                                              title  \\\n664        673                                          Space Jam   \n1824      1907                                              Mulan   \n2902      2987                           Who Framed Roger Rabbit?   \n4923      5018                                           Motorama   \n6793      6902                                      Interstate 60   \n8605     26093         Wonderful World of the Brothers Grimm, The   \n8783     26340  Twelve Tasks of Asterix, The (Les douze travau...   \n9296     27344  Revolutionary Girl Utena: Adolescence of Utena...   \n9825     32031                                             Robots   \n11716    51632                            Atlantis: Milo's Return   \n11751    51939                TMNT (Teenage Mutant Ninja Turtles)   \n13250    64645                                  The Wrecking Crew   \n16055    81132                                             Rubber   \n18312    91335                                      Gruffalo, The   \n22778   108540           Ernest & C\u00e9lestine (Ernest et C\u00e9lestine)   \n22881   108932                                     The Lego Movie   \n25218   117646                     Dragonheart 2: A New Beginning   \n26442   122787                                       The 39 Steps   \n32854   146305                             Princes and Princesses   \n33509   148775                Wizards of Waverly Place: The Movie   \n\n                                                  genres  year  \n664    [Adventure, Animation, Children, Comedy, Fanta...  1996  \n1824   [Adventure, Animation, Children, Comedy, Drama...  1998  \n2902   [Adventure, Animation, Children, Comedy, Crime...  1988  \n4923   [Adventure, Comedy, Crime, Drama, Fantasy, Mys...  1991  \n6793   [Adventure, Comedy, Drama, Fantasy, Mystery, S...  2002  \n8605   [Adventure, Animation, Children, Comedy, Drama...  1962  \n8783   [Action, Adventure, Animation, Children, Comed...  1976  \n9296   [Action, Adventure, Animation, Comedy, Drama, ...  1999  \n9825   [Adventure, Animation, Children, Comedy, Fanta...  2005  \n11716  [Action, Adventure, Animation, Children, Comed...  2003  \n11751  [Action, Adventure, Animation, Children, Comed...  2007  \n13250  [Action, Adventure, Comedy, Crime, Drama, Thri...  1968  \n16055  [Action, Adventure, Comedy, Crime, Drama, Film...  2010  \n18312    [Adventure, Animation, Children, Comedy, Drama]  2009  \n22778  [Adventure, Animation, Children, Comedy, Drama...  2012  \n22881  [Action, Adventure, Animation, Children, Comed...  2014  \n25218  [Action, Adventure, Comedy, Drama, Fantasy, Th...  2000  \n26442  [Action, Adventure, Comedy, Crime, Drama, Thri...  1959  \n32854  [Animation, Children, Comedy, Drama, Fantasy, ...  2000  \n33509  [Adventure, Children, Comedy, Drama, Fantasy, ...  2009  ",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>movieId</th>\n      <th>title</th>\n      <th>genres</th>\n      <th>year</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>664</th>\n      <td>673</td>\n      <td>Space Jam</td>\n      <td>[Adventure, Animation, Children, Comedy, Fanta...</td>\n      <td>1996</td>\n    </tr>\n    <tr>\n      <th>1824</th>\n      <td>1907</td>\n      <td>Mulan</td>\n      <td>[Adventure, Animation, Children, Comedy, Drama...</td>\n      <td>1998</td>\n    </tr>\n    <tr>\n      <th>2902</th>\n      <td>2987</td>\n      <td>Who Framed Roger Rabbit?</td>\n      <td>[Adventure, Animation, Children, Comedy, Crime...</td>\n      <td>1988</td>\n    </tr>\n    <tr>\n      <th>4923</th>\n      <td>5018</td>\n      <td>Motorama</td>\n      <td>[Adventure, Comedy, Crime, Drama, Fantasy, Mys...</td>\n      <td>1991</td>\n    </tr>\n    <tr>\n      <th>6793</th>\n      <td>6902</td>\n      <td>Interstate 60</td>\n      <td>[Adventure, Comedy, Drama, Fantasy, Mystery, S...</td>\n      <td>2002</td>\n    </tr>\n    <tr>\n      <th>8605</th>\n      <td>26093</td>\n      <td>Wonderful World of the Brothers Grimm, The</td>\n      <td>[Adventure, Animation, Children, Comedy, Drama...</td>\n      <td>1962</td>\n    </tr>\n    <tr>\n      <th>8783</th>\n      <td>26340</td>\n      <td>Twelve Tasks of Asterix, The (Les douze travau...</td>\n      <td>[Action, Adventure, Animation, Children, Comed...</td>\n      <td>1976</td>\n    </tr>\n    <tr>\n      <th>9296</th>\n      <td>27344</td>\n      <td>Revolutionary Girl Utena: Adolescence of Utena...</td>\n      <td>[Action, Adventure, Animation, Comedy, Drama, ...</td>\n      <td>1999</td>\n    </tr>\n    <tr>\n      <th>9825</th>\n      <td>32031</td>\n      <td>Robots</td>\n      <td>[Adventure, Animation, Children, Comedy, Fanta...</td>\n      <td>2005</td>\n    </tr>\n    <tr>\n      <th>11716</th>\n      <td>51632</td>\n      <td>Atlantis: Milo's Return</td>\n      <td>[Action, Adventure, Animation, Children, Comed...</td>\n      <td>2003</td>\n    </tr>\n    <tr>\n      <th>11751</th>\n      <td>51939</td>\n      <td>TMNT (Teenage Mutant Ninja Turtles)</td>\n      <td>[Action, Adventure, Animation, Children, Comed...</td>\n      <td>2007</td>\n    </tr>\n    <tr>\n      <th>13250</th>\n      <td>64645</td>\n      <td>The Wrecking Crew</td>\n      <td>[Action, Adventure, Comedy, Crime, Drama, Thri...</td>\n      <td>1968</td>\n    </tr>\n    <tr>\n      <th>16055</th>\n      <td>81132</td>\n      <td>Rubber</td>\n      <td>[Action, Adventure, Comedy, Crime, Drama, Film...</td>\n      <td>2010</td>\n    </tr>\n    <tr>\n      <th>18312</th>\n      <td>91335</td>\n      <td>Gruffalo, The</td>\n      <td>[Adventure, Animation, Children, Comedy, Drama]</td>\n      <td>2009</td>\n    </tr>\n    <tr>\n      <th>22778</th>\n      <td>108540</td>\n      <td>Ernest &amp; C\u00e9lestine (Ernest et C\u00e9lestine)</td>\n      <td>[Adventure, Animation, Children, Comedy, Drama...</td>\n      <td>2012</td>\n    </tr>\n    <tr>\n      <th>22881</th>\n      <td>108932</td>\n      <td>The Lego Movie</td>\n      <td>[Action, Adventure, Animation, Children, Comed...</td>\n      <td>2014</td>\n    </tr>\n    <tr>\n      <th>25218</th>\n      <td>117646</td>\n      <td>Dragonheart 2: A New Beginning</td>\n      <td>[Action, Adventure, Comedy, Drama, Fantasy, Th...</td>\n      <td>2000</td>\n    </tr>\n    <tr>\n      <th>26442</th>\n      <td>122787</td>\n      <td>The 39 Steps</td>\n      <td>[Action, Adventure, Comedy, Crime, Drama, Thri...</td>\n      <td>1959</td>\n    </tr>\n    <tr>\n      <th>32854</th>\n      <td>146305</td>\n      <td>Princes and Princesses</td>\n      <td>[Animation, Children, Comedy, Drama, Fantasy, ...</td>\n      <td>2000</td>\n    </tr>\n    <tr>\n      <th>33509</th>\n      <td>148775</td>\n      <td>Wizards of Waverly Place: The Movie</td>\n      <td>[Adventure, Children, Comedy, Drama, Fantasy, ...</td>\n      <td>2009</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "### Content-based technique tries to figure out what a user's favorite aspects of an item is, and then recommends item that present those aspects."
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<hr>\n\n<a id=\"ref3\"></a>\n\n# Collaborative Filtering"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": " **Collaborative Filtering**, which is also known as **User-User Filtering**.This technique uses other users to recommend items to the input user. It attempts to find users that have similar preferences and opinions as the input and then recommends items that they have liked to the input. There are several methods of finding similar users,I will be using here is going to be based on the **Pearson Correlation Function**.\n\n<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-ML0101EN-SkillsNetwork/labs/Module%205/images/User_Item.png\" width=800px>\n\nThe process for creating a User Based recommendation system is as follows:\n\n-   Select a user with the movies the user has watched\n-   Based on his rating to movies, find the top X neighbours \n-   Get the watched movie record of the user for each neighbour.\n-   Calculate a similarity score using some formula\n-   Recommend the items with the highest score\n\n### Advantages and Disadvantages of Collaborative Filtering\n\n##### Advantages\n\n-   Takes other user's ratings into consideration\n-   Doesn't need to study or extract information from the recommended item\n-   Adapts to the user's interests which might change over time\n\n##### Disadvantages\n\n-   Approximation function can be slow\n-   There might be a low of amount of users to approximate\n-   Privacy issues when trying to learn the user's preferences\n"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# creating an input user to recommend movies\nuserInput = [\n            {'title':'Breakfast Club, The', 'rating':5},\n            {'title':'Toy Story', 'rating':3.5},\n            {'title':'Jumanji', 'rating':2},\n            {'title':\"Pulp Fiction\", 'rating':5},\n            {'title':'Akira', 'rating':4.5}\n         ] \ninputMovies = pd.DataFrame(userInput)\ninputMovies",
            "execution_count": 23,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 23,
                    "data": {
                        "text/plain": "                 title  rating\n0  Breakfast Club, The     5.0\n1            Toy Story     3.5\n2              Jumanji     2.0\n3         Pulp Fiction     5.0\n4                Akira     4.5",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>title</th>\n      <th>rating</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>Breakfast Club, The</td>\n      <td>5.0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>Toy Story</td>\n      <td>3.5</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>Jumanji</td>\n      <td>2.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>Pulp Fiction</td>\n      <td>5.0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>Akira</td>\n      <td>4.5</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "#### Add movieId to input user\n\nWith the input, extract the input movies's ID's from the movies dataframe and add them into it.\n"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# filtering out the rows that contain the input movies' title \ninputId = movies_df[movies_df['title'].isin(inputMovies['title'].tolist())]\ninputId",
            "execution_count": 24,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 24,
                    "data": {
                        "text/plain": "      movieId                title  \\\n0           1            Toy Story   \n1           2              Jumanji   \n293       296         Pulp Fiction   \n1246     1274                Akira   \n1885     1968  Breakfast Club, The   \n\n                                                 genres  year  \n0     [Adventure, Animation, Children, Comedy, Fantasy]  1995  \n1                        [Adventure, Children, Fantasy]  1995  \n293                    [Comedy, Crime, Drama, Thriller]  1994  \n1246             [Action, Adventure, Animation, Sci-Fi]  1988  \n1885                                    [Comedy, Drama]  1985  ",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>movieId</th>\n      <th>title</th>\n      <th>genres</th>\n      <th>year</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>Toy Story</td>\n      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n      <td>1995</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>Jumanji</td>\n      <td>[Adventure, Children, Fantasy]</td>\n      <td>1995</td>\n    </tr>\n    <tr>\n      <th>293</th>\n      <td>296</td>\n      <td>Pulp Fiction</td>\n      <td>[Comedy, Crime, Drama, Thriller]</td>\n      <td>1994</td>\n    </tr>\n    <tr>\n      <th>1246</th>\n      <td>1274</td>\n      <td>Akira</td>\n      <td>[Action, Adventure, Animation, Sci-Fi]</td>\n      <td>1988</td>\n    </tr>\n    <tr>\n      <th>1885</th>\n      <td>1968</td>\n      <td>Breakfast Club, The</td>\n      <td>[Comedy, Drama]</td>\n      <td>1985</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# merg this subset with the input dataframe by title\ninputMovies = pd.merge(inputId, inputMovies)\ninputMovies",
            "execution_count": 25,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 25,
                    "data": {
                        "text/plain": "   movieId                title  \\\n0        1            Toy Story   \n1        2              Jumanji   \n2      296         Pulp Fiction   \n3     1274                Akira   \n4     1968  Breakfast Club, The   \n\n                                              genres  year  rating  \n0  [Adventure, Animation, Children, Comedy, Fantasy]  1995     3.5  \n1                     [Adventure, Children, Fantasy]  1995     2.0  \n2                   [Comedy, Crime, Drama, Thriller]  1994     5.0  \n3             [Action, Adventure, Animation, Sci-Fi]  1988     4.5  \n4                                    [Comedy, Drama]  1985     5.0  ",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>movieId</th>\n      <th>title</th>\n      <th>genres</th>\n      <th>year</th>\n      <th>rating</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>Toy Story</td>\n      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n      <td>1995</td>\n      <td>3.5</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>Jumanji</td>\n      <td>[Adventure, Children, Fantasy]</td>\n      <td>1995</td>\n      <td>2.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>296</td>\n      <td>Pulp Fiction</td>\n      <td>[Comedy, Crime, Drama, Thriller]</td>\n      <td>1994</td>\n      <td>5.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1274</td>\n      <td>Akira</td>\n      <td>[Action, Adventure, Animation, Sci-Fi]</td>\n      <td>1988</td>\n      <td>4.5</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1968</td>\n      <td>Breakfast Club, The</td>\n      <td>[Comedy, Drama]</td>\n      <td>1985</td>\n      <td>5.0</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Dropping information we won't use from the input dataframe\ninputMovies = inputMovies.drop('year', 1)\ninputMovies",
            "execution_count": 26,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 26,
                    "data": {
                        "text/plain": "   movieId                title  \\\n0        1            Toy Story   \n1        2              Jumanji   \n2      296         Pulp Fiction   \n3     1274                Akira   \n4     1968  Breakfast Club, The   \n\n                                              genres  rating  \n0  [Adventure, Animation, Children, Comedy, Fantasy]     3.5  \n1                     [Adventure, Children, Fantasy]     2.0  \n2                   [Comedy, Crime, Drama, Thriller]     5.0  \n3             [Action, Adventure, Animation, Sci-Fi]     4.5  \n4                                    [Comedy, Drama]     5.0  ",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>movieId</th>\n      <th>title</th>\n      <th>genres</th>\n      <th>rating</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>Toy Story</td>\n      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n      <td>3.5</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>Jumanji</td>\n      <td>[Adventure, Children, Fantasy]</td>\n      <td>2.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>296</td>\n      <td>Pulp Fiction</td>\n      <td>[Comedy, Crime, Drama, Thriller]</td>\n      <td>5.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1274</td>\n      <td>Akira</td>\n      <td>[Action, Adventure, Animation, Sci-Fi]</td>\n      <td>4.5</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1968</td>\n      <td>Breakfast Club, The</td>\n      <td>[Comedy, Drama]</td>\n      <td>5.0</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "#### The users who has seen the same movies\n\nWith the movie ID's in the input, we can now get the subset of users that have watched and reviewed the movies in our input."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Filtering out users that have watched movies that the input has watched and storing it\nuserSubset = ratings_df[ratings_df['movieId'].isin(inputMovies['movieId'].tolist())]\nuserSubset.head()",
            "execution_count": 27,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 27,
                    "data": {
                        "text/plain": "     userId  movieId  rating\n19        4      296     4.0\n441      12     1968     3.0\n479      13        2     2.0\n531      13     1274     5.0\n681      14      296     2.0",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>userId</th>\n      <th>movieId</th>\n      <th>rating</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>19</th>\n      <td>4</td>\n      <td>296</td>\n      <td>4.0</td>\n    </tr>\n    <tr>\n      <th>441</th>\n      <td>12</td>\n      <td>1968</td>\n      <td>3.0</td>\n    </tr>\n    <tr>\n      <th>479</th>\n      <td>13</td>\n      <td>2</td>\n      <td>2.0</td>\n    </tr>\n    <tr>\n      <th>531</th>\n      <td>13</td>\n      <td>1274</td>\n      <td>5.0</td>\n    </tr>\n    <tr>\n      <th>681</th>\n      <td>14</td>\n      <td>296</td>\n      <td>2.0</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Groupby the user ID where they all have the same values in the column specified as a parameter\nuserSubsetGroup = userSubset.groupby(['userId'])\nuserSubsetGroup.head()",
            "execution_count": 34,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 34,
                    "data": {
                        "text/plain": "          userId  movieId  rating\n19             4      296     4.0\n441           12     1968     3.0\n479           13        2     2.0\n531           13     1274     5.0\n681           14      296     2.0\n...          ...      ...     ...\n22883679  247738      296     4.0\n22884132  247751        1     4.0\n22884142  247751      296     4.0\n22884164  247751     1274     5.0\n22884355  247753        1     3.0\n\n[196623 rows x 3 columns]",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>userId</th>\n      <th>movieId</th>\n      <th>rating</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>19</th>\n      <td>4</td>\n      <td>296</td>\n      <td>4.0</td>\n    </tr>\n    <tr>\n      <th>441</th>\n      <td>12</td>\n      <td>1968</td>\n      <td>3.0</td>\n    </tr>\n    <tr>\n      <th>479</th>\n      <td>13</td>\n      <td>2</td>\n      <td>2.0</td>\n    </tr>\n    <tr>\n      <th>531</th>\n      <td>13</td>\n      <td>1274</td>\n      <td>5.0</td>\n    </tr>\n    <tr>\n      <th>681</th>\n      <td>14</td>\n      <td>296</td>\n      <td>2.0</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>22883679</th>\n      <td>247738</td>\n      <td>296</td>\n      <td>4.0</td>\n    </tr>\n    <tr>\n      <th>22884132</th>\n      <td>247751</td>\n      <td>1</td>\n      <td>4.0</td>\n    </tr>\n    <tr>\n      <th>22884142</th>\n      <td>247751</td>\n      <td>296</td>\n      <td>4.0</td>\n    </tr>\n    <tr>\n      <th>22884164</th>\n      <td>247751</td>\n      <td>1274</td>\n      <td>5.0</td>\n    </tr>\n    <tr>\n      <th>22884355</th>\n      <td>247753</td>\n      <td>1</td>\n      <td>3.0</td>\n    </tr>\n  </tbody>\n</table>\n<p>196623 rows \u00d7 3 columns</p>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# look at the user with userID=75\nuserSubsetGroup.get_group(75)",
            "execution_count": 35,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 35,
                    "data": {
                        "text/plain": "      userId  movieId  rating\n7507      75        1     5.0\n7508      75        2     3.5\n7540      75      296     5.0\n7633      75     1274     4.5\n7673      75     1968     5.0",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>userId</th>\n      <th>movieId</th>\n      <th>rating</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>7507</th>\n      <td>75</td>\n      <td>1</td>\n      <td>5.0</td>\n    </tr>\n    <tr>\n      <th>7508</th>\n      <td>75</td>\n      <td>2</td>\n      <td>3.5</td>\n    </tr>\n    <tr>\n      <th>7540</th>\n      <td>75</td>\n      <td>296</td>\n      <td>5.0</td>\n    </tr>\n    <tr>\n      <th>7633</th>\n      <td>75</td>\n      <td>1274</td>\n      <td>4.5</td>\n    </tr>\n    <tr>\n      <th>7673</th>\n      <td>75</td>\n      <td>1968</td>\n      <td>5.0</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# Sorting it, so user with movie most in common with the input will have priority\nuserSubsetGroup = sorted(userSubsetGroup,  key=lambda x: len(x[1]), reverse=True)\n\n# check the first users\nuserSubsetGroup[0:3]",
            "execution_count": 36,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 36,
                    "data": {
                        "text/plain": "[(75,\n        userId  movieId  rating\n  7507      75        1     5.0\n  7508      75        2     3.5\n  7540      75      296     5.0\n  7633      75     1274     4.5\n  7673      75     1968     5.0),\n (106,\n        userId  movieId  rating\n  9083     106        1     2.5\n  9084     106        2     3.0\n  9115     106      296     3.5\n  9198     106     1274     3.0\n  9238     106     1968     3.5),\n (686,\n         userId  movieId  rating\n  61336     686        1     4.0\n  61337     686        2     3.0\n  61377     686      296     4.0\n  61478     686     1274     4.0\n  61569     686     1968     5.0)]"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "#### Similarity of users to input user\n\n**Pearson Correlation Coefficient**. It is used to measure the strength of a linear association between two variables. The formula for finding this coefficient between sets X and Y with N values can be seen in the image below. \n\nWhy Pearson Correlation?\n\nPearson correlation is invariant to scaling, i.e. multiplying all elements by a nonzero constant or adding any constant to all elements. For example, if you have two vectors X and Y,then, pearson(X, Y) == pearson(X, 2 \\* Y + 3). This is a pretty important property in recommendation systems because for example two users might rate two series of items totally different in terms of absolute rates, but they would be similar users (i.e. with similar ideas) with similar rates in various scales .\n\n![alt text](https://wikimedia.org/api/rest_v1/media/math/render/svg/bd1ccc2979b0fd1c1aec96e386f686ae874f9ec0 \"Pearson Correlation\")\n\nThe values given by the formula vary from r = -1 to r = 1, where 1 forms a direct correlation between the two entities (it means a perfect positive correlation) and -1 forms a perfect negative correlation. \n\n#### a 1 means that the two users have similar tastes while a -1 means the opposite."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# select subSet of 50 users to iterate through\nuserSubsetGroup = userSubsetGroup[0:50]\n\n# create and store the Pearson Correlation in a dicitnery\npearsonCorrelationDict = {}\n\n#For every user group in our subset\nfor name, group in userSubsetGroup:\n    #Let's start by sorting the input and current user group so the values aren't mixed up later on\n    group = group.sort_values(by='movieId')\n    inputMovies = inputMovies.sort_values(by='movieId')\n    #Get the N for the formula\n    nRatings = len(group)\n    #Get the review scores for the movies that they both have in common\n    temp_df = inputMovies[inputMovies['movieId'].isin(group['movieId'].tolist())]\n    #And then store them in a temporary buffer variable in a list format to facilitate future calculations\n    tempRatingList = temp_df['rating'].tolist()\n    #Let's also put the current user group reviews in a list format\n    tempGroupList = group['rating'].tolist()\n    #Now let's calculate the pearson correlation between two users, so called, x and y\n    Sxx = sum([i**2 for i in tempRatingList]) - pow(sum(tempRatingList),2)/float(nRatings)\n    Syy = sum([i**2 for i in tempGroupList]) - pow(sum(tempGroupList),2)/float(nRatings)\n    Sxy = sum( i*j for i, j in zip(tempRatingList, tempGroupList)) - sum(tempRatingList)*sum(tempGroupList)/float(nRatings)\n    \n    #If the denominator is different than zero, then divide, else, 0 correlation.\n    if Sxx != 0 and Syy != 0:\n        pearsonCorrelationDict[name] = Sxy/sqrt(Sxx*Syy)\n    else:\n        pearsonCorrelationDict[name] = 0\n",
            "execution_count": 40,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "pearsonCorrelationDict.items()",
            "execution_count": 41,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 41,
                    "data": {
                        "text/plain": "dict_items([(75, 0.8272781516947562), (106, 0.5860090386731182), (686, 0.8320502943378437), (815, 0.5765566601970551), (1040, 0.9434563530497265), (1130, 0.2891574659831201), (1502, 0.8770580193070299), (1599, 0.4385290096535153), (1625, 0.716114874039432), (1950, 0.179028718509858), (2065, 0.4385290096535153), (2128, 0.5860090386731196), (2432, 0.1386750490563073), (2791, 0.8770580193070299), (2839, 0.8204126541423674), (2948, -0.11720180773462392), (3025, 0.45124262819713973), (3040, 0.89514359254929), (3186, 0.6784622064861935), (3271, 0.26989594817970664), (3429, 0.0), (3734, -0.15041420939904673), (4099, 0.05860090386731196), (4208, 0.29417420270727607), (4282, -0.4385290096535115), (4292, 0.6564386345361464), (4415, -0.11183835382312353), (4586, -0.9024852563942795), (4725, -0.08006407690254357), (4818, 0.4885967564883424), (5104, 0.7674257668936507), (5165, -0.4385290096535153), (5547, 0.17200522903844556), (6082, -0.04728779924109591), (6207, 0.9615384615384616), (6366, 0.6577935144802716), (6482, 0.0), (6530, -0.3516054232038709), (7235, 0.6981407669689391), (7403, 0.11720180773462363), (7641, 0.7161148740394331), (7996, 0.626600514784504), (8008, -0.22562131409856986), (8086, 0.6933752452815365), (8245, 0.0), (8572, 0.8600261451922278), (8675, 0.5370861555295773), (9101, -0.08600261451922278), (9358, 0.692178738358485), (9663, 0.193972725041952)])"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "pearsonDF = pd.DataFrame.from_dict(pearsonCorrelationDict, orient='index')\npearsonDF.columns = ['similarityIndex']\npearsonDF['userId'] = pearsonDF.index\npearsonDF.index = range(len(pearsonDF))\npearsonDF.head()",
            "execution_count": 42,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 42,
                    "data": {
                        "text/plain": "   similarityIndex  userId\n0         0.827278      75\n1         0.586009     106\n2         0.832050     686\n3         0.576557     815\n4         0.943456    1040",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>similarityIndex</th>\n      <th>userId</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0.827278</td>\n      <td>75</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0.586009</td>\n      <td>106</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0.832050</td>\n      <td>686</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0.576557</td>\n      <td>815</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0.943456</td>\n      <td>1040</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# get the top 10 users that are most similar to the input\ntopUsers=pearsonDF.sort_values(by='similarityIndex', ascending=False)[0:10]\ntopUsers.head()",
            "execution_count": 43,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 43,
                    "data": {
                        "text/plain": "    similarityIndex  userId\n34         0.961538    6207\n4          0.943456    1040\n17         0.895144    3040\n6          0.877058    1502\n13         0.877058    2791",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>similarityIndex</th>\n      <th>userId</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>34</th>\n      <td>0.961538</td>\n      <td>6207</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0.943456</td>\n      <td>1040</td>\n    </tr>\n    <tr>\n      <th>17</th>\n      <td>0.895144</td>\n      <td>3040</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>0.877058</td>\n      <td>1502</td>\n    </tr>\n    <tr>\n      <th>13</th>\n      <td>0.877058</td>\n      <td>2791</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "#### Rating of selected users to all movies\n\nby taking the weighted average of the ratings of the movies using the Pearson Correlation as the weight, we first need to get the movies watched by the users in our **pearsonDF** from the ratings dataframe and then store their correlation in a new column called \\_similarityIndex\". \n"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# Merge these two tables\ntopUsersRating=topUsers.merge(ratings_df, left_on='userId', right_on='userId', how='inner')\ntopUsersRating.head()",
            "execution_count": 44,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 44,
                    "data": {
                        "text/plain": "   similarityIndex  userId  movieId  rating\n0         0.961538    6207        1     3.5\n1         0.961538    6207        2     2.0\n2         0.961538    6207        5     1.5\n3         0.961538    6207       10     2.5\n4         0.961538    6207       16     3.5",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>similarityIndex</th>\n      <th>userId</th>\n      <th>movieId</th>\n      <th>rating</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0.961538</td>\n      <td>6207</td>\n      <td>1</td>\n      <td>3.5</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0.961538</td>\n      <td>6207</td>\n      <td>2</td>\n      <td>2.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0.961538</td>\n      <td>6207</td>\n      <td>5</td>\n      <td>1.5</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0.961538</td>\n      <td>6207</td>\n      <td>10</td>\n      <td>2.5</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0.961538</td>\n      <td>6207</td>\n      <td>16</td>\n      <td>3.5</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Multiplies the similarity by the user's ratings\ntopUsersRating['weightedRating'] = topUsersRating['similarityIndex']*topUsersRating['rating']\ntopUsersRating.head()",
            "execution_count": 45,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 45,
                    "data": {
                        "text/plain": "   similarityIndex  userId  movieId  rating  weightedRating\n0         0.961538    6207        1     3.5        3.365385\n1         0.961538    6207        2     2.0        1.923077\n2         0.961538    6207        5     1.5        1.442308\n3         0.961538    6207       10     2.5        2.403846\n4         0.961538    6207       16     3.5        3.365385",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>similarityIndex</th>\n      <th>userId</th>\n      <th>movieId</th>\n      <th>rating</th>\n      <th>weightedRating</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0.961538</td>\n      <td>6207</td>\n      <td>1</td>\n      <td>3.5</td>\n      <td>3.365385</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0.961538</td>\n      <td>6207</td>\n      <td>2</td>\n      <td>2.0</td>\n      <td>1.923077</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0.961538</td>\n      <td>6207</td>\n      <td>5</td>\n      <td>1.5</td>\n      <td>1.442308</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0.961538</td>\n      <td>6207</td>\n      <td>10</td>\n      <td>2.5</td>\n      <td>2.403846</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0.961538</td>\n      <td>6207</td>\n      <td>16</td>\n      <td>3.5</td>\n      <td>3.365385</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Applies a sum to the topUsers after grouping it up by userId\ntempTopUsersRating = topUsersRating.groupby('movieId').sum()[['similarityIndex','weightedRating']]\ntempTopUsersRating.columns = ['sum_similarityIndex','sum_weightedRating']\ntempTopUsersRating.head()",
            "execution_count": 46,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 46,
                    "data": {
                        "text/plain": "         sum_similarityIndex  sum_weightedRating\nmovieId                                         \n1                   8.661447           32.952520\n2                   8.661447           23.866380\n3                   1.587838            4.869489\n5                   1.728964            2.209733\n6                   2.459502            5.469594",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>sum_similarityIndex</th>\n      <th>sum_weightedRating</th>\n    </tr>\n    <tr>\n      <th>movieId</th>\n      <th></th>\n      <th></th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>1</th>\n      <td>8.661447</td>\n      <td>32.952520</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>8.661447</td>\n      <td>23.866380</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1.587838</td>\n      <td>4.869489</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>1.728964</td>\n      <td>2.209733</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>2.459502</td>\n      <td>5.469594</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Creates an empty dataframe\nrecommendation_df = pd.DataFrame()\n#the weighted average\nrecommendation_df['weighted average recommendation score'] = tempTopUsersRating['sum_weightedRating']/tempTopUsersRating['sum_similarityIndex']\nrecommendation_df['movieId'] = tempTopUsersRating.index\nrecommendation_df.head()",
            "execution_count": 47,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 47,
                    "data": {
                        "text/plain": "         weighted average recommendation score  movieId\nmovieId                                                \n1                                     3.804505        1\n2                                     2.755472        2\n3                                     3.066741        3\n5                                     1.278068        5\n6                                     2.223862        6",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>weighted average recommendation score</th>\n      <th>movieId</th>\n    </tr>\n    <tr>\n      <th>movieId</th>\n      <th></th>\n      <th></th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>1</th>\n      <td>3.804505</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>2.755472</td>\n      <td>2</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>3.066741</td>\n      <td>3</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>1.278068</td>\n      <td>5</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>2.223862</td>\n      <td>6</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Sort it and see the top 10 movies recommended\nrecommendation_df = recommendation_df.sort_values(by='weighted average recommendation score', ascending=False)\nrecommendation_df.head(10)",
            "execution_count": 48,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 48,
                    "data": {
                        "text/plain": "         weighted average recommendation score  movieId\nmovieId                                                \n5135                                       5.0     5135\n98981                                      5.0    98981\n6440                                       5.0     6440\n1211                                       5.0     1211\n5072                                       5.0     5072\n73587                                      5.0    73587\n6807                                       5.0     6807\n1834                                       5.0     1834\n2439                                       5.0     2439\n7042                                       5.0     7042",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>weighted average recommendation score</th>\n      <th>movieId</th>\n    </tr>\n    <tr>\n      <th>movieId</th>\n      <th></th>\n      <th></th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>5135</th>\n      <td>5.0</td>\n      <td>5135</td>\n    </tr>\n    <tr>\n      <th>98981</th>\n      <td>5.0</td>\n      <td>98981</td>\n    </tr>\n    <tr>\n      <th>6440</th>\n      <td>5.0</td>\n      <td>6440</td>\n    </tr>\n    <tr>\n      <th>1211</th>\n      <td>5.0</td>\n      <td>1211</td>\n    </tr>\n    <tr>\n      <th>5072</th>\n      <td>5.0</td>\n      <td>5072</td>\n    </tr>\n    <tr>\n      <th>73587</th>\n      <td>5.0</td>\n      <td>73587</td>\n    </tr>\n    <tr>\n      <th>6807</th>\n      <td>5.0</td>\n      <td>6807</td>\n    </tr>\n    <tr>\n      <th>1834</th>\n      <td>5.0</td>\n      <td>1834</td>\n    </tr>\n    <tr>\n      <th>2439</th>\n      <td>5.0</td>\n      <td>2439</td>\n    </tr>\n    <tr>\n      <th>7042</th>\n      <td>5.0</td>\n      <td>7042</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "movies_df.loc[movies_df['movieId'].isin(recommendation_df.head(10)['movieId'].tolist())]",
            "execution_count": 49,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 49,
                    "data": {
                        "text/plain": "       movieId                                      title  \\\n1185      1211  Wings of Desire (Himmel \u00fcber Berlin, Der)   \n1753      1834                      Spanish Prisoner, The   \n2355      2439                                 Affliction   \n4977      5072                                 Metropolis   \n5040      5135                            Monsoon Wedding   \n6332      6440                                Barton Fink   \n6698      6807         Monty Python's The Meaning of Life   \n6931      7042                 Betty Blue (37\u00b02 le matin)   \n14750    73587                               Soul Kitchen   \n20149    98981                    Arrival of a Train, The   \n\n                                  genres  year  \n1185           [Drama, Fantasy, Romance]  1987  \n1753   [Crime, Drama, Mystery, Thriller]  1997  \n2355                             [Drama]  1997  \n4977                 [Animation, Sci-Fi]  2001  \n5040                   [Comedy, Romance]  2001  \n6332                   [Drama, Thriller]  1991  \n6698                            [Comedy]  1983  \n6931                    [Drama, Romance]  1986  \n14750                           [Comedy]  2009  \n20149                      [Documentary]  1896  ",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>movieId</th>\n      <th>title</th>\n      <th>genres</th>\n      <th>year</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>1185</th>\n      <td>1211</td>\n      <td>Wings of Desire (Himmel \u00fcber Berlin, Der)</td>\n      <td>[Drama, Fantasy, Romance]</td>\n      <td>1987</td>\n    </tr>\n    <tr>\n      <th>1753</th>\n      <td>1834</td>\n      <td>Spanish Prisoner, The</td>\n      <td>[Crime, Drama, Mystery, Thriller]</td>\n      <td>1997</td>\n    </tr>\n    <tr>\n      <th>2355</th>\n      <td>2439</td>\n      <td>Affliction</td>\n      <td>[Drama]</td>\n      <td>1997</td>\n    </tr>\n    <tr>\n      <th>4977</th>\n      <td>5072</td>\n      <td>Metropolis</td>\n      <td>[Animation, Sci-Fi]</td>\n      <td>2001</td>\n    </tr>\n    <tr>\n      <th>5040</th>\n      <td>5135</td>\n      <td>Monsoon Wedding</td>\n      <td>[Comedy, Romance]</td>\n      <td>2001</td>\n    </tr>\n    <tr>\n      <th>6332</th>\n      <td>6440</td>\n      <td>Barton Fink</td>\n      <td>[Drama, Thriller]</td>\n      <td>1991</td>\n    </tr>\n    <tr>\n      <th>6698</th>\n      <td>6807</td>\n      <td>Monty Python's The Meaning of Life</td>\n      <td>[Comedy]</td>\n      <td>1983</td>\n    </tr>\n    <tr>\n      <th>6931</th>\n      <td>7042</td>\n      <td>Betty Blue (37\u00b02 le matin)</td>\n      <td>[Drama, Romance]</td>\n      <td>1986</td>\n    </tr>\n    <tr>\n      <th>14750</th>\n      <td>73587</td>\n      <td>Soul Kitchen</td>\n      <td>[Comedy]</td>\n      <td>2009</td>\n    </tr>\n    <tr>\n      <th>20149</th>\n      <td>98981</td>\n      <td>Arrival of a Train, The</td>\n      <td>[Documentary]</td>\n      <td>1896</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "",
            "execution_count": null,
            "outputs": []
        }
    ],
    "metadata": {
        "kernelspec": {
            "name": "python3",
            "display_name": "Python 3.7",
            "language": "python"
        },
        "language_info": {
            "name": "python",
            "version": "3.7.9",
            "mimetype": "text/x-python",
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "pygments_lexer": "ipython3",
            "nbconvert_exporter": "python",
            "file_extension": ".py"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 1
}