PY0101EN-4-3-Pandas Load Data.ipynb

{"cells":[{"cell_type":"markdown","metadata":{},"source":["<center>\n","    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/IDSNlogo.png\" width=\"300\" alt=\"cognitiveclass.ai logo\"  />\n","</center>\n","\n","# Introduction to Pandas in Python\n","\n","Estimated time needed: **15** minutes\n","\n","## Objectives\n","\n","After completing this lab you will be able to:\n","\n","*   Use Pandas to access and view data\n"]},{"cell_type":"markdown","metadata":{},"source":["<h2>Table of Contents</h2>\n","<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n","    <ul>\n","        <li><a href=\"https://https://data/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01set/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01\">About the Dataset</a></li>\n","        <li><a href=\"https://pandas/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01\">Introduction of <code>Pandas</code></a></li>\n","        <li><a href=\"data\">Viewing Data and Accessing Data</a></li>\n","        <li><a href=\"https://quiz/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01\">Quiz on DataFrame</a></li>\n","    </ul>\n","\n","</div>\n","\n","<hr>\n"]},{"cell_type":"markdown","metadata":{},"source":["<h2 id=\"dataset\">About the Dataset</h2>\n"]},{"cell_type":"markdown","metadata":{},"source":["The table has one row for each album and several columns.\n","\n","<ul>\n","    <li><b>artist</b>: Name of the artist</li>\n","    <li><b>album</b>: Name of the album</li>\n","    <li><b>released_year</b>: Year the album was released</li>\n","    <li><b>length_min_sec</b>: Length of the album (hours,minutes,seconds)</li>\n","    <li><b>genre</b>: Genre of the album</li>\n","    <li><b>music_recording_sales_millions</b>: Music recording sales (millions in USD) on <a href=\"http://www.song-database.com/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01\">[SONG://DATABASE]</a></li>\n","    <li><b>claimed_sales_millions</b>: Album's claimed sales (millions in USD) on <a href=\"http://www.song-database.com/\">[SONG://DATABASE]</a></li>\n","    <li><b>date_released</b>: Date on which the album was released</li>\n","    <li><b>soundtrack</b>: Indicates if the album is the movie soundtrack (Y) or (N)</li>\n","    <li><b>rating_of_friends</b>: Indicates the rating from your friends from 1 to 10</li>\n","</ul>\n","\n","You can see the dataset here:\n","\n","<font size=\"1\">\n","<table font-size:xx-small>\n","  <tr>\n","    <th>Artist</th>\n","    <th>Album</th> \n","    <th>Released</th>\n","    <th>Length</th>\n","    <th>Genre</th> \n","    <th>Music recording sales (millions)</th>\n","    <th>Claimed sales (millions)</th>\n","    <th>Released</th>\n","    <th>Soundtrack</th>\n","    <th>Rating (friends)</th>\n","  </tr>\n","  <tr>\n","    <td>Michael Jackson</td>\n","    <td>Thriller</td> \n","    <td>1982</td>\n","    <td>00:42:19</td>\n","    <td>Pop, rock, R&B</td>\n","    <td>46</td>\n","    <td>65</td>\n","    <td>30-Nov-82</td>\n","    <td></td>\n","    <td>10.0</td>\n","  </tr>\n","  <tr>\n","    <td>AC/DC</td>\n","    <td>Back in Black</td> \n","    <td>1980</td>\n","    <td>00:42:11</td>\n","    <td>Hard rock</td>\n","    <td>26.1</td>\n","    <td>50</td>\n","    <td>25-Jul-80</td>\n","    <td></td>\n","    <td>8.5</td>\n","  </tr>\n","    <tr>\n","    <td>Pink Floyd</td>\n","    <td>The Dark Side of the Moon</td> \n","    <td>1973</td>\n","    <td>00:42:49</td>\n","    <td>Progressive rock</td>\n","    <td>24.2</td>\n","    <td>45</td>\n","    <td>01-Mar-73</td>\n","    <td></td>\n","    <td>9.5</td>\n","  </tr>\n","    <tr>\n","    <td>Whitney Houston</td>\n","    <td>The Bodyguard</td> \n","    <td>1992</td>\n","    <td>00:57:44</td>\n","    <td>Soundtrack/R&B, soul, pop</td>\n","    <td>26.1</td>\n","    <td>50</td>\n","    <td>25-Jul-80</td>\n","    <td>Y</td>\n","    <td>7.0</td>\n","  </tr>\n","    <tr>\n","    <td>Meat Loaf</td>\n","    <td>Bat Out of Hell</td> \n","    <td>1977</td>\n","    <td>00:46:33</td>\n","    <td>Hard rock, progressive rock</td>\n","    <td>20.6</td>\n","    <td>43</td>\n","    <td>21-Oct-77</td>\n","    <td></td>\n","    <td>7.0</td>\n","  </tr>\n","    <tr>\n","    <td>Eagles</td>\n","    <td>Their Greatest Hits (1971-1975)</td> \n","    <td>1976</td>\n","    <td>00:43:08</td>\n","    <td>Rock, soft rock, folk rock</td>\n","    <td>32.2</td>\n","    <td>42</td>\n","    <td>17-Feb-76</td>\n","    <td></td>\n","    <td>9.5</td>\n","  </tr>\n","    <tr>\n","    <td>Bee Gees</td>\n","    <td>Saturday Night Fever</td> \n","    <td>1977</td>\n","    <td>1:15:54</td>\n","    <td>Disco</td>\n","    <td>20.6</td>\n","    <td>40</td>\n","    <td>15-Nov-77</td>\n","    <td>Y</td>\n","    <td>9.0</td>\n","  </tr>\n","    <tr>\n","    <td>Fleetwood Mac</td>\n","    <td>Rumours</td> \n","    <td>1977</td>\n","    <td>00:40:01</td>\n","    <td>Soft rock</td>\n","    <td>27.9</td>\n","    <td>40</td>\n","    <td>04-Feb-77</td>\n","    <td></td>\n","    <td>9.5</td>\n","  </tr>\n","</table></font>\n"]},{"cell_type":"markdown","metadata":{},"source":["<hr>\n"]},{"cell_type":"markdown","metadata":{},"source":["<h2 id=\"pandas\">Introduction of <code>Pandas</code></h2>\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Dependency needed to install file \n","\n","!pip install xlrd"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Import required library\n","\n","import pandas as pd"]},{"cell_type":"markdown","metadata":{},"source":["After the import command, we now have access to a large number of pre-built classes and functions. This assumes the library is installed; in our lab environment all the necessary libraries are installed. One way pandas allows you to work with data is a dataframe. Let's go through the process to go from a comma separated values (<b>.csv</b>) file to a dataframe. This variable <code>csv_path</code> stores the path of the <b>.csv</b>, that is  used as an argument to the <code>read_csv</code> function. The result is stored in the object <code>df</code>, this is a common short form used for a variable referring to a Pandas dataframe.\n"]},{"cell_type":"code","execution_count":null,"metadata":{"scrolled":true},"outputs":[],"source":["# Read data from CSV file\n","\n","csv_path = 'https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/data/TopSellingAlbums.csv'\n","df = pd.read_csv(csv_path)"]},{"cell_type":"markdown","metadata":{},"source":["We can use the method <code>head()</code> to examine the first five rows of a dataframe:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Print first five rows of the dataframe\n","\n","df.head()"]},{"cell_type":"markdown","metadata":{},"source":["We use the path of the excel file and the function <code>read_excel</code>. The result is a data frame as before:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Read data from Excel File and print the first five rows\n","\n","xlsx_path = 'https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/PY0101EN/Chapter%204/Datasets/TopSellingAlbums.xlsx'\n","\n","df = pd.read_excel(xlsx_path)\n","df.head()"]},{"cell_type":"markdown","metadata":{},"source":["We can access the column <b>Length</b> and assign it a new dataframe <b>x</b>:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Access to the column Length\n","\n","x = df[['Length']]\n","x"]},{"cell_type":"markdown","metadata":{},"source":["The process is shown in the figure:\n"]},{"cell_type":"markdown","metadata":{},"source":["<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/images/DataEgOne.png\" width=\"750\" />\n"]},{"cell_type":"markdown","metadata":{},"source":["<hr>\n"]},{"cell_type":"markdown","metadata":{},"source":["<h2 id=\"data\">Viewing Data and Accessing Data</h2>\n"]},{"cell_type":"markdown","metadata":{},"source":["You can also get a column as a series. You can think of a Pandas series as a 1-D dataframe. Just use one bracket:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Get the column as a series\n","\n","x = df['Length']\n","x"]},{"cell_type":"markdown","metadata":{},"source":["You can also get a column as a dataframe. For example, we can assign the column <b>Artist</b>:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Get the column as a dataframe\n","\n","x = df[['Artist']]\n","type(x)"]},{"cell_type":"markdown","metadata":{},"source":["You can do the same thing for multiple columns; we just put the dataframe name, in this case, <code>df</code>, and the name of the multiple column headers enclosed in double brackets. The result is a new dataframe comprised of the specified columns:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Access to multiple columns\n","\n","y = df[['Artist','Length','Genre']]\n","y"]},{"cell_type":"markdown","metadata":{},"source":["The process is shown in the figure:\n"]},{"cell_type":"markdown","metadata":{},"source":["<img src = \"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/images/DataEgTwo.png\" width=\"1100\" />\n"]},{"cell_type":"markdown","metadata":{},"source":["One way to access unique elements is the <code>iloc</code> method, where you can access the 1st row and the 1st column as follows:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Access the value on the first row and the first column\n","\n","df.iloc[0, 0]"]},{"cell_type":"markdown","metadata":{},"source":["You can access the 2nd row and the 1st column as follows:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Access the value on the second row and the first column\n","\n","df.iloc[1,0]"]},{"cell_type":"markdown","metadata":{},"source":["You can access the 1st row and the 3rd column as follows:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Access the value on the first row and the third column\n","\n","df.iloc[0,2]"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Access the value on the second row and the third column\n","df.iloc[1,2]"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":[]},{"cell_type":"markdown","metadata":{},"source":["This is shown in the following image\n"]},{"cell_type":"markdown","metadata":{},"source":["<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/images/2_loc.PNG\" width=\"750\" />\n"]},{"cell_type":"markdown","metadata":{},"source":["You can access the column using the name as well, the following are the same as above:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Access the column using the name\n","\n","df.loc[1, 'Artist']"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Access the column using the name\n","\n","df.loc[1, 'Artist']"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Access the column using the name\n","\n","df.loc[0, 'Released']"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Access the column using the name\n","\n","df.loc[1, 'Released']"]},{"cell_type":"markdown","metadata":{},"source":["<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/images/1_loc.png\" width=\"750\" />\n"]},{"cell_type":"markdown","metadata":{},"source":["You can perform slicing using both the index and the name of the column:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Slicing the dataframe\n","\n","df.iloc[0:2, 0:3]"]},{"cell_type":"markdown","metadata":{},"source":["<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/images/4_loc.png\" width=\"750\" />\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Slicing the dataframe using name\n","\n","df.loc[0:2, 'Artist':'Released']"]},{"cell_type":"markdown","metadata":{},"source":["<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/images/3_loc.png\" width=\"750\" />\n"]},{"cell_type":"markdown","metadata":{},"source":["<hr>\n"]},{"cell_type":"markdown","metadata":{},"source":["<h2 id=\"quiz\">Quiz on DataFrame</h2>\n"]},{"cell_type":"markdown","metadata":{},"source":["Use a variable <code>q</code> to store the column <b>Rating</b> as a dataframe\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Write your code below and press Shift+Enter to execute\n"]},{"cell_type":"markdown","metadata":{},"source":["<details><summary>Click here for the solution</summary>\n","\n","```python\n","q = df[['Rating']]\n","q\n","    \n","```\n","\n","</details>\n"]},{"cell_type":"markdown","metadata":{},"source":["Assign the variable <code>q</code> to the dataframe that is made up of the column <b>Released</b> and <b>Artist</b>:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Write your code below and press Shift+Enter to execute\n"]},{"cell_type":"markdown","metadata":{},"source":["<details><summary>Click here for the solution</summary>\n","\n","```python\n","q = df[['Released', 'Artist']]\n","q\n","    \n","```\n","\n","</details>\n"]},{"cell_type":"markdown","metadata":{},"source":["Access the 2nd row and the 3rd column of <code>df</code>:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Write your code below and press Shift+Enter to execute\n"]},{"cell_type":"markdown","metadata":{},"source":["<details><summary>Click here for the solution</summary>\n","\n","```python\n","df.iloc[1, 2]\n","    \n","```\n","\n","</details>\n"]},{"cell_type":"markdown","metadata":{},"source":["Use the following list to convert the dataframe index <code>df</code> to characters and assign it to <code>df_new</code>; find the element corresponding to the row index <code>a</code> and column  <code>'Artist'</code>. Then select the rows <code>a</code> through <code>d</code> for the column  <code>'Artist'</code>\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["new_index=['a','b','c','d','e','f','g','h']\n","\n"]},{"cell_type":"markdown","metadata":{},"source":["<details><summary>Click here for the solution</summary>\n","\n","```python\n","df_new=df\n","df_new.index=new_index\n","df_new.loc['a', 'Artist']\n","df_new.loc['a':'d', 'Artist']\n","    \n","```\n","\n","</details>\n"]},{"cell_type":"markdown","metadata":{},"source":["<hr>\n","<h2>The last exercise!</h2>\n","<p>Congratulations, you have completed your first lesson and hands-on lab in Python. \n","<hr>\n"]},{"cell_type":"markdown","metadata":{},"source":["## Authors:\n","\n","[Joseph Santarcangelo](https://www.linkedin.com/in/joseph-s-50398b136/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork19487395-2022-01-01)\n","\n","Joseph Santarcangelo has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n","\n","## Change Log\n","\n","| Date (YYYY-MM-DD) | Version | Changed By | Change Description                 |\n","| ----------------- | ------- | ---------- | ---------------------------------- |\n","| 2022-01-10        | 2.1     | Malika     | Removed the readme for GitShare    |\n","| 2020-08-26        | 2.0     | Lavanya    | Moved lab to course repo in GitLab |\n","| 2020-11-24        | 3.0     | Nayef      | Added new images                   |\n","|                   |         |            |                                    |\n","\n","<hr/>\n","\n","## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n"]}],"metadata":{"anaconda-cloud":{},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.6"}},"nbformat":4,"nbformat_minor":2}