1036628816-Pandas.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.4.2"
    },
    "toc": {
      "colors": {
        "hover_highlight": "#DAA520",
        "running_highlight": "#FF0000",
        "selected_highlight": "#FFD700"
      },
      "moveMenuLeft": true,
      "nav_menu": {},
      "navigate_menu": true,
      "number_sections": true,
      "sideBar": true,
      "threshold": 4,
      "toc_cell": false,
      "toc_section_display": "block",
      "toc_window_display": false
    },
    "colab": {
      "name": "Pandas.ipynb",
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jdpulgarine/Evaluacion2020-1/blob/master/1036628816-Pandas.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d8BGX58hxE06",
        "colab_type": "text"
      },
      "source": [
        "# Pandas"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zm9tRsn7xE0_",
        "colab_type": "text"
      },
      "source": [
        "From http://pandas.pydata.org/pandas-docs/stable/\n",
        "\n",
        "pandas is a Python package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data both easy and intuitive. It aims to be the fundamental high-level building block for doing practical, real world data analysis in Python. Additionally, it has the broader goal of becoming the most powerful and flexible open source data analysis / manipulation tool available in any language. It is already well on its way toward this goal.\n",
        "\n",
        "See also:\n",
        "\n",
        "* https://github.com/restrepo/data-analysis\n",
        "  * https://classroom.github.com/g/sSMBdBqN\n",
        "  * https://classroom.github.com/a/PcbQBE7F\n",
        "* https://github.com/restrepo/PythonTipsAndTricks\n",
        "\n",
        "A good and practice book about `Pandas` possibilities is:\n",
        "\n",
        "[__Python for Data Analysis__](https://drive.google.com/open?id=0BxoOXsn2EUNIWExXbVc4SDN0YTQ)<br/>\n",
        "Data Wrangling with Pandas, NumPy, and IPython<br/>\n",
        "_By William McKinney_\n",
        "\n",
        "\n",
        "This other is about aplications based on `Pandas`:\n",
        "![image.png](https://covers.oreillystatic.com/images/0636920030515/cat.gif) [Introduction to Machine Learning with Python](https://drive.google.com/open?id=0BxoOXsn2EUNISGhrdEZ3S29fS3M)<br/>\n",
        "A Guide for Data Scientists\n",
        "By Sarah Guido, Andreas Müller\n",
        "\n",
        "`Pandas` can be used in a similar way to `R`, which is based on similar data structures. `Pandas` also can replace the use of graphical interfaces to access spreadsheets like Excel"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zd4jo7KLxE1C",
        "colab_type": "text"
      },
      "source": [
        "## Standard way to load the module"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K3BSlyLBxE1E",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I7Hbe9JEadOC",
        "colab_type": "text"
      },
      "source": [
        "## Basic structure: DataFrame"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IE_Vzycqajls",
        "colab_type": "text"
      },
      "source": [
        "An flat _spreadsheet_ can be seen in terms of the types of variables of `Python` just as dictionary of lists, where each column of the spreadsheet is a pair key-list of the dictionary \n",
        "\n",
        "|   |  A   |  B   |\n",
        "|---|:----:|:----:|\n",
        "| 1 | even | odd  |\n",
        "| 2 |   0  | 1    |\n",
        "| 3 |   2  | 3    |\n",
        "| 4 |   4  | 5    |\n",
        "| 5 |   6  | 7    |\n",
        "| 6 |   8  | 9    |"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lxEc0jlHauyc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "numbers={\"even\": [0,2,4,6,8],   #  First  key-list\n",
        "         \"odd\" : [1,3,5,7,9] }  #  Second key-list"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HeN8rw1JxE1N",
        "colab_type": "text"
      },
      "source": [
        "## Data structures"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UiL0O2SNcCfG",
        "colab_type": "text"
      },
      "source": [
        "`Pandas` has two new data structures:\n",
        "1. `DataFrame` which are similar to numpy arrays but with some assigned key. For example, for the previous case\n",
        "```python\n",
        "import numpy as np\n",
        "np.array([[0,1],\n",
        "          [2,3],\n",
        "          [4,5],\n",
        "          [6,7],\n",
        "          [8,9] \n",
        "         ])\n",
        "```\n",
        "1. `Series` which are enriched  to dictionaries, as the ones defined for the rows of the previous example: `{'even':0,'odd':1}`.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jaWfZyimxE1O",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "The rows in a two-dimensional `DataFrame` corresponds to `Series` with similar keys, while the columns are also Series with the indices as keys. \n",
        "\n",
        "An example of a  `DataFrame` is a spreadsheet, as the one before."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uRdBR4BXc8WY",
        "colab_type": "text"
      },
      "source": [
        "### `DataFrame`\n",
        "`Pandas` can convert a dictionary of lists, like the `numbers` dictionary before, into a `DataFrame`, which is just an spreadsheet but interpreted at the programming level:\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dC00VECAd_Dr",
        "colab_type": "code",
        "outputId": "320b93a9-0f21-4109-8eb1-60a34dc8da74",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "import pandas as pd\n",
        "pd.set_option('display.max_colwidth',200)\n",
        "df=pd.DataFrame(numbers)\n",
        "df"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>even</th>\n",
              "      <th>odd</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>4</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>6</td>\n",
              "      <td>7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>8</td>\n",
              "      <td>9</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   even  odd\n",
              "0     0    1\n",
              "1     2    3\n",
              "2     4    5\n",
              "3     6    7\n",
              "4     8    9"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zUZ1BcuneYQf",
        "colab_type": "text"
      },
      "source": [
        "It is equivalent to:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nqtL07nJecD6",
        "colab_type": "code",
        "outputId": "7e1ecdbc-beed-4ef1-fe5b-7de4e2e59aa9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "pd.DataFrame.from_dict(numbers)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>even</th>\n",
              "      <th>odd</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>4</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>6</td>\n",
              "      <td>7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>8</td>\n",
              "      <td>9</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   even  odd\n",
              "0     0    1\n",
              "1     2    3\n",
              "2     4    5\n",
              "3     6    7\n",
              "4     8    9"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1ftuPJd5eiu6",
        "colab_type": "text"
      },
      "source": [
        "See below for other possibilities of [creating Pandas DataFrames from lists and dictionaries](https://fisica.udea.edu.co:4443/user/restrepo/notebooks/prog/cursos/data-analysis/Pandas.ipynb#Intialization-from-lists-and-dictionaries)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "myuwII1Ed9PE",
        "colab_type": "text"
      },
      "source": [
        "The main advantage of the `DataFrame`, `df`, is that it can be managed without a graphical interface."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5A0YScKRehta",
        "colab_type": "text"
      },
      "source": [
        "We can check the shape of the `DataFrame`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RV-ivs5Weyzq",
        "colab_type": "code",
        "outputId": "edb11553-e61a-474b-924e-f2fb2f0a4980",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "df.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(5, 2)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZMc0puXbe3nD",
        "colab_type": "text"
      },
      "source": [
        "####  Export DataFrame to other formats\n",
        "* To export to excel:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ioWKY1FAe-rK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df.to_excel('example.xlsx',index=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yIdeiNi_fBPJ",
        "colab_type": "text"
      },
      "source": [
        "__Activity__: Open the resulting spreadsheet in Google Drive, publish it and open from the resulting link with Pandas in the next cell"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZqIbSbLvf0wm",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 266
        },
        "outputId": "d9ea38ca-1205-40c2-eccd-a29cd3bed889"
      },
      "source": [
        "df=pd.read_excel('https://docs.google.com/spreadsheets/d/e/2PACX-1vS9nHd8gYZUXiOCjjBg5bxDGcHJ55pqqG-oIdL0GHziCZ4bA_6sPP-32GZ7Sq8WaQ/pub?output=xlsx')\n",
        "df"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>even</th>\n",
              "      <th>odd</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>4</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>6</td>\n",
              "      <td>7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>8</td>\n",
              "      <td>9</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>10</td>\n",
              "      <td>11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>12</td>\n",
              "      <td>13</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   even  odd\n",
              "0     0    1\n",
              "1     2    3\n",
              "2     4    5\n",
              "3     6    7\n",
              "4     8    9\n",
              "5    10   11\n",
              "6    12   13"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WUiuUq22cP4A",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 161
        },
        "outputId": "64f3f1b9-40a7-4522-fdd8-d844de164a2d"
      },
      "source": [
        "df['even']"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0     0\n",
              "1     2\n",
              "2     4\n",
              "3     6\n",
              "4     8\n",
              "5    10\n",
              "6    12\n",
              "Name: even, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c_bj3IfLxE1Q",
        "colab_type": "text"
      },
      "source": [
        "### `Series`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H2LIeZbCgoL5",
        "colab_type": "text"
      },
      "source": [
        "Each column of the DataFrame is now an augmented dictionary called `Series`, with the indices as the keys of the `Series`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "klrbIsoGxE1R",
        "colab_type": "text"
      },
      "source": [
        "A `Pandas` `Series` object can be just initialized from a `Python` dictionary:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FoTT2jkqgrpR",
        "colab_type": "code",
        "outputId": "6d95befb-04d7-4cd1-8414-322336a9e4d1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "type( df['even'] )"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "pandas.core.series.Series"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Iy8h3XICgz9w",
        "colab_type": "text"
      },
      "source": [
        "The keys are the index of the `DataFrame`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BTIxbKvhg6m2",
        "colab_type": "code",
        "outputId": "c0fb58ab-2969-4e91-c053-a1215f933dcc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "df.even[3]"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "6"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7MIKjfslg7Xx",
        "colab_type": "text"
      },
      "source": [
        "Each row is also a series"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rhiEFGhOg-kx",
        "colab_type": "code",
        "outputId": "85c91413-7e9b-4cbb-ba95-2e97faea3c4b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 71
        }
      },
      "source": [
        "df.loc[0]"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "even    0\n",
              "odd     1\n",
              "Name: 0, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ki2GW4yohKl5",
        "colab_type": "text"
      },
      "source": [
        "with keys: `'even'` and `'odd'`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JPLVdBrAhFCJ",
        "colab_type": "code",
        "outputId": "14bb4eff-9aa7-40d5-a1cf-f9d2736c81af",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "df.loc[0]['even']"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8NuiZ2NIhGjx",
        "colab_type": "text"
      },
      "source": [
        "or attributes `even` and `odd`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KmY9Jrk_hZyx",
        "colab_type": "code",
        "outputId": "2e210676-f6ab-4004-91a5-f28e806f0420",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "df.loc[0].odd"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vOxxMKJshhKJ",
        "colab_type": "text"
      },
      "source": [
        "One specific cell value can be reached with the index and the key:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MexLUmZLhph5",
        "colab_type": "code",
        "outputId": "f3af26b1-7023-4a30-b6bb-d4c54f9c1627",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "df.loc[2,'odd']"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "5"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LrhKw9EQxE1T",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 89
        },
        "outputId": "3992ca18-dc37-4f2e-9367-53c4f4ca5312"
      },
      "source": [
        "s=pd.Series({'Name':'Juan Valdez',\n",
        "             'Nacionality':'Colombia',\n",
        "             'Age':23})\n",
        "s"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Name           Juan Valdez\n",
              "Nacionality       Colombia\n",
              "Age                     23\n",
              "dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TzNtVUV6h36n",
        "colab_type": "text"
      },
      "source": [
        "*Note* that the key name can be used also as an attribute.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ip9ADEo7h5vA",
        "colab_type": "code",
        "outputId": "a4e367e5-a390-435b-8956-a12bd2ba92fc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 126
        }
      },
      "source": [
        "df.odd"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    1\n",
              "1    3\n",
              "2    5\n",
              "3    7\n",
              "4    9\n",
              "Name: odd, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z5JBXWlkh8Z3",
        "colab_type": "text"
      },
      "source": [
        "> The __power__ of Pandas rely in that their main data structures: `DataFrames` and `Series`, are enriched with many useful methods and attributes.\n",
        "\n",
        "### [Official definition of Pandas](http://pandas.pydata.org/pandas-docs/stable/)\n",
        "\n",
        "> Pandas is a Python package providing __fast__, __flexible__, and __expressive__ _data structures_ designed to make working with “relational” or “labeled” data both easy and intuitive. It aims to be the fundamental high-level building block for doing practical, real world data analysis in Python. Additionally, it _has the broader goal of becoming the most powerful and flexible open source data analysis / manipulation tool_ available in any language. It is already well on its way toward this goal.\n",
        "\n",
        "* \"relational\": the list of data is identified with some unique index (like a `SQL` table)\n",
        "* \"labeled\": the list is identified with a key, like the previous `odd` or `even` keys.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sDC5HnPOicTg",
        "colab_type": "text"
      },
      "source": [
        "For example. A double bracket `[[...]]`, can be used to filter data.\n",
        "\n",
        "A row in a two-dimensional `DataFrame` corresponds to `Series` with the same keys of the `DataFrame`, but with single values instead of a list "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UfyqbhjXigZc",
        "colab_type": "code",
        "outputId": "6c7f9bf1-a3ee-483c-e7cf-94b9a7d42af1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 78
        }
      },
      "source": [
        "df.loc[[0]]"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>even</th>\n",
              "      <th>odd</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   even  odd\n",
              "0     0    1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-dANKUEoi78M",
        "colab_type": "code",
        "outputId": "908fe434-920b-4ece-a21c-d2c84a28b053",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 90
        }
      },
      "source": [
        "print( 'the row has' )\n",
        "print( '                  keys: {} and values: {}'.format( list( df.loc[[0]].keys() ),df.loc[[0]].values[0]  ) )\n",
        "print( \"like the dictionay:\" )\n",
        "print( \"                      { 'even' : 0, 'odd' : 1 }\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "the row has\n",
            "                  keys: ['even', 'odd'] and values: [0 1]\n",
            "like the dictionay:\n",
            "                      { 'even' : 0, 'odd' : 1 }\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p7MgQWMijI0W",
        "colab_type": "text"
      },
      "source": [
        "To filter a column:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nT_SkKMIjMWY",
        "colab_type": "code",
        "outputId": "f9f45db6-cf96-4ba2-e454-904d6385bd99",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 198
        }
      },
      "source": [
        "df[['odd']]"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>odd</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>9</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   odd\n",
              "0    1\n",
              "1    3\n",
              "2    5\n",
              "3    7\n",
              "4    9"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ElU5h9tnxE1Y",
        "colab_type": "text"
      },
      "source": [
        "### More on `Series`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pg20NksOjdQW",
        "colab_type": "text"
      },
      "source": [
        "A `Pandas` `Series` object can be just initialized from a `Python` dictionary:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qjuk_nn0jgFH",
        "colab_type": "code",
        "outputId": "e439a5a7-e843-4328-cbb5-c452261a6546",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 89
        }
      },
      "source": [
        "s=pd.Series({'Name':'Juan Valdez','Nacionality':'Colombia','Age':23})\n",
        "s"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Name           Juan Valdez\n",
              "Nacionality       Colombia\n",
              "Age                     23\n",
              "dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pAtixllExE1Z",
        "colab_type": "code",
        "outputId": "e08322c3-81dd-4e1a-9d5a-190c0fbe2c4d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "s['Name']"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'Juan Valdez'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pYb112eqxE1e",
        "colab_type": "text"
      },
      "source": [
        "but also as containers of name spaces!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6_GfsFVwxE1f",
        "colab_type": "code",
        "outputId": "94eba5d9-d28d-41f0-89f2-00344a72128e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "s.Name"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'Juan Valdez'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Kab6pfyMxE1k",
        "colab_type": "text"
      },
      "source": [
        "## `DataFrame` initialization"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4zhJVNq1pTTe",
        "colab_type": "text"
      },
      "source": [
        "### Initization from an existing spreadsheet. \n",
        "This can be locally in your computer o from some downloadable  link"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yM4oF9r-pWsw",
        "colab_type": "code",
        "outputId": "316dcf54-3578-4373-9064-d3d8d19d89b7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 111
        }
      },
      "source": [
        "df=pd.read_excel('http://bit.ly/spreadsheet_xlsx')\n",
        "df"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Nombre</th>\n",
              "      <th>Edad</th>\n",
              "      <th>Compañia</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Juan Valdez</td>\n",
              "      <td>23</td>\n",
              "      <td>Café de Colombia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Álvaro Uribe Vélez</td>\n",
              "      <td>65</td>\n",
              "      <td>Senado de la República</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "               Nombre  Edad                Compañia\n",
              "0         Juan Valdez    23        Café de Colombia\n",
              "1  Álvaro Uribe Vélez    65  Senado de la República"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QM4z4MoXpjLj",
        "colab_type": "text"
      },
      "source": [
        "To make a downloadable link for any spread sheet in Google Drive, follow the sequence:\n",
        "```\n",
        "File → Publish to the web...→ Entire Document → Web page → Microsoft excel (xlsx)\n",
        "```\n",
        "as illustrated in the figure:\n",
        "![GS](https://github.com/restrepo/data-analysis/blob/master/img/img1.png?raw=1)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "29uXQfWjpx66",
        "colab_type": "code",
        "outputId": "7c58a4ca-69b9-4ee2-9187-96aee19ee940",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 108
        }
      },
      "source": [
        "df.loc[0,'Edad']=32\n",
        "df"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Nombre</th>\n",
              "      <th>Edad</th>\n",
              "      <th>Compañia</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Juan Valdez</td>\n",
              "      <td>32</td>\n",
              "      <td>Café de Colombia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Álvaro Uribe Vélez</td>\n",
              "      <td>65</td>\n",
              "      <td>Senado de la República</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "               Nombre  Edad                Compañia\n",
              "0         Juan Valdez    32        Café de Colombia\n",
              "1  Álvaro Uribe Vélez    65  Senado de la República"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eeZzLpDUpqbS",
        "colab_type": "text"
      },
      "source": [
        "*After* some modification"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TRjW0Y4Rpv-i",
        "colab_type": "text"
      },
      "source": [
        "it can be saved again as an `excel file` with the option to not create a column of indices: `index=False`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QZUHeScjp6cC",
        "colab_type": "text"
      },
      "source": [
        "### Intialization from lists and dictionaries"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5ulKgKBRxE1m",
        "colab_type": "text"
      },
      "source": [
        "#### Inizialization from Series\n",
        "We start with an empty `DataFrame`:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J-F92xHmp8v7",
        "colab_type": "text"
      },
      "source": [
        "Creating Pandas DataFrame from list and dictionaries [offers many alternatives](http://pbpython.com/pandas-list-dict.html)\n",
        "\n",
        "![creating dataframes](http://pbpython.com/images/pandas-dataframe-shadow.png)\n",
        "\n",
        "#### Row oriented way\n",
        "* In addition to the dictionary of lists [already illustrated at the beginning]() that in this case corresponds to:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7IOuJCeMqBRj",
        "colab_type": "code",
        "outputId": "4217e6e6-7fbd-4f90-f969-4714383a1850",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 111
        }
      },
      "source": [
        "pd.DataFrame({'Nombre'   : ['Juan Valdez','Álvaro Uribe Vélez'],\n",
        "              'Edad'     : [32,            65                 ],\n",
        "              'Compañia' : ['Café de Colombia','Senado de la República']})"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Nombre</th>\n",
              "      <th>Edad</th>\n",
              "      <th>Compañia</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Juan Valdez</td>\n",
              "      <td>32</td>\n",
              "      <td>Café de Colombia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Álvaro Uribe Vélez</td>\n",
              "      <td>65</td>\n",
              "      <td>Senado de la República</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "               Nombre  Edad                Compañia\n",
              "0         Juan Valdez    32        Café de Colombia\n",
              "1  Álvaro Uribe Vélez    65  Senado de la República"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nJLhj4G4qJZb",
        "colab_type": "text"
      },
      "source": [
        "* We can obtain the DataFrame from list of items"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p755U93VqKLC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "pd.DataFrame.from_items([ [ 'Nombre'  , ['Juan Valdez','Álvaro Uribe Vélez']],\n",
        "                          [ 'Edad'    , [  32,            65               ]],\n",
        "                          [ 'Compañia', ['Café de Colombia','Senado de la República']] ])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J6tp94TlqO_S",
        "colab_type": "text"
      },
      "source": [
        "* We can obtain the `DataFrame` from dictionary"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wx-JJgqIqQAD",
        "colab_type": "code",
        "outputId": "580eeee2-c714-4ede-ebc0-faae7a170333",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 142
        }
      },
      "source": [
        "pd.DataFrame( {'Nombre':'Juan Valdez',        'Edad': 32   ,'Compañia':'Café de Colombia'      },\n",
        "              {'Nombre':'Álvaro Uribe Vélez', 'Edad': 65   ,'Compañia':'Senado de la República'}\n",
        "            )"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Nombre</th>\n",
              "      <th>Edad</th>\n",
              "      <th>Compañia</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>Nombre</th>\n",
              "      <td>Juan Valdez</td>\n",
              "      <td>32</td>\n",
              "      <td>Café de Colombia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Edad</th>\n",
              "      <td>Juan Valdez</td>\n",
              "      <td>32</td>\n",
              "      <td>Café de Colombia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Compañia</th>\n",
              "      <td>Juan Valdez</td>\n",
              "      <td>32</td>\n",
              "      <td>Café de Colombia</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "               Nombre  Edad          Compañia\n",
              "Nombre    Juan Valdez    32  Café de Colombia\n",
              "Edad      Juan Valdez    32  Café de Colombia\n",
              "Compañia  Juan Valdez    32  Café de Colombia"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jPsRspsXxE1n",
        "colab_type": "code",
        "outputId": "cb0aeb13-fa1a-480d-a214-fdb27b8f0228",
        "colab": {}
      },
      "source": [
        "df=pd.DataFrame()\n",
        "df"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "Empty DataFrame\n",
              "Columns: []\n",
              "Index: []"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9p6og3fXfptN",
        "colab_type": "text"
      },
      "source": [
        "Actividad: Crear el DataFrame desde una lista de diccionarios y agregar una tercera fila"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pOVo5mmkf3fp",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 142
        },
        "outputId": "fff5c45b-1c53-4946-8a6f-125908d13a75"
      },
      "source": [
        "df= pd.DataFrame([{'Nombre': 'Juan Valdez', 'Edad' : 32, 'Compañía':'Café de colombia'},\n",
        "                  {'Nombre':'Alvaro Uribe Vélez', 'Edad' :64, 'Compañía':'Senado de la República'},\n",
        "                  {'Nombre':'Margarita Rosa de Francisco', 'Edad' :63, 'Compañía':'Caracol TV'}\n",
        "                  \n",
        "])\n",
        "df"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Nombre</th>\n",
              "      <th>Edad</th>\n",
              "      <th>Compañía</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Juan Valdez</td>\n",
              "      <td>32</td>\n",
              "      <td>Café de colombia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Alvaro Uribe Vélez</td>\n",
              "      <td>64</td>\n",
              "      <td>Senado de la República</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Margarita Rosa de Francisco</td>\n",
              "      <td>63</td>\n",
              "      <td>Caracol TV</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                        Nombre  Edad                Compañía\n",
              "0                  Juan Valdez    32        Café de colombia\n",
              "1           Alvaro Uribe Vélez    64  Senado de la República\n",
              "2  Margarita Rosa de Francisco    63              Caracol TV"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_ylhD2zjqdby",
        "colab_type": "text"
      },
      "source": [
        "### Initialization from sequential rows as  Series\n",
        "We start with an empty `DataFrame`:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "r6V0qilXqxBB",
        "colab_type": "code",
        "outputId": "9d29bcac-132f-4e8c-97df-91f9c2c94f70",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "df=pd.DataFrame()\n",
        "df.empty"
      ],
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f0OJapjvxE1q",
        "colab_type": "text"
      },
      "source": [
        "We can append a dictionary (or Series) as a row of the `DataFrame`, provided that we always use the option: `ignore_index=True`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AgR5uPbxrA6q",
        "colab_type": "code",
        "outputId": "bc6fc916-2087-4a86-a917-d7452573fb8b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 80
        }
      },
      "source": [
        "d={'Name':'Juan Valdez','Nacionality':'Colombia','Age':23}\n",
        "df=df.append(d,ignore_index=True)\n",
        "df"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Age</th>\n",
              "      <th>Nacionality</th>\n",
              "      <th>Name</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>23.0</td>\n",
              "      <td>Colombia</td>\n",
              "      <td>Juan Valdez</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    Age Nacionality         Name\n",
              "0  23.0    Colombia  Juan Valdez"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "32Oh_stDxE1w",
        "colab_type": "text"
      },
      "source": [
        "We can fix the type of data of the `'Age'` column"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-Jk5bSRGxE1y",
        "colab_type": "code",
        "outputId": "60d0ba16-bf2f-4d16-f19d-b1b7d2a71692",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "type(df.loc[0,'Age'])"
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "numpy.float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vl9RKEBXxE12",
        "colab_type": "code",
        "outputId": "c4f5cbb8-10f0-47bf-e540-efc40d3fb369",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 80
        }
      },
      "source": [
        "df['Age']=df.Age.astype(int)\n",
        "df"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Age</th>\n",
              "      <th>Nacionality</th>\n",
              "      <th>Name</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>23</td>\n",
              "      <td>Colombia</td>\n",
              "      <td>Juan Valdez</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Age Nacionality         Name\n",
              "0   23    Colombia  Juan Valdez"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "npH4wLrNxE17",
        "colab_type": "text"
      },
      "source": [
        "To add a second file we build another `dict`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nTwRmub3xE18",
        "colab_type": "code",
        "outputId": "5c5fa816-2ad7-4b31-bd96-a03e49c8c38a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 161
        }
      },
      "source": [
        "d={}\n",
        "for k in ['Name','Nacionality','Age','Company']:\n",
        "    var=input('{}:\\n'.format(k))\n",
        "    d[k]=var"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Name:\n",
            "Francisco\n",
            "Nacionality:\n",
            "Argentina\n",
            "Age:\n",
            "82\n",
            "Company:\n",
            "Vaticano\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t8Xzsb0qxE1-",
        "colab_type": "text"
      },
      "source": [
        "#### Exercises\n",
        "* Display the resulting `Series` in the screen:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "58Cku5HjxE1_",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 107
        },
        "outputId": "c67aacf3-74fd-447a-a5ef-0e742ec0ed64"
      },
      "source": [
        "pd.Series(d)"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Name           Francisco\n",
              "Nacionality    Argentina\n",
              "Age                   82\n",
              "Company         Vaticano\n",
              "dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sSPUdkvtxE2C",
        "colab_type": "text"
      },
      "source": [
        "* Append to the previous `DataFrame` and visualize it:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vRRypFa_xE2D",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 111
        },
        "outputId": "05e9fc08-dd5c-4995-a707-f33070d60b1c"
      },
      "source": [
        "df=df.append(d,ignore_index=True)\n",
        "df"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Age</th>\n",
              "      <th>Nacionality</th>\n",
              "      <th>Name</th>\n",
              "      <th>Company</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>23</td>\n",
              "      <td>Colombia</td>\n",
              "      <td>Juan Valdez</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>82</td>\n",
              "      <td>Argentina</td>\n",
              "      <td>Francisco</td>\n",
              "      <td>Vaticano</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  Age Nacionality         Name   Company\n",
              "0  23    Colombia  Juan Valdez       NaN\n",
              "1  82   Argentina    Francisco  Vaticano"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wLUED3NWxE2G",
        "colab_type": "text"
      },
      "source": [
        "* Fill NaN with empty strings"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "28rp4v3BxE2I",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 111
        },
        "outputId": "ccf18e39-45a0-42e9-a5fd-f853c515a429"
      },
      "source": [
        "df.loc[0,'Company']=\"Café de Colombia\"\n",
        "df"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Age</th>\n",
              "      <th>Nacionality</th>\n",
              "      <th>Name</th>\n",
              "      <th>Company</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>23</td>\n",
              "      <td>Colombia</td>\n",
              "      <td>Juan Valdez</td>\n",
              "      <td>Café de Colombia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>82</td>\n",
              "      <td>Argentina</td>\n",
              "      <td>Francisco</td>\n",
              "      <td>Vaticano</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  Age Nacionality         Name           Company\n",
              "0  23    Colombia  Juan Valdez  Café de Colombia\n",
              "1  82   Argentina    Francisco          Vaticano"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K3feFAfYxE2K",
        "colab_type": "text"
      },
      "source": [
        "* Save `Pandas` `DataFrame` as an Excel file"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fdxuA_FAxE2L",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df.to_excel('trabajadores.xlsx',index=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evWZKwEPxE2O",
        "colab_type": "text"
      },
      "source": [
        "* Load pandas DataFrame from the saved file in Excel"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xl7KUNVCxE2P",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 111
        },
        "outputId": "bd2e05bf-3c00-494a-c79f-cce55873f1c9"
      },
      "source": [
        "pd.read_excel('trabajadores.xlsx')"
      ],
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Age</th>\n",
              "      <th>Nacionality</th>\n",
              "      <th>Name</th>\n",
              "      <th>Company</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>23</td>\n",
              "      <td>Colombia</td>\n",
              "      <td>Juan Valdez</td>\n",
              "      <td>Café de Colombia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>82</td>\n",
              "      <td>Argentina</td>\n",
              "      <td>Francisco</td>\n",
              "      <td>Vaticano</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Age Nacionality         Name           Company\n",
              "0   23    Colombia  Juan Valdez  Café de Colombia\n",
              "1   82   Argentina    Francisco          Vaticano"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pAZISlqKxE2T",
        "colab_type": "text"
      },
      "source": [
        "### Common operations upon `DataFrames`\n",
        "See https://github.com/restrepo/PythonTipsAndTricks"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IBgQ8cojxE2U",
        "colab_type": "text"
      },
      "source": [
        "* __To fill a specific cell__"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a00HZIQhxE2V",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df.loc[0,'Company']='Federación de Caferos'"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g6f2psm3xE2X",
        "colab_type": "code",
        "outputId": "888e8cee-6bfd-4313-86ed-2d7eeead7579",
        "colab": {}
      },
      "source": [
        "df"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Age</th>\n",
              "      <th>Nacionality</th>\n",
              "      <th>Name</th>\n",
              "      <th>Company</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>23.0</td>\n",
              "      <td>Colombia</td>\n",
              "      <td>Juan Valdez</td>\n",
              "      <td>Federación de Caferos</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    Age Nacionality         Name                Company\n",
              "0  23.0    Colombia  Juan Valdez  Federación de Caferos"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rzAKwxQhxE2a",
        "colab_type": "text"
      },
      "source": [
        "## Other formats to saving and read files\n",
        "We are interested in format which keeps the tags of the columns, like `'Nombre', 'Edad', 'Compañia'`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CbgkbfTKvIkO",
        "colab_type": "code",
        "outputId": "431510dc-5cde-4f89-841e-d8f607d6f167",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 108
        }
      },
      "source": [
        "df=pd.read_excel('http://bit.ly/spreadsheet_xlsx')\n",
        "df"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Nombre</th>\n",
              "      <th>Edad</th>\n",
              "      <th>Compañia</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Juan Valdez</td>\n",
              "      <td>23</td>\n",
              "      <td>Café de Colombia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Álvaro Uribe Vélez</td>\n",
              "      <td>65</td>\n",
              "      <td>Senado de la República</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "               Nombre  Edad                Compañia\n",
              "0         Juan Valdez    23        Café de Colombia\n",
              "1  Álvaro Uribe Vélez    65  Senado de la República"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w8hdIGOvu3Ee",
        "colab_type": "text"
      },
      "source": [
        "#### CSV"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xzNVZKNquw3I",
        "colab_type": "text"
      },
      "source": [
        ""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "90PSoWzUxE2b",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df.to_csv('hoja.csv',index=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "55n8K23es3Na",
        "colab_type": "code",
        "outputId": "4998873f-b713-4817-f20b-32d1afa8d2c6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 78
        }
      },
      "source": [
        "df"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Age</th>\n",
              "      <th>Nacionality</th>\n",
              "      <th>Name</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>23</td>\n",
              "      <td>Colombia</td>\n",
              "      <td>Juan Valdez</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Age Nacionality         Name\n",
              "0   23    Colombia  Juan Valdez"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ViJ62k6BxfOm",
        "colab_type": "text"
      },
      "source": [
        "We can check the explicit file format with"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fGmx6K8SxRB1",
        "colab_type": "code",
        "outputId": "3fbcc04e-34bf-4f26-aa06-b41191b58f5e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 89
        }
      },
      "source": [
        "print(df.to_csv(None,index=False))"
      ],
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Age,Nacionality,Name,Company\n",
            "23,Colombia,Juan Valdez,Café de Colombia\n",
            "82,Argentina,Francisco,Vaticano\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s1rbHbLmuv2g",
        "colab_type": "text"
      },
      "source": [
        "#### JSON"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JwWj-SSVv6Aj",
        "colab_type": "text"
      },
      "source": [
        "This format keeps the lists and dictionaries at the storage level"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OaMV4GdOwmBo",
        "colab_type": "code",
        "outputId": "98f54bc6-41fc-4f35-b392-0bad8545f265",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 111
        }
      },
      "source": [
        "df=pd.DataFrame([{\"Name\":\"Donald Trump\",\"Age\":73},{\"Name\":\"Barak Obama\", \"Age\":58}])\n",
        "df"
      ],
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Name</th>\n",
              "      <th>Age</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Donald Trump</td>\n",
              "      <td>73</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Barak Obama</td>\n",
              "      <td>58</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "           Name  Age\n",
              "0  Donald Trump   73\n",
              "1   Barak Obama   58"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ajc79wC9z2b0",
        "colab_type": "text"
      },
      "source": [
        "This format allow us to keep exactly the very same list of dictionaries structure!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "00a9jBWAzg29",
        "colab_type": "code",
        "outputId": "9e60bc42-72ce-4129-e65e-f1826f0a0ae9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "print(df.to_json(None,orient='records'))"
      ],
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[{\"Name\":\"Donald Trump\",\"Age\":73},{\"Name\":\"Barak Obama\",\"Age\":58}]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xwPu1tn30hQ8",
        "colab_type": "text"
      },
      "source": [
        "__Activity__: \n",
        "* Save to a file instead of `None` and open the file with some editor. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KtKmXfQg1Z4z",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df.to_json('ex.json',orient='records')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5S7OR7X11acD",
        "colab_type": "text"
      },
      "source": [
        "* Add a break-line at the end of the first dictionary and try to\n",
        "load the resulting file with `pd.read_json`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xOaIuF_y1b5C",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5vSq9daI0F7R",
        "colab_type": "text"
      },
      "source": [
        "JSON allows for some flexibility in the break-lines structure:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qT8Lnlrgy0M9",
        "colab_type": "code",
        "outputId": "6af77407-b8fb-4525-f4ef-75ee0c7cb463",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 111
        }
      },
      "source": [
        "pd.read_json('''\n",
        "             [{\"Name\":\"Donald Trump\",\"Age\":73},\n",
        "              {\"Name\":\"Barak Obama\", \"Age\":58}]\n",
        "            ''')"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Name</th>\n",
              "      <th>Age</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Donald Trump</td>\n",
              "      <td>73</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Barak Obama</td>\n",
              "      <td>58</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "           Name  Age\n",
              "0  Donald Trump   73\n",
              "1   Barak Obama   58"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w6PyR85f1mfD",
        "colab_type": "text"
      },
      "source": [
        "For large databases it is convinient just to accumulate dictionaries in a sequential form:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "y2f8y66fyC5S",
        "colab_type": "code",
        "outputId": "c45744db-8662-45b1-b833-cb0c0590a06c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        }
      },
      "source": [
        "print(df.to_json(None,orient='records',lines=True))"
      ],
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{\"Name\":\"Donald Trump\",\"Age\":73}\n",
            "{\"Name\":\"Barak Obama\",\"Age\":58}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Bvhzs_GRzNWM",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df=pd.read_json('''\n",
        "             {\"Name\":\"Donald Trump\",\"Age\":73}\n",
        "             {\"Name\":\"Barak Obama\", \"Age\":58}\n",
        "            ''',orient='records',lines=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sYfWnw1e2wfi",
        "colab_type": "text"
      },
      "source": [
        "__Activity__: \n",
        "* Save to a file instead of `None`, with options: `orient='records',lines=True`, and open the file with some editor. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S9wv93dM3KER",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df.to_json('kx4.json',orient='records',lines=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bGYW8Xl-3Kdi",
        "colab_type": "text"
      },
      "source": [
        "* Add a similar dictionary in the next new line, and try to\n",
        "load the resulting file with `pd.read_json` with options: `orient='records',lines=True`. \n",
        "   * WARNING: Use doble-quotes `\"` to write the keys od the new\n",
        "dictionary"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "icGCveT34HDA",
        "colab_type": "text"
      },
      "source": [
        "Any Python string need to be converted first to double-quotes before to be used as JSON string.\n",
        "\n",
        "__Example__"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mPRCSHbG4dRx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "numbers={\"even\": [0,2,4,-6,8],   #  First  key-list\n",
        "         \"odd\" : [1,3,-5,7,9] }  #  Second key-list"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "An-yzTut4gtw",
        "colab_type": "code",
        "outputId": "33cd5dd5-9ae8-499c-c610-079b1efbbd05",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "str(numbers)"
      ],
      "execution_count": 50,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\"{'even': [0, 2, 4, -6, 8], 'odd': [1, 3, -5, 7, 9]}\""
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 50
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BFCE3Hq34ktw",
        "colab_type": "text"
      },
      "source": [
        "This string can be writing in the `JSON` format by replacing the single quotes, ' , by  duoble quotes, \":"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PJVD2Btc4nJx",
        "colab_type": "code",
        "outputId": "74309c2b-905b-4553-fd48-574ad0771dfb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "str(numbers).replace(\"'\",'\"')"
      ],
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'{\"even\": [0, 2, 4, -6, 8], \"odd\": [1, 3, -5, 7, 9]}'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zDssJstU45Qi",
        "colab_type": "text"
      },
      "source": [
        "and now can be used as an JSON input"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HxbaP5T542_h",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "65d98719-5cd2-43df-a2f7-a7ecafa9140a"
      },
      "source": [
        "df=pd.read_json(  str(numbers).replace(\"'\",'\"') )\n",
        "df"
      ],
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>even</th>\n",
              "      <th>odd</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>4</td>\n",
              "      <td>-5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>-6</td>\n",
              "      <td>7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>8</td>\n",
              "      <td>9</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   even  odd\n",
              "0     0    1\n",
              "1     2    3\n",
              "2     4   -5\n",
              "3    -6    7\n",
              "4     8    9"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 53
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EtqUn4t85Hlp",
        "colab_type": "text"
      },
      "source": [
        "__Activity__: Try to read the string as JSON without make the double-quote replacement"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "lg5vwraLoEdO",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 322
        },
        "outputId": "688f8a4f-005e-470d-9dae-3c542bb7f674"
      },
      "source": [
        "df=pd.read_json(str(numbers))"
      ],
      "execution_count": 54,
      "outputs": [
        {
          "output_type": "error",
          "ename": "ValueError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-54-3b7845049c2d>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mdf\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_json\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnumbers\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/pandas/io/json/_json.py\u001b[0m in \u001b[0;36mread_json\u001b[0;34m(path_or_buf, orient, typ, dtype, convert_axes, convert_dates, keep_default_dates, numpy, precise_float, date_unit, encoding, lines, chunksize, compression)\u001b[0m\n\u001b[1;32m    590\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mjson_reader\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    591\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 592\u001b[0;31m     \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mjson_reader\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    593\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mshould_close\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    594\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/pandas/io/json/_json.py\u001b[0m in \u001b[0;36mread\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    715\u001b[0m             \u001b[0mobj\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_object_parser\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_combine_lines\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msplit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"\\n\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    716\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 717\u001b[0;31m             \u001b[0mobj\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_object_parser\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    718\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    719\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mobj\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/pandas/io/json/_json.py\u001b[0m in \u001b[0;36m_get_object_parser\u001b[0;34m(self, json)\u001b[0m\n\u001b[1;32m    737\u001b[0m         \u001b[0mobj\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    738\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mtyp\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m\"frame\"\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 739\u001b[0;31m             \u001b[0mobj\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mFrameParser\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mjson\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mparse\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    740\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    741\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mtyp\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m\"series\"\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0mobj\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/pandas/io/json/_json.py\u001b[0m in \u001b[0;36mparse\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    847\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    848\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 849\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_parse_no_numpy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    850\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    851\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mobj\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/pandas/io/json/_json.py\u001b[0m in \u001b[0;36m_parse_no_numpy\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   1091\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0morient\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m\"columns\"\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1092\u001b[0m             self.obj = DataFrame(\n\u001b[0;32m-> 1093\u001b[0;31m                 \u001b[0mloads\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mjson\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mprecise_float\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprecise_float\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1094\u001b[0m             )\n\u001b[1;32m   1095\u001b[0m         \u001b[0;32melif\u001b[0m \u001b[0morient\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m\"split\"\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mValueError\u001b[0m: Expected object or value"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s2uApSIi8PjI",
        "colab_type": "text"
      },
      "source": [
        "## Filters\n",
        "The main application of labeled data for data analysis is the possibility to make filers, or cuts, to obtain specific reduced datasets to further analysis"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zt2S0aln8_4s",
        "colab_type": "code",
        "outputId": "169e3351-8db1-4a9b-eb6c-3684d1249708",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 111
        }
      },
      "source": [
        "df[df.even.abs()>4]"
      ],
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>even</th>\n",
              "      <th>odd</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>-6</td>\n",
              "      <td>7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>8</td>\n",
              "      <td>9</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   even  odd\n",
              "3    -6    7\n",
              "4     8    9"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eVSePCQB9GFe",
        "colab_type": "code",
        "outputId": "596533f7-6b9e-4b47-b110-5e0aff429561",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 78
        }
      },
      "source": [
        "#and\n",
        "df[(df.even>0) & (df.odd<0)]"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>even</th>\n",
              "      <th>odd</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>4</td>\n",
              "      <td>-5</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   even  odd\n",
              "2     4   -5"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 116
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dVktA6ze9wI2",
        "colab_type": "code",
        "outputId": "62806bab-da66-4945-a717-bc6078ca9731",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 108
        }
      },
      "source": [
        "#or\n",
        "df[(df.even<0) | (df.odd<0)]"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>even</th>\n",
              "      <th>odd</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>4</td>\n",
              "      <td>-5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>-6</td>\n",
              "      <td>7</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   even  odd\n",
              "2     4   -5\n",
              "3    -6    7"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 117
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xHsCPml65i54",
        "colab_type": "text"
      },
      "source": [
        "## The `apply` method\n",
        "The advantage of the spreadsheet paradigm is that the columns can be transformed with functions. All the typical functions avalaible for a spreadsheet are already implemented like the method `.abs()` used before, or the method: `.sum()`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pPLWFFLA6NX4",
        "colab_type": "code",
        "outputId": "c38db211-777b-41d2-8fa8-94d58e9730ba",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "df.even.sum()"
      ],
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "8"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XCUwgF0f6WFP",
        "colab_type": "text"
      },
      "source": [
        "__Activity__: Explore the avalaible methods by using the completion system of the notebook after the last semicolon of `df.even.`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JDqDXVoF6mOv",
        "colab_type": "text"
      },
      "source": [
        "In addittion, for the `DataFrame` paradigm, we can easy implement any other function directly at the programming level either along the columns or along the rows"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "378RNNb97OQs",
        "colab_type": "text"
      },
      "source": [
        "### Column-level `apply`\n",
        "We just select the column and apply the direct or implicit function:\n",
        "* Pre-defined function"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L210XcRK7qYn",
        "colab_type": "code",
        "outputId": "db5a2c2b-94b2-4fe4-e022-7fa9f169b7a7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 126
        }
      },
      "source": [
        "df.even.apply(abs)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    0\n",
              "1    2\n",
              "2    4\n",
              "3    6\n",
              "4    8\n",
              "Name: even, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 104
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JX0nA6swDphj",
        "colab_type": "text"
      },
      "source": [
        "* Implicit function"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oGEcVFrA72LX",
        "colab_type": "code",
        "outputId": "245ccc64-a7b0-4601-ff9a-a87c0beea422",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 126
        }
      },
      "source": [
        "df.even.apply(lambda n:isinstance(n,int))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    True\n",
              "1    True\n",
              "2    True\n",
              "3    True\n",
              "4    True\n",
              "Name: even, dtype: bool"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 87
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8wFVd7Sk7irH",
        "colab_type": "text"
      },
      "source": [
        "### Row-level apply\n",
        "The foll row is passed as dictionary to the explicit or implicit function when `apply` is used for the full `DataFrame` and the option `axis=1` is used at the end"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z6GLMMuEEUwB",
        "colab_type": "code",
        "outputId": "47763d9f-1fbf-414c-e55d-74fc7fe0bf5d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 126
        }
      },
      "source": [
        "df.apply(lambda row: row['even']+row['odd'])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0     1\n",
              "1     5\n",
              "2    -1\n",
              "3     1\n",
              "4    17\n",
              "dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 119
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z2NsLagWs-Jg",
        "colab_type": "text"
      },
      "source": [
        "## Non-relational databases"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2NdcwWa5tuzv",
        "colab_type": "text"
      },
      "source": [
        "We have shown that the simple two dimensional spreadsheets where each cell values is a simple type like string, integer, or float, can be represented as a dictionary of lists values or a list of dictionary column-value assignment. \n",
        "\n",
        "We can go further and allow to store in the value itself a more general data structure, like nested lists and dictionaries. This allows advanced data-analysis when the `apply` methos is used to operate inside the nested lists or dictionaries.\n",
        "\n",
        "See for example:\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wB4hBVkhoPwR",
        "colab_type": "text"
      },
      "source": [
        "##  Machine web\n",
        "There are really three kinds of web\n",
        "* The normal web, \n",
        "* The deep web,\n",
        "* _The machine web_. The web for machine readable responses. It is served in `JSON` or `XML`  formats, which preserve programming objects.\n",
        "\n",
        "For example, consider the following normal web page:\n",
        "\n",
        "http://inspirehep.net/search?p=doi:10.1103/PhysRevLett.122.132001\n",
        "\n",
        "about a Scientific paper with people from the University of Antioquia. A _machine web_ version can be easily obtained in JSON just by attaching the extra parameter `&of=recjson`, and direcly loaded from Pandas, which works like a _browser for the third web_:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-cg2w7wsoo7H",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df=pd.read_json('http://inspirehep.net/search?p=doi:10.1103/PhysRevLett.122.132001&of=recjson')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MPiuuciYo90S",
        "colab_type": "code",
        "outputId": "5c049b40-16ec-45f4-cce3-158546834ea3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 323
        }
      },
      "source": [
        "df"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>comment</th>\n",
              "      <th>reference</th>\n",
              "      <th>abstract</th>\n",
              "      <th>creation_date</th>\n",
              "      <th>imprint</th>\n",
              "      <th>primary_report_number</th>\n",
              "      <th>keywords</th>\n",
              "      <th>publication_info</th>\n",
              "      <th>coyright</th>\n",
              "      <th>subject</th>\n",
              "      <th>cataloguer_info</th>\n",
              "      <th>physical_description</th>\n",
              "      <th>number_of_citations</th>\n",
              "      <th>title</th>\n",
              "      <th>persistent_identifiers_keys</th>\n",
              "      <th>corporate_name</th>\n",
              "      <th>files</th>\n",
              "      <th>system_control_number</th>\n",
              "      <th>filenames</th>\n",
              "      <th>source_of_acquisition</th>\n",
              "      <th>number_of_reviews</th>\n",
              "      <th>version_id</th>\n",
              "      <th>FIXME_OAI</th>\n",
              "      <th>authors</th>\n",
              "      <th>doi</th>\n",
              "      <th>number_of_authors</th>\n",
              "      <th>license</th>\n",
              "      <th>accelerator_experiment</th>\n",
              "      <th>number_of_comments</th>\n",
              "      <th>url</th>\n",
              "      <th>recid</th>\n",
              "      <th>collection</th>\n",
              "      <th>thesaurus_terms</th>\n",
              "      <th>filetypes</th>\n",
              "      <th>prepublication</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Submitted to Phys. Rev. Lett. All figures and tables can be found at http://cms-results.web.cern.ch/cms-results/public-results/publications/BPH-18-007 (CMS Public Pages). This version corrects the...</td>\n",
              "      <td>[{'title': 'Phys.Rev.Lett.,81,2432', 'year': '1998', 'doi': 'doi:10.1103/PhysRevLett.81.2432', 'order_number': '1', 'authors': 'F. Abe'}, {'title': 'Phys.Rev.,D49,5845', 'year': '1994', 'doi': 'do...</td>\n",
              "      <td>[{'number': 'arXiv', 'summary': 'Signals consistent with the B$^+_\\mathrm{c}$(2S) and B$^{*+}_\\mathrm{c}$(2S) states are observed in proton-proton collisions at $\\sqrt{s} =$ 13 TeV, in an event sa...</td>\n",
              "      <td>2019-02-05T04:03:36</td>\n",
              "      <td>{'date': '2019-04-03'}</td>\n",
              "      <td>[arXiv:1902.00571, None, CMS-BPH-18-007, CERN-EP-2019-014]</td>\n",
              "      <td>[{'institute': 'publisher', 'term': 'Elementary Particles and Fields'}]</td>\n",
              "      <td>{'volume': '122', 'pagination': '132001', 'title': 'Phys.Rev.Lett.', 'year': '2019'}</td>\n",
              "      <td>{'date': '2019', 'holder_contact': 'Article', 'holder': 'CERN'}</td>\n",
              "      <td>[{'source': 'INSPIRE', 'term': 'Experiment-HEP'}, {'source': 'arXiv', 'term': 'hep-ex'}, {'source': 'INSPIRE', 'term': 'Experiment-HEP'}]</td>\n",
              "      <td>{'creation_date': '2019-02-05'}</td>\n",
              "      <td>{'pagination': '17'}</td>\n",
              "      <td>30</td>\n",
              "      <td>{'title': 'Observation of Two Excited B$^+_\\mathrm{c}$ States and Measurement of the B$^+_\\mathrm{c}$(2S) Mass in pp Collisions at $\\sqrt{s} =$ 13 TeV'}</td>\n",
              "      <td>[recid, system_control_number, doi]</td>\n",
              "      <td>[{'collaboration': 'CMS'}]</td>\n",
              "      <td>[{'comment': None, 'status': '', 'magic': ['XML 1.0 document text', 'application/xml; charset=utf-8', 'XML 1.0 document text', 'application/xml; charset=utf-8', 'application/xml'], 'description': ...</td>\n",
              "      <td>[{'institute': 'INSPIRETeX', 'value': 'Sirunyan:2019osb'}, {'institute': 'arXiv', 'value': 'oai:arXiv.org:1902.00571'}, {'institute': 'CDS', 'value': '2655521'}, {'institute': 'HAL', 'value': 'hal...</td>\n",
              "      <td>[fulltext, fulltext.xml, 10, 10.1103_PhysRevLett.122.132001.pdf, 10.1103_PhysRevLett.122.132001, 10.1103_PhysRevLett, 10.1103_PhysRevLett.122, 1902, 1902.00571, 1902.00571.pdf, scoap3-fulltext, sc...</td>\n",
              "      <td>{'source_of_acquisition': 'arXiv', 'date': '2019-02-13T03:35:35.632671', 'accession_number': '1447042'}</td>\n",
              "      <td>0</td>\n",
              "      <td>2019-08-15T02:03:52</td>\n",
              "      <td>{'set': ['CERN', 'INSPIRE:HEP', 'CERN:arXiv', 'CDS'], 'id': 'oai:inspirehep.net:1718338'}</td>\n",
              "      <td>[{'INSPIRE_number': 'INSPIRE-00312131', 'affiliation': 'Yerevan Phys. Inst.', 'first_name': 'Albert M', 'last_name': 'Sirunyan', 'full_name': 'Sirunyan, Albert M'}, {'INSPIRE_number': 'INSPIRE-001...</td>\n",
              "      <td>10.1103/PhysRevLett.122.132001</td>\n",
              "      <td>2279</td>\n",
              "      <td>[{'url': 'http://creativecommons.org/licenses/by/4.0/', 'material': 'preprint', 'license': 'CC BY 4.0'}, {'imposing': 'APS', 'url': 'https://creativecommons.org/licenses/by/4.0/', 'license': 'Crea...</td>\n",
              "      <td>{'experiment': 'CERN-LHC-CMS'}</td>\n",
              "      <td>0</td>\n",
              "      <td>{'url': 'https://physics.aps.org/synopsis-for/10.1103/PhysRevLett.122.132001', 'description': 'Physics Synopsis'}</td>\n",
              "      <td>1718338</td>\n",
              "      <td>[{'primary': 'HEP'}, {'primary': 'Citeable'}, {'primary': 'CORE'}, {'primary': 'Fermilab'}, {'primary': 'Published'}]</td>\n",
              "      <td>[{'term': 'p p: scattering'}, {'term': 'p p: colliding beams'}, {'term': 'B/c+: hadroproduction'}, {'term': 'B/c: excited state'}, {'term': 'B/c+: hadronic decay'}, {'term': 'excited state: hadron...</td>\n",
              "      <td>[xml, pdf, pdf, pdf, png, png, png]</td>\n",
              "      <td>{'date': '2019-02-01'}</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                                                                                                                                                                                   comment  ...          prepublication\n",
              "0  Submitted to Phys. Rev. Lett. All figures and tables can be found at http://cms-results.web.cern.ch/cms-results/public-results/publications/BPH-18-007 (CMS Public Pages). This version corrects the...  ...  {'date': '2019-02-01'}\n",
              "\n",
              "[1 rows x 35 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 130
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SyVgDj1ApPtP",
        "colab_type": "text"
      },
      "source": [
        "We can see that the column `authors` is quite nested: Is a list of dictionaries with the full information for each one of the authors of the article."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BybpMovNsB7N",
        "colab_type": "code",
        "outputId": "b3e5be05-9b6b-44c5-eab4-e97592403819",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 73
        }
      },
      "source": [
        "df.authors"
      ],
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    [{'INSPIRE_number': 'INSPIRE-00312131', 'affiliation': 'Yerevan Phys. Inst.', 'first_name': 'Albert M', 'last_name': 'Sirunyan', 'full_name': 'Sirunyan, Albert M'}, {'INSPIRE_number': 'INSPIRE-001...\n",
              "Name: authors, dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 58
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8zr5Nsxwsikt",
        "colab_type": "text"
      },
      "source": [
        "__Activity__: Check that the lenght of the auhors list coincides with the `number_of_authors` \n",
        "<!-- df.authors.apply(len),df.number_of_authors.values -->"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "I1IvH1K-s46d",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "outputId": "a94cb11c-761c-4225-97b3-6285f74f174f"
      },
      "source": [
        "df.authors.apply(len)"
      ],
      "execution_count": 60,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    2279\n",
              "Name: authors, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 60
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "U8iwN3x3tKlN",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "a5883e7e-835c-4671-f93d-92337b8438a7"
      },
      "source": [
        "df.loc[0,'number_of_authors']"
      ],
      "execution_count": 61,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2279"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 61
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VL9DGK7tr1Fa",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 73
        },
        "outputId": "f031fa38-fb6c-41a7-99cf-c62b4ef2e53a"
      },
      "source": [
        "df.authors.str[0]"
      ],
      "execution_count": 62,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    {'INSPIRE_number': 'INSPIRE-00312131', 'affiliation': 'Yerevan Phys. Inst.', 'first_name': 'Albert M', 'last_name': 'Sirunyan', 'full_name': 'Sirunyan, Albert M'}\n",
              "Name: authors, dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 62
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i22ywhohsAVo",
        "colab_type": "text"
      },
      "source": [
        " We can use all the previous methods to extract the authors from `'Antioquia U.'`:\n",
        " \n",
        " Note: For a dictionary, `d` is safer to use `d.get('key')` instead of just `d['key']` to obtain some `key`, because not error is generated if the requested `key` does not exists at all "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1cx7xy2rpOzg",
        "colab_type": "code",
        "outputId": "deb081de-85d8-4560-c361-03400c82c8cb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        }
      },
      "source": [
        "df.authors.apply(lambda l:  #implicit function\n",
        "                           [ d.get('full_name') #safer way to obtain a `key` value\n",
        "                             for d in l   #comprehension list \n",
        "                              if isinstance(d,dict) and d.get('affiliation')=='Antioquia U.' #condition\n",
        "                           ] \n",
        "                             if isinstance(l,list) else None #  Be sure that cell have the proper format\n",
        "                 )"
      ],
      "execution_count": 63,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    [Mejia Guisao, Jhovanny, Ruiz Alvarez, José David]\n",
              "Name: authors, dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 63
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-YpfjPAdu19J",
        "colab_type": "text"
      },
      "source": [
        "A simpler version is possible,  but is more prompted to error when used through multiple column entries:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oM80j8zdvAdq",
        "colab_type": "code",
        "outputId": "0a5bc9e0-cd9e-44b0-86eb-b4f409298b8b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        }
      },
      "source": [
        "df.authors.apply(lambda l: [ d['full_name'] for d in l   \n",
        "                              if d['affiliation']=='Antioquia U.'\n",
        "                           ] )"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    [Mejia Guisao, Jhovanny, Ruiz Alvarez, José David]\n",
              "Name: authors, dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 157
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1Z96Ju2gwQO0",
        "colab_type": "text"
      },
      "source": [
        "For further details see: https://github.com/restrepo/inspire/blob/master/gfif.ipynb\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": true,
        "id": "D7SNm2ABxE22",
        "colab_type": "text"
      },
      "source": [
        "##  ACTIVITIES\n",
        "See:\n",
        "* https://github.com/ajcr/100-pandas-puzzles\n",
        "* https://github.com/guipsamora/pandas_exercises"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": true,
        "id": "fWjYdB7LxE23",
        "colab_type": "text"
      },
      "source": [
        "## Final remarks\n",
        "With basic scripting and Pandas we already have a solid environment to analyse data. We introduce the other libraries motivated with the extending the capabilities of Pandas"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "erRi7J01xE24",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}