Álvaro López - Ejercicios Bloque 1 - Descarga de datos.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Descarga de datos **\n",
    "\n",
    "Descargar y descomprimir el fichero .tar.gz de los casos del Titanic:\n",
    "http://www.cs.toronto.edu/~delve/data/titanic/desc.html (investigar las\n",
    "librerías para hacerlo)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Descargado correctamente: C:\\Users\\LVARO~1\\AppData\\Local\\Temp\\tmp14opy4v8\n"
     ]
    }
   ],
   "source": [
    "import tarfile, io\n",
    "import urllib.request\n",
    "#La librería requests no soporta descargas de FTP\n",
    "\n",
    "titanicTargzUri = 'ftp://ftp.cs.toronto.edu/pub/neuron/delve/data/tarfiles/titanic.tar.gz'\n",
    "\n",
    "try:\n",
    "    localTargz, headers = urllib.request.urlretrieve(titanicTargzUri)\n",
    "    tar = tarfile.open(localTargz)\n",
    "    tar.extractall()\n",
    "    tar.close()\n",
    "    print(\"Descargado correctamente: \" + localTargz)\n",
    "    \n",
    "except urllib.error.URLError as e:\n",
    "    print(e.reason)\n",
    "except:\n",
    "    print(\"Error al descargar:\", sys.exc_info()[0])\n",
    "\n",
    "#Referencias\n",
    "#https://docs.python.org/3/library/tarfile.html\n",
    "#https://docs.python.org/3/howto/urllib2.html\n",
    "#http://stackoverflow.com/questions/11768214/python-download-a-file-over-an-ftp-server\n",
    "#https://docs.python.org/3/tutorial/errors.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Utilizando wget, descargar varios ficheros de los N-gramas de Google, teniendo\n",
    "las URL de los ficheros en una lista en Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-all-2gram-20120701-xq.gz\n",
      "Almacenando: googlebooks-eng-all-2gram-20120701-xq\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-all-2gram-20120701-yb.gz\n",
      "Almacenando: googlebooks-eng-all-2gram-20120701-yb\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-all-2gram-20120701-yy.gz\n",
      "Almacenando: googlebooks-eng-all-2gram-20120701-yy\n"
     ]
    }
   ],
   "source": [
    "#import wget\n",
    "#Anaconda no incluye wget\n",
    "import urllib.request\n",
    "import gzip\n",
    "\n",
    "ngrams = ['http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-all-2gram-20120701-xq.gz',\n",
    "         'http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-all-2gram-20120701-yb.gz',\n",
    "         'http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-all-2gram-20120701-yy.gz']\n",
    "\n",
    "try:\n",
    "    for uri in ngrams:\n",
    "        print(uri)\n",
    "        with urllib.request.urlopen(uri) as response:\n",
    "            with gzip.GzipFile(fileobj=response) as uncompressed:\n",
    "                #Obtener el nombre del fichero descargado para almacenarlo en disco\n",
    "                filename = uri.split('/')[-1]\n",
    "                outFilePath = filename[:-3]\n",
    "                print(\"Almacenando: \" + outFilePath)\n",
    "                with open(outFilePath, 'wb') as outfile:\n",
    "                    outfile.write( uncompressed.read() )\n",
    "except:\n",
    "    print(\"Error descargando ngrams\")\n",
    "    \n",
    "#Referencias\n",
    "#http://stackoverflow.com/questions/7243750/download-file-from-web-in-python-3\n",
    "#http://stackoverflow.com/questions/10552188/python-split-url-to-find-image-name-and-extension\n",
    "#http://stackoverflow.com/questions/20635245/using-gzip-module-with-python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(**) Utilizando requests, descargar cotizaciones de Bitcoin de:\n",
    "https://blockchain.info/es/api/exchange_rates_api Nos devuelve un diccionario\n",
    "con las cotizaciones. Se pide encontrar la mayor diferencia entre valores de\n",
    "compra y venta."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Máxima diferencia entre compra y venta: 283.4000000001397 [KRW]\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "import json\n",
    "\n",
    "uri = \"https://blockchain.info/es/ticker\"\n",
    "r = requests.get(uri)\n",
    "if r.ok:\n",
    "    json_data = json.loads(r.text)\n",
    "    #print(float(json_data['CHF']['buy']))\n",
    "    \n",
    "    maxDif = float(\"-inf\")\n",
    "    maxCode = \"Unknown\"\n",
    "    \n",
    "    for curCode, curInfo in json_data.items():\n",
    "        buy = float(curInfo['buy'])\n",
    "        sell = float(curInfo['sell'])\n",
    "        #Buscamos la maxima diferencia en valor absoluto\n",
    "        curDif = abs(buy - sell)\n",
    "        #print(str(buy) + \" \" + str(sell) + \" [\" + str(curDif)  + \"]\")\n",
    "        if curDif > maxDif:\n",
    "            maxDif = curDif\n",
    "            maxCode = curCode\n",
    "        \n",
    "    print(\"Máxima diferencia entre compra y venta: \" + str(maxDif) + \" [\" + maxCode + \"]\")\n",
    "    \n",
    "#Referencias\n",
    "#http://stackoverflow.com/questions/19483351/converting-json-string-to-dictionary-not-list\n",
    "#http://stackoverflow.com/questions/5147292/python-read-json-and-loop-dictionary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(**) Obtener los ficheros de palabras que comienzan por “tr” de la página de\n",
    "descarga de los N-gramas de Google."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "44 enlaces obtenidos para -tr\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-all-2gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-all-3gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-all-4gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-all-5gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-us-all-2gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-us-all-3gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-us-all-4gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-us-all-5gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-gb-all-2gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-gb-all-3gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-gb-all-4gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-gb-all-5gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-fiction-all-2gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-fiction-all-3gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-fiction-all-4gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-eng-fiction-all-5gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-chi-sim-all-2gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-chi-sim-all-3gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-chi-sim-all-4gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-chi-sim-all-5gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-fre-all-2gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-fre-all-3gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-fre-all-4gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-fre-all-5gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-ger-all-2gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-ger-all-3gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-ger-all-4gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-ger-all-5gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-heb-all-2gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-heb-all-3gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-heb-all-4gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-heb-all-5gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-ita-all-2gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-ita-all-3gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-ita-all-4gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-ita-all-5gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-rus-all-2gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-rus-all-3gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-rus-all-4gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-rus-all-5gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-spa-all-2gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-spa-all-3gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-spa-all-4gram-20120701-tr.gz\n",
      "http://storage.googleapis.com/books/ngrams/books/googlebooks-spa-all-5gram-20120701-tr.gz\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "import re\n",
    "from lxml import html\n",
    "\n",
    "ngramsUri = \"http://storage.googleapis.com/books/ngrams/books/datasetsv2.html\"\n",
    "#Ficheros de palabras que comienzan por tr\n",
    "ngramsFilter = \"-tr\"\n",
    "\n",
    "r = requests.get(ngramsUri)\n",
    "if r.ok:\n",
    "    #Obtener todos los enlaces de la página. También se podría usar BeautifulSoup, pero no está incluida.\n",
    "    webpage = html.fromstring(r.content)\n",
    "    links = webpage.xpath('//a/@href')\n",
    "    \n",
    "    trWords = []\n",
    "    for link in links:\n",
    "        if link.find(ngramsFilter) != -1:\n",
    "            trWords.append(link)\n",
    "    \n",
    "    print(str(len(trWords)) + \" enlaces obtenidos para \" + ngramsFilter)\n",
    "    for link in trWords:\n",
    "        print(link)\n",
    "#Referencias\n",
    "#http://www.pythonforbeginners.com/code/regular-expression-re-findall\n"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}