Homework 3 - HuBMAP.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8977401c-2f3d-42f6-ba9c-e1d275e6c9c1",
   "metadata": {},
   "source": [
    "# Homework 3\n",
    "<img src=\"https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fwww.chem.purdue.edu%2Fmedia%2Fnews%2F2018%2Fimages%2Fhubmap.jpg&f=1&nofb=1&ipt=d9be5726927fa5f4b7445361cab50fc40c0804467bfe8abbbe5dc900ef6fc888&ipo=images\" width=\"25%\" />\n",
    "\n",
    "The Human BioMolecular Atlas Program is a consortium composed of diverse research teams funded by the Common Fund at the National Institutes of Health . HuBMAP values secure, open sharing, and collaboration with other consortia and the wider research community.\n",
    "\n",
    "Name:\n",
    "\n",
    "Andrew ID:\n",
    " \n",
    "a. Load the `data_file` as a data frame and name it `df`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ba4fef73-ccd8-4d4a-833c-956fc7cf6ae7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# INSERT CODE HERE\n",
    "data_file = '/bil/workshops/2022/data-science/hubmap-data.tsv'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f321743f-f82f-439c-ae29-ea91c957dc3f",
   "metadata": {},
   "source": [
    "b. Print the keys to in the dataframe `df`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e55df1fd-1532-421c-9637-587870023c85",
   "metadata": {},
   "outputs": [],
   "source": [
    "# INSERT CODE HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed348f86-0d54-408a-bdd8-d3f4c98920e1",
   "metadata": {},
   "source": [
    "3. Globus and Globus ready. The columns `globus` and `globus_ready` are empty. The purpose of this exercise is to complete both columns.\n",
    "\n",
    "    a. Create a method called `get_globus_link`. The function takes the value in columns `path` and `dataset_uuid` and returns a `URL`. This `URL` needs to be added to the corresponding cell in the `globus` column. For example for input arguments `TLH_S3_L006_R1_001.fastq.gz` (`path`) and `c65efd23-bbc4-459a-ac60-d3cde705193d` (`dataset_uuid`)  the method should return the string `https://g-d00e7b.09193a.5898.dn.glob.us/7277b2460f9c548004496508684a90ef/c65efd23-bbc4-459a-ac60-d3cde705193d/TLH_S3_L006_R1_001.fastq.gz`. Use it populate the appropiate cell in column `globus`.\n",
    "\n",
    "    b. Create a method called `check_globus_link`. The function takes the value in column `globus` and checks if the link is reachable. For example, for input argument `https://g-d00e7b.09193a.5898.dn.glob.us/7277b2460f9c548004496508684a90ef/c65efd23-bbc4-459a-ac60-d3cde705193d/TLH_S3_L006_R1_001.fastq.gz`, then method should populate the cell in column `globus_ready` with `True` if the link reachable. `False` otherwise. \n",
    "    \n",
    "**Hints**: \n",
    "* Don't download the file, just download the file header. Use the results of the response populate the appropiate cell in column `globus_ready`.  \n",
    "* Use an iterator to iterate over the rows of the dataframe. Feel free to use any other Pythonic methods as long as it produces the proper result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d0f0abfd-dc67-4119-a1d2-b2a46fcf0b9d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# INSERT CODE HERE\n",
    "def get_globus_link( dataset_uuid, path ):\n",
    "    raise NotImplementedError()\n",
    "    \n",
    "def check_globus_link( url ):\n",
    "    raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "616d44f3-9d59-4ce0-88da-3afcbece4a02",
   "metadata": {},
   "source": [
    "4. Assets and assets ready. The columns `assets` and `assets_ready` are empty. The purpose of this exercise is to complete both columns.\n",
    "\n",
    "    a. Create a method called `get_assets_link`. The function takes the value in columns `path` and `dataset_uuid` and returns a `URL`. For example for input arguments `ometiffs/separate/VAN0005-RK-4-172-IMS_PosMode_mz834.601.ome.tiff` and `d9e3c80a32567cde9b61e38ce8693559`  the method should return the string `https://assets.hubmapconsortium.org/d9e3c80a32567cde9b61e38ce8693559/ometiffs/separate/VAN0005-RK-4-172-IMS_PosMode_mz834.601.ome.tiff`. Use it populate the appropiate cell in column `assets`.\n",
    "\n",
    "    b. Create a method called `check_assets_link`. The function takes the value in column `assets` and check if the link is reachable. For example, for input argument `https://assets.hubmapconsortium.org/d9e3c80a32567cde9b61e38ce8693559/ometiffs/separate/VAN0005-RK-4-172-IMS_PosMode_mz834.601.ome.tiff`, then method should populate the cell in column `assets_ready` with `True`. `False` otherwise. Use it populate the appropiate cell in column `assets_ready`.\n",
    "    \n",
    "**Hints**: \n",
    "* Don't download the file, just download the file header. Use the results of the response populate the appropiate cell in column `globus_ready`.  \n",
    "* Use an iterator to iterate over the rows of the dataframe. Feel free to use any other Pythonic methods as long as it produces the proper result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ce24b7e1-f77d-4b47-8766-ec397c63dac6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# INSERT CODE HERE\n",
    "def get_assets_link( dataset_uuid, path ):\n",
    "    raise NotImplementedError()\n",
    "    \n",
    "def check_assets_link( url ):\n",
    "    raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e5d8361-c133-4a11-8ced-c4a54d20864b",
   "metadata": {},
   "source": [
    "5. Add a column named `extension` to the dataframe. Create a method called `extract_extension`. The function takes the value in column `path` and returns the file extension. For example, for file `TLH_S3_L006_R1_001.fastq.gz`, the method should return the string `fastq.gz`. Use it populate the appropiate cell in column `extension`.\n",
    "\n",
    "Keep in mind that\n",
    "* If the file is named `<filename>.ome.tiff` or `<filename>.ome.tif`, then the method should return `ome.tiff`.\n",
    "* If the file is named `<filename>.fastq.gz`, then the method should return `fastq.gz`.\n",
    "\n",
    "*Hint*: Use the `pathlib` library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "728a6208-c306-4258-bd04-6f24f785f426",
   "metadata": {},
   "outputs": [],
   "source": [
    "# INSERT CODE HERE\n",
    "def extract_extension( path ):\n",
    "    raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef8d0dba-6b68-470c-ae7b-1fcb65e0ae11",
   "metadata": {},
   "source": [
    "6. Save dataframe to the current working directory as `hubmap1.tsv` using the method `to_tsv`. \n",
    "\n",
    "**Hint** \n",
    "* Set `sep=`\\t'`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b314f7b7-b010-4792-b925-ed7d3f04de5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# INSERT CODE HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96b8d59e-040f-48ee-9443-78d156f208b3",
   "metadata": {},
   "source": [
    "7. Create histogram using the column `globus_ready` with seaborn. Compute the number of working and broken links, and create a histogram with 2 bins.\n",
    "\n",
    "* The title should be `Reachable hyperlinks in Globus`.\n",
    "* No x-label.\n",
    "* Set y-label to `Counts`\n",
    "* Add labels to both bins, `Reachable` for working links and `Unreachable` for broken links."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "88bd32f5-a20b-49c6-9a99-0b5838cd0c4a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# INSERT CODE HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56bb63db-c293-4548-a3fb-d9a0c7b5e9f1",
   "metadata": {},
   "source": [
    "8. Perform the same task as the previous but with the columns `assets_ready` with seaborn. However instead of plotting a histrogram, plot a pie chart.\n",
    "\n",
    "* The title should be `Reachable hyperlinks using the Assets API`.\n",
    "* No x-label.\n",
    "* Set y-label to `Counts`\n",
    "* Add labels to both bins, `Reachable` for working links and `Unreachable` for broken links."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "b32208af-e9c6-49e6-9199-f5c00c32d9f1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# INSERT CODE HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c8921b6-ec53-40b0-b1ee-1f4fcb8ac53a",
   "metadata": {},
   "source": [
    "9. Create a waffleplot using the column `extension`. \n",
    "\n",
    "* The title should be `File extensions`.\n",
    "* Set legend using the file extensions as labels\n",
    "\n",
    "**Hint**\n",
    "* Check this [post](https://www.geeksforgeeks.org/generate-waffle-chart-using-pywaffle-in-python/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "82730574-422e-448f-9021-1a6baca8634b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# INSERT CODE HERE"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}