demos/2023-02-01/mit-feb1-demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5c9b2c23",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# MIT (TA1): From Paper and Code to annotated Petri Nets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0701ccf7",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### Mike Cafarella, Chunwei Liu, Markos Markakis, Peter Chen"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2719d0e6",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 0. Preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "d07d53d5",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import ast, json, requests, os\n",
    "from IPython import display\n",
    "\n",
    "API_ROOT = \"http://100.26.10.46/\"\n",
    "GPT_KEY = \"\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4520a23",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### Starting with the SIDARTHE [paper](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7175834/pdf/41591_2020_Article_883.pdf) provided in Scenario 2,  we can run COSMOS (thanks Enrique Noriega from UArizona!) to extract a JSON with entries like this:\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "afd351d8",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'pdf_name': '41591_2020_Article_883.pdf', 'dataset_id': 'documents_5Feb', 'page_num': 1, 'img_pth': '/mytmp/images/41591_2020_Article_883.pdf_1_pad', 'pdf_dims': [0.0, 0.0, 612.0, 792.0], 'bounding_box': [103.0, 51.0, 1341.0, 212.0], 'classes': ['Section Header', 'Page Footer', 'Figure', 'Body Text', 'Other', 'Reference text', 'Equation', 'Page Header', 'Abstract', 'Table Caption', 'Figure Caption', 'Table', 'Table Note', 'Equation label'], 'scores': [-5.5183234215, -7.2636179924, -7.3330821991, -8.6994543076, -8.9685325623, -9.0146894455, -9.3787899017, -9.9470386505, -10.0574893951, -10.3444986343, -10.5733766556, -11.2742881775, -11.5064592361, -12.4296512604], 'content': '', 'postprocess_cls': 'Page Header', 'postprocess_score': 0.8623757958, 'detect_cls': 'Section Header', 'detect_score': -5.5183234215}\n"
     ]
    }
   ],
   "source": [
    "with open(\"documents_sidarthe--COSMOS-data.json\", \"r\") as f:\n",
    "    cosmos = f.read()\n",
    "    print(ast.literal_eval(cosmos)[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ef4c4a5",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### We can run a local script to consolidate the \"content\" fields to get just the text of the paper:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "9c565790",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Modelling the COVID-19 epidemic and implementation of population-wide Modelling the COVID-19 epidemic and implementation of population-wide interventions in Italy Giulia Giordano 1 ✉, Franco Blanchini2, Raffaele Bruno3,4, Patrizio Colaneri5,6, Alessandro Di Filippo3, Angela Di Matteo3 and Marta Colaneri3 In Italy, 128,948 confirmed cases and 15,887 deaths of peo- discharged due to recovery and 15,887 had died7. In the early days of ple  who  tested  positive  for  SARS-CoV-2  were  registered  \n"
     ]
    }
   ],
   "source": [
    "with open(\"sidarthe.txt\", \"r\") as f:\n",
    "    text = f.read()\n",
    "    print(text.replace('\\n', ' ')[:500])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae2884dd",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### From the COSMOS output, we can also keep metadata like the paper name and doi for later:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "bcd142ee",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"pdf_name\": \"41591_2020_Article_883.pdf\", \"DOI\": \"https://doi.org/10.1038/s41591-020-0883-7\"}\n"
     ]
    }
   ],
   "source": [
    "with open(\"sidarthe_info.json\", \"r\") as f:\n",
    "    info = json.load(f)\n",
    "    info_str = json.dumps(info)\n",
    "    print(info_str)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae6e9ef4",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 1. Extracting variables and annotating them from the text and the DKG"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c55b8b4",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### Using our API (powered by GPT-3), we can extract variables from the paper alongside a list of possible definitions, and ground each of these variables to the MIRA DKG (thanks Harvard team!). If you're interested, the JSON format of our intermediate output can be found [here](https://github.com/mikecafarella/mitaskem/blob/main/JSONformat.md)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "4b84abd1",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "with open(\"sidarthe_short.txt\", \"r\") as f:\n",
    "        text = f.read()           \n",
    "        dct_extract = {\"text\":text, \"gpt_key\":GPT_KEY}           \n",
    "        json_str = requests.post(API_ROOT + \"annotation/find_text_vars/\", params=dct_extract).text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "f3ca99aa",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'type': 'variable',\n",
       "  'name': 'S',\n",
       "  'id': 'v0',\n",
       "  'text_annotations': [' Susceptible (uninfected)'],\n",
       "  'dkg_annotations': [['ncit:C171133', 'COVID-19 Infection'],\n",
       "   ['ido:0000514', 'susceptible population']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'I',\n",
       "  'id': 'v1',\n",
       "  'text_annotations': [' Infected (asymptomatic or pauci-symptomatic infected, undetected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'D',\n",
       "  'id': 'v2',\n",
       "  'text_annotations': [' Diagnosed (asymptomatic infected, detected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'A',\n",
       "  'id': 'v3',\n",
       "  'text_annotations': [' Ailing (symptomatic infected, undetected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'R',\n",
       "  'id': 'v4',\n",
       "  'text_annotations': [' Recognized (symptomatic infected, detected)'],\n",
       "  'dkg_annotations': [['ncit:C171133', 'COVID-19 Infection'],\n",
       "   ['ncit:C28554', 'Dead']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'T',\n",
       "  'id': 'v5',\n",
       "  'text_annotations': [' Threatened (infected with life-threatening symptoms, detected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'H',\n",
       "  'id': 'v6',\n",
       "  'text_annotations': [' Healed (recovered)'],\n",
       "  'dkg_annotations': [['ncit:C171133', 'COVID-19 Infection'],\n",
       "   ['ncit:C28554', 'Dead']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'E',\n",
       "  'id': 'v7',\n",
       "  'text_annotations': [' Extinct (dead)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'α',\n",
       "  'id': 'v8',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)'],\n",
       "  'dkg_annotations': []},\n",
       " {'type': 'variable',\n",
       "  'name': 'β',\n",
       "  'id': 'v9',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)'],\n",
       "  'dkg_annotations': [['doid:0080928', 'dialysis-related amyloidosis'],\n",
       "   ['vo:0005114', 'β-propiolactone-inactivated SARS-CoV vaccine']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'γ',\n",
       "  'id': 'v10',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)'],\n",
       "  'dkg_annotations': [['askemo:0000013', 'recovery rate'],\n",
       "   ['vo:0004915', 'vaccine specific interferon-γ immune response']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'δ',\n",
       "  'id': 'v11',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)ε'],\n",
       "  'dkg_annotations': [['askemo:0000011', 'progression rate'],\n",
       "   ['vo:0005123', 'VSVΔG-MERS vaccine']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'θ',\n",
       "  'id': 'v12',\n",
       "  'text_annotations': [' Probability rate of detection relative to symptomatic cases'],\n",
       "  'dkg_annotations': []},\n",
       " {'type': 'variable',\n",
       "  'name': 'ζ',\n",
       "  'id': 'v13',\n",
       "  'text_annotations': [' Probability rate at which an infected subject not aware of being infected develops clinically relevant symptoms'],\n",
       "  'dkg_annotations': []},\n",
       " {'type': 'variable',\n",
       "  'name': 'η',\n",
       "  'id': 'v14',\n",
       "  'text_annotations': [' Probability rate at which an infected subject aware of being infected develops clinically relevant symptomsμ'],\n",
       "  'dkg_annotations': []},\n",
       " {'type': 'variable',\n",
       "  'name': 'ν',\n",
       "  'id': 'v15',\n",
       "  'text_annotations': [' Rate at which detected infected subjects develop life-threatening symptoms'],\n",
       "  'dkg_annotations': []},\n",
       " {'type': 'variable',\n",
       "  'name': 'τ',\n",
       "  'id': 'v16',\n",
       "  'text_annotations': [' Mortality rate for infected subjects with life-threatening symptoms'],\n",
       "  'dkg_annotations': []},\n",
       " {'type': 'variable',\n",
       "  'name': 'λ',\n",
       "  'id': 'v17',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': []},\n",
       " {'type': 'variable',\n",
       "  'name': 'κ',\n",
       "  'id': 'v18',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': []},\n",
       " {'type': 'variable',\n",
       "  'name': 'ξ',\n",
       "  'id': 'v19',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': []},\n",
       " {'type': 'variable',\n",
       "  'name': 'ρ',\n",
       "  'id': 'v20',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': []},\n",
       " {'type': 'variable',\n",
       "  'name': 'σ',\n",
       "  'id': 'v21',\n",
       "  'text_annotations': [' Rate of recovery for infected subjectsNone'],\n",
       "  'dkg_annotations': []}]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ast.literal_eval(json_str)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfb7bbbd",
   "metadata": {},
   "source": [
    "## 2. Adding annotations from dataset columns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6938a2a",
   "metadata": {},
   "source": [
    "#### Alongside the text, we might also have discovered a collection of datasets that we think might be relevant:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "ba00cfb6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "countyFIPS,County Name,State,StateFIPS,2020-01-22\n",
      "0,\"Statewide Unallocated\",\"AL\",\"01\",0\n",
      "1001,\"Autauga County \",\"AL\",\"01\",0\n"
     ]
    }
   ],
   "source": [
    "with open(\"../../resources/dataset/covid_confirmed_usafacts.csv\") as f:\n",
    "    dataset_1 = f.read()\n",
    "    print(dataset_1[:150])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "5d42a7de",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "countyFIPS,County Name,State,StateFIPS,2020-01-22\n",
      "0,\"Statewide Unallocated\",\"AL\",\"01\",0\n",
      "1001,\"Autauga County \",\"AL\",\"01\",0\n"
     ]
    }
   ],
   "source": [
    "with open(\"../../resources/dataset/covid_deaths_usafacts.csv\") as f:\n",
    "    dataset_2 = f.read()\n",
    "    print(dataset_2[:150])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e8df028",
   "metadata": {},
   "source": [
    "#### Let's collect just the column names into a single file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "c63f494e",
   "metadata": {},
   "outputs": [],
   "source": [
    "dir = \"../../resources/dataset/\"\n",
    "with open(os.path.join(dir,\"headers.txt\"), \"w+\") as fw:\n",
    "    for filename in os.listdir(dir):\n",
    "        file = os.path.join(dir, filename)\n",
    "        if os.path.isfile(file) and file.endswith(\".csv\"):\n",
    "            fw.write(\"{}:\\t{}\".format(filename, open(file, \"r\").readline()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "1c449c42",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "COVID-19_Reported_Patient_Impact_and_Hospital_Capacity_by_State_Archive_Repository.csv:\tUpdate Date,Days Since Update,User,Rows,Row Change,Columns,Column Change,Metadata Published,Metadata Updates,Column Level Metadata,Column Level Metadata Updates,Archive Link\n",
      "covid_confirmed_usafacts.csv:\tcountyFIPS,County Name,State,StateFIPS,2020-01-22\n",
      "covid_deaths_usafacts.csv:\tcountyFIPS,County Name,State,StateFIPS,2020-01-22\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open(os.path.join(dir,\"headers.txt\")) as f:\n",
    "    dataset_str = f.read()\n",
    "    print(dataset_str[:419])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90543cb7",
   "metadata": {},
   "source": [
    "#### Now we can call our `annotation/link_datasets_to_vars` endpoint to map variables discovered earlier to any matching dataset columns (GIGO warning here):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "d4d891aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "dct_cols = {\"json_str\":json_str, \"dataset_str\": dataset_str, \"gpt_key\":GPT_KEY}           \n",
    "json_str = requests.post(API_ROOT + \"annotation/link_datasets_to_vars/\", params=dct_cols).text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "5d492104",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'type': 'variable',\n",
       "  'name': 'S',\n",
       "  'id': 'v0',\n",
       "  'text_annotations': [' Susceptible (uninfected)'],\n",
       "  'dkg_annotations': [['ncit:C171133', 'COVID-19 Infection'],\n",
       "   ['ido:0000514', 'susceptible population']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'negative'],\n",
       "   ['covid_tracking.csv', 'totalTestResults']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'I',\n",
       "  'id': 'v1',\n",
       "  'text_annotations': [' Infected (asymptomatic or pauci-symptomatic infected, undetected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positive'],\n",
       "   ['covid_tracking.csv', 'probableCases']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'D',\n",
       "  'id': 'v2',\n",
       "  'text_annotations': [' Diagnosed (asymptomatic infected, detected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positive'],\n",
       "   ['covid_tracking.csv', 'totalTestResults']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'A',\n",
       "  'id': 'v3',\n",
       "  'text_annotations': [' Ailing (symptomatic infected, undetected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'hospitalizedCurrently'],\n",
       "   ['covid_tracking.csv', 'hospitalizedCumulative']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'R',\n",
       "  'id': 'v4',\n",
       "  'text_annotations': [' Recognized (symptomatic infected, detected)'],\n",
       "  'dkg_annotations': [['ncit:C171133', 'COVID-19 Infection'],\n",
       "   ['ncit:C28554', 'Dead']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positive'],\n",
       "   ['covid_tracking.csv', 'death']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'T',\n",
       "  'id': 'v5',\n",
       "  'text_annotations': [' Threatened (infected with life-threatening symptoms, detected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'hospitalizedCurrently'],\n",
       "   ['covid_tracking.csv', 'hospitalizedCumulative']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'H',\n",
       "  'id': 'v6',\n",
       "  'text_annotations': [' Healed (recovered)'],\n",
       "  'dkg_annotations': [['ncit:C171133', 'COVID-19 Infection'],\n",
       "   ['ncit:C28554', 'Dead']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'recovered'],\n",
       "   ['nychealth.csv', 'recovered']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'E',\n",
       "  'id': 'v7',\n",
       "  'text_annotations': [' Extinct (dead)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'death'],\n",
       "   ['nychealth.csv', 'DEATH_COUNT']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'α',\n",
       "  'id': 'v8',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positiveIncrease'],\n",
       "   ['covid_tracking.csv', 'negativeIncrease']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'β',\n",
       "  'id': 'v9',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)'],\n",
       "  'dkg_annotations': [['doid:0080928', 'dialysis-related amyloidosis'],\n",
       "   ['vo:0005114', 'β-propiolactone-inactivated SARS-CoV vaccine']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positiveIncrease'],\n",
       "   ['covid_tracking.csv', 'negativeIncrease']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'γ',\n",
       "  'id': 'v10',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)'],\n",
       "  'dkg_annotations': [['askemo:0000013', 'recovery rate'],\n",
       "   ['vo:0004915', 'vaccine specific interferon-γ immune response']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'totalTestResultsSource'],\n",
       "   ['covid_tracking.csv', 'totalTestResults']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'δ',\n",
       "  'id': 'v11',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)ε'],\n",
       "  'dkg_annotations': [['askemo:0000011', 'progression rate'],\n",
       "   ['vo:0005123', 'VSVΔG-MERS vaccine']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'totalTestResultsSource'],\n",
       "   ['covid_tracking.csv', 'totalTestResults']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'θ',\n",
       "  'id': 'v12',\n",
       "  'text_annotations': [' Probability rate of detection relative to symptomatic cases'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'dateModified'],\n",
       "   ['covid_tracking.csv', 'checkTimeEt']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'ζ',\n",
       "  'id': 'v13',\n",
       "  'text_annotations': [' Probability rate at which an infected subject not aware of being infected develops clinically relevant symptoms'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positiveIncrease'],\n",
       "   ['covid_tracking.csv', 'negativeIncrease']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'η',\n",
       "  'id': 'v14',\n",
       "  'text_annotations': [' Probability rate at which an infected subject aware of being infected develops clinically relevant symptomsμ'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'dateModified'],\n",
       "   ['covid_tracking.csv', 'checkTimeEt']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'ν',\n",
       "  'id': 'v15',\n",
       "  'text_annotations': [' Rate at which detected infected subjects develop life-threatening symptoms'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'hospitalizedCurrently'],\n",
       "   ['covid_tracking.csv', 'hospitalizedCumulative']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'τ',\n",
       "  'id': 'v16',\n",
       "  'text_annotations': [' Mortality rate for infected subjects with life-threatening symptoms'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'death'],\n",
       "   ['nychealth.csv', 'DEATH_COUNT']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'λ',\n",
       "  'id': 'v17',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'recovered'],\n",
       "   ['Hospitals_and_Medical_Centers.csv', 'dis_status']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'κ',\n",
       "  'id': 'v18',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'recovered'], '']},\n",
       " {'type': 'variable',\n",
       "  'name': 'ξ',\n",
       "  'id': 'v19',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'recovered'],\n",
       "   ['Hospitals_and_Medical_Centers.csv', 'dis_status']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'ρ',\n",
       "  'id': 'v20',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'recovered'], '']},\n",
       " {'type': 'variable',\n",
       "  'name': 'σ',\n",
       "  'id': 'v21',\n",
       "  'text_annotations': [' Rate of recovery for infected subjectsNone'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': ['', '']}]"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ast.literal_eval(json_str)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60ca157e",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 3. Extracting LaTeX from formula images"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1639ae0a",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "####  Here is a formula image from the SIDARTHE paper:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "2c77b882",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAA2CAIAAACz0U7UAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAEnQAABJ0Ad5mH3gAABW+SURBVHhe7Z13XFNX+8Bzs5ElW5YgSASERJYsEeHnLLZatVq1y7qrr6toHbUqttqqrdW2rr7OurVYa2srDqwiIAJCGAEEB3sTCCFk3Ht/F70gZCeQRN+e7z987skNPOd5nvOc5znn3AuEoigBAAAAdAYR/wkAAAC6AUQZAACgW0CUAQAAugVEGQAAoFtAlAEAALoFRBkAAKBbQJQBAAC6BUQZAACgW0CUAQAAugVEGQAAoFtAlAEAALoFRBkAAKBbeh9lJM9Sfz3z28NGGL8GAACA7vQ2yvDub5kwavqsKRHzv+fgTQAAANCN3kYZRMDjS1ACQcRvEeBNAAAA0I3ev1+mOf38zwkVg99ZMNnDGG8CAACALnq/LmMeOD12/UrNQgyKVF/ffzS9VXWAq7977Mg/1Qh+9T/L664Q9eUHqIPerKwfxzPAHhMKPzu1aOYJwbChJhDepBjLgICGvbPXXnz2P7y4/LorRCP5AeqgHyvrz/GwiqmPkHCfldQI8AvF8BLjoqMWXK6D8WuViOqvLQyJ+j61Bb9+VWgvTjz4xWcHHrYieAMqbheq3auXvO4KUUv+9qqUYzs2Ll8wd3Hstgtp6vf134vWVkYkLc9KKtvxKyXoz/H6LMrw83aGWHhtT1bRu8qrS7y9FiY2dI1Mtai5uYLpveh2o2bf0h0IXHVxVXjwlB/zWjpEEjflnNgwbZijqZntrAQeLmRDwvoQbwbD4yUMxtBxKy9Lmeh1V4hK+RG44caud33cwzaczm7glfw8051qPCT2YjkINCrRysri9B1RDkNjM4UqvqVPx+ubKIPAFQenDiRCxvOON+JN8oAF91f5Oc89osLD2gXSoQqBKw9OdRuz7l4b3mBgWu6t8TBifZspwvpUeW/3xMHPM06IOvyjc9Xd+oYIa+/uirHAqlKIEvTBsfwGMf5BJ+opRFYfr4pCVMqPwLV/bhhhYcKKu1n/4h7ureXOJMjGZ02WqmEA0MLKwoqTb9pRaWaTLjUpU6+eHa9vokztjWVuFGwwUUZvSJPgbbJIsr4f6+C14oFASf/5GUfmjJt/UbYP3DufelhG/syRHqgGoSVl/RBav/C5hy4f+8THtCOK0C1Zc7++Xi0jHTdx+UASRKKGHCiUlVy1QmB+3v4PI1f/ysOvu/EKKESF/Ahc/+faMDOicUzcAyHehgqKdw6nEkm0sEOPNJJcfP/LUePWp2Jx/VUAEed+Hh60474adUnv0NDKLVeW+pAhApHqu/OBElXp2/H6IMogYvbnYR5BQS4kAuQ17RQfb5YGbk/6xNN87MYMxWFImLxzrDWJPusnOSEWEWetHmYeseJ2l78aFD4nYd/88AGmjsHvLt6w58T1Eq78WaH08GQjiGDFWJYmM3WrVAgsfLg12onWb/TJCjm3GFwhquSHi8+870gm2vrGpvNf9v1FlCFSBm/5R6MhKkxYxQxedO3VsD5mmqRPvNzXJ6hehuwlGlmZn70t2C3I34VKgMwWnVG4eqJ/x+v9HhPy6MSW34wW717kR4XQpicFpR2H9OTAvX368hPnMW/5kvAGaVBJTvzJ1CaSZ1CYraxYENlrwthBuZdO3m97FbZL6SbEsse0mfEpd8/s+3LZ+6PdzOVqUpiTVShCIUvPYQyq9DK+SoUI2JfOJFVYeIb6y9GH4RWiXH5R9YXVn56vRGymrv3Ur9/LvsNcXisBE/hVMOJrgPpWRpGyY5uOufznp49Z5hAqeFpQomgzSP+O19soA3MTtu8omrFpAdOT4UCGmh8XPJZ/BliUfu0O3y40xIeMN8ggqkhOKeKb2AUGeMq9hzosMphWdetmmhBvMBwt7B8Xbmlce2rnaGcK3iQPFKnOZlfDBLIr09cEb+tCpUKQ0uTUYjHRKTDUDUuC5WBYhSiXv/X6ti+vVIlshn64cJpddycTVVTUYPMQZGLSUWn+qxE2N/ccK5KW2tpWCX7RibpWrrm2/afHMbEdA3EAkSAp5XAUfMEAjtdLS4vu7/kye9hnC8ONjTw8XCgkIb+osFhaTxgo/DQ5tczSy99TZkrHqLm7dZwfyz86Lq0d5jdenR/E9PVlvRX7B/5xJ8bDWAxSZfo9hUFaP4gbE9bMPR/1zc7RAxRNBjioIJtd0AYRjRnMIVK3KlUI93rcm35M5htb/8Fq36I/VgT6+jJZUVv+rME/78SAClFuUFHFub2nCiUEo1FzF/j0uAEuZnOaEQKZZO/kpHC+0QGStD0zw0PC5+zO7KYr0Z0dU8NCwt7fkYo36B5h2Y1vP1s4/Y0RPi5WzoHr2cIX6QBSmXzgo/BB1gMGMPxjU/k9cgR1rIyIsvZsToj8bHVAP9pghgsZQhsf5T+VV1UYxPF6FWXanx7fchiZ9/l0GyKBaObhbk9FxI8LOXKSGaQt6yFHOIAxxEzOH0QQs+BPNm2c4kWBUUrYRzv37zt45Nip/RvfwD/vhGzJcLclPsnJFeENSkAETdUVZRpRUd3UrvJoI4rUXli9gTtz//JQU7xJMe2P2ZwmMZE8xIdphDd1okQhWOFrFTzviy9mekAIRPVZtGvf/oOHj506vmqCHX5HJ5oopI9RalBJ9rGjiU1imnnUtBmuPcMr98GDYsw1TZw8BpnjTXoAhctuXLqWkpZPHzCoSx5UUvTX+RspaXlG9h54k+6hOY9euWVdBP0pp5RrNzzSl4YNdW7SDzNHjFl+NqNagqKNz/JLeT3cUA0rw4VHN8eT56yYjmUxhP4e7nZEiPs4v6QN/7g7BnE8OT6iNo2Xv/yuefyG9306qgaI4sFwpRHQ9pL8QtkIJ6p4UtGGWjvYy5v9ifassW+9GUDnCVCSfcTkKVERYUEBPo4yKx0QxcHBltJS/rROVQgVVR2b7D7QeaBGOA9ym3K+QsWvrkmI28WZuGnp8z6rgpuVUyxG+tkM9Roo3W8lCoGINv7jJk0I6Mdrgc0cQidNiR4RNtzfZ6BshaG+QvocJfKjcPGVK7nYPEoyaU34OnZlN1YsX3Y4uRGrlwYMC5GbBOkIuDk5ObuVTPcLDTfDmzq6kHg3n0+h+4eP7I836QUi3ZggEqEQjRUxgo403dgybc4Ro823SpsbHiWc2Hf40k+Te+bIKq0MN17dtjPv7Q1LGM8LHCMGw4UCiQRFhUVyqgqDOJ72UaYlbc/2P5yXrB33Yk6HiFaDB1uRCJKy/Hz+85buiKqq6hDIwtpGtm8vQNrSM/LaqP2Yfn40vEkWyNKqP0lQV1kDq1h2otp/fK1RINaQ1qar7zoqErADFOYc+Op6ZOx/vNQaIaL8bE47ClkOYXp2TFk9UKkQ3oNMjhi18Q3ylvnuS9RWSJ+jRH5RecItdisKUXyjJ4UNY3XH26K1qklCgIyDxkbp89FaXurdDJ7YymtEkEOXwyPVt25ntcNW3hEhjtqPAi1A+GlpWTwiZWhwBP3OtndW3Aw+fu3wB8F2NGPX0e8vnh09iIrf2IkKKwuTdn+V5r58yTg8VpItB7vZ0hDx0yJO64uW7hjE8bSsjVG45NDnh4rJAy+tnXWls41byEUJaGNR3hMRyuo5DhG+oB2BaEYKI0hbbkZOi9iC6eejOJGGiP3odKJEwFdd2HRAIvV14d98e//ZxgknYyzwa6WgSAObXYEQSE6+LEsZN1alEFFOOpuPkN39A5UUZmoqBKsXEv57Or1OrG4ogiiuER+8N9Iev5SHEvlbs7ILhDCR4jN9xdI5ft1zPpi965cGhEDrH/lGjLzdC+zXCh9dPX61sEVWVOTxw/oqUfzuXWyZ4QFRTb0nfjjBjY5fyyDKvptWB5MCgyPcO5czUaQ8/lxSG0pihUcNkr/G2UFr0bVTV3N5MgscKFKW08Slx3+/K1vGySCylff4WRM8FZm2NTMxuVZoPijEMm3Zqr8CDv0eF4bNzopRbuX2ksObfsoWe9h/OvMO3oTy8ltFBBQqzisUE0Kkkm59Ol4X2kVxpPzyV3uzB74zY6S7SxeuXt6uxkRCS2lBiWwyg8AIAcLAL6VBKjOySiWQAyvIVbHJMWmJJAKCoe5w6Vv4N87+bRX9JlNJjO8GKsrOKhCgEJXBkldeKVUIitRkZJQjRCvfQIYyB1RPIShcm5t8764GJKVzavEvK0Kh/HApp7gZIRhb+/n33CvEKqlf4zPFKNEz5qPxcktn7Mv80vQ7d3ApepDELue3VObK70ZSRjlfYe6OwuVJ954gRNOAkUFdaUJLyr6DifUEouXwUYHSucNLEF5pplxxkpKyKvntpTnJ+HUP7iRnPBUoHH6SvJtJpWIskqdtX3GuxcjBob9SC3eg0MooUvdr3O4nHlPfHjkYH4UYrt5eg8yJBLicky9viVR/jvcS/NyMJsD85OW+VhM2ZUidLuw8cMX46p70gSvu9aVOJPqcI4qeP+Cf/mAgEaLP2leh9Mhzw543rCzcP0ltV3hmUXfAwuSlXlZzjzfh16poK/g6ENMG1UfuKUzlCoFbf5thS6P2G32qUtHJqRcYTCGK5RfeXM3EHNRheNyjnrI3J69lUIlkut936VocmdX+VJ64/sQEcwqlX/Qv5bhAiKTo2/f9nehEmtnECzXKNSwfrU/lIZLirRH9IcgoYFSIGZEAkRyWnqtS6vMYCq3ceG+dt3nA7kwpfUrYO0ZhAd7ac5Xs00wGcTwtchmYfWjz6caxS5ewpKZoqoMXw5qKSMoecXh4UycUW1tLEtLcKP/twKiEk5HVRCC7sQKslAiEwk3cZgnd0tZCVfQX1f6+cnxEaIhGhEaOj02oUTglShrZuaVUSytFqaY0rdm5RWKEZuI1dIicyk25QgScDDZXZO7m76NMHxoopM9RLD/a0tKxt0ExMTftNl+iSO2ve88Ui0nDP942N0BdHfYJvJSkTJ7Y2ntEEH5wBy44vqeAascXobaskcFKq5U+R1yXeDurlUwLWLDvhyWBFgS46uyWr9N7bl1LocjKKFx0YOMRSszqD1hS+iQ5eTGwIr2lrKBYeiAaxvE0jjKi6otxO1MCF6weJ7N+BFG9hmDFMSouzssX4204NGcXRypUXy3/LTjihsyHJQIjcybLuyN7bW9rf9EuBQpX1zRITB1d7ZVVVR1QLP2mzF+4SEMWzJvio1htkrq6JnFz+dM69UpRCSc7vw0lWHgwGd1OvnahVCFIbebDJ2LIjhno0bG8pUgfGiikz1EsP2Ri0rFtj0jE3Xc4uGl7v71UZuY6e/PmMS+3efSBKOtuWj1CcgoIfbH+wk3ftTFxiJcwtxkhe0ZGOZAIvCbui1v1QEPirQd8ibVPdKRHwKrtH7phOuT8d/tPeQonN4VWRp5diNub6jRn9RTZVT8j7yGDKESJqKggX3qv2TCOh+c06tIUv8jb1HbqZfl5puDsRy5YGugSvaeyZxaIiB+sYpoyJh9txht6gGVxjiSoI8cWt6Qdnhf9zn65hZOk4fh4E7rSBzJ1CJ+9lUklmjpP3JNQ3Koqx0Xgil2jrSAC0WfWBblPrypVSPvvi7GqmBLzZa6whb3v45HzDz6R+wcNqBAl8pcenWoMEWyZa9liPJ2GBenrgi0ppv5x17V+s4yWFRNWoWwJN4cI5JGf3sUKV0Fp/Jyod89x/prjYgSRXL+4JRCUnIzdcAm/W220rZj45z92IxLIkavvYcIgcPWxWR3nd/rZTb5YqtCGcq0safp7zmDjodN+wR917wnc9vvMATQCRJvxwzOpzw3ieBrkMijCvb9/4bLDBZZDRwRay5nzUaSF3/FoA1qZduZcRo9cDSL7hATZNBawi0VykkNRTU0DgtYXX5g7gvneMesv9s5/ueHYDT4n95HI0j/UR700rY+huQWw7Gi8sj+Wj/WwMDa3sOhvbmE7aOjIjzZekD3+1Jh26HQKF+sqImzjy5s1lCpEUl3dhBDgzHNLw31jrg5Y99U8V7l2MqBClMhvF/PuBAd6U/H9h5UdPUclpaeXfri30Gnl0fj1o601Tp57B9yYlJKDFSSS1CNLZ86dOWrURpsVuydC7JwqIYXm6dr/77Xrs2JWvonfrWMQUfrNuzUoyT58TMfeG0S0mxG3ZqQFpa3mytY1Z6sU5DOyVpZw2XvnLzlZIh4aEdHxXhEZJLy2NgJKQEXJp4/k9nzbpmEcD482KoCfnVvMcu6P50cQub+D99wf87pFMvGDPe94OZp1JVBEirlH8Op73R7GLTsx3YIeelDe8/5Nfy2yJxFI9IGTVp8rVvRMNyrJ/HqkqeWUyxq+d6fvkBReXBZgK7PDBFFCP/mrU2q4/OKywCFOJl2KgMjm9oyYdQmyC56KFdIeP9+dRIDMXf5v83ms7FKEgRWiWH74yR9rAm3onuPX7z/49bzRzKA3VsXn9vLNflrmMvWX59kQidaOjkYQRLcM3BD/BHPa9qf7o/qTMdtYM2acztfmuWrtcpnWzE3eVMhkwHtdrzrDBk720dmudKwAMGZOWPzdbwUy2UEPKyNwzfGFQY7mlBfuRSSbugydejz/pQkQce6OKUx705f+RzN3HvdZAv7xc/TveH3zfhl1ENefnWRjNv37EtksC5FUpSUkcqqVbT0gksfbo6x93z3DxRsMA9z6LP3Wn1f+vJmaVVhR31BR8M/OaYNtfT/L7qwO1EeJQoRVGdcTc+qUbsUYXCFK5McQtzxO/vv3qzdTCqoUzhuaoF2UEd6IZZKJlkvOPCvMzOn2uli4jvPPX9fTK7QVrW/f/MAtSb50+pfzV+49aZIe+bqwsv4dT39RBitK/1jqPXDENx2ziea0cXYEmXpsTmzFr18ZGuPneI/9ofsr8tTmdVdIr+TXEGHi2oAR/7mlUZRBJEWbQs2pxmPkvielN8DC+ytYnpsS+ybKKEE3Vta34+kzyqDC8hMTHYdopTLelSU+zHd+qdV28bAPgXl557775kRiGWYjuDV31yS/Rb/IXa1WzeuukF7IrzHtFdnsZ0onWRmENYfHmJLtgzYXap5pqkJcmvWwQjNxtEBXVtaz4+k1ymAV3cPdY72ivy2WLQmVws/dNcIp6nCBhl/TCXDJgRg6hJXEll7BIV72VhGLf+uFE7zuCtFSfv1QeXZ2fyIpbJlmGdCrgy6trFfH03OUQRFx0Q9vs2b/yFE/WUOE+dvG+M47LHWU1GDw2T+McTaGILK1e/SnB1KaejfPvO4K0UJ+/YDA1QenOpFIjmv+1Eeq1efo2sr6dDx9RxkMUUPiysgINY9OIHDtbysj31x3s5eDuW9BJK2Njfy+Mv/rrhCN5NcbwpqrG2bPeG/JIc5r+M8S9GNlvTmeAaIMhqjmn60L4+49/2dGyqm5vnXZN7cbXin/1QGvu0LUlx+gDnqzsn4cr/f/jR8AAACUoeejmAAA4F8HiDIAAEC3gCgDAAB0C4gyAABAt4AoAwAAdAuIMgAAQLeAKAMAAHQLiDIAAECXEAj/Dwkt7mE7JHMJAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "display.Image(\"../../resources/images/SIDARTHE/sidarthe_dAdt.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a0ac4f7",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### As we demoed last week, we can extract LaTeX from such formula images (powered by `pix2tex`), also through a public API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "1574de16",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['\"{\\\\\\\\dot{A}}(t)=\\\\\\\\zeta I(t)-(\\\\\\\\theta+\\\\\\\\mu+\\\\\\\\kappa)A(t)\"', '\"{\\\\\\\\dot{D}}(t)=\\\\\\\\varepsilon I(t)-(\\\\\\\\eta+\\\\\\\\rho)D(t)\"', '\"\\\\\\\\dot{\\\\\\\\bar{F}}(t)\\\\\\\\,\\\\\\\\longrightarrow\\\\\\\\,\\\\\\\\tau\\\\\\\\,T(t)\"', '\"H(t)=i I(t)+\\\\\\\\theta b(t)+\\\\\\\\kappa d(t)+i R(t)+\\\\\\\\sigma T(t)\"', '\"\\\\\\\\scriptstyle I_{\\\\\\\\theta\\\\\\\\theta}=\\\\\\\\delta\\\\\\\\theta\\\\\\\\theta\\\\\\\\theta+I_{\\\\\\\\theta\\\\\\\\theta}+\\\\\\\\gamma\\\\\\\\xi\\\\\\\\theta_{1}+\\\\\\\\delta\\\\\\\\theta_{1})-\\\\\\\\epsilon+\\\\\\\\xi+\\\\\\\\delta I_{\\\\\\\\theta}\"', '\"R(t)=\\\\\\\\eta D(t)+\\\\\\\\theta A(t)-(\\\\\\\\nu+\\\\\\\\xi)R(t)\"', '\"\\\\\\\\delta(t)=-S(t)(a l(t)+\\\\\\\\theta D(t)+r A(t)+\\\\\\\\delta t(t)\"', '\"{\\\\\\\\hat{T}}(t)=\\\\\\\\mu A(t)+\\\\\\\\nu R(t)-(\\\\\\\\sigma+\\\\\\\\tau){\\\\\\\\hat{T}}(t)\"']\n"
     ]
    }
   ],
   "source": [
    "directory = '../../resources/images/SIDARTHE'\n",
    "latex_strs = []\n",
    " \n",
    "for filename in os.listdir(directory):\n",
    "    f = os.path.join(directory, filename)\n",
    "    latex_str = !python3 img_latex.py -p {f} # This is a local script that resizes the image and calls the public API.\n",
    "    latex_strs.append(latex_str[0])\n",
    "\n",
    "print(latex_strs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6cfbd2ba",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 4. Linking variables from the LaTeX formulas to variables from the text"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da7f5ef9",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### We just extracted all these equations in LaTeX, which include variables. Let's link these variables to the variables we found from the text in part 1.  Again, the output (internal)JSON format can be found [here](https://github.com/mikecafarella/mitaskem/blob/main/JSONformat.md)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "7fbda461",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "for latex_str in latex_strs:\n",
    "    dct_latex = {\"json_str\":json_str, \"formula\": latex_str, \"gpt_key\":GPT_KEY}           \n",
    "    json_str = requests.post(API_ROOT + \"annotation/link_latex_to_vars/\", params=dct_latex).text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "b5d7c652",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'type': 'variable',\n",
       "  'name': 'S',\n",
       "  'id': 'v0',\n",
       "  'text_annotations': [' Susceptible (uninfected)'],\n",
       "  'dkg_annotations': [['ncit:C171133', 'COVID-19 Infection'],\n",
       "   ['ido:0000514', 'susceptible population']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'negative'],\n",
       "   ['covid_tracking.csv', 'totalTestResults']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'I',\n",
       "  'id': 'v1',\n",
       "  'text_annotations': [' Infected (asymptomatic or pauci-symptomatic infected, undetected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positive'],\n",
       "   ['covid_tracking.csv', 'probableCases']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'D',\n",
       "  'id': 'v2',\n",
       "  'text_annotations': [' Diagnosed (asymptomatic infected, detected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positive'],\n",
       "   ['covid_tracking.csv', 'totalTestResults']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'A',\n",
       "  'id': 'v3',\n",
       "  'text_annotations': [' Ailing (symptomatic infected, undetected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'hospitalizedCurrently'],\n",
       "   ['covid_tracking.csv', 'hospitalizedCumulative']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'R',\n",
       "  'id': 'v4',\n",
       "  'text_annotations': [' Recognized (symptomatic infected, detected)'],\n",
       "  'dkg_annotations': [['ncit:C171133', 'COVID-19 Infection'],\n",
       "   ['ncit:C28554', 'Dead']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positive'],\n",
       "   ['covid_tracking.csv', 'death']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'T',\n",
       "  'id': 'v5',\n",
       "  'text_annotations': [' Threatened (infected with life-threatening symptoms, detected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'hospitalizedCurrently'],\n",
       "   ['covid_tracking.csv', 'hospitalizedCumulative']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'H',\n",
       "  'id': 'v6',\n",
       "  'text_annotations': [' Healed (recovered)'],\n",
       "  'dkg_annotations': [['ncit:C171133', 'COVID-19 Infection'],\n",
       "   ['ncit:C28554', 'Dead']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'recovered'],\n",
       "   ['nychealth.csv', 'recovered']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'E',\n",
       "  'id': 'v7',\n",
       "  'text_annotations': [' Extinct (dead)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'death'],\n",
       "   ['nychealth.csv', 'DEATH_COUNT']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'α',\n",
       "  'id': 'v8',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positiveIncrease'],\n",
       "   ['covid_tracking.csv', 'negativeIncrease']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'β',\n",
       "  'id': 'v9',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)'],\n",
       "  'dkg_annotations': [['doid:0080928', 'dialysis-related amyloidosis'],\n",
       "   ['vo:0005114', 'β-propiolactone-inactivated SARS-CoV vaccine']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positiveIncrease'],\n",
       "   ['covid_tracking.csv', 'negativeIncrease']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'γ',\n",
       "  'id': 'v10',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)'],\n",
       "  'dkg_annotations': [['askemo:0000013', 'recovery rate'],\n",
       "   ['vo:0004915', 'vaccine specific interferon-γ immune response']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'totalTestResultsSource'],\n",
       "   ['covid_tracking.csv', 'totalTestResults']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'δ',\n",
       "  'id': 'v11',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)ε'],\n",
       "  'dkg_annotations': [['askemo:0000011', 'progression rate'],\n",
       "   ['vo:0005123', 'VSVΔG-MERS vaccine']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'totalTestResultsSource'],\n",
       "   ['covid_tracking.csv', 'totalTestResults']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'θ',\n",
       "  'id': 'v12',\n",
       "  'text_annotations': [' Probability rate of detection relative to symptomatic cases'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'dateModified'],\n",
       "   ['covid_tracking.csv', 'checkTimeEt']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'ζ',\n",
       "  'id': 'v13',\n",
       "  'text_annotations': [' Probability rate at which an infected subject not aware of being infected develops clinically relevant symptoms'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positiveIncrease'],\n",
       "   ['covid_tracking.csv', 'negativeIncrease']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'η',\n",
       "  'id': 'v14',\n",
       "  'text_annotations': [' Probability rate at which an infected subject aware of being infected develops clinically relevant symptomsμ'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'dateModified'],\n",
       "   ['covid_tracking.csv', 'checkTimeEt']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'ν',\n",
       "  'id': 'v15',\n",
       "  'text_annotations': [' Rate at which detected infected subjects develop life-threatening symptoms'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'hospitalizedCurrently'],\n",
       "   ['covid_tracking.csv', 'hospitalizedCumulative']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'τ',\n",
       "  'id': 'v16',\n",
       "  'text_annotations': [' Mortality rate for infected subjects with life-threatening symptoms'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'death'],\n",
       "   ['nychealth.csv', 'DEATH_COUNT']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'λ',\n",
       "  'id': 'v17',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'recovered'],\n",
       "   ['Hospitals_and_Medical_Centers.csv', 'dis_status']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'κ',\n",
       "  'id': 'v18',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'recovered'], '']},\n",
       " {'type': 'variable',\n",
       "  'name': 'ξ',\n",
       "  'id': 'v19',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'recovered'],\n",
       "   ['Hospitals_and_Medical_Centers.csv', 'dis_status']]},\n",
       " {'type': 'variable',\n",
       "  'name': 'ρ',\n",
       "  'id': 'v20',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'recovered'], '']},\n",
       " {'type': 'variable',\n",
       "  'name': 'σ',\n",
       "  'id': 'v21',\n",
       "  'text_annotations': [' Rate of recovery for infected subjectsNone'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': ['', '']},\n",
       " {'type': 'equation',\n",
       "  'latex': '{\\\\dot{A}}(t)=\\\\zeta I(t)-(\\\\theta+\\\\mu+\\\\kappa)A(t)',\n",
       "  'id': 'e15029274875602918169',\n",
       "  'matches': {'A': ['v0', 'v3'],\n",
       "   't': ['v16', 'v12'],\n",
       "   'ζ': ['v13', 'v16'],\n",
       "   'I': ['v1', 'v16'],\n",
       "   'θ': ['v12', 'v16'],\n",
       "   'μ': ['v14', 'v15'],\n",
       "   'κ': ['v18', 'v19']}},\n",
       " {'type': 'equation',\n",
       "  'latex': '{\\\\dot{D}}(t)=\\\\varepsilon I(t)-(\\\\eta+\\\\rho)D(t)',\n",
       "  'id': 'e9160306732573434510',\n",
       "  'matches': {'D': ['v0', 'v2'],\n",
       "   'I': ['v1', 'v16'],\n",
       "   't': ['v16', 'v12'],\n",
       "   'ε': ['v12', 'v14'],\n",
       "   'η': ['v14', 'v16'],\n",
       "   'ρ': ['v20', 'v19']}},\n",
       " {'type': 'equation',\n",
       "  'latex': '\\\\dot{\\\\bar{F}}(t)\\\\,\\\\longrightarrow\\\\,\\\\tau\\\\,T(t)',\n",
       "  'id': 'e4327158603912984336',\n",
       "  'matches': {' \\\\dot{\\x08ar{F}}(t)': ['v15', 'v16'],\n",
       "   '\\tau': ['v16', 'v15'],\n",
       "   'T(t)': ['v16', 'v5']}},\n",
       " {'type': 'equation',\n",
       "  'latex': 'H(t)=i I(t)+\\\\theta b(t)+\\\\kappa d(t)+i R(t)+\\\\sigma T(t)',\n",
       "  'id': 'e18402757488225498028',\n",
       "  'matches': {'H(t)': ['v6'],\n",
       "   'I(t)': ['v1'],\n",
       "   'b(t)': ['v9', 'v16'],\n",
       "   'd(t)': ['v16', 'v11'],\n",
       "   'R(t)': ['v4', 'v16'],\n",
       "   'T(t)': ['v16', 'v5'],\n",
       "   'i': ['v1', 'v8'],\n",
       "   'θ': ['v12', 'v16'],\n",
       "   'κ': ['v18', 'v19'],\n",
       "   'σ': ['v21', 'v19']}},\n",
       " {'type': 'equation',\n",
       "  'latex': '\\\\scriptstyle I_{\\\\theta\\\\theta}=\\\\delta\\\\theta\\\\theta\\\\theta+I_{\\\\theta\\\\theta}+\\\\gamma\\\\xi\\\\theta_{1}+\\\\delta\\\\theta_{1})-\\\\epsilon+\\\\xi+\\\\delta I_{\\\\theta}',\n",
       "  'id': 'e13111876941200814871',\n",
       "  'matches': {'I_θθ': ['v1', 'v12'],\n",
       "   'δθθθ': ['v11', 'v12'],\n",
       "   'γξθ1': ['v10', 'v19'],\n",
       "   'δθ1': ['v11', 'v12'],\n",
       "   'ε': ['v12', 'v14'],\n",
       "   'ξ': ['v19', 'v16'],\n",
       "   'δI_θ': ['v11', 'v12']}},\n",
       " {'type': 'equation',\n",
       "  'latex': 'R(t)=\\\\eta D(t)+\\\\theta A(t)-(\\\\nu+\\\\xi)R(t)',\n",
       "  'id': 'e14600561528505332591',\n",
       "  'matches': {'R(t)': ['v4', 'v16'],\n",
       "   'D(t)': ['v2'],\n",
       "   'A(t)': ['v8', 'v16'],\n",
       "   'η': ['v14', 'v16'],\n",
       "   'θ': ['v12', 'v16'],\n",
       "   'ν': ['v15', 'v16'],\n",
       "   'ξ': ['v19', 'v16']}},\n",
       " {'type': 'equation',\n",
       "  'latex': '\\\\delta(t)=-S(t)(a l(t)+\\\\theta D(t)+r A(t)+\\\\delta t(t)',\n",
       "  'id': 'e2849391391462718456',\n",
       "  'matches': {'δ': ['v11', 'v16'],\n",
       "   'S': ['v0', 'v1'],\n",
       "   'a': ['v8', 'v9'],\n",
       "   'l': ['v17', 'v18'],\n",
       "   'θ': ['v12', 'v16'],\n",
       "   'D': ['v0', 'v2'],\n",
       "   'r': ['v4', 'v20'],\n",
       "   'A': ['v0', 'v3'],\n",
       "   't': ['v16', 'v12']}},\n",
       " {'type': 'equation',\n",
       "  'latex': '{\\\\hat{T}}(t)=\\\\mu A(t)+\\\\nu R(t)-(\\\\sigma+\\\\tau){\\\\hat{T}}(t)',\n",
       "  'id': 'e6404018658125356859',\n",
       "  'matches': {'T': ['v16', 'v12'],\n",
       "   'μ': ['v14', 'v15'],\n",
       "   'A': ['v0', 'v3'],\n",
       "   'ν': ['v15', 'v16'],\n",
       "   'R': ['v0', 'v1'],\n",
       "   'σ': ['v21', 'v19'],\n",
       "   'τ': ['v16', 'v12']}}]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ast.literal_eval(json_str)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65a888fa",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 5. Getting a Petri net (as a pyascet) from code "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ab7283a",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### Let's now turn our attention to code. We have a python function that describes the SIDARTHE dynamics:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "e2969b62",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# define SIDARTHE_model\n",
      "def SIDARTHE_model(y, t, alpha, beta, gamma, delta, epsilon, mu, zeta, lamda, eta, rho, theta, kappa, nu, xi, sigma, tau):\n",
      "    S, I, D, A, R, T, H, E = y\n",
      "    dSdt = -S*(alpha(t)*I + beta(t)*D + gamma(t)*A + delta(t)*R)\n",
      "    dIdt = S*(alpha(t)*I + beta(t)*D + gamma(t)*A + delta(t)*R) - (epsilon(t) + zeta(t) + lamda(t))*I\n",
      "    dDdt = epsilon(t)*I - (eta(t) + rho(t))*D\n",
      "    dAdt = zeta(t)*I - (theta(t) + mu(t) + kappa(t))*A\n",
      "    dRdt = eta(t)*D + theta(t)*A - (nu(t) + xi(t))*R\n",
      "    dTdt = mu(t)*A + nu(t)*R - (sigma(t) + tau(t))*T\n",
      "    dHdt = lamda(t)*I + rho(t)*D + kappa(t)*A + xi(t)*R + sigma(t)*T\n",
      "    dEdt = tau(t)*T\n",
      "    \n",
      "    return dSdt, dIdt, dDdt, dAdt, dRdt, dTdt, dHdt, dEdt\n"
     ]
    }
   ],
   "source": [
    "with open(\"../../resources/jan_evaluation/scenario_2_sidarthe/sidarthe_code.py\", \"r\") as f:\n",
    "    code = f.read()\n",
    "print(code)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "785c8fa3",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### Using calls to the public MIT API, we can get Petri net components (places, transitions, hypothesized arcs) from this piece of code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "bc6e7dcb",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['S', ' I', ' D', ' A', ' R', ' T', ' H', ' E']"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dict_petri = {\"code\": code, \"gpt_key\": GPT_KEY}\n",
    "places = requests.post(API_ROOT + \"petri/get_places\", params=dict_petri).text\n",
    "ast.literal_eval(places)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "4f48fde8",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['alpha',\n",
       " ' beta',\n",
       " ' gamma',\n",
       " ' delta',\n",
       " ' epsilon',\n",
       " ' mu',\n",
       " ' zeta',\n",
       " ' lamda',\n",
       " ' eta',\n",
       " ' rho',\n",
       " ' theta',\n",
       " ' kappa',\n",
       " ' nu',\n",
       " ' xi',\n",
       " ' sigma',\n",
       " ' tau']"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transitions = requests.post(API_ROOT + \"petri/get_transitions\", params=dict_petri).text\n",
    "ast.literal_eval(transitions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "a2558515",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['S', ' I'],\n",
       " ['I', ' D'],\n",
       " ['I', ' A'],\n",
       " ['I', ' R'],\n",
       " ['D', ' E'],\n",
       " ['A', ' R'],\n",
       " ['A', ' T'],\n",
       " ['R', ' H'],\n",
       " ['T', ' H'],\n",
       " ['I', ' H'],\n",
       " ['D', ' H'],\n",
       " ['A', ' H'],\n",
       " ['R', ' H'],\n",
       " ['T', ' E']]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arcs = requests.post(API_ROOT + \"petri/get_arcs\", params=dict_petri).text\n",
    "ast.literal_eval(arcs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21bc9dbc",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### We can then convert these outputs into a py-acset (thanks to Justin Lieffers from Arizona for some of the conversion code and to Owen Lynch for the py-acset code!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "473da719",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "dict_acset = {\"places_str\": places, \"transitions_str\": transitions, \"arcs_str\": arcs}\n",
    "pyacset_str = requests.post(API_ROOT + \"petri/get_pyacset\", params=dict_acset).text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "c2a4aaa3",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'S': [{'sname': 'S', 'uid': 1},\n",
       "  {'sname': 'I', 'uid': 2},\n",
       "  {'sname': 'D', 'uid': 3},\n",
       "  {'sname': 'A', 'uid': 4},\n",
       "  {'sname': 'R', 'uid': 5},\n",
       "  {'sname': 'T', 'uid': 6},\n",
       "  {'sname': 'H', 'uid': 7},\n",
       "  {'sname': 'E', 'uid': 8}],\n",
       " 'T': [{'tname': 'alpha', 'uid': 10},\n",
       "  {'tname': ' beta', 'uid': 11},\n",
       "  {'tname': ' gamma', 'uid': 12},\n",
       "  {'tname': ' delta', 'uid': 13},\n",
       "  {'tname': ' epsilon', 'uid': 14},\n",
       "  {'tname': ' mu', 'uid': 15},\n",
       "  {'tname': ' zeta', 'uid': 16},\n",
       "  {'tname': ' lamda', 'uid': 17},\n",
       "  {'tname': ' eta', 'uid': 18},\n",
       "  {'tname': ' rho', 'uid': 19},\n",
       "  {'tname': ' theta', 'uid': 20},\n",
       "  {'tname': ' kappa', 'uid': 21},\n",
       "  {'tname': ' nu', 'uid': 22},\n",
       "  {'tname': ' xi', 'uid': 23},\n",
       "  {'tname': ' sigma', 'uid': 24},\n",
       "  {'tname': ' tau', 'uid': 25}],\n",
       " 'I': [{'it': 1, 'is': 1},\n",
       "  {'it': 2, 'is': 2},\n",
       "  {'it': 3, 'is': 2},\n",
       "  {'it': 4, 'is': 2},\n",
       "  {'it': 5, 'is': 3},\n",
       "  {'it': 6, 'is': 4},\n",
       "  {'it': 7, 'is': 4},\n",
       "  {'it': 8, 'is': 5},\n",
       "  {'it': 9, 'is': 6},\n",
       "  {'it': 10, 'is': 2},\n",
       "  {'it': 11, 'is': 3},\n",
       "  {'it': 12, 'is': 4},\n",
       "  {'it': 13, 'is': 5},\n",
       "  {'it': 14, 'is': 6}],\n",
       " 'O': [{'ot': 1, 'os': 2},\n",
       "  {'ot': 2, 'os': 3},\n",
       "  {'ot': 3, 'os': 4},\n",
       "  {'ot': 4, 'os': 5},\n",
       "  {'ot': 5, 'os': 8},\n",
       "  {'ot': 6, 'os': 5},\n",
       "  {'ot': 7, 'os': 6},\n",
       "  {'ot': 8, 'os': 7},\n",
       "  {'ot': 9, 'os': 7},\n",
       "  {'ot': 10, 'os': 7},\n",
       "  {'ot': 11, 'os': 7},\n",
       "  {'ot': 12, 'os': 7},\n",
       "  {'ot': 13, 'os': 7},\n",
       "  {'ot': 14, 'os': 8}]}"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ast.literal_eval(pyacset_str)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59df09f4",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 6. Linking the annotations to the py-acset and paper info"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b11fcced",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### Finally, we bring everything together: for every place and transition in the pyacset, let's map it to the annotations from earlier:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "1f50f236",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "dct_link = {\"pyacset_str\":pyacset_str, \"annotations_str\":json_str, \"info_str\":info_str}           \n",
    "metadata_str = requests.post(API_ROOT + \"annotation/link_annos_to_pyacset/\", params=dct_link).text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "61bcdb1b",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'1': {'type': 'variable',\n",
       "  'name': 'S',\n",
       "  'id': 'v0',\n",
       "  'text_annotations': [' Susceptible (uninfected)'],\n",
       "  'dkg_annotations': [['ncit:C171133', 'COVID-19 Infection'],\n",
       "   ['ido:0000514', 'susceptible population']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'negative'],\n",
       "   ['covid_tracking.csv', 'totalTestResults']],\n",
       "  'equation_annotations': {'\\\\delta(t)=-S(t)(a l(t)+\\\\theta D(t)+r A(t)+\\\\delta t(t)': ['S']},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '2': {'type': 'variable',\n",
       "  'name': 'I',\n",
       "  'id': 'v1',\n",
       "  'text_annotations': [' Infected (asymptomatic or pauci-symptomatic infected, undetected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positive'],\n",
       "   ['covid_tracking.csv', 'probableCases']],\n",
       "  'equation_annotations': {'{\\\\dot{A}}(t)=\\\\zeta I(t)-(\\\\theta+\\\\mu+\\\\kappa)A(t)': ['I'],\n",
       "   '{\\\\dot{D}}(t)=\\\\varepsilon I(t)-(\\\\eta+\\\\rho)D(t)': ['I']},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '3': {'type': 'variable',\n",
       "  'name': 'D',\n",
       "  'id': 'v2',\n",
       "  'text_annotations': [' Diagnosed (asymptomatic infected, detected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positive'],\n",
       "   ['covid_tracking.csv', 'totalTestResults']],\n",
       "  'equation_annotations': {'{\\\\dot{D}}(t)=\\\\varepsilon I(t)-(\\\\eta+\\\\rho)D(t)': ['D'],\n",
       "   '\\\\delta(t)=-S(t)(a l(t)+\\\\theta D(t)+r A(t)+\\\\delta t(t)': ['D']},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '4': {'type': 'variable',\n",
       "  'name': 'A',\n",
       "  'id': 'v3',\n",
       "  'text_annotations': [' Ailing (symptomatic infected, undetected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'hospitalizedCurrently'],\n",
       "   ['covid_tracking.csv', 'hospitalizedCumulative']],\n",
       "  'equation_annotations': {'{\\\\dot{A}}(t)=\\\\zeta I(t)-(\\\\theta+\\\\mu+\\\\kappa)A(t)': ['A'],\n",
       "   '\\\\delta(t)=-S(t)(a l(t)+\\\\theta D(t)+r A(t)+\\\\delta t(t)': ['A'],\n",
       "   '{\\\\hat{T}}(t)=\\\\mu A(t)+\\\\nu R(t)-(\\\\sigma+\\\\tau){\\\\hat{T}}(t)': ['A']},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '5': {'type': 'variable',\n",
       "  'name': 'R',\n",
       "  'id': 'v4',\n",
       "  'text_annotations': [' Recognized (symptomatic infected, detected)'],\n",
       "  'dkg_annotations': [['ncit:C171133', 'COVID-19 Infection'],\n",
       "   ['ncit:C28554', 'Dead']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positive'],\n",
       "   ['covid_tracking.csv', 'death']],\n",
       "  'equation_annotations': {'{\\\\hat{T}}(t)=\\\\mu A(t)+\\\\nu R(t)-(\\\\sigma+\\\\tau){\\\\hat{T}}(t)': []},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '6': {'type': 'variable',\n",
       "  'name': 'T',\n",
       "  'id': 'v5',\n",
       "  'text_annotations': [' Threatened (infected with life-threatening symptoms, detected)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'hospitalizedCurrently'],\n",
       "   ['covid_tracking.csv', 'hospitalizedCumulative']],\n",
       "  'equation_annotations': {'{\\\\hat{T}}(t)=\\\\mu A(t)+\\\\nu R(t)-(\\\\sigma+\\\\tau){\\\\hat{T}}(t)': []},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '7': {'type': 'variable',\n",
       "  'name': 'H',\n",
       "  'id': 'v6',\n",
       "  'text_annotations': [' Healed (recovered)'],\n",
       "  'dkg_annotations': [['ncit:C171133', 'COVID-19 Infection'],\n",
       "   ['ncit:C28554', 'Dead']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'recovered'],\n",
       "   ['nychealth.csv', 'recovered']],\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '8': {'type': 'variable',\n",
       "  'name': 'E',\n",
       "  'id': 'v7',\n",
       "  'text_annotations': [' Extinct (dead)'],\n",
       "  'dkg_annotations': [['ido:0000511', 'infected population'],\n",
       "   ['ncit:C171133', 'COVID-19 Infection']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'death'],\n",
       "   ['nychealth.csv', 'DEATH_COUNT']],\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '10': {'type': 'variable',\n",
       "  'name': 'α',\n",
       "  'id': 'v8',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positiveIncrease'],\n",
       "   ['covid_tracking.csv', 'negativeIncrease']],\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '11': {'type': 'variable',\n",
       "  'name': 'β',\n",
       "  'id': 'v9',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)'],\n",
       "  'dkg_annotations': [['doid:0080928', 'dialysis-related amyloidosis'],\n",
       "   ['vo:0005114', 'β-propiolactone-inactivated SARS-CoV vaccine']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positiveIncrease'],\n",
       "   ['covid_tracking.csv', 'negativeIncrease']],\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '12': {'type': 'variable',\n",
       "  'name': 'γ',\n",
       "  'id': 'v10',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)'],\n",
       "  'dkg_annotations': [['askemo:0000013', 'recovery rate'],\n",
       "   ['vo:0004915', 'vaccine specific interferon-γ immune response']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'totalTestResultsSource'],\n",
       "   ['covid_tracking.csv', 'totalTestResults']],\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '13': {'type': 'variable',\n",
       "  'name': 'δ',\n",
       "  'id': 'v11',\n",
       "  'text_annotations': [' Transmission rate (the probability of disease transmission in a single contact multiplied by the average number of contacts per person)ε'],\n",
       "  'dkg_annotations': [['askemo:0000011', 'progression rate'],\n",
       "   ['vo:0005123', 'VSVΔG-MERS vaccine']],\n",
       "  'data_annotations': [['covid_tracking.csv', 'totalTestResultsSource'],\n",
       "   ['covid_tracking.csv', 'totalTestResults']],\n",
       "  'equation_annotations': {'\\\\delta(t)=-S(t)(a l(t)+\\\\theta D(t)+r A(t)+\\\\delta t(t)': ['δ']},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '14': {'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '15': {'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '16': {'type': 'variable',\n",
       "  'name': 'ζ',\n",
       "  'id': 'v13',\n",
       "  'text_annotations': [' Probability rate at which an infected subject not aware of being infected develops clinically relevant symptoms'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'positiveIncrease'],\n",
       "   ['covid_tracking.csv', 'negativeIncrease']],\n",
       "  'equation_annotations': {'{\\\\dot{A}}(t)=\\\\zeta I(t)-(\\\\theta+\\\\mu+\\\\kappa)A(t)': ['ζ']},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '17': {'type': 'variable',\n",
       "  'name': 'λ',\n",
       "  'id': 'v17',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'recovered'],\n",
       "   ['Hospitals_and_Medical_Centers.csv', 'dis_status']],\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '18': {'type': 'variable',\n",
       "  'name': 'η',\n",
       "  'id': 'v14',\n",
       "  'text_annotations': [' Probability rate at which an infected subject aware of being infected develops clinically relevant symptomsμ'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'dateModified'],\n",
       "   ['covid_tracking.csv', 'checkTimeEt']],\n",
       "  'equation_annotations': {'{\\\\dot{D}}(t)=\\\\varepsilon I(t)-(\\\\eta+\\\\rho)D(t)': ['η'],\n",
       "   'R(t)=\\\\eta D(t)+\\\\theta A(t)-(\\\\nu+\\\\xi)R(t)': ['η']},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '19': {'type': 'variable',\n",
       "  'name': 'ρ',\n",
       "  'id': 'v20',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'recovered'], ''],\n",
       "  'equation_annotations': {'{\\\\dot{D}}(t)=\\\\varepsilon I(t)-(\\\\eta+\\\\rho)D(t)': ['ρ']},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '20': {'type': 'variable',\n",
       "  'name': 'θ',\n",
       "  'id': 'v12',\n",
       "  'text_annotations': [' Probability rate of detection relative to symptomatic cases'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'dateModified'],\n",
       "   ['covid_tracking.csv', 'checkTimeEt']],\n",
       "  'equation_annotations': {'{\\\\dot{A}}(t)=\\\\zeta I(t)-(\\\\theta+\\\\mu+\\\\kappa)A(t)': ['θ'],\n",
       "   'H(t)=i I(t)+\\\\theta b(t)+\\\\kappa d(t)+i R(t)+\\\\sigma T(t)': ['θ'],\n",
       "   'R(t)=\\\\eta D(t)+\\\\theta A(t)-(\\\\nu+\\\\xi)R(t)': ['θ'],\n",
       "   '\\\\delta(t)=-S(t)(a l(t)+\\\\theta D(t)+r A(t)+\\\\delta t(t)': ['θ']},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '21': {'type': 'variable',\n",
       "  'name': 'κ',\n",
       "  'id': 'v18',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'recovered'], ''],\n",
       "  'equation_annotations': {'{\\\\dot{A}}(t)=\\\\zeta I(t)-(\\\\theta+\\\\mu+\\\\kappa)A(t)': ['κ'],\n",
       "   'H(t)=i I(t)+\\\\theta b(t)+\\\\kappa d(t)+i R(t)+\\\\sigma T(t)': ['κ']},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '22': {'type': 'variable',\n",
       "  'name': 'ν',\n",
       "  'id': 'v15',\n",
       "  'text_annotations': [' Rate at which detected infected subjects develop life-threatening symptoms'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'hospitalizedCurrently'],\n",
       "   ['covid_tracking.csv', 'hospitalizedCumulative']],\n",
       "  'equation_annotations': {'R(t)=\\\\eta D(t)+\\\\theta A(t)-(\\\\nu+\\\\xi)R(t)': ['ν'],\n",
       "   '{\\\\hat{T}}(t)=\\\\mu A(t)+\\\\nu R(t)-(\\\\sigma+\\\\tau){\\\\hat{T}}(t)': ['ν']},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '23': {'type': 'variable',\n",
       "  'name': 'ξ',\n",
       "  'id': 'v19',\n",
       "  'text_annotations': [' Rate of recovery for infected subjects'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'recovered'],\n",
       "   ['Hospitals_and_Medical_Centers.csv', 'dis_status']],\n",
       "  'equation_annotations': {'\\\\scriptstyle I_{\\\\theta\\\\theta}=\\\\delta\\\\theta\\\\theta\\\\theta+I_{\\\\theta\\\\theta}+\\\\gamma\\\\xi\\\\theta_{1}+\\\\delta\\\\theta_{1})-\\\\epsilon+\\\\xi+\\\\delta I_{\\\\theta}': ['ξ'],\n",
       "   'R(t)=\\\\eta D(t)+\\\\theta A(t)-(\\\\nu+\\\\xi)R(t)': ['ξ']},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '24': {'type': 'variable',\n",
       "  'name': 'σ',\n",
       "  'id': 'v21',\n",
       "  'text_annotations': [' Rate of recovery for infected subjectsNone'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': ['', ''],\n",
       "  'equation_annotations': {'H(t)=i I(t)+\\\\theta b(t)+\\\\kappa d(t)+i R(t)+\\\\sigma T(t)': ['σ'],\n",
       "   '{\\\\hat{T}}(t)=\\\\mu A(t)+\\\\nu R(t)-(\\\\sigma+\\\\tau){\\\\hat{T}}(t)': ['σ']},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'},\n",
       " '25': {'type': 'variable',\n",
       "  'name': 'τ',\n",
       "  'id': 'v16',\n",
       "  'text_annotations': [' Mortality rate for infected subjects with life-threatening symptoms'],\n",
       "  'dkg_annotations': [],\n",
       "  'data_annotations': [['covid_tracking.csv', 'death'],\n",
       "   ['nychealth.csv', 'DEATH_COUNT']],\n",
       "  'equation_annotations': {'{\\\\hat{T}}(t)=\\\\mu A(t)+\\\\nu R(t)-(\\\\sigma+\\\\tau){\\\\hat{T}}(t)': ['τ']},\n",
       "  'file': '41591_2020_Article_883.pdf',\n",
       "  'doi': 'https://doi.org/10.1038/s41591-020-0883-7'}}"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ast.literal_eval(metadata_str)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94a9cfcd",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### Data in this format can be ingested, visualized and edited by TA4! Let's export the related files:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "8cf547c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"output-mit-pyacset.json\", \"w+\") as f:\n",
    "    json.dump(ast.literal_eval(pyacset_str), f)\n",
    "with open(\"output-mit-metadata.json\", \"w+\") as f:\n",
    "    json.dump(ast.literal_eval(metadata_str), f)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d66094a4",
   "metadata": {},
   "source": [
    "## 7. Interacting with the University of Arizona codepaths"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "439e54ac",
   "metadata": {},
   "source": [
    "#### The University of Arizona team can also produce an annotated py-acset as an output. We can integrate the two outputs by matching on the names of places and transitions, to get a more complete picture of the model. The metadata extracted by both teams can then be accessible by using the associated `uid` of each place/transition as a key into the metadata JSON file."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}