data-sets/arxiv/create-arxiv-corpus.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7776cf85",
   "metadata": {},
   "source": [
    "# Custom arXiv corpus for Annif\n",
    "## Subject vocabulary\n",
    "\n",
    "To create the subject vocabulary we first need the categories that are used in the arXiv articles. The categories can be browsed at [Category Taxonomy page](https://arxiv.org/category_taxonomy). However, they are defined in a Python source file in a [arXiv's GitHub repository](https://github.com/arXiv/arxiv-base/blob/develop/arxiv/taxonomy/definitions.py), which can be easily downloaded. \n",
    "\n",
    "For downloading we use command-line tool `curl`, and to use the (terminal) command line from within Jupyter Notebook cell the command is prepended with `!`: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "00015482",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "100 90754  100 90754    0     0   223k      0 --:--:-- --:--:-- --:--:--  223k\n"
     ]
    }
   ],
   "source": [
    "# Download a file with specific commit\n",
    "# https://github.com/arXiv/arxiv-base/commit/8b5f5c404ebd5aa0c11db6b4db967d7290c33948\n",
    "! curl -H \"Accept: application/vnd.github.VERSION.raw\" https://api.github.com/repos/arXiv/arxiv-base/contents/arxiv/taxonomy/definitions.py?ref=8b5f5c404ebd5aa0c11db6b4db967d7290c33948 > definitions.py"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8fb0fa7",
   "metadata": {},
   "source": [
    "Now we have the `definitions.py` file, where the categories are defined as a Python dictionary, and the dictionary can be conveniently imported into this Notebook:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "73a6b327",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('acc-phys',\n",
       "  {'name': 'Accelerator Physics',\n",
       "   'in_archive': 'acc-phys',\n",
       "   'is_active': False,\n",
       "   'is_general': False}),\n",
       " ('adap-org',\n",
       "  {'name': 'Adaptation, Noise, and Self-Organizing Systems',\n",
       "   'in_archive': 'adap-org',\n",
       "   'is_active': False,\n",
       "   'is_general': False}),\n",
       " ('alg-geom',\n",
       "  {'name': 'Algebraic Geometry',\n",
       "   'in_archive': 'alg-geom',\n",
       "   'is_active': False,\n",
       "   'is_general': False})]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from definitions import CATEGORIES\n",
    "\n",
    "\n",
    "# Check out three of the categories:\n",
    "list(CATEGORIES.items())[0:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd67fffe",
   "metadata": {},
   "source": [
    "The categories are identified with codes like `acc-phys`, `adap-org` and `alg-geom`, and they have also more human-readable names. Next we use the categories to construct a [subject vocabulary](https://github.com/NatLibFi/Annif/wiki/Subject-vocabulary-formats) in TSV format.\n",
    "\n",
    "In Annif subject vocabulary the subjects are identified by URIs. As the arXiv taxonomy does not use URIs, we create (dummy) URIs based on the category codes by prepending each category code with `https://arxiv.org/list/`, which makes the URIs act as URLs to pages on arXiv.org website. In the Annif vocabulary format the URIs are surrounded with angle brackets (`<>`). For the labels of the subjects we use the category names. \n",
    "\n",
    "There are [some categories](https://github.com/arXiv/arxiv-base/blob/163c1f9ddb60a017b21ff03190662ea171884530/arxiv/taxonomy/definitions.py#L2100-L2147) apparently used for testing purposes and some have been deprecated (they have field `'is_active': False`), so we omit them from our vocabulary.\n",
    "\n",
    "The vocabulary is saved to the file `arxiv-vocab.tsv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8bd2586a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<https://arxiv.org/list/astro-ph.CO>\tCosmology and Nongalactic Astrophysics\n",
      "<https://arxiv.org/list/astro-ph.EP>\tEarth and Planetary Astrophysics\n",
      "<https://arxiv.org/list/astro-ph.GA>\tAstrophysics of Galaxies\n",
      "<https://arxiv.org/list/astro-ph.HE>\tHigh Energy Astrophysical Phenomena\n",
      "<https://arxiv.org/list/astro-ph.IM>\tInstrumentation and Methods for Astrophysics\n",
      "<https://arxiv.org/list/astro-ph.SR>\tSolar and Stellar Astrophysics\n",
      "<https://arxiv.org/list/cond-mat.dis-nn>\tDisordered Systems and Neural Networks\n",
      "<https://arxiv.org/list/cond-mat.mes-hall>\tMesoscale and Nanoscale Physics\n",
      "<https://arxiv.org/list/cond-mat.mtrl-sci>\tMaterials Science\n",
      "<https://arxiv.org/list/cond-mat.other>\tOther Condensed Matter\n",
      "155\n"
     ]
    }
   ],
   "source": [
    "URI_BASE = 'https://arxiv.org/list/'\n",
    "\n",
    "with open('arxiv-vocab.tsv', 'w', encoding='utf-8') as vocab_file:\n",
    "    for category, data in CATEGORIES.items():\n",
    "        if data['in_archive'] == 'test':\n",
    "            continue  # Skip categories belonging to the 'test' archive\n",
    "        if not data['is_active']:\n",
    "            continue  # Skip deprecated categories\n",
    "        print('<' + URI_BASE + category + '>\\t' + data['name'], file=vocab_file)\n",
    "\n",
    "\n",
    "# We can check the first 10 lines of the file with `head` command:\n",
    "! head arxiv-vocab.tsv\n",
    "# And count lines in the file with `wc -l` command:\n",
    "! wc -l < arxiv-vocab.tsv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa90f792",
   "metadata": {},
   "source": [
    "Now we have a subject vocabulary with 155 subjects for arXiv articles.\n",
    "\n",
    "\n",
    "## Document corpus\n",
    "\n",
    "The actual document corpus still needs to be constructed. With this tutorial is provided a sample (random 100k articles) from an arXiv dataset that is distributed in [Kaggle](https://www.kaggle.com/Cornell-University/arxiv) (you can freely download the full data-set of 1.7M+ articles if you wish). \n",
    "\n",
    "Our data-set is compressed with `gzip`, so the first step is to unpack it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "102cb26c",
   "metadata": {},
   "outputs": [],
   "source": [
    "! gzip -dk arxiv-metadata-oai-snapshot-100k-articles.json.gz"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69b39c73",
   "metadata": {},
   "source": [
    "Now we have the metadata in the file `arxiv-metadata-oai-snapshot-100k-articles.json`. Take a look at the file by printing the first two lines of it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e63bad03",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"id\":\"0711.2512\",\"submitter\":\"Mark Hertzberg\",\"authors\":\"Mark P. Hertzberg (MIT), Shamit Kachru (Stanford), Washington Taylor\\n  (MIT), Max Tegmark (MIT)\",\"title\":\"Inflationary Constraints on Type IIA String Theory\",\"comments\":\"22 pages, 1 figure; v3: Updated to match version published in JHEP,\\n  references added\",\"journal-ref\":\"JHEP 0712:095,2007\",\"doi\":\"10.1088/1126-6708/2007/12/095\",\"report-no\":\"MIT-CTP-3905, SLAC-PUB-12999\",\"categories\":\"hep-th astro-ph gr-qc\",\"license\":\"http://arxiv.org/licenses/nonexclusive-distrib/1.0/\",\"abstract\":\"  We prove that inflation is forbidden in the most well understood class of\\nsemi-realistic type IIA string compactifications: Calabi-Yau compactifications\\nwith only standard NS-NS 3-form flux, R-R fluxes, D6-branes and O6-planes at\\nlarge volume and small string coupling. With these ingredients, the first\\nslow-roll parameter satisfies epsilon >= 27/13 whenever V > 0, ruling out both\\ninflation (including brane/anti-brane inflation) and de Sitter vacua in this\\nlimit. Our proof is based on the dependence of the 4-dimensional potential on\\nthe volume and dilaton moduli in the presence of fluxes and branes. We also\\ndescribe broader classes of IIA models which may include cosmologies with\\ninflation and/or de Sitter vacua. The inclusion of extra ingredients, such as\\nNS 5-branes and geometric or non-geometric NS-NS fluxes, evades the assumptions\\nused in deriving the no-go theorem. We focus on NS 5-branes and outline how\\nsuch ingredients may prove fruitful for cosmology, but we do not provide an\\nexplicit model. We contrast the results of our IIA analysis with the rather\\ndifferent situation in IIB.\\n\",\"versions\":[{\"version\":\"v1\",\"created\":\"Fri, 16 Nov 2007 20:58:20 GMT\"},{\"version\":\"v2\",\"created\":\"Tue, 27 Nov 2007 17:21:48 GMT\"},{\"version\":\"v3\",\"created\":\"Fri, 18 Jul 2008 20:02:22 GMT\"}],\"update_date\":\"2009-10-07\",\"authors_parsed\":[[\"Hertzberg\",\"Mark P.\",\"\",\"MIT\"],[\"Kachru\",\"Shamit\",\"\",\"Stanford\"],[\"Taylor\",\"Washington\",\"\",\"MIT\"],[\"Tegmark\",\"Max\",\"\",\"MIT\"]]}\n",
      "{\"id\":\"1807.03918\",\"submitter\":\"Pamela Harris\",\"authors\":\"Daniel Gotshall, Pamela E. Harris, Dawn Nelson, Maria D. Vega, and\\n  Cameron Voigt\",\"title\":\"Bin Decompositions\",\"comments\":\"13 pages, 1 figures, 1 table\",\"journal-ref\":\"Involve 12 (2019) 503-519\",\"doi\":\"10.2140/involve.2019.12.503\",\"report-no\":null,\"categories\":\"math.CO\",\"license\":\"http://arxiv.org/licenses/nonexclusive-distrib/1.0/\",\"abstract\":\"  It is well known that every positive integer can be expressed as a sum of\\nnonconsecutive Fibonacci numbers provided the Fibonacci numbers satisfy $F_n\\n=F_{n-1}+F_{n-2}$ for $n\\\\geq 3$, $F_1 =1$ and $F_2 =2$. In this paper, for any\\n$n,m\\\\in\\\\mathbb{N}$ we create a sequence called the $(n,m)$-bin sequence with\\nwhich we can define a notion of a legal decomposition for every positive\\ninteger. These sequences are not always positive linear recurrences, which have\\nbeen studied in the literature, yet we prove, that like positive linear\\nrecurrences, these decompositions exist and are unique. Moreover, our main\\nresult proves that the distribution of the number of summands used in the\\n$(n,m)$-bin legal decompositions displays Gaussian behavior.\\n\",\"versions\":[{\"version\":\"v1\",\"created\":\"Wed, 11 Jul 2018 01:28:45 GMT\"}],\"update_date\":\"2019-02-06\",\"authors_parsed\":[[\"Gotshall\",\"Daniel\",\"\"],[\"Harris\",\"Pamela E.\",\"\"],[\"Nelson\",\"Dawn\",\"\"],[\"Vega\",\"Maria D.\",\"\"],[\"Voigt\",\"Cameron\",\"\"]]}\n"
     ]
    }
   ],
   "source": [
    "! head -n 2 arxiv-metadata-oai-snapshot-100k-articles.json"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c64b2f17",
   "metadata": {},
   "source": [
    "The file is in [newline-delimited JSON](http://ndjson.org/) format: each line has one JSON object representing the metadata of an article. Each article will become a document in our corpus.\n",
    "\n",
    "The JSON object for the article metadata has several fields (`id`, `submitter`, `author` etc.), but we are interested in three of them:\n",
    "- `categories` include the category codes which will be mapped to our vocabulary subjects and used as the (gold-standard) subjects of a document,\n",
    "- `title` and `abstract` are used to form the text content of a document.\n",
    "\n",
    "First we construct a Python dictionary, which will be used to map category code to URI:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "1ff66971",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'<https://arxiv.org/list/bayes-an>'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cat2uri = {}\n",
    "\n",
    "for category in CATEGORIES.keys():\n",
    "    cat2uri[category] = '<' + URI_BASE + category + '>'\n",
    "\n",
    "# For example 'bayes-an' catecory code is mapped to '<https://arxiv.org/list/bayes-an>' URI:\n",
    "cat2uri['bayes-an']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b9684cb",
   "metadata": {},
   "source": [
    "The JSON objects can be parsed with Python using `loads()` funtion of the `json` library, so we import the library.\n",
    "\n",
    "As there are newline characters (`\\n`) in the titles and abstracts we also define a simple function for replacing them and all other whitespace characters with spaces (tabulator characters would cause problems in the corpus)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "f2b4da95",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'...chromodynamics is presented for...'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import json\n",
    "\n",
    "\n",
    "def cleanup(text):\n",
    "    return \" \".join(text.split())\n",
    "\n",
    "# For example:\n",
    "cleanup('...chromodynamics is\\npresented for...')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52ad5497",
   "metadata": {},
   "source": [
    "We aim to form not a single corpus, but three separate data-sets:\n",
    "- train set (for training models; 60-70 % of documents)\n",
    "- validation set (for optimizing models hyperparameters; 20-15 % of documents)\n",
    "- test set (for evaluating models performance; 20-15 % of documents)\n",
    "\n",
    "We could just split the articles randomly to these sets, but as there is the \"creation time\" of articles available in the metadata, we perform the split based on it: the oldest articles go to the train set and newest to the test set, and the articles with age in between go to the validation set. This way the evaluation results are on more realistic grounds as the evaluation setting mimics real usage: the documents to use for evaluation are those created only after a model has been trained (including hyperparameter optimization) and deployed in use.\n",
    "\n",
    "For deciding the points in time to use as the data-set boundaries it is useful to know the whole timespan of the creation dates of the articles. We use regular expressions and the `re` library to find the four-digit part (`'\\d\\d\\d\\d'`) from the `created` field (from a datetime string looking e.g. like `\"Tue, 27 Nov 2007 17:21:48 GMT\"`), which is the creation year. The year is stored in a set, and then the first and last year is printed out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "04d1f212",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Earliest: 1990\n",
      "Latest: 2021\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "years = set()\n",
    "\n",
    "with open('arxiv-metadata-oai-snapshot-100k-articles.json', 'r', encoding='utf-8') as input_metadata_file:\n",
    "    for line in input_metadata_file:\n",
    "        article_metadata = json.loads(line)\n",
    "        datetime_str = article_metadata['versions'][0]['created']\n",
    "        year = re.findall('\\d\\d\\d\\d', datetime_str)[0]\n",
    "        years.add(year)\n",
    "\n",
    "print('Earliest: ' + min(years))\n",
    "print('Latest: ' + max(years))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff0f4ac6",
   "metadata": {},
   "source": [
    "The articles we have are created in 1990-2021. At first we can just try some years, e.g. 2010 and 2015, as the boundaries between the three sets, and then refine the boundaries to get the desired ratios of documents for the sets (it is found out that years 2015 and 2018 produce approximate 60-20-20 ratios).\n",
    "\n",
    "\n",
    "Now we are ready to create the corpus!\n",
    "\n",
    "\n",
    "We use [Short text document corpus format](https://github.com/NatLibFi/Annif/wiki/Document-corpus-formats#short-text-document-corpus-tsv-file), which is the same as the subject vocabulary format: \n",
    "a TSV file, where the first column contains the text of the document, and the second column contains a whitespace-separated list of subject URIs for the document.\n",
    "\n",
    "The text of the documents is formed by concatenating the title and abstract. If an article would have an category that is not in our vocabulary, a warning is printed out (`Unknown category: XXX`) and that article is omitted from the data-sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a783a108",
   "metadata": {},
   "outputs": [],
   "source": [
    "max_train_year = 2015  # refined after few tries\n",
    "max_validation_year = 2018  # refined after few tries\n",
    "\n",
    "\n",
    "# Open the JSON metadata file for reading:\n",
    "with open('arxiv-metadata-oai-snapshot-100k-articles.json', 'r', encoding='utf-8') as input_metadata_file:\n",
    "    # Open the TSV data-sets files for writing:\n",
    "    with \\\n",
    "        open('arxiv-train.tsv', 'w', encoding='utf-8') as output_train_file, \\\n",
    "        open('arxiv-validate.tsv', 'w', encoding='utf-8') as output_validate_file, \\\n",
    "        open('arxiv-test.tsv', 'w', encoding='utf-8') as output_test_file:\n",
    "        # Loop line-by-line:\n",
    "        for line in input_metadata_file:\n",
    "            article_metadata = json.loads(line)\n",
    "            text = cleanup(\n",
    "                article_metadata['title'] + article_metadata['abstract'])\n",
    "\n",
    "            # Get the URIs from arXiv categories:\n",
    "            try:\n",
    "                uris = [cat2uri[cat] for cat in article_metadata['categories'].split()]\n",
    "            except KeyError:\n",
    "                print('Unknown category: ' + article_metadata['categories'])\n",
    "                continue\n",
    "\n",
    "            # Get the creation year of article:\n",
    "            datetime_str = article_metadata['versions'][0]['created']\n",
    "            year = int(re.findall('\\d\\d\\d\\d', datetime_str)[0])\n",
    "\n",
    "            # Print to train, validate or test set file:\n",
    "            if year <= max_train_year:\n",
    "                print(text + '\\t' + ' '.join(uris), file=output_train_file)\n",
    "            elif max_train_year < year <= max_validation_year:\n",
    "                print(text + '\\t' + ' '.join(uris), file=output_validate_file)\n",
    "            else:\n",
    "                print(text + '\\t' + ' '.join(uris), file=output_test_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "09cfb67d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "59140 arxiv-train.tsv\n",
      "20097 arxiv-validate.tsv\n",
      "20763 arxiv-test.tsv\n"
     ]
    }
   ],
   "source": [
    "# Check how many documents there are in each corpus file, and refine the data-set time boundaries if needed in the cell above:\n",
    "! wc -l arxiv-train.tsv\n",
    "! wc -l arxiv-validate.tsv\n",
    "! wc -l arxiv-test.tsv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "bbe59ed5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Inflationary Constraints on Type IIA String Theory We prove that inflation is forbidden in the most well understood class of semi-realistic type IIA string compactifications: Calabi-Yau compactifications with only standard NS-NS 3-form flux, R-R fluxes, D6-branes and O6-planes at large volume and small string coupling. With these ingredients, the first slow-roll parameter satisfies epsilon >= 27/13 whenever V > 0, ruling out both inflation (including brane/anti-brane inflation) and de Sitter vacua in this limit. Our proof is based on the dependence of the 4-dimensional potential on the volume and dilaton moduli in the presence of fluxes and branes. We also describe broader classes of IIA models which may include cosmologies with inflation and/or de Sitter vacua. The inclusion of extra ingredients, such as NS 5-branes and geometric or non-geometric NS-NS fluxes, evades the assumptions used in deriving the no-go theorem. We focus on NS 5-branes and outline how such ingredients may prove fruitful for cosmology, but we do not provide an explicit model. We contrast the results of our IIA analysis with the rather different situation in IIB.\t<https://arxiv.org/list/hep-th> <https://arxiv.org/list/astro-ph> <https://arxiv.org/list/gr-qc>\n",
      "An ultra-relativistic outflow from a neutron star accreting gas from a companion Collimated relativistic outflows, or jets, are amongst the most energetic and relativistic phenomena in the Universe. They are associated with supermassive black holes in distant active galactic nuclei (AGN), accreting black holes and neutron stars in binary systems and are believed to be responsible for gamma-ray bursts. The physics of these jets,however, remains something of a mystery in that their bulk velocities, compositions and energetics remain poorly determined. Here we report the discovery of an ultra-relativistic outflow from a binary accreting neutron star accreting gas within a binary system. The velocity of the outflow is comparable to the fastest-moving flows observed from active galactic nuclei, and its strength is modulated by the rate of accretion of material onto the neutron star. Shocks are energized further downstream in the flow, which are themselves moving with mildly relativistic bulk velocities and are the sites of the observed synchrotron emission from the jet. We conclude that the generation of such highly relativistic outflows does not require properties unique to black holes, such as an event horizon.\t<https://arxiv.org/list/astro-ph>\n",
      "Slightly generalized Maxwell classical electrodynamics can be applied to inneratomic phenomena In order to extend the limits of classical theory application in the microworld some weak generalization of Maxwell electrodynamics is suggested. It is shown that slightly generalized classical Maxwell electrodynamics can describe the intraatomic phenomena with the same success as relativistic quantum mechanics can do. Group-theoretical grounds for the description of fermionic states by bosonic system are presented briefly. The advantages of generalized electrodynamics in intraatomic region in comparison with standard Maxwell electrodynamics are demonstrated on testing example of hydrogen atom. We are able to obtain some results which are impossible in the framework of standard Maxwell electrodynamics. The Sommerfeld - Dirac formula for the fine structure of the hydrogen atom spectrum is obtained on the basis of such Maxwell equations without appealing to the Dirac equation. The Bohr postulates and the Lamb shift are proved to be the consequences of the equations under consideration. The relationship of the new model with the Dirac theory is investigated. Possible directions of unification of such electrodynamics with gravity are mentioned.\t<https://arxiv.org/list/hep-th>\n"
     ]
    }
   ],
   "source": [
    "# See how few of the documents in the train-set look like:\n",
    "! head -n 3 arxiv-train.tsv"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}