2_zero_shot.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c9b10ed5",
   "metadata": {},
   "source": [
    "# Part 2 - Zero-shot summaries\n",
    "\n",
    "In this part we will use Hugging Face's high-level Pipeline API to create summaries with a pre-trained model. There are three main steps involved when you pass some text to a pipeline:\n",
    "\n",
    "1) The text is preprocessed into a format the model can understand.\n",
    "\n",
    "2) The preprocessed inputs are passed to the model.\n",
    "\n",
    "3) The predictions of the model are post-processed, so you can make sense of them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b15df7cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import pipeline\n",
    "summarizer = pipeline(\"summarization\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a83d1ef7",
   "metadata": {},
   "source": [
    "This line of code allows us to see which model is being used by default. We can also find this information in the source code for pipelines:https://github.com/huggingface/transformers/blob/master/src/transformers/pipelines/__init__.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23c9993c",
   "metadata": {},
   "outputs": [],
   "source": [
    "summarizer.model.config.__getattribute__('_name_or_path')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22c21726",
   "metadata": {},
   "source": [
    "The model for the standard summarisation task is https://huggingface.co/sshleifer/distilbart-cnn-12-6, which has been specifically trained on 2 datasets: https://huggingface.co/datasets/cnn_dailymail and https://huggingface.co/datasets/xsum. We will keep using this model, but if we wanted to use a different model we could easily do this by specifing it like below. All the models that are trained for summarisation can be viewed here: https://huggingface.co/models?pipeline_tag=summarization&sort=downloads"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7e68038a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# summarizer = pipeline(\"summarization\", model='facebook/bart-large-cnn')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a40b298",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "df_test = pd.read_csv('data/test.csv')\n",
    "ref_summaries = list(df_test['summary'])\n",
    "texts = list(df_test['text'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc122d7e",
   "metadata": {},
   "source": [
    "Testing the pipeline with an abstract from the test dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3b18f9db",
   "metadata": {},
   "outputs": [],
   "source": [
    "texts[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "758aabb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "summarizer(texts[0], max_length=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1e4ba28",
   "metadata": {},
   "source": [
    "Running the pipeline over all 2,000 examples. Because this will take a while we print a counter to keep track of the progress. This should take around 50 minutes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "187429e7",
   "metadata": {},
   "outputs": [],
   "source": [
    "candidate_summaries = []\n",
    "\n",
    "for i, text in enumerate(texts):\n",
    "    if i % 100 == 0:\n",
    "        print(i)\n",
    "    candidate = summarizer(text, min_length=5, max_length=20)\n",
    "    candidate_summaries.append(candidate[0]['summary_text'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c569312",
   "metadata": {},
   "source": [
    "Saving the candidate summaries in case we want to investigate further."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e607d234",
   "metadata": {},
   "outputs": [],
   "source": [
    "file = open(\"summaries/zero-shot-summaries.txt\", \"w\")\n",
    "for s in candidate_summaries:\n",
    "    file.write(s + \"\\n\")\n",
    "file.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d4613a38",
   "metadata": {},
   "outputs": [],
   "source": [
    "candidate_summaries[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea2f8780",
   "metadata": {},
   "source": [
    "Calculating the ROUGE scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f225268c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from datasets import load_metric\n",
    "metric = load_metric(\"rouge\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1f849be6",
   "metadata": {},
   "outputs": [],
   "source": [
    "def calc_rouge_scores(candidates, references):\n",
    "    result = metric.compute(predictions=candidates, references=references, use_stemmer=True)\n",
    "    result = {key: round(value.mid.fmeasure * 100, 1) for key, value in result.items()}\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7bd7585d",
   "metadata": {},
   "outputs": [],
   "source": [
    "calc_rouge_scores(candidate_summaries, ref_summaries)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32aa2755",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_pytorch_latest_p37",
   "language": "python",
   "name": "conda_pytorch_latest_p37"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}