Summarize

Written by: Ling Li Ya

This is a text summarizing tool written in Python.

User Guide

Some things to note:

• The application is accessible here.
• However, due to limited free-tier server resources, the application may crash, so it is advisable that you run this project locally.
• You might not be able to run the abstractive models after reaching a character limit in HuggingFace Accelerated Inference API. Therefore, it is advisable that you use the Notebooks for replicating our results in the documentation.
• Note that you might not be able to run Pegasus on the notebook successfully due to the amount of resources required, so it is advisable that you run only the Pegasus model through the application interface.

To run the project locally, please refer to the guide below.

Setup Tutorial Video (Windows)

SummarizeLocalSetup.mp4

for the detailed steps in word, refer to sections below

1. Downloading the project

Either download the .zip file in Google Classroom from our GitHub.

Then unzip the .zip file. You will see the file summarize-main.

2. Install prerequisites

You need Python and Node.js installed. Open up command prompt (cmd) and type in the code below.

To check whether Python is installed:

$ python

You will see this is it is installed. Note that your version might be different.

Type exit() to exit the Python shell if it is installed.

To check whether Node.js is installed:

$ node

You will see this is it is installed. Note that your version might be different.

Otherwise, download Python and/or Node.js here. Run the installer and follow its instructions. Verify your installation.

3. Install project Python dependencies

Double click on summarize-main. Single click on the summarize folder, hold down your shift key, and right click on the folder. Select Open PowerShell window here.

A PowerShell window will pop up. Then right click on the Makefile in the file explorer and open it with Notepad.

Something like this will pop up:

These are the commands to install all the project Python dependencies. Simply copy the command and paste them in the PowerShell window. If you encounter this warning message:

Simply retype the command with an additional flag pip install -r requirements.txt --use-feature-in-tree-build. Then let it run.

4. Install our summarize library

We have made our application into a Python library and you need to install it with the command below:

5. Run the backend server

Be sure that you select the command under the server-dev instead of server-prod.

6. Prepare the frontend client

Open up another PowerShell window this time by holding shift and right clicking the server folder.

After you have installed Node.js, run the following command to install pnpm.

$ npm install -g pnpm

After installing pnpm, type cd client to go into the client folder in the new PowerShell window.

Then return to your Notepad and run the command pnpm i in the PowerShell window. It will take 10 - 20 seconds to install.

7. Run the frontend client

Run this command in the PowerShell window to launch the application on localhost:3333

You will see this:

8. Adding API token

To use BART, T5 and Pegasus, you need an API token. We will private message you an API token because it is not supposed to be public.

At the summarize-main project root, right click on an empty space to add a new .txt named .env.

Click on yes for this warning:

Open the .env file in Notepad. Type in HUGGING_FACE_API_TOKEN_={your_api_token}. It will look something like this:

Save the file then refresh the Summarize web application page.

You will be able to use the models now.

Code folders

• summarize - The python library for all the algorithm
• server - The backend server using FastAPI
• client - The frontend app using Vue3

Misc folders

• notebooks - A folder to keep all our jupyter notebooks testground
• data - A folder to keep all datasets needed to train or test the algorithm
• docs - Keep our documentation files