slr-kit is a tool that is intended to assist in the development of a Systematic Literature Review (SLR) of scientific papers related to a specific scientific field.
The tool is based on a semi-supervised workflow that leverages automatic techniques based on text mining to automatize key steps in the workflow, while some manual operations are necessary to provide the expert knowledge to the process. slr-kit aims at making the manual operations as efficient as possible.
It analyzes the abstracts of scientific papers to derive the topics associated to the papers, allowing the clustering of papers by topic and the derivation of useful statistics about the set of papers and their topics, such as the trend of publications per topic during the years, and the amount of papers published on different journals by topic.
As a byproduct, slr-kit can be used to manage a list of terms related on a specific topic, curated by an expert, which can be useful in many NLP applications.
- Installation
- Usage
- The workflow on data from scratch (basic version)
- Using existing data
- Links to other docs
slrkit is available on PyPi and can thus be installed as
pip install slrkit
The documentation below refers to the installed program though pip.
Clone the repository:
git clone <repository address>
Install the dependencies:
pip install -r requirements.txt
In this case, the command can be run, from within the repository directory, as
python3 slrkit/slrkit.py
After installing slrkit and its dependencies, the corpora for NLTK must be downloaded.
The following command can be used:
python -m nltk.downloader popular
If slr-kit was installed from the git repository, you can launch the slrkit command.
Use the -h or --help option to generate the following help:
$ slrkit --help
usage: slrkit [-h] [-C path] [--version]
{init,import,journals,acronyms,preprocess,terms,fawoc,postprocess,topics,report,record,stopwords,build,readme}
...
slrkit project handling tool
options:
-h, --help show this help message and exit
-C path Change directory to 'path' before running the specified command.
--version, -V show program's version number and exit
slrkit commands:
{init,import,journals,acronyms,preprocess,terms,fawoc,postprocess,topics,report,record,stopwords,build,readme}
init Initialize a slr-kit project
import Import a bibliographic database converting to the csv format used by slr-kit.
journals Subcommand to extract and filter a list of journals. This command accepts a
subcommand. If none is given, "extract" is assumed.
acronyms Extract acronyms from texts.
preprocess Run the preprocess stage in a slr-kit project
terms Subcommand to extract and handle lists of terms in a slr-kit project. This
command accepts a subcommand. If none is given, "generate" is assumed.
fawoc Run fawoc in a slr-kit project. This command accepts a subcommand. If none is
given "terms" is assumed.
postprocess Run the postprocess stage in a slr-kit project
topics Subcommand to extract the topics from the documents in a slrkit project. This
command requires a subcommand.
report Run the report creation script in a slr-kit project.
record Record a snapshot of the project in the underlying git repository
stopwords Extracts the terms classified as stopwords from the terms file
build Run all the commands needed to re-create all the files after cloning a slr-kit
project.
readme Creates a README.md file for the project using information from the META.toml
file. The README.md is also committed to the git repository.
Each sub-command is explained in details the slrkit.md documentation file.
The organization of the information is based on projects.
A project is basically a directory containing all the necessary data related to the SLR.
Such data are manipulated using the slrkit commands.
All the data, being final or intermediate results, are stored in easy to inspect CSV files.
The workflow supported by slr-kit includes several optional steps that can be done to refine the final results.
Here it is reported the mandatory steps to obtain the desired reports starting from the bibliographic material.
The basic steps are:
- initialization of the project (automatic)
- import of bibliographic data (automatic)
- preprocessing of the abstracts (automatic)
- extraction of terms (automatic)
- classification of terms using FAWOC (manual)
- topics extraction (automatic)
- generation of reports (automatic)
Anytime during the various steps, you can run the command
slrkit record
that will produce a snapshot of the current information required by an slr-kit project. Note that the snapshots are maintained as standard git repositories. Therefore, you can use the regular git commands to operate on the repository (git log, git status, git checkout, etc.). Moreover, you can use the usual git push/pull to move the dataset to/from remotes.
Create a new directory to host the slr-kit project, and move into the newly created directory (we use my-project as example project):
mkdir my-project && cd my-project
Initialize the project:
slrkit init my-project
At the moment, only the RIS file format is supported as source data file. The RIS format is one of the many available on, e.g., Scopus. Make sure that the file contains the Abstract of the papers.
Copy the bibliographic data into the project directory, naming it my-project.ris.
To make slrkit aware of the name of the bibliographic data, set the value of the input_file parameter into the file configuration file slr-kit.conf/import.toml (this file was generated by the init command) as follows:
input_file = "my-project.ris"
To import the data into the project, run the command:
slrkit import
This operation performs several text processing operations on the abstracts, such as stemming and removal of punctuation, preparing the text for further processing.
The command to run the preprocessing is:
slrkit preprocess
Extract the list of terms from the abstracts:
slrkit terms
Run FAWOC and manually classify the terms:
slrkit fawoc terms
FAWOC is an interactive terminal program that allows to quickly (well, some time is required depending on the amount of terms) classify the terms.
For this basic example, you can distinguish the terms between RELEVANT (pressing r) and NOISE (press n) terms.
If you make some mistake, press u to undo.
This operation processes the documents by managing the terms classified in the previous stage. Run it with
slrkit postprocess
Run the text mining procedure with the command to generate the topics:
slrkit topics
Alternatively, you may want to run an optimization in the generation of the topics with:
slrkit topics optimize
This operation takes significantly more time, but produces results that are far more accurate.
HINT: Use the optimization everytime a "production ready" result is required.
Generate the reports with:
slrkit report
The reports are stored in a newly generated unique directory whose name has the format <timestamp>_<UUID>_reports/.
In the directory, you will find some .md, .tex and .png files that report the graphs and tables with the results.
The reports are generated according to some templates in Markdown and LaTeX format for being easily edited or directly included in scientific papers or websites.
slr-kit can also be used on an existing dataset, i.e., a set of papers that were collected by someone else and already (even partially) elaborated.
In this example, we assume that the dataset was properly recorded using slrkit record, and all the terms have been already classified.
A dataset is kept versioned using git. Therefore, if the dataset is shared in some online repository, you can simply download it with
git clone <repository>
A repository containing the classification of a dataset about Natural Language Processing is available here: https://github.com/robolab-pavia/slrkit_NLP_Nocera.
First, you need to re-build all the preprocessed data. This is done with one single command:
slrkit build
At this point, if necessary, you can refine the classification of terms by running the usual command
slrkit fawoc terms
Now you are ready to generate the topics and the reports with the same commands explained above:
slrkit topics optimize
followed by
slrkit report
- documentation of the commands of
slr-kit - documentation of the scripts
- FAWOC
- utilities and analysis tools: TBA