Source Code of Salience Rank Keyphrase Extraction Algorithm

by Nedelina Teneva, April 15, 2017

Reference

Nedelina Teneva and Weiwei Cheng. Salience Rank: Efficient Keyphrase Extraction with Topic Modeling. The 55th Annual Meeting of the Association for Computational Linguistics (ACL-17). Vancouver, Canada.

Dependencies

nltk 2.0
matplotlib 1.3
networkx 1.10
numpy 1.8

Files

runner.py: executes the main function
ranks.py: implementation of various key phrase extraction algorithms
tagger.py: POS tagging infrastructure
utils.py: various utilities functions
process.py: infrastructure for dataset processing

Directories

data: contains the two standard datasets Inspec (Hulth. 2003. Improved automatic keyword extraction given more linguistic knowledge) and 500N (Marujo et al. 2013. Supervised topical key phrase extraction of news stories using crowdsourcing, light filtering and co-reference normalization).
lda: The TPR and DR algorithms rely on two LDA output files (which can be obtained with any standard LDA implementation).
- Each line of lda-topicsXvocab*.txt contains the topic distribution over the vocabulary for each document (documents are sorted alphabetically by filename).
- Each line of lda-docxXtopics*.txt contains the proportion of each topic for each document (documents are sorted alphabetically by filename).
results: the results for the two datasets are output here after executing runner.py

Running the code

select an algorithm by setting the algorithm variable on line 2 in runner.py
run the code as follows: $ python runner.py