Accompanying code for our EMNLP 2017 paper providing evaluation scripts and a baseline for the presented corpus. Please use the following citation:
@inproceedings{TUD-CS-2017-0153,
title = {Bringing Structure into Summaries: Crowdsourcing a Benchmark Corpus of Concept Maps},
author = {Falke, Tobias and Gurevych, Iryna},
booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
pages = {(to appear)},
year = {2017},
location = {Copenhagen, Denmark},
website = {https://www.ukp.tu-darmstadt.de/data/summarization/concept-map-summaries},
}
Abstract: Concept maps can be used to concisely represent important information and bring structure into large document collections. Therefore, we study a variant of multi-document summarization that produces summaries in the form of concept maps. However, suitable evaluation datasets for this task are currently missing. To close this gap, we present a newly created corpus of concept maps that summarize heterogeneous collections of web documents on educational topics. It was created using a novel crowdsourcing approach that allows us to efficiently determine important elements in large document collections. We release the corpus along with a baseline system and proposed evaluation protocol to enable further research on this variant of summarization.
The dataset described in the paper can be found here:
Contacts
- Tobias Falke, lastname@aihphes.tu-darmstadt.de
- https://www.ukp.tu-darmstadt.de
- https://www.aiphes.tu-darmstadt.de
Don't hesitate to send us an e-mail or report an issue, if something is broken (and it shouldn't be) or if you have further questions.
This repository contains experimental software and is published for the sole purpose of giving additional background details on the respective publication.
This repository contains two folders:
evalevaluation scripts for the corpusbaselineimplementation of the described baseline system
For details, please refer to the corresponding README files.