TwitterNEED
============

We offer three twitter datasets that are mainly designed for named entity extraction (NEE) and disambiguation (NED) tasks.

-The first set is mainly created by and annotated for NEE [1]. We did the annotations for NED. Each NE is assigned to one appropriate entity page. We gave higher priority to Wikipedia pages. If Wikipedia has no page for the entity we link it to a home page or profile page. The dataset composed of four subsets of tweets; one public timeline subset and four subsets of targeted tweets revolves around economic recession, Australian Bushfiresand and gas explosion in Bozeman, MT.

-The second dataset is the one used in [2] which is relatively small in size of tweets but rich in number of NE.

- The third dataset is the training set provided for the #Microposts 2014 challenge [3].

Statistics about the two data sets are shown in the following table:

                      1st Dataset     2nd Dataset     3rd Dataset             

#Tweets                   1603        162             2339                      
#Mentions                 1585        510             3819
#Wiki Entities            1233(78%)   483(94%)        3819 (100%)
#Non-Wiki Entities        274(17%)    19(4%)          0 (0%)
#Mentions with no Entity  78(5%)      8(2%)           0 (0%)

CITATION: If you want to use the 1st dataset for NEE task please cite reference[1]. If you want to use the 2nd dataset for NEE task please cite reference[2]. If you want to use the 3rd dataset you should cite reference[3].

CONTACT: Mena B. Habib (m.b.habib AT ewi.utwente.nl).

References
============
[1] Locke, B. and Martin, J. (2009). Named entity recognition: Adapting to microblogging. Senior Thesis, University of Colorado. 
[2] Habib, M. B. and van Keulen, M. (2012). Unsupervised improvement of named entity extraction in short informal context using disambiguation clues. In Proceedings of the Workshop on Semantic Web and Information Extraction (SWAIE 2012), pages 1–10.
[3] A. E. Cano Basave, G. Rizzo, A. Varga, M. Rowe, M. Stankovic, and A.-S. Dadzie (2014). Making Sense of Microposts (#Microposts2014) Named Entity Extraction & Linking Challenge. In Proceedings of #Microposts2014, pages 54–60, 2014.