Thai-NNER (Thai Nested Named Entity Recognition Corpus)

This repository contains the code associated with the paper Thai Nested Named Entity Recognition Corpus presented at ACL2022(findings).

Abstract / Motivation

This work presents the first Thai Nested Named Entity Recognition (N-NER) dataset. Thai N-NER consists of 264,798 mentions, 104 classes, and a maximum depth of 8 layers obtained from news articles and restaurant reviews, a total of 4894 documents. Our work, to the best of our knowledge, presents the largest non-English N-NER dataset and the first non-English one with fine-grained classes.

How to use?

Installation

To get started, install the library:

pip install thai_nner

Model Preparation

First, download the necessary resources (models, datasets, and pre-trained language models) from here and use the convert_model2use.py script to prepare it.

python convert_model2use.py -i 0906_214036/checkpoint.pth -o model.pth

Sample Usage

import os
os.environ['CUDA_VISIBLE_DEVICES'] = "0"
from thai_nner import NNER

nner = NNER("model.pth")
tags = nner.get_tag("วันนี้วันที่ 5 เมษายน 2565 เป็นวันที่อากาศดีมาก")
print(tags)

Examples & Testing

• Python Library Demo
• Functionality Testing

Training and Testing

Train

python train.py --device 0,1 -c config.json

Test

python test_nne.py --resume [PATH]/checkpoint.pth

Tensorboard

tensorboard --logdir [PATH]/save/log/

Results

Citation

If you find our work useful, please consider citing:

@inproceedings{buaphet-etal-2022-thai,
    title = "{T}hai Nested Named Entity Recognition Corpus",
    author = "Buaphet, Weerayut  and
      Udomcharoenchaikit, Can  and
      Limkonchotiwat, Peerat  and
      Rutherford, Attapol  and
      Nutanong, Sarana",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2022",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.findings-acl.116",
    doi = "10.18653/v1/2022.findings-acl.116",
    pages = "1473--1486",
    abstract = "",
}

License

The project is licensed under CC-BY-SA 3.0.

Acknowledgements

• Dataset Credits: The Thai N-NER corpus owes its inception partly to the Digital Economy Promotion Agency (depa) Digital Infrastructure Fund MP-62-003 and the Siam Commercial Bank. The dataset is named as scb-nner-th-2022.
• Training Code Inspiration: Adapted from Tensorflow-Project-Template by Mahmoud Gemy.