Add CRFCut sentence segmentation

[cstorm125]

CRFCut -- Thai sentence segmentation with conditional random field, default trained on TED dataset

• ORCHID - space-correct accuracy 87% vs 95% state-of-the-art (Zhou et al, 2016; https://www.aclweb.org/anthology/C16-1031.pdf)
• TED dataset - space-correct accuracy 82%

See development notebooks at https://github.com/vistec-AI/ted_crawler;
POS features are not used due to unreliable POS tagging available

[pep8speaks]

Hello @cstorm125! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2019-12-20 17:27:11 UTC

[bact]

1. python-crfsuite should be included in setup.py's install_requires (and maybe in requirements.txt as well, see next point)

• The code currently passed the test only because in the test the python-crfsuite was installed as dependencies of attacut - but in real settings, user may not install the optional attacut.

2. How big is the model? If it's not that big I would suggest to include it in the module, so we will less dependent on the network during runtime.

• This will make it possible to have crfcut included as a pythainlp core, as a default engine for sentence segmentation.
• If this is really the case, python-crfsuite should be in requirements.txt too.
• The model file should be included in setup.py's package_data
• Once the model included, _download() function should be removed.

[cstorm125]

@bact The model is 5MB so I agree we can include it as battery. Is everyone okay with having some model files in the library? @artificiala @wannaphongcom

[bact]

For reference on model size, see #298

I think 5 MB is ok.

[bact]

I have add few words to STARTERS and ENDERS. May require retrain and reupload of the model.
Apart of that, I think we're good to go.

Please also kindly update the table here: #298 thx

Great work! Another step towards full pipeline.

[wannaphong]

💯

[bact]

TODO: Next step is to convert https://github.com/vistec-AI/ted_crawler/blob/master/sentenceseg_ted.ipynb to a commandline script and maybe put it in bin/ directory. So people can train their own model. Follow up on #344.