What is the best way to train spaCy for NER

[cbonadio]

Hello

I am developing a bot for a retailer and I would like to know what is the best way to train spaCy to do Name Entity recognition. The bot will be in the Portuguese language ( I know there is no full support for it ).

With the current implementation of the português language, is it enough to get a reasonable NER result (I will provide all the data for the training) ?

If I want to do ner on the following text “I am looking for a bike” and would like to extract "bike" as an entity labeled “PRODUCT”, should my training data has all the variations on that place for the items that the retailer has, like:

“I am looking for a bike”
“I am looking for a car”
“I am looking for a book”

(bike, car, book) are items on the retailer catalog and should be labeled as “PRODUCT”

or it is better to have more variations like

“I want a bike”
“do you have a car”
“I am searching for a book”

or variations and volume like:

“I want a bike”
“I want a car”
“I want a book”
“do you have a bike”
“do you have a car”
“do you have a book”
“I am searching for a bike”
“I am searching for a car”
“I am searching for a book”

Thanks

[honnibal]

Hi,

I've given a longer reply on a related question in #773 , to start helping people on this in one place.

The question of how best to provide examples to the model speaks to much deeper questions about NLP and ML. There's currently no one-size-fits-all solution to this, and it's not specifically a spaCy issue.

Closing this so we can keep discussion in one place.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.