cleaners always return [everyvoice.utils.lower, everyvoice.utils.collapse_whitespace, everyvoice.utils.nfc_normalize] regardless of user selection

[MENGZHEGENG]

Regardless of user's selection in wizard,
thecleaners in everyvoice-shared-text.yaml always return
[everyvoice.utils.lower, everyvoice.utils.collapse_whitespace, everyvoice.utils.nfc_normalize]

[MENGZHEGENG]

This is particularly a problem for SENĆOŦEN, as F is part of it ipa representation, and automatically applying everyvoice.utils.lower will lead to issues.

[roedoejet]

@marctessier recently found that this error caused training to fail entirely (@marctessier could you paste the error that you get here?). It would be good if we could catch these types of errors and return a helpful error message.

[marctessier]

This cause training STR to fail with the attached message when running EV stock. Also a core dump file is created.

BUT like mentioned If we remove everyvoice.utils.lower from the list of cleaners before running preprocessing . Training will work :-) !

ex: everyvoice-shared-text.yaml:cleaners: [everyvoice.utils.collapse_whitespace, everyvoice.utils.nfc_normalize]

STR_STOCK.e2205168.txt