You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've come across a minor bug in the newly added detokenization routine, where some inputs result in a java.lang.UnsupportedOperationException. Example:
It seems like this could be easily fixed by initialising the list to be output differently. For now, I'm circumventing the problem by always prepending an empty string to the input:
com.twitter.penguin.korean.TwitterKoreanProcessor.detokenize(List("", "이", "제품을", "사용하겠습니다"))
// works
This is neither critical nor urgent, but it would be nice if this could be fixed in some of the future releases of this great library.
The text was updated successfully, but these errors were encountered:
I've come across a minor bug in the newly added detokenization routine, where some inputs result in a
java.lang.UnsupportedOperationException
. Example:It seems like this could be easily fixed by initialising the list to be output differently. For now, I'm circumventing the problem by always prepending an empty string to the input:
This is neither critical nor urgent, but it would be nice if this could be fixed in some of the future releases of this great library.
The text was updated successfully, but these errors were encountered: