Changelog

tsproisl · Dec 22, 2023 · 59cc625 · 59cc625
1 parent 3eb5247
commit 59cc625
Showing 1 changed file with 7 additions and 1 deletion.
diff --git a/CHANGES.txt b/CHANGES.txt
@@ -2,7 +2,13 @@
 
 ## Version 2.4.0, ???? ##
 
-- 
+- New feature: SoMaJo can output character offsets for tokens,
+  allowing for stand-off tokenization. Pass `character_offsets=True`
+  to the constructor or use the option `--character-offsets` on the
+  command line to enable the feature. The character offsets are
+  determined by aligning the tokenized output with the input,
+  therefore activating the feature incurs a noticeable runtime
+  increase.
 
 ## Version 2.3.1, 2023-09-23 ##