Add Left and Right Pointing Angle Brackets as punctuation to ancient …

…Greek (#12829) * Update universe.json * Update universe.json add some missing commas in the greCy's description. * Update punctuation.py Add mathematical left and right angle brackets as punctuation for ancient Greek for better tokenization.
explosion · Jul 20, 2023 · 4f8daa4 · 4f8daa4
1 parent ea54d17
commit 4f8daa4
Showing 1 changed file with 2 additions and 0 deletions.
diff --git a/spacy/lang/grc/punctuation.py b/spacy/lang/grc/punctuation.py
@@ -15,6 +15,7 @@
  [
  "†",
  "⸏",
+ "〈",
  ]
  + LIST_PUNCT
  + LIST_ELLIPSES
@@ -31,6 +32,7 @@
  + [
  "†",
  "⸎",
+ "〉",
  r"(?<=[\u1F00-\u1FFF\u0370-\u03FF])[\-\.⸏]",
  ]
 )