Skip to content

Commit

Permalink
Merge #275
Browse files Browse the repository at this point in the history
275: Support markdown formatted codeblocks r=ManyTheFish a=6543

# Pull Request

## Related issue
Fixes go-gitea/gitea#29740 (comment)

## What does this PR do?
add ``` ` ``` as seperator to support markdown formated text

## PR checklist
Please check if your PR fulfills the following requirements:
> Does this PR fix an existing issue, or have you listed the changes applied in the PR description...

well I can file a issue if desired but I think this is a minor change?

- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes
      -> I hope so ;)


Co-authored-by: 6543 <6543@obermui.de>
  • Loading branch information
meili-bors[bot] and 6543 committed Mar 14, 2024
2 parents 460d2a1 + e4fa5fc commit 8bd1936
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion charabia/src/separators.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
/// - Zp Paragraph Separator
/// - Zs Space Separator
/// plus ". ", ", " and ។ល។" (៘ decomposition) to categorize them as hard separators
/// and "`" to understand markdown formatted text
#[rustfmt::skip]
pub const DEFAULT_SEPARATORS: &[&str] = &[
". ", ", ", "_", "‿", "⁀", "⁔", "︳", "︴", "﹍", "﹎", "﹏", "_", "-", "֊", "־", "᐀", "᠆", "‐", "‒", "–",
Expand Down Expand Up @@ -58,7 +59,7 @@ pub const DEFAULT_SEPARATORS: &[&str] = &[
"𑪠", "𑪡", "𑪢", "𑱁", "𑱂", "𑱃", "𑱄", "𑱅", "𑱰", "𑱱", "𑻷", "𑻸", "𑿿", "𒑰", "𒑱", "𒑲", "𒑳", "𒑴", "𖩮",
"𖩯", "𖫵", "𖬷", "𖬸", "𖬹", "𖬺", "𖬻", "𖭄", "𖺗", "𖺘", "𖺙", "𖺚", "𖿢", "𛲟", "𝪇", "𝪈", "𝪉", "𝪊", "𝪋",
"𞥞", "𞥟", "\n", "\r", "\u{2029}", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ",
" ", " "
" ", " ", "`"
];

#[rustfmt::skip]
Expand Down

0 comments on commit 8bd1936

Please sign in to comment.