MD013: Unicode character width

[m15a]

Writing texts using non-ASCII characters in markdown would be daily practice worldwide. One, like me, would want to lint markdown files written in non-ASCII characters.

Regarding MD013, the current implementation of markdownlint seems checking character count of each line, based on regular expression. However, character width of unicode characters varies. For example, CJK characters often have double width.

Imagine that you configured MD013 as

$ cat .markdownlint.yml
MD013:
  line_length: 40

and you have example.md:

# An old story

<!-- Line below is of length 56, so it will be warned by markdownlint. -->
Long, long ago there lived, an old man and an old woman.

<!-- Line below has total unicode width 70 but won't be warned as its character counts 35. -->
むかしむかし，あるところに，おじいさんとおばあさんがくらしていました。

It is expected that both lines 4 and 7 are warned by MD013. However,

$ markdownlint example.md
example.md:4:41 MD013/line-length Line length [Expected: 40; Actual: 56]

Related: psf/black#1197

[DavidAnson]

In my mind, rules about line length are meant to ensure that everything fits on the screen or that line lengths are consistent when using a monospaced font. With that understanding, the current implementation of this rule seems correct: it counts the number of visible characters. Trying to base a rule like this on the underlying encoding/representation of the data does not seem generally useful.

Something that complicates things is shown in your example: some characters may render wider then the default monospace character width. However, that is a rendering behavior that will vary by font, program, and operating system - and does not seem like it could be addressed by a rule.

As such, I feel the current implementation is valid.

[m15a]

In my mind, rules about line length are meant to ensure that everything fits on the screen or that line lengths are consistent when using a monospaced font.

Agree, but displayed line lengths are inconsistent when writing CJK text since majority of monospace CJK fonts are actually duospaced.

However, that is a rendering behavior that will vary by font, program, and operating system - and does not seem like it could be addressed by a rule.

Probably I want an option, rather than changing a rule, to customize how to measure line length, based on either the number of characters or unicode width.

[DavidAnson]

Do you know if there is a RegExp character class to identify "wide" characters?

[m15a]

Hmm, no. I searched for and found cjk-regex, which matches CJK characters but not all CJK characters are necessarily double width (e.g., single width ｱｲｳｴｵ and double width アイウエオ).

[DavidAnson]

This part of the Unicode spec seems relevant:

• https://unicode.org/reports/tr11/

As do these packages:

• https://github.com/vangie/east-asian-width
• https://github.com/susisu/meaw

However, I have a strict "no dependencies" rule and I don't see a clean way of referencing this data otherwise.