fix(transformers): Ignore empty lines on notation transformer range #755
Conversation
❌ Deploy Preview for shiki-matsu failed.
|
❌ Deploy Preview for shiki-next failed.
|
|
Also I realized this is a good opportunity to improve the current behaviors of notation transformers. Like: Normal console.log("hello") // [!code xxx] original commentMultiple transformers console.log("hello") // [!code highlight] [!code world:hello] original comment(maybe we can support the previous impact next line if the current line is empty (this PR) // [!code highlight] [!code world:hello]
console.log("hello") // original commentkeep current line // [!code highlight] [!code current]
console.log("hello") // [!code highlight]
it requires some extra work but won't cause a break change except the current proposal of "Ignoring empty lines" |
|
@antfu I'm fixing some edge cases and I found one of the test cases: function hello(indentSize, type) {
if (indentSize === 4 && type !== 'tab') {
console.log('Each next indentation will increase on 4 spaces'); // [!code error] // [!code focus]
}
}gives the mismatch:
I thought it was the problem of my changes, but it seems like the original snapshot was wrong. I'll commit the changes for now (it's probably a breaking change, although docs didn't mention that explicitly) |
| /** | ||
| * Regex that matches code comments | ||
| */ | ||
| const regex = /((?:\/\/|\/\*|<!--|[#"']|--|%%|;{1,2}|%{1,2})\s*)(\S.*?)(-->|\*\/|$)/ |
There was a problem hiding this comment.
I am not sure if we should hard-code the regex, this makes the code less flexiable
There was a problem hiding this comment.
Hmm do you mean to allow passing the entire regex, including the part for matching comments? (like the previous code)
I assume createCommentNotationTransformer is for matching comment notations, it sounds reasonable to me to add the regex for matching comment inside the utility. I can revert it if you wanted
| const isEmptyLine = line.children.length === 1 | ||
| text.value = text.value.replace(regex, (_, prefix, text, end) => { | ||
| // no other tokens except the comment | ||
| const replaced = onMatch.call(this, text, line, last, lines, |
There was a problem hiding this comment.
If the second arg text is coming from the match, why do we need to change it from match to text? Doesn't it make the utils less flexiable?
Instead of making a new function as breaking changes, couldn't we just allow onMatch to return a string instead of boolean?
There was a problem hiding this comment.
the new match here refers to the entire comment (// [!code] highlight), but we also want to remove the comment if the comment content becomes empty, e.g. // .
so we pass the text instead ([!code] highlight), the transformer replaces comment content, which allows us to detect if the new text is empty. This can also improve the flexibility, and remove duplicated regex for matching code comments.
As you see, the createCommentNotationTransformer now has the algorithm for matching comments. We can improve it to support other comment syntaxes.
For the problem about breaking change, I think we could also create a wrapper over the original function, would like to hear your opinions about this.
|
I'm doing some experiments on notation transformers, and I found it's possible to redesign the current algorithm of notation transformers to support JSX comments e.g. #770, and optimize performance by reading only the last few tokens of each line. I putted these changes here, but this introduces breaking change as some backward incompatible changes are made, including "Ignore empty lines on notation transformer range" (the aim of this PR). I'm thinking to port the changes to a PR, if you're up to it. |
|
Feel free to always create the PR (if you already work on it), so we can discuss about the implementations. Sorry I was taking a long day offs, need to catch up and review a lot things |
|
Thanks, I'll open a better PR later with more details and explanations. Appreciate your hard work |
Description
Start counting from next line of the comment if the current line is empty.
For example:
Linked Issues
#619
Additional context
I made some changes to the design of
createCommentNotationTransformerso it matches the comment token without a given regex, I've documented it in case for other contributors who doesn't understand it well.note: forgot to reset local repo after the last PR was merged, apologies for the messy commit history.