Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(markdown/parser): support thematic break block parser #3982

Merged
merged 23 commits into from
Sep 25, 2024

Conversation

suxin2017
Copy link
Contributor

Summary

support thematic break block parser
https://spec.commonmark.org/0.31.2/#thematic-breaks

Test Plan

snap test

@github-actions github-actions bot added A-Parser Area: parser A-Tooling Area: internal tools labels Sep 19, 2024
Copy link

codspeed-hq bot commented Sep 19, 2024

CodSpeed Performance Report

Merging #3982 will degrade performances by 6.85%

Comparing suxin2017:feature_thematic_block (27000a6) with main (ecf1f16)

Summary

❌ 1 regressions
✅ 106 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark main suxin2017:feature_thematic_block Change
dojo_11880045762646467684.js[cached] 7.9 ms 8.5 ms -6.85%

@dyc3 dyc3 self-requested a review September 19, 2024 12:19
Copy link
Contributor

@dyc3 dyc3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a couple of implementation questions.

crates/biome_markdown_parser/src/syntax.rs Show resolved Hide resolved
xtask/codegen/markdown.ungram Outdated Show resolved Hide resolved
xtask/codegen/markdown.ungram Outdated Show resolved Hide resolved
xtask/codegen/markdown.ungram Outdated Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rename of the nodes should be done in a separate PR, because it changes a lot of stuff and even though it's all codegen, it makes the review harder because we can't understand where are the new nodes.

xtask/codegen/markdown.ungram Outdated Show resolved Hide resolved
xtask/codegen/markdown.ungram Outdated Show resolved Hide resolved
@suxin2017
Copy link
Contributor Author

I think I have modified most of the problems. I wrote it again. Could you please help me see it again? thank you @ematipico @dyc3

Copy link
Member

@ematipico ematipico left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are few things to address, yet. Also, could you please address this comment? https://github.com/biomejs/biome/pull/3982/files#r1766848010

crates/biome_markdown_parser/src/lexer/mod.rs Outdated Show resolved Hide resolved
@@ -15,19 +19,54 @@ pub(crate) fn parse_block_list(p: &mut MarkdownParser) {
while !p.at(T![EOF]) {
parse_any_block(p);
}
m.complete(p, MARKDOWN_BLOCK_LIST);
m.complete(p, MD_BLOCK_LIST);
}

pub(crate) fn parse_any_block(p: &mut MarkdownParser) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parsing functions should always return a ParsedSyntax. Not returning a ParsedSyntax is considered a side effect, which we want to avoid

crates/biome_markdown_parser/src/syntax.rs Outdated Show resolved Hide resolved
crates/biome_markdown_parser/src/token_source.rs Outdated Show resolved Hide resolved
@suxin2017
Copy link
Contributor Author

There are few things to address, yet. Also, could you please address this comment? https://github.com/biomejs/biome/pull/3982/files#r1766848010

Sorry for the rename change in this pr. This is because this is the first concrete implementation of the lexer to the parser, so the change is in this pr

crates/biome_markdown_parser/src/lexer/mod.rs Outdated Show resolved Hide resolved
@@ -63,6 +62,30 @@ impl<'source> MarkdownTokenSource<'source> {
}
}
}

/// Returns the number of whitespace characters before the current token until the first new line.
/// tab will be counted as 4 spaces
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a link to the spec that says that? We need to justify certain decisions

@ematipico ematipico merged commit ab0f96e into biomejs:main Sep 25, 2024
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Parser Area: parser A-Tooling Area: internal tools
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants