refactor(post/tag): render tag before content #4171

SukkaW · 2020-03-04T08:01:54Z

What does it do?

Continuation of #3543 (comment) and #4161

Currently post rendering process is something like this: #4161 (comment). But basically, hexo will render markdown first, then the tag (using nunjucks). So hexo has to remove all swig tags before markdown rendering, and restore them after markdown rendering is finished.

In this PR, I make hexo renders tag first, then markdown, so that remove and restore swig tag are not necessary any more, resulted in 6% faster in generation speed. And the post rendering related test cases are passed as well.

I wonder why Hexo has to render tag after markdown at the beginning.

How to test

git clone -b render-tag-before-md https://github.com/sukkaw/hexo.git
cd hexo
npm install
npm test

Screenshots

Pull request tasks

Add test cases for the changes.
Passed the CI test.

Issue resolved: #2821

lib/hexo/post.js

SukkaW · 2020-03-04T08:16:14Z

I am considering fixing #3346 in this PR as well since that issue is also related with post rendering process.

Update:

@stevenjoezhang I have added a test case for #3346. It seems that issue has been fixed.

stevenjoezhang · 2020-03-04T09:25:06Z

{% pullquote warning %}
Text
{% endpullquote %}
# Title 0
{% pullquote warning %}
Text
{% endpullquote %}

{% pullquote warning %}
Text
{% endpullquote %}

# Title 1

{% pullquote warning %}
Text
{% endpullquote %}

{% pullquote warning %}Text{% endpullquote %}
# Title 2
{% pullquote warning %}Text{% endpullquote %}

{% pullquote warning %}Text{% endpullquote %}

# Title 3

{% pullquote warning %}Text{% endpullquote %}

The bug reported in #4161 was reproduced. I am confirming if this is a bug (or feature) of Nunjucks
https://github.com/mozilla/nunjucks/blob/40dfdf0ea3e6ba43c3a180eca57d935c36d188c2/nunjucks/src/parser.js#L640-L649

lib/plugins/filter/before_post_render/backtick_code_block.js

seaoak · 2020-03-05T04:49:27Z

Thank you for this great patch.

I wonder why Hexo has to render tag after markdown at the beginning.

Me, too. 😉

However, since #4171 (comment) is still there, we have to find out a way to solve this.

Generally speaking, in Markdown, any block content is separated by empty line(s) each other.

And custom tags in Nunjucks is not intended for Markdown block content only.
So Nunjucks does not generate additional newline characters by default.

That is,

{% pullquote warning %}
Text
{% endpullquote %}
# Title 0
{% pullquote warning %}
Text
{% endpullquote %}

should be written as folows:

{% pullquote warning %}
Text
{% endpullquote %}

# Title 0

{% pullquote warning %}
Text
{% endpullquote %}

If we want to add a newline character automatically for Nunjucks custom tags in Hexo,
the codes in lib/extend/tag.js should be modified as follows:

diff --git a/lib/extend/tag.js b/lib/extend/tag.js
index 31f9f2aa..883fdb5f 100644
--- a/lib/extend/tag.js
+++ b/lib/extend/tag.js
@@ -77,7 +77,7 @@ class NunjucksBlock extends NunjucksTag {
   }

   run(context, args, body) {
-    return this._run(context, args, trimBody(body));
+    return this._run(context, args, trimBody(body)) + '\n';
   }
 }

@@ -111,7 +111,7 @@ class NunjucksAsyncBlock extends NunjucksBlock {
       body = () => result || '';

       this._run(context, args, trimBody(body)).then(result => {
-        callback(err, result);
+        callback(err, result && `${result}\n`);
       });
     });
   }

I'm not sure whether this behavior is desirable in Hexo.
Even so, this modification should be independent of this patch.

- Fix swig tag inside markdown content

SukkaW · 2020-03-05T09:30:57Z

Even so, this modification should be independent of this patch.

@segayuu

IMHO, it is fine to include this patch inside this PR since it is related with post rendering issue fix.

Update:

@stevenjoezhang

I have applied the patch provided by @seaoak and it seems that #4171 (comment) has been fixed.
Related test cases is added as well.

'{' & '}' already escaped in backtick code filter

SukkaW · 2020-03-06T11:32:45Z

lib/extend/tag.js

-    const cache = [];
-
-    const escapeContent = str => `<!--${placeholder}${cache.push(str) - 1}-->`;
-
-    str = str.replace(/<pre><code.*>[\s\S]*?<\/code><\/pre>/gm, escapeContent);


Hexo only uses tag.render in post.js, which is after backtick_code_filter is executed.

In backtick_code_filter, { & } are already escape, so it is fine to render with nunjucks directly.

@seaoak

Apply suggestions from code review by @seaoak

SukkaW · 2020-03-07T09:52:24Z

This line will be simpler because result must be a string.

@seaoak Updated.

jiangtj · 2020-03-09T09:13:50Z

lib/extend/tag.js

@@ -77,7 +75,7 @@ class NunjucksBlock extends NunjucksTag {
  }

  run(context, args, body) {
-    return this._run(context, args, trimBody(body));
+    return this._run(context, args, trimBody(body)) + '\n';


Why add \n?

Look at the following test cases, which one is expected?

const str = [ 'begin', '{% test foo bar %}', 'test content', '{% endtest %}', 'end' ].join('\n'); const result = await tag.render(str); // there is only one `\n` before and after result.should.eql('begin\nfoo bar test content\nend'); // Or should there be two `\n` at the end? result.should.eql('begin\nfoo bar test content\n\nend');

This extra \n is to separete a Nunjucks tag from immediate-after content.
So I expect two \n at the end.

Please refer the above comment #4171 (comment)

stevenjoezhang · 2020-03-09T10:39:11Z

Another issue: #4087
Indented code blocks will be parsed and rendered by Nunjucks.

SukkaW · 2020-03-10T07:29:36Z

@stevenjoezhang

For indented code blocks there is no better ways. For backtick code blocks we already have a filter.

Maybe a indented code filter? Any line started with 4 spaced or \t should be wrapped in <pre><code>?

seaoak · 2020-03-10T10:24:19Z

@SukkaW @stevenjoezhang

To escape Nunjucks tags in indented code blocks properly,
we might need a new before_post_render filter (same as backtick code blocks)

I try to make a patch of such a filter:
seaoak/hexo:feature/escape_indented_code_block_before_render
https://github.com/seaoak/hexo/tree/feature/escape_indented_code_block_before_render

With merging it, this @SukkaW 's patch resolves the issue #4087.

Do you think about this new filter?
If it is useful, I continue to try to make it. 😄

SukkaW · 2020-03-10T10:31:21Z

To escape Nunjucks tags in indented code blocks properly,
we might need a new before_post_render filter (same as backtick code blocks)

I'd rather not to do that.

Only escape { and } which is inside code block, and after code block is highlighted.

Also, you can draft a PR (without actually creating it) from your branch.

SukkaW · 2020-03-24T02:42:44Z

@seaoak

From #4179 we might consider it is necessary for hexo to render markdown first, right?

seaoak · 2020-03-25T07:50:35Z

@SukkaW Yes. I think so.

SukkaW · 2020-03-25T08:58:31Z

@seaoak So we should then focus on how to make swig escaping quicker.

refactor(post): render tag before content

2ef9c9b

SukkaW requested review from segayuu, jiangtj, stevenjoezhang, curbengh and a team March 4, 2020 08:01

SukkaW mentioned this pull request Mar 4, 2020

{% %} should not be rendered in a post #3346

Closed

SukkaW commented Mar 4, 2020

View reviewed changes

lib/hexo/post.js Outdated Show resolved Hide resolved

stevenjoezhang reviewed Mar 4, 2020

View reviewed changes

lib/hexo/post.js Show resolved Hide resolved

test(post): add cases for issue hexojs#3346

32d5638

SukkaW requested a review from stevenjoezhang March 4, 2020 08:35

SukkaW force-pushed the render-tag-before-md branch from 30fb8fd to c21e2d0 Compare March 4, 2020 09:23

SukkaW commented Mar 4, 2020

View reviewed changes

lib/plugins/filter/before_post_render/backtick_code_block.js Outdated Show resolved Hide resolved

SukkaW added 2 commits March 5, 2020 17:27

refactor(post): simplify the <escape> regex pattern

54d45e3

refactor(tag): add extra blankline in tag.run()

1b0819a

- Fix swig tag inside markdown content

SukkaW force-pushed the render-tag-before-md branch from c21e2d0 to 1b0819a Compare March 5, 2020 09:29

SukkaW requested a review from seaoak March 5, 2020 09:31

SukkaW changed the title ~~refactor(post): render tag before content~~ refactor(post/tag): render tag before content Mar 5, 2020

SukkaW force-pushed the render-tag-before-md branch 2 times, most recently from b770791 to 1b0819a Compare March 5, 2020 09:43

stevenjoezhang previously approved these changes Mar 6, 2020

View reviewed changes

refactor(tag): remove escape

1bc7e78

'{' & '}' already escaped in backtick code filter

SukkaW dismissed stevenjoezhang’s stale review via 1bc7e78 March 6, 2020 11:21

SukkaW requested a review from stevenjoezhang March 6, 2020 11:21

SukkaW commented Mar 6, 2020

View reviewed changes

This comment has been minimized.

Sign in to view

fix(tag): simplify error handle

3bdf42d

Apply suggestions from code review by @seaoak

jiangtj reviewed Mar 9, 2020

View reviewed changes

seaoak mentioned this pull request Mar 11, 2020

Escape indented code block before render #4179

Closed

2 tasks

stevenjoezhang mentioned this pull request Mar 11, 2020

label 放在列表首部会导致后面内容不渲染 theme-next/hexo-theme-next#1407

Open

3 tasks

SukkaW mentioned this pull request Apr 11, 2020

test(post): adding extra test cases #4238

Merged

2 tasks

SukkaW mentioned this pull request Apr 21, 2020

perf(post): simplify codeblock escape #4254

Merged

2 tasks

SukkaW closed this May 31, 2020

SukkaW mentioned this pull request Aug 11, 2020

fix(#4460): refactor post escape #4472

Merged

2 tasks

ChrAlpha mentioned this pull request Aug 18, 2020

unespected hexoPostRenderEscape appeared #4481

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(post/tag): render tag before content #4171

refactor(post/tag): render tag before content #4171

SukkaW commented Mar 4, 2020 •

edited by stevenjoezhang

Loading

SukkaW commented Mar 4, 2020 •

edited

Loading

stevenjoezhang commented Mar 4, 2020

seaoak commented Mar 5, 2020 •

edited

Loading

SukkaW commented Mar 5, 2020 •

edited

Loading

SukkaW Mar 6, 2020 •

edited

Loading

This comment has been minimized.

SukkaW commented Mar 7, 2020 •

edited

Loading

jiangtj Mar 9, 2020

seaoak Mar 11, 2020

stevenjoezhang commented Mar 9, 2020

SukkaW commented Mar 10, 2020 •

edited

Loading

seaoak commented Mar 10, 2020

SukkaW commented Mar 10, 2020 •

edited

Loading

SukkaW commented Mar 24, 2020

seaoak commented Mar 25, 2020

SukkaW commented Mar 25, 2020

refactor(post/tag): render tag before content #4171

refactor(post/tag): render tag before content #4171

Conversation

SukkaW commented Mar 4, 2020 • edited by stevenjoezhang Loading

What does it do?

How to test

Screenshots

Pull request tasks

SukkaW commented Mar 4, 2020 • edited Loading

stevenjoezhang commented Mar 4, 2020

seaoak commented Mar 5, 2020 • edited Loading

SukkaW commented Mar 5, 2020 • edited Loading

SukkaW Mar 6, 2020 • edited Loading

Choose a reason for hiding this comment

This comment has been minimized.

SukkaW commented Mar 7, 2020 • edited Loading

jiangtj Mar 9, 2020

Choose a reason for hiding this comment

seaoak Mar 11, 2020

Choose a reason for hiding this comment

stevenjoezhang commented Mar 9, 2020

SukkaW commented Mar 10, 2020 • edited Loading

seaoak commented Mar 10, 2020

SukkaW commented Mar 10, 2020 • edited Loading

SukkaW commented Mar 24, 2020

seaoak commented Mar 25, 2020

SukkaW commented Mar 25, 2020

SukkaW commented Mar 4, 2020 •

edited by stevenjoezhang

Loading

SukkaW commented Mar 4, 2020 •

edited

Loading

seaoak commented Mar 5, 2020 •

edited

Loading

SukkaW commented Mar 5, 2020 •

edited

Loading

SukkaW Mar 6, 2020 •

edited

Loading

SukkaW commented Mar 7, 2020 •

edited

Loading

SukkaW commented Mar 10, 2020 •

edited

Loading

SukkaW commented Mar 10, 2020 •

edited

Loading