Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tools: simplify tools/doc/preprocess.js #19539

Closed
wants to merge 2 commits into from
Closed

tools: simplify tools/doc/preprocess.js #19539

wants to merge 2 commits into from

Conversation

vsemozhetbyt
Copy link
Contributor

@vsemozhetbyt vsemozhetbyt commented Mar 22, 2018

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • commit message follows commit guidelines

This PR seems to be a big refactoring, but it is rather simple and trims the script down almost by half, streamlining the logic significantly.

Investigated status quo in preprocess.js

tools/doc/preprocess.js seems to either contain artifacts from some very old doc system state or to be designed for a much more complicated doc system than the current one, just in case.

  1. preprocess.js has two aims: strip @// comments in doc sources + replace @include directives with outer file content.
  2. preprocess.js is designed to support repeated @include directives so it uses caching.
  3. preprocess.js is designed to support nested @include directives so it calls its main exported function recursively.
  4. preprocess.js is designed to support docs with various file extensions (.md, .markdown, etc).

Investigated status quo in doc building system and doc sources

  1. Only included doc/api/_toc.md has @// comments .
  2. Only doc/api/index.md and doc/api/all.md have @include directives. None of these cases has repeated or nested directives, so neither caching nor recursive processing is needed here.
  3. Our docs have only .md extension.
  4. Except for the main use in tools/doc/generate.js, preprocess.js is also used in tools/doc/html.js and test/doctool/test-doctool-html.js. Neither of these cases means repeated or nested directives as well.
  5. The only cause for recursive calls is comment stripping in main and included docs. This is a very simple operation needed only for one included doc, so it can be inlined (for both main and included docs to be on the safe side for future comment additions).

Simplification strategy

  1. Remove caching.
  2. Remove recursion.
  3. Remove auxiliary functions and export the single remaining function.
  4. Inline comment stripping.
  5. Expect only .md file extension.

I've built the docs on Windows (using #19330) before and after these changes and both doc sets are identical + doctool tests are OK.

@nodejs-github-bot nodejs-github-bot added doc Issues and PRs related to the documentations. tools Issues and PRs related to the tools directory. labels Mar 22, 2018
@vsemozhetbyt
Copy link
Contributor Author

CI-lite: https://ci.nodejs.org/job/node-test-pull-request-lite/289/

Let me know if we need full CI for this.


function processIncludes(inputFile, input, cb) {
const includes = input.match(includeExpr);
if (includes === null) return cb(null, input);
if (includes === null) return cb(null, input.replace(commentExpr, ''));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I think dominant style is to put if body in a new line.

@vsemozhetbyt
Copy link
Contributor Author

vsemozhetbyt commented Mar 23, 2018

Added a fixup commit: put long if bodies in a new line.
CI-lite: https://ci.nodejs.org/job/node-test-pull-request-lite/301/

@vsemozhetbyt vsemozhetbyt added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Mar 23, 2018
@vsemozhetbyt
Copy link
Contributor Author

Not sure if this can be fast-tracked. Feel free to add the label if so.

vsemozhetbyt added a commit that referenced this pull request Mar 25, 2018
PR-URL: #19539
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>
@vsemozhetbyt
Copy link
Contributor Author

Landed in cde98ce

@vsemozhetbyt vsemozhetbyt deleted the tools-doc-preprocess-simplify branch March 25, 2018 20:15
targos pushed a commit that referenced this pull request Mar 27, 2018
PR-URL: #19539
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>
MylesBorins pushed a commit that referenced this pull request Aug 17, 2018
PR-URL: #19539
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>
@MylesBorins MylesBorins mentioned this pull request Aug 17, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. doc Issues and PRs related to the documentations. tools Issues and PRs related to the tools directory.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants