-
-
Notifications
You must be signed in to change notification settings - Fork 8.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docusaurus Faster #10556
Comments
siteConfig.future.experimental_faster.rspackBundler
#10402
Thanks @slorber, sharing comparison results for our OpenAPI plugin site. Cold Build No Optimization
Cold Build With Experimental Faster Enabled
Edit from @slorber: results above are still based on Webpack. See PaloAltoNetworks/docusaurus-openapi-docs#1003 for the real upgrade. Benchmark shows demo site building:
|
This comment was marked as outdated.
This comment was marked as outdated.
The Rspack support has been merged: #10402 You can try canary Here's an example PR upgrading the React Native website: facebook/react-native-website#4268 How to benchmarkOne way to benchmark this is to use hyperfine with an env variable: const config = {
future: {
experimental_faster: (process.env.DOCUSAURUS_FASTER ?? 'true') === 'true',
},
} For cold builds: hyperfine --prepare 'yarn clear' --runs 5 'DOCUSAURUS_FASTER=false yarn build' 'DOCUSAURUS_FASTER=true yarn build' For warm rebuilds: DOCUSAURUS_FASTER=false yarn build
hyperfine --runs 5 'DOCUSAURUS_FASTER=false yarn build' 'DOCUSAURUS_FASTER=true yarn build' Troubleshooting slow buildsIt's possible that enabling this new option does not give you the expected speed increase. Perf loggerYou can get a more verbose performance logger with the RsdoctorAnother option is to use Rsdoctor, which works for both webpack and Rspack. It can tell you which loader/plugin/phase is slow while bundling your app. For many Docusaurus sites (including ours), the slow loaders are postcss/css-loader, mdx loader, and the image loaders of our ideal image plugin. Despite these slow loaders, Docusaurus Faster builds 3x faster. It is possible that a Docusaurus plugin you use slows down your build. Rsdoctor might be able to give you insight on that. We'll probably release an official Rsdoctor plugin for Docusaurus, but in the meantime here's a Docusaurus plugin we built to integrate it into our own site: https://github.com/facebook/docusaurus/blob/main/website/src/plugins/rsdoctor/RsdoctorPlugin.ts |
Tested with a relatively large internal documentation site. PS > Get-ChildItem -File -Recurse | Group-Object -Property 'Extension' | Sort-Object -Property 'Count' -Descending | Format-Table -Property 'Count', 'Name'
Count Name
----- ----
748 .md
553 .png
71 .jpg
24 .json
10 .webp
8 .svg
5 .order
4 .py
3 .drawio
3 .html
3 .pdf
2 .mdx
2 .xml
1 .xlsx
1
1 .avif
1 .zip
PS > Locally, Windows, Yarn v4.5.1, NodeJS v20.18.0. Used following command in PowerShell for timing the build: yarn docusaurus clear; $Stopwatch = [System.Diagnostics.Stopwatch]::StartNew(); yarn build; $Stopwatch.Stop(); $Stopwatch.Elapsed.ToString()
We're pretty stoked on that speedup. 😊 |
which version of docusaurus comes with this experimental config? since i can't get it to run in 3.5.2
|
Answering myself, it is I'm seeing
Nice job Update: build is down from +3 minutes to 30 seconds |
Support for building projects via rspack has been added in the latest Canary version of Docusaurus, which significantly improves compilation speed. ```bash ❯ hyperfine --prepare 'pnpm clear' --runs 5 'DOCUSAURUS_FASTER=false pnpm build' 'DOCUSAURUS_FASTER=true pnpm build' Benchmark 1: DOCUSAURUS_FASTER=false pnpm build Time (mean ± σ): 70.715 s ± 17.150 s [User: 134.096 s, System: 31.651 s] Range (min … max): 52.330 s … 88.497 s 5 runs Benchmark 2: DOCUSAURUS_FASTER=true pnpm build Time (mean ± σ): 30.094 s ± 4.572 s [User: 64.061 s, System: 28.384 s] Range (min … max): 25.832 s … 37.655 s 5 runs Summary DOCUSAURUS_FASTER=true pnpm build ran 2.35 ± 0.67 times faster than DOCUSAURUS_FASTER=false pnpm build ``` see: facebook/docusaurus#10556 (comment) ```release-note None ```
I ran some informal benchmarks on a Macbook M3 Pro building our Docusaurus site. I cleared the cache after each build. Seeing a 2.5x speedup.
Our CI runners are significantly slower than my Macbook, so I'm estimating that it will reduce the build time by 2 minutes in the CI pipeline. |
Wanted to test it on a docusaurus page that uses stage 3 decorators. Is it (or will it be) possible to customize the configuration of swc? (I have no experience so far with swc, but what i understand from the docs, the following should work: {
"jsc": {
"parser": {
"syntax": "typescript",
"decorators": true
}
}
} ) |
I'm not 100% sure but maybe adding a .swcrc would work? |
Thanks for the advice @slorber . I tried it, but somehow swc does not pick up the .swcrc configuration. Here is a little repro: https://codesandbox.io/p/devbox/sharp-bird-jd7tf5 With The used code for the decorators can be compiled by the swc playground when enabling the decorators. decorated code exampletype AccessorDescriptor = {
get: () => number;
set: (value: number) => void;
};
type DecoratorContext = {
name: string;
kind: string;
};
function accessorDecorator(
{ get, set }: AccessorDescriptor,
{ name, kind }: DecoratorContext
) {
if (kind === "accessor") {
return {
init(initialValue: any) {
return initialValue ?? 0;
},
get() {
const value = get.call(this);
// Additional logic can be added here
return value;
},
set(newValue: any) {
const oldValue = get.call(this);
console.log("set", name, "=", newValue);
// Additional logic can be added here
set.call(this, newValue);
},
};
}
}
class ExampleClass {
@accessorDecorator accessor exampleField: number = 1;
}
export default ExampleClass; Maybe the options must be passed to swc explicitely 👉 https://github.com/facebook/docusaurus/blob/main/packages/docusaurus-faster/src/index.ts#L25-L34 ? Update 5.11.2024The workaround from #10556 (comment) works: const config = {
future: {
experimental_faster: {
swcJsLoader: false, /* set it to false here */
swcJsMinimizer: true,
swcHtmlMinimizer: true,
lightningCssMinimizer: true,
rspackBundler: true,
mdxCrossCompilerCache: true,
},
},
webpack: {
jsLoader: (isServer) => {
const defaultOptions = require("@docusaurus/faster").getSwcLoaderOptions({isServer});
return {
loader: 'builtin:swc-loader', // (only works with Rspack)
options: {
...defaultOptions,
jsc: {
parser: {
...defaultOptions.jsc.parser,
decorators: true
},
transform: {
...defaultOptions.jsc.transform,
decoratorVersion: '2022-03'
}
}
},
}
},
} Thanks @slorber |
I don't know, will need to investigate @lebalz Note that you might still be able to use the old custom js loader option mentioned here: #4765 (comment) Didn't try, but pseudo-code: const config = {
webpack: {
jsLoader: (isServer) => ({
loader: 'builtin:swc-loader', // (only works with Rspack)
options: require("@docusaurus/faster").getSwcLoaderOptions({isServer}),
}),
},
} |
Docusaurus v3.6 is out with the Docusaurus Faster options 🥳 🎉 |
Would it be possible to add an option in Maybe It'd also be useful to disable using Docusaurus Faster whenever it becomes the default. |
@o-l-a-v I'm not sure it's necessary. Depending on initial feedback I plan to make it soon the default of our init template to increase adoption, feedback, bugfixes, and finally making it the new default infrastructure. |
Hi, I found a new problem after turning on faster mode. This may be related to rspack. This problem only occurs in the development environment (pnpm start). ❯ DOCUSAURUS_PERF_LOGGER=true pnpm start
> @halo-dev/docs@0.0.0 start /Users/ryanwang/Workspace/github/ruibaby/docs
> docusaurus start
-----------------------------------------------------------------------------------------------------------------------------------------------
Update available 3.6.0-canary-6132 → 3.6.0
To upgrade Docusaurus packages with the latest version, run the following command:
`npm i @docusaurus/core@latest @docusaurus/faster@latest @docusaurus/plugin-client-redirects@latest @docusaurus/preset-classic@latest
@docusaurus/theme-classic@latest @docusaurus/theme-common@latest @docusaurus/theme-mermaid@latest @docusaurus/module-type-aliases@latest`
-----------------------------------------------------------------------------------------------------------------------------------------------
[INFO] Starting the development server...
[PERF] Load site > Load context - 115.72 ms - (48mb -> 64mb)
[PERF] Load site > Load plugins > Init plugins - 191.18 ms - (64mb -> 78mb)
[PERF] Load site > Load plugins > Load plugins content > Load classic@default > translatePluginContent() - 8.34 ms - (79mb -> 81mb)
[PERF] Load site > Load plugins > Load plugins content > Load pages@default > loadContent() - 9.00 ms - (79mb -> 81mb)
[PERF] Load site > Load plugins > Load plugins content > Load pages@default - 19.53 ms - (79mb -> 80mb)
[PERF] Load site > Load plugins > Load plugins content > Load classic@default > getDefaultCodeTranslationMessages() - 11.98 ms - (81mb -> 81mb)
[PERF] Load site > Load plugins > Load plugins content > Load classic@default - 20.62 ms - (79mb -> 81mb)
[PERF] Load site > Load plugins > Load plugins content > Load docs@default > loadContent() - 138.61 ms - (79mb -> 86mb)
[PERF] Load site > Load plugins > Load plugins content > Load docs@default > contentLoaded() - 39.50 ms - (87mb -> 95mb)
[PERF] Load site > Load plugins > Load plugins content > Load docs@default - 180.91 ms - (79mb -> 95mb)
[PERF] Load site > Load plugins > Load plugins content - 181.09 ms - (79mb -> 95mb)
[PERF] Load site > Load plugins - 374.37 ms - (64mb -> 95mb)
[PERF] Load site > Create site files - 46.84 ms - (96mb -> 100mb)
[PERF] Load site - 538.29 ms - (48mb -> 100mb)
[SUCCESS] Docusaurus website is running at: http://localhost:3000/
● Client ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ (83%) sealing chunk ids Panic occurred at runtime. Please file an issue on GitHub with the backtrace below: https://github.com/web-infra-dev/rspack/issues
Message: byte index 2 is not a char boundary; it is inside '务' (bytes 0..3) of `务端-1-d-0-f-04c`
Location: crates/rspack_core/src/utils/compile_boolean_matcher.rs:325
Backtrace omitted.
Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
ELIFECYCLE Command failed.
[1] 97491 abort DOCUSAURUS_PERF_LOGGER=true pnpm start |
I enabled |
Thanks a lot @slorber, I upgraded today on a Docusaurus website with around 60 Markdown files including Mermaid charts, in Github Actions pipeline it went from average 1m30 build time to 12s ! Impressive ! 💯 |
To add to the list of wins - with faster enabled on one of my sites with 4651 markdown files, the build job went from taking up to 15 minutes to build down to only ~7.5 minutes 🥳 . Thanks so much for the effort y'all put into optimizing this. |
The build from an internal website here went from 4m18s to 1m8s. A great deal, thank you very much! |
Migrated a small doc today very easily, no problem. Build time went from 1m29s to 0m14s! 🚀 Thanks! |
Incredible and impressive improvement. Tested at same environment on same machine with clean build command
Took only 20% (19.712s / 98.320s) build time compared to previous version. |
Our Spectro Cloud documentation build time went from 1 hour and 15 minutes to 35 min 🎉 For context, our builds run on a self-hosted GitHub runner with 8 CPU 16 GB Memory and 150 GB SSD. |
Our Grafana plugin-tools docs builds times went from 36s to 11s build time. Every little helps! Really appreciate the efforts everyone made here. 👏👏👏 |
Important security notice for Docusaurus Faster usersVersion 1.1.7 of More info here: The 1.1.7 version was published yesterday. The Rspack team reacted quickly to deprecate it and publish a safe 1.1.8 version within 1h after the 1.1.7 malicious publish.
If you are unlucky and installed it yesterday during that 1h period, you may have installed that compromised Rspack version. This may also happen when upgrading dependencies or re-generating your package manager lock file. You can check the
Alternatively, you can look at If your Docusaurus site is using Rspack v1.1.7, make sure to upgrade to v1.1.8 immediately, and check for unusual activity on your system (analysis of the malicious script) More info here: Note that the large majority of Docusaurus users should not be affected:
|
Is this plugin forward compatible? Do older versions support this feature? |
Docusaurus v3.6 is out with the Docusaurus Faster options 🥳 🎉
https://docusaurus.io/blog/releases/3.6
Docusaurus Faster
This is a meta-issue for the Docusaurus Faster project.
The goal is to modernize our infrastructure and greatly reduce the time and memory it takes to build a Docusaurus site in production mode, to fix this long-standing issue #4765.
Before using these flags, you will need to add an extra package encapsulating all the opt-in dependencies:
It includes multiple experimental PRs adding new
siteConfig.future.experimental_faster
options:swcJsLoader
: Use SWC to transpile JS (instead of Babel).swcJsMinimizer
: Use SWC to minify JS (instead of Terser).swcHtmlMinimizer
: Use SWC to minify HTML and inlined JS/CSS (instead of html-minifier-terser).lightningCssMinimizer
: Use Lightning CSS to minify CSS (instead of cssnano and clean-css).rspackBundler
: Use Rspack to bundle your app (instead of webpack).mdxCrossCompilerCache
: Compile MDX files only once for both browser/Node.js environments instead of twice.We recommend turning them on all at once with this simple boolean shortcut:
But you can also turn them on/off one by one to troubleshoot potential problems.
Note that the
rspackBundler
option requiresswcJsLoader
,swcJsMinimizer
, andlightningCssMinimizer
to be enabled first.It is possible that certain combinations of options do not work, and our CI doesn't cover all possible permutations (only
true
/false
).It also includes notable memory improvements:
What to expect
With
experimental_faster: true
, our Docusaurus website builds:Although rebuilds (with warm
./node_modules/.cache
) are not faster (yet), we recommend you enable theexperimental_faster: true
option to future-proof your site against our new dependencies and report us potential problems you encounter.Our core plugins/themes are all compatible with this new option, and we expect most community plugins to be compatible out-of-the-box. Some plugins might need minor adjustments to make them compatible with Rspack, and we'll help the community to make our ecosystem compatible with Rspack.
This option is relatively safe to use in production: we have deployed our own website with it already:
For Docusaurus plugins authors
Some plugins implementing the
configureWebpack()
lifecycle hook might need to adjust their code to add support for Rspack.I will write a guide for them in this sister issue: #10572
Upgrade PRs
Example upgrade PRs to look at, including some benchmarks.
Community websites:
Popular plugins:
Contributing to this issue
We want our community to share feedback on this Docusaurus Faster project.
Please try to adopt it as part of the v3.6+ / canary releases, and tell us how it works for you.
If you encounter problems, please let us know here, and we might create dedicated issues if needed.
This general issue is a great place to follow the Docusaurus Faster project overall, as we try to stabilize it and aim to make it the default for an upcoming major version.
The text was updated successfully, but these errors were encountered: