Large sidebars (i.e autogenerating all folders) causes build slowdowns due to repeated getSidebar calls

[KianNH]

What version of starlight are you using?

0.25.2

What version of astro are you using?

4.13.4

What package manager are you using?

npm

What operating system are you using?

Mac

What browser are you using?

Chrome

Describe the Bug

On Starlight sites with a large (thousands of items) sidebar, which may be filtered with a Sidebar override (i.e to the current folder), individual pages will build slower as the sidebar size increases.

For example, with cloudflare/cloudflare-docs as an example, there are ~4,000 pages and each top-level folder within src/content/docs is an autogenerate entry for the sidebar.

As part of generateRouteData, getSidebar is called on each page with the current page's path (to mark a given page as current). With a very large sidebar, this can represent a large amount of the time spent rendering a page.

NodeJS Dev Tools profile:

Full build flamegraph:

I will try and follow-up with a proper MRE in StackBlitz!

Link to Minimal Reproducible Example

https://github.com/cloudflare/cloudflare-docs

Participation

• I am willing to submit a pull request for this issue.

[delucis]

Thanks for the issue @KianNH! I tried @HiDeoo’s suggestion of caching parts of our treeify() function and saw some nice benefits although not totally game changing. Building the Cloudflare docs on my 2019 Intel MacBook Pro, it knocked about 50 seconds off a 16-minute build, and @HiDeoo saw a similar gain with a faster build (about 50 seconds off a 7m30s build).

@HiDeoo is working on a more comprehensive refactor that is showing very promising benchmarks in early tests (cutting the ~7m30s build in half to ~3m45s locally). Hopefully we’ll have more to share soon.