autogenerate table of contents for Docusaurus 2

[endiliey]

🚀 Feature

Have you read the Contributing Guidelines on issues?

yes

Motivation

In v1, we have <AUTOGENERATED_TABLE_OF_CONTENTS>. syntax that generates table of content for us

https://docusaurus.io/docs/en/doc-markdown#generating-table-of-contents

However, we haven't implemented it back in v2 because we use MDX instead of remarkable now. If anyone from the community is interested to work on this, PR is welcome.

Requirement

1. Write a remark plugins that convert <AUTOGENERATED_TABLE_OF_CONTENTS>. to a generated list of table of content
2. Add the remark plugins here: https://github.com/facebook/Docusaurus/tree/master/packages/docusaurus-mdx-loader/src
3. Add tests for it
4. Add that remark plugin to our jmdx-loader default remarkPlugins option
5. Test locally with v2 website that it can transform the syntax to autogenerated toc

Let's say this is our markdown file

<AUTOGENERATED_TABLE_OF_CONTENTS>

### `docusaurus.function(a, b)`

Text describing my function

### `docdoc(file)`

Text describing my function

The remark manual test could be

var vfile = require('to-vfile')
var remark = require('remark')
var toc = require('../path/to/your/remark/plugin')

remark()
  .use(toc)
  .process(vfile.readSync('example.md'), function(err, file) {
    if (err) throw err
    console.log(String(file))
  })

Result

- `docusaurus.function(a, b)`
- `docdoc(file)`

### `docusaurus.function(a, b)`

Text describing my function

### `docdoc(file)`

Text describing my function

[yangshun]

Would recommend checking out MDX and see if they have first-class support for this - The docs is currently WIP but this might mean they've already built it out.

[endiliey]

but we still need custom plugin because our autogenerated_table_of_contents syntax is unique.

Edit: the logic is unique too

[yangshun]

We'll change our syntax then. It's not too big of a deal to search and replace. This is a major update after all.

[endiliey]

Checked their source code. They don't have it. They are just manually typing Table of Contents.

They recommend usinjg remark-toc which has a different behavior with our TOC

Remark-toc

## Table of Contents

## Bravo

### Charlie

## Delta

become

## Table of Contents

-   [Bravo](#bravo)

    -   [Charlie](#charlie)

-   [Delta](#delta)

## Bravo

### Charlie

## Delta

Docusaurus

## Options
<AUTOGENERATED_TABLE_OF_CONTENTS>

## Bravo

### Charlie

## Delta

become

## Options

-   [Bravo](#bravo)

-   [Charlie](#charlie)

## Bravo

### Charlie

## Delta

Ours is more of a partial TOC.

But its indeed wrong to mark this as difficulty:starter. I'm going to assign myself

It's also nicer to make it backward compatible as possible

[wgao19]

This is probably more suited for a future discussion, but would like to throw in an opinion that maybe we would like to consider the following two types of table of contents.

Hoping @endiliey to consider these use cases when designing the API :)))) So that we can implement those variations as themes in the future.

1. A book-style ToC on top of the main page like RequireJS

This will be useful when the page content is rather rich and each page is relatively separate with one another

2. Extend the sidebar like docz

This counts as ToC because the headers of the extended section are the headers from that page's content.

[endiliey]

Sorry for the confusion. We actually have two types of toc. One is the right side
It generates table of content for the current document

User can swizzle Docitem and modify it as their wish, you can show the table of contents on the left, bottom or even inside the page.

https://github.com/facebook/Docusaurus/blob/f0d5313d488fcb3264286ab46fdabdb2c92a88d8/packages/docusaurus-theme-classic/src/theme/DocItem/index.js#L56-L62

However this one is referring to another type of special syntax by Docusaurus. It is some kind of TOC, but more of a partial one.

This might explain better

[amimas]

The overall TOC of the entire page on the right hand side is great!! I'd really love to see the ability to auto generate a partial TOC for a particular section only as shown by @endiliey in the above comment's last screenshot. I don't believe we can do that right now in V1.

[endiliey]

@amimas actually that's what <AUTOGENERATED_TABLE_OF_CONTENTS> in v1 do

See https://github.com/jaredpalmer/formik/blame/next/docs/api/withFormik.md#L65-L68

Result:

[amimas]

@amimas actually that's what <AUTOGENERATED_TABLE_OF_CONTENTS> in v1 do

Actually I find the current implementation a bit limited. Here's why and let me know what your thoughts are. Maybe it can be more flexible in V2.

• Current implementation in V1 requires a specific pattern: an H2 heading that has one or more H3 headings that is wrapped in backticks

  This limits the potential of having partial table of content for a specific section. Requiring backticks in H3 headings means I cannot have a partial toc for a section that might not necessarily refer to any command/functions/methods (these may be in the details of the corresponding sections)

  Requiring H2 followed by H3 also limits the usage of partial toc. I may want a partial toc of a H3 headings followed by several H4 headings and so on.

• In V1 the syntax, <AUTOGENERATED_TABLE_OF_CONTENTS>, seems to generate a TOC of the entire page that follows the pattern: H2 followed by several H3, each of which are wrapped in backticks

  This does not seem to work for partial toc. For example: I have multiple H2 headings. Each of those has several H3 headings wrapped in backticks. If I add <AUTOGENERATED_TABLE_OF_CONTENTS> to the first H2 heading, it generates a TOC for all the H3 headings instead of only those that are within the scope of the first H2 heading where I added <AUTOGENERATED_TABLE_OF_CONTENTS>.

  I'm not sure if that was intended feature or if it's a bug.

[endiliey]

@amimas you are right ! I just went to re-read https://github.com/facebook/Docusaurus/blob/1aa0ea1152b0133830d127d11b123fcea261deb4/packages/docusaurus-1.x/lib/core/toc.js#L60-L71 and it seems that it's what really happening under the hood. That feature was implemented way before i joined the team, so i'm not really sure of the intention.
It seems to be intended ...

To keep backward compatibility, i'll implement it like v1 for this feature.

Technically in v2 you can just write remark plugins to do this and pass it to the plugin-docs option.

[endiliey]

Or maybe we should really discuss. Whether to deprecate this weird behavior or not. Backward compatibility isn't the highest priority because we're doing major version after all.

Since we allow passing remark plugins in v2, I am not even sure if we should add this to v2.

I am more towards deprecating this <AUTOGENERATED_TABLE_OF_CONTENTS> that seems to be weird. It increases parsing time and slow down the build process.

Having two table of contents is strange. I wonder if having table of contents on the right side is actually enough?

[yangshun]

Since this will be part of a major upgrade, I would opt for doing the right thing over preserving non-ideal backwards compatibility.

…

On Fri, Jun 7, 2019, 9:17 AM Endi ***@***.***> wrote: Or maybe we should really discuss. Whether to deprecate this weird behavior or not. Backward compatibility isn't the highest priority because we're doing major version after all. Since we allow passing remark plugins in v2, I am not even sure if we should add this too v2. I am more towards deprecating this <AUTOGENERATED_TABLE_OF_CONTENTS> that seems to be weird. It increases parsing time and slow down the build process. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1549>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKBCHJZLMU3KHIWKTVB6BLPZKCZXANCNFSM4HSGVYZQ> .

[endiliey]

I think the right thing here is quite subjective for that part.

It isn't clear whether that feature is intended or not. The industry standard is to have table of contents for the whole page, which we already did in the right side.

If they don't want it in the right side, they can swizzle and put it in the middle like this.

Here, i swizzled and moved the toc and change even the style

[endiliey]

If we were to stick to industry standard, https://github.com/remarkjs/remark-toc is also another solution

It adds a Table of Contents to a Markdown document. You can specify depth too (not just h2 or h3 like v1. Adding it to v2 is also as simple as passing remark plugins to v2 docs options.

I'm going to close this issue out as I feel we should do the right thing. If there is a need for <AUTOGENERATED_TABLE_OF_CONTENTS>. open up a new issue. If many are in favor, we will implement it

[amimas]

I personally think having an auto generated toc for the entire page like what we have in V1 is the right thing to do and it should be part of the core instead of removing it entirely in V2. The location of this overall TOC is subjective but I like the current implementation of having it on the right side out of the box; it gives a clear distinction between the right sidebar and the overall toc for a given page without cluttering the sidebar on the left side.

The <AUTOGENERATED_TABLE_OF_CONTENTS> is also another useful feature for generating partial toc for a particular section only. Perhaps it can be renamed to be something else. If we want that to be done using a plugin, that sounds okay too but that's also something I think a lot of users will benefit from. So, maybe this can be part of the core too and have a clear documentation explaining the usage/intention.

[endiliey]

I personally think having an auto generated toc for the entire page like what we have in V1 is the right thing to do and it should be part of the core instead of removing it entirely in V2

Its not removed, check https://v2.docusaurus.io

The thing is, adding a feature like partial toc in markdown by default will slow down the markdown parsing time. For site like jest which has thousands of pages, it can increase dev time a lot (every seconds saved is valuable)

It should be an opt-in, and anyway it can be added easily by adding few lines of code in your config. I know how to do it with remark-toc. Perhaps I will add that to documentation in the future when we release v2 and teach people about it.

[wgao19]

Will add to docs #1578 2019年6月8日 12:52 +0800、Endi <notifications@github.com>のメール:

…

> I personally think having an auto generated toc for the entire page like what we have in V1 is the right thing to do and it should be part of the core instead of removing it entirely in V2 Its not removed, check https://v2.docusaurus.io The thing is, adding a feature like partial toc in markdown by default will slow down the markdown parsing time. For site like jest which has thousands of pages, it can increase dev time a lot (every seconds saved is valuable) It should be an opt-in, and anyway it can be added easily by adding few lines of code in your config. I know how to do it with remark-toc. Perhaps I will add that to documentation in the future when we release v2 and teach people about it. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[endiliey]

Did this get updated to docs ? @wgao19

I think we need to show its deprecated somehow

[wgao19]

@endiliey do you mind adding the guide in #2036?

[endiliey]

I think simply pointing it to this issue for reasoning is more sensible since the conversation above can explain it well.

[wcauchois]

Would it be possible to add some more instructions to the blurb at https://v2.docusaurus.io/docs/migrating-from-v1-to-v2/#remove-autogenerated_table_of_contents? It directed me to this issue but it's unclear to me how to add remark-toc to my "remarkPlugins" configuration (that's not in the default docosaurus.config.js - where do I put that key?) or use it. Any simple explanation or pointer to a simple example would be super appreciated.

[frank-lam]

@endiliey Hi, I am want to to generate the toc of h4/h5/h6 ( h2 to h6) . Now just display the h2/h3. Is there any config parms to achive this feasure.

[slorber]

inline table of contents inside md docs (full or partial) will be possible thanks to #3904