auto generated API docs for Typescript users with Typedoc

[joshribakoff]

💥 Proposal

I've created a basic typedoc plugin, it simply loads the HTML created by the "minimal" HTML template, grabs the various "sections" and passes them to a component the end user can override and "dangerously renders the HTML" inside the docusaurus header & footer. It also supports mono-repos with multiple packages that need to create typedocs for each package. It can also support versioning, just create a separate folder for each typedoc output.

I believe this is much more maintainable & robust than what most docusaurus users seem to be doing, which is to manually create these types of docs. Therefore, I would propose it be added as an official plugin. I can make a PR if you agree.

The docusaurus plugin takes in an options object describing the path(s) and glob pattern(s) to the typedoc generated output, users must run typedoc as a separate command to generate the output, such as in their CI build step that publishes their docs, or commit it (such as each time they create a docusaurus "version")

Have you read the Contributing Guidelines on issues?

yes

Screenshots

Example of typedoc generated HTML being pulled into docs site:

Example of typedoc CSS copy pasted into docs custom CSS:

Dark mode can also easily be supported, with a little time spent tweaking the CSS:

Custom Component

Users can render whatever they want. The sections are grabbed separately using cheerio in the docusaurus plugin

import React from 'react';
import Layout from '@theme/Layout';

export default ({ __content, __menu }) => {
  return (
    <Layout title="API docs">
      <div class="container">
        <div class="row">
          <div class="col col--8">
            <div dangerouslySetInnerHTML={{ __html: __content }} />
          </div>
          <div class="col col--4">
            <div dangerouslySetInnerHTML={{ __html: __menu }} />
          </div>
        </div>
      </div>
    </Layout>
  );
};

[joshribakoff]

Link to my custom plugin (I can convert to TS & cleanup if the proposal is accepted): rx-store/rx-store#81

[slorber]

Hi,

That looks interesting.

As I'm working on i18n support I don't have much time to review this currently.

But I've seen @phryneas doing some interesting things on improving TS documentation for Redux in Docusaurus. https://twitter.com/phry/status/1302662810707660801

That would be nice to have a more general discussion on how we could provide good TS support on Docusaurus as TS is clearly becoming more popular very fast. Ideally, the plugins should be compatible with each others.

[joshribakoff]

Yep, I followed @phryneas on Twitter after they posted that. I also plan to utilize that in my docs, as converting code samples & manually checking them is error prone.

Auto generated API docs, and tested & type checked code samples in docs are both affordances I got used to in other language(s) & framework(s) other than JS / React, its nice to see the tooling catching up :)

Even if a user is not using TS / TSdoc, I don't see any real reason why API docs cannot be generated from static analysis like this, and code samples extracted & executed. I'm presuming they are using the TS compiler API to load the extracted string(s) and access the diagnostics.

Both auto generated API docs, and type checked/tested code snippets should be possible (even if the user doesn't use TS, and instead is using JS).

One area where it does cause a new complication is if both features are added, as ideally we'd also extract the code snippets that are in the TSdoc comments themselves, and run those through @phryneas tool as well, but that could be a "phase 2" feature / stretch goal.

[phryneas]

Heyo :)

I'm pretty sure these two things won't get in each other's way - I wrote the stuff I wrote because the @reduxjs/toolkit docs are very hand-written and I wanted to have some deduplication between docs & code comments without sacrificing that style. Our types are so complex that no generated API doc would ever make sense for them - believe me, I've tried.
So: pretty different use case.

As for the code checking & transpiling: The easiest way to get that here of course were if this would output mdx instead for html - but I believe that's not really the way this is going here, right? I've looked into certain tools mentioned in the tsdoc repo and they all seemed to go other ways.
So the second option would be to just use the Compiler class. It's not exported right now, but that's totally something I could do.
It is instantiated once using some config and then you just call compiler.compile(files) with a set of "virtual files" to simulate imports etc.
I'm using this logic to extract those "virtual files" from the codeblocks - again, I could export that.

If you want to do some experimentation on that, best way would be to just ping me on Discord - I'm phryneas#4779 over there and I'm always up for such shennanigans :)

PS: by the way, if you want to see some of those docs in action, I'm preparing a PR over here: reduxjs/redux-toolkit#707

[joshribakoff]

Sounds good I’ll ping you. I actually have flip flopped on whether api docs should be generated or hand written

For example, in RX Store I have a store() function which creates the store, and returns a component. The generated api docs did not pickup the manager component, it only initially picked up the store() closure that wraps it. I initially presumed our types would be too complex as well for this issue and other issues, for example we use mapped types and generic function signature type aliases that in turn describe functions that are generic

However, after some refactoring I’ve found it’s easy and beneficial for numerous reasons to “rework my code around the tooling”, initially I rejected the notion of changing my code based on the tool, however TsDoc is not specific to typedoc, editors use it for intellisense. I feel like the refactors I’m doing not only improve the typedoc output, it’s more explicit and robust to have for example a Manager type alias, which is something we didn’t have before By exporting more types it surfaces new concepts in the generated docs and allows users who consume my typings to have access to more descriptive typings. I was also able to flag the createManager function as internal, not only further cleaning up the generated docs but also making the code itself more readable for new users in my opinion, who can now see which things are meant for end consumption even just from reading the code (hopefully)

Also, much of what goes into TSDoc is still hand written by the developers, it’s just able to infer some portion of that and provide more structure.

I checked and the typedoc tool only seems to output json and HTML. The json seemed to be almost as verbose as an AST, so naturally I just reached for cheerio to scrape the HTML and shove that HTML into a page hackily. Well likely need to clean that up if we want this to be an official plugin (eg not injecting HTML inside of json), so although I’m happy to volunteer to make the official plugin I may have some questions as I work through cleaning up the plugin.

Also I didn’t mean to imply our plugins would conflict in any way, quite the opposite, it’s just that if you’re using Ts doc you’ll have code snippets in both the TS doc comments and the docusaurus markdown, so it would be great to have tight integration. The code samples could probably easily be extracted from the tsdoc using the reference implementation of the tsdoc parser, or could also probably be grabbed with cheerio from the HTML, and then supplied to your tool. One open question would be whether your tool should modify the typedoc generated HTML to splice in JavaScript examples of each code snippet

[phryneas]

Yeah, the thing is that we actually went the other way of "code for tooling": The types allow for an amazing IDE/IntelliJ experience.
But to do that, there's stuff like workarounds for slightly different behaviour of different TypeScript versions etc. in there. Types to code ratio should be around 3:1 and almost all of those types are generic and conditional. No use in showing that to any user :D

JSON and HTML output only is a shame. The problem will be that the html "code" is probably already processed for syntax highlighting, so it will contain lots of markup. Since my tool already uses the TsDoc parser and it is pretty easy to query it for "give me the code examples for interface X", it would probably be better to go that way.

Or we could look into typedoc plugins.

Going one way, there already seems to be one that generates markdown: https://www.npmjs.com/package/typedoc-plugin-markdown.
Also, going further through npm, there seems to be already a plugin to integrate with docusaurus, although that is apparently for docusaurus 1. But maybe some ideas could be taken there.

Going the other way, I'm sure it's possible to write a typedoc plugin that mangles code blocks before formatting them. So something like the MarkedLinksPlugin which seems to modify the markdown before it is rendered to html.

[anshulrgoyal]

Typedoc have a plugin for docusaurus v2. It can outputs markdown and sidebar.json

[slorber]

Oh that's cool. I'm leaving for holidays tomorrow so I don't have time to follow this issue but hope you'll find your way to have good TS support on your sites in the meantime!

https://github.com/tgreyuk/typedoc-plugin-markdown/tree/master/packages/docusaurus-plugin-typedoc

Maybe @tgreyuk will be interested to discuss this topic as well? :)

[anshulrgoyal]

I am converting a typedoc website to docusaurus.

[ark120202]

@phryneas have you seen twoslash by @orta, which is used on the new TypeScript website? It looks quite similar to the type-checking part of your remark plugin

[joshribakoff]

Two slash doesn’t document apis it’s purpose built for showing where errors occur in typescript snippets

The typedoc plugin looks interesting I’ll check it out!

[phryneas]

@ark120202 Yeah, they both do type-checking but with very different focus.

Mine focuses of actually creating JavaScript code in the end, and on type-checking against an existing code base (your library) and a node_-_modules folder. It even simulates a file system, so each code block is a "folder" and can contain multiple files that reference each other.

Twoslash on the other hand focuses from what I've seen on self-contained examples without external references and on showing off errors, not on transpilation.

[felipecrs]

I can think of 2 different scenarios:

1. Create TypeDoc docs that integrate with Docusaurus through Markdown:
   typedoc-plugin-markdown does this very well! /cc @tgreyuk

2. Create TypeDoc docs which integrate with Docusaurus through other means:
   There is nothing currently that I know of, but I think it's interesting because it would enable fancier visual stuff such as how the standard TypeDoc's API server shows (there are even icons there!), without being limited to Markdown.

[milesj]

I went ahead and wrote this: https://github.com/milesj/docusaurus-plugin-typedoc-api

Still a work in progress but the bulk of it is done.

[Josh-Cena]

Closing because this could be delegated to external plugins