Directory structure improvements

[Zamiell]

Hello,

In my TypeScript project, I have a class called DefaultMap.

When I use Typedoc + typedoc-plugin-markdown with the hideBreadcrumbs value set to false, it generates the following output:

[isaacscript-common](../README.md) / [Modules](../modules.md) / [classes/DefaultMap](../modules/classes_DefaultMap.md) / DefaultMap

# Class: DefaultMap<Key, Value, Args\>

[classes/DefaultMap](../modules/classes_DefaultMap.md).DefaultMap

This is the documentation for the DefaultMap. Blah blah blah.

When I use Typedoc + typedoc-plugin-markdown with the hideBreadcrumbs value set to true, it generates the following output:

# Class: DefaultMap<Key, Value, Args\>

[classes/DefaultMap](../modules/classes_DefaultMap.md).DefaultMap

This is the documentation for the DefaultMap. Blah blah blah.

Here, we can see that the plugin removed the first "breadcrumb" line, but it did not remove the link to the parent module on the 5th line. Is this not defined as a "breadcrumb"? Shouldn't the plugin also be removing this line? It seems to me that if the end-user wants breadcrumb lines to be removed, it should also be removing this line.

This same behavior seems to apply to interfaces and enums in addition to classes.

[tgreyuk]

Hi - thanks. It is not quite a breadcrumb, - more of a module tree helper - but I take your point. It kind of begs the question what is the point of that line at all. I will review it as looking to make some other ui enhancements (other suggestions welcome :)).

[Zamiell]

Thank you very much!

I have a lot of other suggestions actually. :)

Biggest thing for me is I want to be able to specify the title of the page and the nested location, but maybe that belongs in a separate issue.

[tgreyuk]

Biggest thing for me is I want to be able to specify the title of the page and the nested location, but maybe that belongs in a separate issue.

Do you have an example of what you might expect?

[Zamiell]

Hi Tom,

Gladly. 😊 Imagine that we have a basic TypeScript library that provides some helper functions for fruit.

It has helper functions for:

• apples
• bananas
• passion fruit

It also exposes a FruitType enum like this:

enum FruitType {
  Apple,
  Banana,
  PassionFruit,
}

So, the file structure of our TypeScipt project would look like this:

project/
└── src/
    ├── index.ts
    ├── functions/
    |   ├── apples.ts
    |   ├── bananas.ts
    |   └── passionFruit.ts
    └── enums/
        └── FruitType.ts

Now, we want to generate some API documentation automatically for the project. The obvious choice is to use TypeDoc. Of course, we also want to use your awesome typedoc-plugin-markdown (so that we can feed it into Docusaurus), because the default TypeDoc webpage is ugly and sucks. 😊

However, when we execute npx typedoc --plugin typedoc-plugin-markdown --entryPoints src/index.ts, we get the following output:

project/
└── docs/
    ├── README.md
    └── modules.md
    └── enums/    
        └── FruitType.md

This is no good. Helpfully, the FruitType enum is in a separate markdown file, but TypeDoc combined all the functions together into one monolithic "modules.md" file. That's going to be bad UX for our users.

Instead, we want to have a separate Docusaurus page for apples, bananas, and passion fruit. So, let's try invoking TypeDoc by specifying all of the entry points specifically:

npx typedoc \
  --plugin typedoc-plugin-markdown \
  --entryPoints src/functions/apples.ts \
  --entryPoints src/functions/bananas.ts \
  --entryPoints src/functions/passionFruit.ts \
  --entryPoints src/enums/FruitType.ts \

We get the following output:

project/
└── docs/
    ├── README.md
    ├── modules.md
    └── modules/    
    |   ├── functions_apples.md
    |   ├── functions_bananas.md
    |   ├── functions_passionFruit.md
    |   └── enums_FruitType.md
    └── enums/    
        └── enums_FruitType.FruitType.md

Much better! But some problems remain.

Problem 1 - Sections

In our example project, we want there to be two different sections on the Docusaurus left-hand side bar:

• Functions
• Enums

These will correspond to the source directories of "src/functions" and "src/enums", respectively.

However, we don't want to maintain a manual mapping of source files --> Docusaurus sidebar locations. That's going to be hard to maintain and will get out of date.

Instead, Docusaurus has a helpful feature where it will automatically make a sidebar that corresponds to an input file system. You enable this with type: "autogenerated". This sounds like what we want.

However, a problem: TypeDoc messed up the folder structure. In other words, we started with:

src/functions/passionFruit.ts

And we ended up with:

docs/modules/functions_passionFruit.md

But what we really want is for TypeDoc to respect the structure and for it to generate a Markdown file like this:

docs/functions/passionFruit.md

Feature request 1: typedoc-plugin-markdown should generate Markdown files in paths that correspond to the original source files by default.

But of course, that might not make sense for all TypeScript projects. What we really want is to give the end-user control as to where the generated Markdown files should go.

Feature request 2: typedoc-plugin-markdown should look for a @path JSDoc tag in the module-level JSDoc comment, and then respect that. For example, at the top of passionFruit.ts, I would want to have something like this:

/**
 * This is a collection of helper functions regarding passion fruit.
 *
 * @module
 * @path functions/arbitrarySubDirectory/passionFruit.md
 */

/** This is the documentation for my "usePassionFruit" helper function. Blah blah blah. */
function usePassionFruit() {}

Problem 2 - Markdown Titles

At the top of every Markdown file, there are breadcrumbs - but that's easy enough to disable with the helpful hideBreadcrumbs option.

Once we've done that, the first line of the "functions_passionFruit.md" becomes:

# Module: functions/passionFruit

Here, the word "Module" is superfluous. In our project, every individual Docusaurus page is going to be a module. So, we can get rid of the "Module: " prefix.

Additionally, the "functions/" is superfluous, because it can already be seen from the left-hand sidebar that we are in the category of "Functions". So, we can also get rid of the "functions/" prefix.

But even having a title of # passionFruit is still wrong. Conventionally, TypeScript files should use a camelCase naming scheme, matching the camelCase naming scheme for variables. But for the end-user documentation, we don't want the title of the page to match the camelCase file name - we want to display it as "Passion Fruit".

In this trivial example, we might want typedoc-plugin-markdown to automatically detect camelCase words and convert it to "Camel Case" accordingly. But of course, that's terrible as general-purpose behavior, and prone to error. Better to let the end-user be in control of what they want the resulting title of the Markdown page to be. For example, you could imagine that I want to have a file called "passionFruit.ts", but the name of the resulting documentation page to be "Passion Fruit (Argentinian)", to flag to the end-users that these passion fruit functions only work on passion fruits that come from Argentina.

TypeDoc already exposes a feature to rename a module: the @module JSDoc tag. For example, we would put the following at the top of the "passionFruit.ts" file:

/**
 * This is a collection of helper functions regarding passion fruit.
 *
 * @module Passion Fruit (Argentinian)
 */

This works great. The resulting markdown file name becomes:

docs/modules/Passion_Fruit__Argentinian_.md

And the Markdown title becomes:

# Module: Passion Fruit (Argentinian)

So all that's needed is for the plugin to remove the "Module: " prefix.

Feature request 3: Remove the "Module: " prefix in the title, either always, or only when the "@module" name is specified.

However, even if the "Module: " prefix was removed, we still wouldn't be able to use the "@module" feature without a fix for problem 1 above. This is because we've lost the meta-data along the way that specifies that this is a Markdown file that belongs in the "Functions" section of the sidebar.

Thus, in order for modules to have arbitrary names, it's essential that we have feature request 1/2 above.

Problem 3 - Duplication

Yet another issue in our Docusaurus website has to do with our FruitType enum.

Notice that in our generated output, there are two separate pages that were generated for FruitType:

1. docs/enums/enums_FruitType.FruitType.md
2. docs/modules/enums_FruitType.md

The first page contains all the "real" information about the enum, such as the values for all the enum members. The second page is just a stub that has a link to the first page, with nothing else of interest. Having the second page be generated at all is a bug. We just want to have the first file be in the "Enums" part of the left-hand side bar, and that's it.

Feature request 3: typedoc-plugin-markdown should not generate spurious pages like this. (Specifically, it happens with types, interfaces, enums, and classes.)

This can be stated more generically: typedoc-plugin-markdown should put everything that is specified in a module entry point on the same Markdown page. I think that this is what we really want typedoc-plugin-markdown to be doing, as this behavior prevents other types of bugs.

Consider the case where I have a module entry point with a few functions, and a few tiny interfaces. In this scenario, TypeDoc would automatically put the interfaces on separate pages. That's bad, because the interfaces / types aren't big enough to warrant putting them on their own dedicated pages. And it makes it harder for end-users to understand what the functions are doing, as they are now warping back and forth between entirely different pages.

The end-user should have control of this. If it would be useful for interfaces/types/enums/classes to be own their own Docusaurus pages, then the end-user should put them in a dedicated entry-point. Easy!

[tgreyuk]

Thanks for detailing this very useful. There is also some history with this in #317.

I am currently refactoring quite a lot of things (removing handlebars, making it easy to implement custom theme etc) with a bunch of other stuff haven't had time to fix - so will look at addressing with the next major version.

Problem 1 - Sections

typedoc-plugin-markdown should generate Markdown files in paths that correspond to the original source files by default.

• You can use --allReflectionsHaveOwnDocument option which will put each symbol on its own page and relevant directory. I don't like this name and looking a making it more flexible. It doesn't fully resolve your issue but is a start.

typedoc-plugin-markdown should look for a @path JSDoc tag in the module-level JSDoc comment, and then respect that. For example, at the top of passionFruit.ts, I would want to have something like this:

• Controlling directory structure by @path is interesting. There is also a @category option https://typedoc.org/tags/category/ which perhaps could also be used to separate the structure, putting each category in its own folder perhaps.

Problem 2 - Markdown Titles

Remove the "Module: " prefix in the title, either always, or only when the "@module" name is specified.

The titles were defined in this way to follow the html theme but agree not needed. Also with next version it should be easy to to override the title component with a custom component in whatever format is required.

Problem 3 - Duplication

typedoc-plugin-markdown should not generate spurious pages like this. (Specifically, it happens with types, interfaces, enums, and classes.)

Agreed that there should be a mechanism to keep all symbols of a module in a single page. Have been looking at it.

[Zamiell]

Tom, thanks for the quick reply.

You can use --allReflectionsHaveOwnDocument option which will put each symbol on its own page and relevant directory. I don't like this name and looking a making it more flexible. It doesn't fully resolve your issue but is a start.

This seems like a really useful option for some use-cases. However, for our basic fruit library, it doesn't help us. When we turn on the option here, it puts each individual helper function on its own Docusaurus page. This isn't what we want - we will have thousands of individual helper functions, and this will result in a left-hand sidebar that will be impossible to navigate.

We need to keep the structure outlined previously: an "autogenerated" Docusaurus sidebar, with "sections" corresponding to directories, and one page per file/module (not one page per reflection).

[Zamiell]

There is also a @category option https://typedoc.org/tags/category/ which perhaps could also be used to separate the structure, putting each category in its own folder perhaps.

We might be able to piggy-back off of the "@category" tag, but I don't think it is a very good idea. Correct me if I am wrong, but the tag seems to be meant to go on individual exports, not a top-level module JSDoc comment.

For example, in our fruit library, say that for some reason, we wanted to refactor the "apple.ts", "banana.ts", and "passionFruit.ts" functions together into a combined "functions.ts".

If we did that, it might still be possible to still get the same Docusaurus output as before, because we could manually annotate @category Apple on the apple functions, and so on.

In this context, "@category" is a shorthand for "this is the module that I want the function to belong to". But this concept doesn't quite map on to what we want to do at a module-level. We don't need to specify which module that a module belongs to. That doesn't make any sense - it's already a module, and it doesn't belong to any other modules. Rather, we want to specify where the module is going to live on the file system.

[tgreyuk]

Ok been making some progress on next branch and made some updates to the build. Please see example project - https://github.com/tgreyuk/typedoc-plugin-markdown/tree/next/examples/fruit-api .

In this case we have exported 2 modules, enums and functions with their own distinct directories. Within the function module we are exporting apples, bananas and passionfruit modules. You may be wondering why enums.md is neccessary, but each module does need an entry point when there is no navigation. But we could hoist all symbols from in a module in one file.

I will carry on playing around but any input would be great. Have a look at the example project and feel free to add some further use cases in there with an MR.

Here is the directory structure:

[Zamiell]

Hell yeah, looking fantastic so far!

Some small tweaks are needed, I think we need to add "hideBreadcrumbs": true, to typedoc.json.

Also, what does "githubPages": false mean?

Also, for the apples function page, the Markdown title is "# apples" instead of "# Apples", so it looks like we need to add "@module Apples" to the JSDoc comment for the file I presume.

I tried to clone the repo and build it like this:

git clone git@github.com:tgreyuk/typedoc-plugin-markdown.git
cd typedoc-plugin-markdown
git checkout next
yarn
cd examples/fruit-api
npx typedoc

But I got a bunch of errors:

error The plugin D:\Repositories\typedoc-plugin-markdown\node_modules\typedoc-bitbucket-theme could not be loaded.
error Error: Cannot find module 'D:\Repositories\typedoc-plugin-markdown\node_modules\typedoc-bitbucket-theme\dist\index.js'. Please verify that the package.json has a valid "main" entry
    at tryPackage (node:internal/modules/cjs/loader:353:19)
    at Function.Module._findPath (node:internal/modules/cjs/loader:566:18)
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:27)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at loadPlugins (D:\Repositories\typedoc-plugin-markdown\node_modules\typedoc\dist\lib\utils\plugins.js:15:30)
    at Application.bootstrap (D:\Repositories\typedoc-plugin-markdown\node_modules\typedoc\dist\lib\application.js:86:33)
    at Object.<anonymous> (D:\Repositories\typedoc-plugin-markdown\node_modules\typedoc\bin\typedoc:26:5)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)

Not sure what to do from here.

[tgreyuk]

Not sure what to do from here.

update the branch and then run the docs script (instead of npx typedoc):

yarn run docs 

basically need to build the package locally first.

Also, what does "githubPages": false mean?

https://typedoc.org/guides/options/#githubpages

I am going to continue looking at output - feel free to add some code samples to play around with.

[Zamiell]

I tried today, and its still broken.

Steps to reproduce:

git clone git@github.com:tgreyuk/typedoc-plugin-markdown.git
cd typedoc-plugin-markdown
git checkout next
yarn
cd examples/fruit-api
yarn run docs
yarn run docs

The first yarn run docs succeeds, but the second one fails. It gives an NX error.

[Zamiell]

In our current fruit API, we are exporting by name, like this:

functions/index.ts

export * as apples from './apples';
export * as bananas from './bananas';
export * as passionFruits from './passionFruits';

This format seems to be needed in order to help TypeDoc + the plugin sort the exports into separate Markdown files.

However, this is not how we actually want to structure our library. Notice that when an end-user consumes our fruit API library as-is, their code will end up looking like the following:

main.ts

import { eatApple } from "fruit-api/dist/functions/apples";

eatApple();

This import statement is not good - it's not conventional in JavaScript/TypeScript code to be importing files from a "dist" path like this. Instead, we want the import path for our end-users to look like this:

import { eatApple } from "fruit-api";

In other words, we want our library to export everything from the root, in a flat structure. That provides the best end-user experience.

As far as TypeDoc and the plugin is concerned, is it absolutely necessary that we have to use named exports like this?

It would be optimal here if we could just export everything flatly at the root (i.e. without any named exports), and then have the plugin simply create output that mirrors the file system. In other words:

• /src/functions/apple.ts --> /docs/functions/apple.ts
• /src/enums/FruitType.ts --> /docs/enums/FruitType.ts

[Zamiell]

Another problem: It looks like the TypeDoc plugin isn't respecting the @module JSDoc tag.

In other words, if we put the following at the top of passionFruits.ts:

/**
 * This is the documentation for passion fruit functions.
 *
 * @module Passion Fruit
 */

Then the first line of the resulting passionFruit.md file will still be "# passionFruit" instead of "# Passion Fruit".

[Zamiell]

Another thing:

In the current setup, TypeDoc creates index files like "functions.md". But these won't be needed for us, because we will be able to see all of the functions easily using the Docusaurus left sidebar.

Furthermore, generating all of these "index" files results in a buggy-looking Docusaurus sidebar with duplicated items like this:

So how about a plugin option called "makeModuleIndexes". If set to false, then the plugin would automatically delete files like "functions.md" for us, so that they don't clutter the output.

[Zamiell]

Also, looks like all of the "Defined in" links are broken.

e.g.

Old:

New:

[tgreyuk]

Is your project on a public repo - it would be good to test against a real project.

[Zamiell]

Yes, I'm using the plugin here: https://github.com/IsaacScript/isaacscript/blob/main/packages/isaacscript-common/docs.sh
And here: https://github.com/IsaacScript/isaacscript/blob/main/packages/isaac-typescript-definitions/docs.sh

[Zamiell]

More bugs:

The source code looks like this:

import { CornerType } from "../enums/CornerType";

export interface Corner {
  readonly type: CornerType;
  readonly position: Readonly<Vector>;
}

So it has 2 fields, but for some reason neither of them are documented.

Edit - you can see it in the Fruit API too, the enum members of FruitType are not generated.

[Zamiell]

Tom,

Any progress on this? I'm eagerly awaiting the leveled-up version of the plugin.

[tgreyuk]

https://www.typedoc-plugin-markdown.org/docs/options#output-options