feat: add react-metadata package

[joshblack]

Follow up to work in: #5948 and #6048

tl;dr

This work proposes a new way of documenting for our React library. This new approach will:

• Automate the generation of documentation for the props of components (no more writing docs.json files)
• Provide complete and always up-to-date information about components
• Allow for seamlessly interacting with types on our website (no more trying to figure out what a type is, you can just drill down into it instead)
• Unlock the ability to allow for generation of documentation for any component (such as for shared components)

About

This Pull Request adds an internal package, @primer/react-metadata, that helps with getting the TypeScript information from a JavaScript module. The goal is to use this to automate the generation of our docs.json files and corresponding component metadata 🤞 There is an example using this package in packages/react/script/generate-metadata.mts

This exploration has been incredibly difficult due to the challenge in getting structured type information from a file in a way that works with some of our wishlist items for the docs site. Ideally, we'd be able to capture the prop information for components in order to display them on the site. This information could look like:

type ExampleProps = {
  /** Example prop description */
  className?: string;
};

And we would like this type to serialize into a format that we could use to generate prop tables on the website. However, component prop types are not always this simple. Some examples include:

• Intersection types

  // How do we correctly "flatten" this intersection type in order to determine
  // what to show in a docs page?
  type ExampleProps = A & B & { className?: string }

• Type references

  // How do we correctly "unwrap' type references that decorate types
  // that we provide
  type ExampleProps = React.PropsWithChildren<{ className?: string }>

• Expanded types

  // How do we either "expand" types like `Variant` or reference them
  // (similar to React spectrum) so that the types are fully available
  // on the docs site
  type Variant = 'default' | 'condensed' | 'spacious'
  
  type ExampleProps = {
    variant?: Variant
  }

To make matters more difficult, we would also like to provide structured type information about non-component related exports, as well 😰 Hooks are a great example where it can be helpful to automate our *.docs.json files through our source code and provide a good experience for these our site.

To this end, the react-metadata package here attempts to provide a structured representation of the symbols in a file. The hope is to be able to use the typescript compiler in order to collect structured data that can then be consumed to automatically generate great documentation pages for components, hooks, and more 🎉

Implementation

The getMetadataFromSourceFile function relies on several different TypeScript concepts in order to parse the information that we want into corresponding types. In particular:

• ts.Node to understand the AST of the file
• ts.Symbol to help with things like functions or interfaces that have multiple definitions
• ts.Type to get the underlying type information when this is unavailable from a ts.Node. This is helpful for inferred types for functions

When going through a file, we get the exported ts.Node nodes and get type information from them using either ts.Node or drilling down into ts.Type, as needed. We emit a custom format for representing types. This can change over time but is a rough combination of ts.Node and ts.Type types. The goal is to be able to format things as-authored in our docs site versus offering the underlying type value. For example:

function useExample(a?: number) {}

When printing this hook, we would most likely want to display it close to how it was authored. Otherwise, if we use the type information exclusively, it would look like:

function useExample(a: number | undefined): void

While this type is strictly true (this is what the ? token represents) it seemed unlikely that we would want to expose this in all cases. This is something that we can definitely change in the future though.

In many cases, we won't be able to rely strictly on ts.Node as types like functions and interaces can be merged or have types that are inferred versus explicitly annotated. In these cases, we need to get the type of the underlying symbol and parse our type information from a ts.Type instead of a ts.Node. For example:

function test1() {}

function test2() {
  return 1
}

function test3() {
  return <div />
}

In these examples, we do not have an explicit return value and cannot use the ts.Node for these function declarations to parse our type information. Instead, we need to use the TypeScript compiler to access the inferred ts.Type.

Remaining work

• Provide functions for formatting and discovering types from metadata, it should be easy to generate the props table for a component
• Finding out the best way to pull component types out and structure them in the style of our components.json file
• Potentially explore adding support for metadata for a component (like stories, tests, etc)
• Integrate JSDoc comments into each node (where appropriate)
• Add support for interface types
• Find a way to correctly associate TypeReference nodes to provide a way to "drill down" in to type references in our docs site

Next steps

If folks on the team are encouraged by this approach, we could put together a pitch to begin the work towards automatically generating our docs.json files. Most likely we would want to do this gradually, opening this up per-component to make sure that the package supports all the syntax that we need. This work would be completed when all of our docs.json files are automatically generated using this package.

[changeset-bot]

⚠️ No Changeset found

Latest commit: e908bd0

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[github-actions]

👋 Hi, this pull request contains changes to the source code that github/github depends on. If you are GitHub staff, we recommend testing these changes with github/github using the integration workflow. Thanks!

[github-actions]

size-limit report 📦

Path	Size
packages/react/dist/browser.esm.js	96.22 KB (0%)
packages/react/dist/browser.umd.js	96.48 KB (0%)