Spike: generating a CRD reference

[fhennig]

Many operators are missing a CRD reference for the ProductCluster resource. Instead of manually creating it and having to maintain it, it would be nice if it was possible to generate it from the rust code/the generated CRD files.

This should be spiked for one operator and then put into the templating if possible.

Tested tools

• https://github.com/ahmetb/gen-crd-api-reference-docs
  • uses .go source files as a basis to generate the documention -> doesn't work for us
  • The generated docs look really nice though: https://fluxcd.io/flux/components/source/api/#source.toolkit.fluxcd.io/v1beta2.GitRepositorySpec
  • The generated docs are divided into sub-objects. The names of these sub objects are not found in the CRD anymore - they are only in the go code (Or in our Rust code). We might need to generate from the Rust code too.
• https://github.com/giantswarm/crd-docs-generator
  • I could generate something, it generates a markdown file by default.
  • I don't know if this is the generator or the generated CRD itself, bit it generates a lot of fields that are just object.
  • Inlined enums are rendered in a shitty way.
• https://github.com/srfrnk/crd-api-doc-gen
  • Not tested, but also uses the CRD file as a basis. Also doesn't look maintained

I noticed that if I generate docs for i.e. the DruidCluster, it also generates the docs for all the embedded types, i.e. the S3Resource. I think those should be documented separately.

Conclusion

Good documentation cannot be generated from the CRD file, as a lot of information is missing from it. The one tool that generates nice docs uses the underlying source files as a basis for the docs. This only works with go files.

There is no good tool to do this at the moment.

My preferred solution would be something that integrates with the rust code, like the go solution. That seems way more work than initially thought though.

---

The config I used for the CRD docs generator:

template_path: templates/crd.template

output_path: output

source_repositories:
  - url: https://github.com/stackabletech/druid-operator
    organization: stackabletech
    short_name: druid-operator
    commit_reference: main
    crd_paths:
      - deploy/crd
    cr_paths:
    metadata:
      druidclusters.druid.stackable.tech:
        owner:
          - https://github.com/orgs/giantswarm/teams/team-honeybadger
        topics:
          - apps

[fhennig]

Update: Sebastian found this: https://github.com/crdsdev/doc

I.e.: https://doc.crds.dev/github.com/stackabletech/trino-operator@0.8.0

@sbernauer

[maltesander]

This looks pretty good. This probably also means we should improve our documentation on the exposed structs?

[fhennig]

It does look nice. Not sure how we'd integrate it with the docs, but maybe it can just be visually distinct, it doesn't matter. We can give it the stackable colorscheme though.

Yes, we'd need to make the docs nicer :D

Once thing that it doesn't do but that would be cool is factoring out common elements. I.e. the opa configuration or an S3 connection, that's a common fragment that would only need to be documented once. But I think that can only be seen in the rust code.

[sbernauer]

We should do that regardless of that, but yes in that case most notably ;)

[fhennig]

I think it would be nice to use this for our docs, I'm making a new ticket for it though, because this one has gotten a bit messy.

#328