Docs: Auto-generate human-readable version of Gutenberg block grammar

[mcsf]

Description

Related: #6030, as another effort to make the Gutenberg block grammar parser-agnostic.

I'm opening this PR to get the idea out, and so we can iterate on presentation. GitHub's flavor of Markdown doesn't like style tags, but this is the HTML generated with the intended styling:

Changes

• Block grammar: Reduce and rename for clarity, add example display name for Block_Attributes rule
• Docs: Add autogenerated doc on Gutenberg block grammar
  • Add PEG.js grammar to parse Gutenberg block grammar
  • Add bin/generate-public-grammar.sh adapter to generate doc from PEG.js syntax tree

/cc @youknowriad @dmsnell @mtias

[dmsnell]

wow! this is really developed-out, isn't it!

we also have a parser for the grammar already built, and I'm curious why the changes are made to the grammar in this PR, which seems like it's conflating a few issues.

const parser = require( './node_modules/pegjs/lib/parser.js' );
const fs = require( 'fs' )
const grammarSource = fs.readFileSync( './blocks/api/post.pegjs', 'utf8' );
const grammar = parser.parse( grammarSource );

with these four lines we get a full AST of the grammar which we can use to provide an alternate pretty-printer and I don't think that will involve as much code to maintain

[dmsnell]

Also check out existing projects:

• http://dundalek.com/GrammKit/

[mcsf]

Thanks for looking so quickly!

wow! this is really developed-out, isn't it!

:) Not really, but bash-assisted gluing helps me to prototype more quickly.

we also have a parser for the grammar already built

True, but we are to assume that to be part of the public interface of the pegjs module? If so, I agree it could be pretty handy. The other bit that made me go for a grammar fork was the ease with which all code blocks can be dropped during parsing, as seen in pegjs-strip.

That said, the whole diff is telling of my path as I was learning the bits (at some point I was consuming pegjs-strip's output and transforming it with successive sed commands in a shell script 🙈). I'd be happy to simplify tons using pegjs's internal parser if that use is deemed OK.

GrammKit

I'd seen that, and it's pretty cool, but I'm not currently convinced those visualizations would be useful.

[dmsnell]

True, but we are to assume that to be part of the public interface of the pegjs module?

For now it seems solid enough. That project will always need to parse the pegjs because it's actually building the parser. If it were to ever change we should be able to extract it easily enough.

I'm not currently convinced those visualizations would be useful.

That's fair, although it covers most of what we spoke about. The point was that we can parse the .pegjs file and print out the AST in a way that's probably simpler, more reliable, and more flexible than mangling strings.

[dmsnell]

Also, I think some of the renames in the grammar are good ideas - would make a good independent PR! 😄

[mcsf]

Also, I think some of the renames in the grammar are good ideas - would make a good independent PR! 😄

Woot: #6127

[mcsf]

@dmsnell: took me a while to circle back to this, but I've switched to an all-JS solution, as discussed.

[mcsf]

@mtias: how do you see us formatting this document? What is the least we must do here before merging?

[dmsnell]

get it in an iterate 👍

[mtias]

Can we proceed with exposing this in https://wordpress.github.io/gutenberg/ ?