Generator for asciidoc representation of all ECS fields

[webmat]

This code generates both the index page for the fields section, and the detailed page for each fieldset.

TODO

• Fix fieldset order: Base first, then alphabetic
• Fix field order: preserve original order from schema files
• Default short description for fieldsets to description. Raise if description has newlines.
• Render table with complete field set details
• Revert back to alphabetical sorting for fields
• Render the footnotes as well?
• Mention where a field set is expected to be reused.
  • In fieldset, list acceptable destination fieldsets.
  • In destination fieldset, mention which fieldsets are expected here, as contractions linking to their canonical documentation.
• Replace markdown in schemas with asciidoc markup (e.g. hyperlinks)

How to visualize this PR

Via GitHub's rendering of asciidoc, in my branch

• Field Index Page
• Field Details Page

Or in the ECS repo

make generate
make docs

http://localhost:8000/ecs-fields.html 👀

[webmat]

@dedemorton Hey this uses the 3 column layout you were proposing in elastic/beats#9519. I'm doing so with a twist: we're packing a bit of information vertically, for each row:

• Middle column is short description (1 line) and on line below, the example for the field
• Rightmost column is ECS level followed by the field's datatype.

I'm not in love with how exactly this first attempt looks, but I think it's a good start :-)

cc @karenzone

[webmat]

Index page:

Details page:

[webmat]

Updated screenshots.

Index page

Notice Base is now first.

Fieldset details page

• Still has summary table at the beginning
• Below, each field is detailed in full
• Trimmed down the summary table, since we now have details below
• Fields are now displayed in the same order as they're listed in schemas/*.yml

[dedemorton]

@webmat I like the second layout is better (the table in the first iteration was a little confusing visually when you added the level and type info). It would be nice for the entries in the table to link to the full description under field details.

@karenzone I pinged you about this in slack, but wanted to ask if you can make sure this content builds OK with the asciidoctor version of the doc build.

tl;dr for @webmat: we're eventually migrating to asciidoctor, which is more restrictive than asciidoc.

[webmat]

@dedemorton Yes, linking to the detail section was next on my TODO :-) As for the table, I agree the second is better. But I'm, still experimenting to find just the right way to integrate the level, I'm still not satisfied. So ideas are welcome there as well.

I'm more than happy to build using both versions of the tooling, to make sure things will remain ok when we switch over. Just point me in the right direction :-)

[webmat]

Actually since ECS now has short descriptions everywhere, 4 columns renders pretty good. My favourite below is "type last". I'm also attaching a bunch of 3-col experiments as a zip. I really prefer 4 cols.

level last

type last

3 columns

3-col.zip

[webmat]

Actually, here's another format that I think works best. Going with this for now.

I call this one Mike ;-)

[webmat]

HTTP is really cramped, though 🤔

[dedemorton]

HTTP is really cramped, though

Yah the overall layout of tables in really bad. Unfortunately the knobs for adjusting cell padding in asciidoc don't work with our doc build. I haven't tested on asciidoctor. I've been wanting a css change for quite awhile, but that's not happened yet.

To build with asciidoctor, run the docker-based version of the doc build (build_docs instead of build_docs.pl) and include the --asciidoctor flag. Sometimes one problem can cause many messages, so don't lose hope. :-) Reach out to Nik E if you have questions about the asciidoctor build.

[webmat]

I realize I haven't posted a full page rendering of the listing of field sets, yet. With the new short descriptions, the page looks great :-)

Also, moving back to the layout initially proposed in #334, which has all of the information directly in the table (no need for details section below):

[webmat]

First experiment displaying what field sets may be nested under host. Feedback welcome.

[webmat]

Looking for ideas on how to best word the section about field reuses at the bottom of the fieldset detail page.

Here's a screenshot of user.*. This fieldset is special, it's the only one with another fieldset that can be nested under it (group) and that is expected to be nested in other places.

Input on the wording of this section is particularly welcome 😆 This gets the job done, but it's not concise.

[webmat]

@ruflin As of now, this PR doesn't render footnotes yet. Only agent and cloud have one, and I'm not sure what to do with it.

I wonder if we should drop this attribute altogether. If we need to flesh out details about a section, we should do it in the main description, I think.

What do you think?

[webmat]

@karenzone Your input would be welcome on how best to display information about field reuse.

Where a field set is expected is currently two sentences (just like in the readme), whereas what's expected to be nested under a field set is a table.

Should we do all sentences, no tables? Should we do two tables? Only one table, with an added column that explains which direction the reuse goes?

See the screenshot above, where user.* is on both sides of the field reuse :-)

[webmat]

Ok, this is ready for final review, IMO.

We can revisit how reusability is rendered as a separate PR. What we currently have there is functional, though.

[ruflin]

I skimmed through the code and looked at the docs locally. What is most important for me in this PR is that we have the docs in asciidocs and can iterate on top of it. There are a few things I would change but lets discuss it later as it's just about pages and formatting, not the general idea.