stdlib: markdown conversor to erlang+html

[kikofernandez]

conversor from markdown to erlang+html. shell_docs needs documentation in erlang+html format, so that it can render the documentation attributes correctly in the shell. documentation attributes are written in markdown and, with this erlang+html conversion tool, shell_docs can interpret documentation attributes.

[github-actions]

CT Test Results

    5 files    540 suites   1h 25m 28s ⏱️
4 216 tests 4 123 ✅  93 💤 0 ❌
9 975 runs  9 853 ✅ 122 💤 0 ❌

Results for commit 767ea13.

♻️ This comment has been updated with latest results.

To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass.

See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally.

Artifacts

• Complete CT logs (Download Logs)
• HTML Documentation (Download HTML Docs)
• Windows Installer

// Erlang/OTP Github Action Bot

[josevalim]

Could you have forgotten to push doc_html.erl? :)

Btw, we have a module in Elixir for converting markdown to ansi/shell. I assume that if you only care about ansi/shell, it is actually a subset of the problem of converting all of markdown to html: https://github.com/elixir-lang/elixir/blob/main/lib/elixir/lib/io/ansi/docs.ex

Tests are here: https://github.com/elixir-lang/elixir/blob/main/lib/elixir/test/elixir/io/ansi/docs_test.exs

[kikofernandez]

@josevalim thanks! The thing is that not all of my tests pass, so I did not bother to push what I had...
We are not immediately going from markdown to ansi/shell, the approach that we thought could be easier was to go from markdown to erlang+html format (https://www.erlang.org/doc/man/shell_docs#type-chunk_element_attr), so that we do not need to deal with much of the formatting.

We will see how it goes :) Otherwise, we probably will re-use parts of the Elixir code :) That said, this PR was highly influenced by the Elixir docs.ex module, where docs.ex is great work and really understandable!

[lukaszsamson]

From the LSP side it would be more convenient if we could skip erlang+html and get markdown. Currently LSPs need to implement a reverse conversion from erlang+html to markdown

[josevalim]

I believe the docs chunk will have markdown for Erlang from OTP 27 onwards if using -doc.

[kikofernandez]

Yes, docs chunks will have markdown. The erlang+html is just for transforming the erlang shell, so that one can write h(lists). The function h/1 is the one receiving markdown and converting to erlang+html, because that's what it knows how to render

[garazdawi]

@lukaszsamson From an LSPs perspective you may want to use the new edoc_html_to_markdown module added in Erlang/OTP 27 that converts application/html+erlang to markdown. This is for projects that use edoc to create their documentation, or Erlang/OTP before 27.

But, as the others here have said, for all of the Erlang/OTP docs after 27 you will get the markdown directly from beam file using code:get_doc/1. There will of course always remain applications out there that uses edoc, so being able to handle both formats is a good thing.

[lukaszsamson]

@lukaszsamson From an LSPs perspective you may want to use the new edoc_html_to_markdown module added in Erlang/OTP 27 that converts application/html+erlang to markdown. This is for projects that use edoc to create their documentation, or Erlang/OTP before 27.

But, as the others here have said, for all of the Erlang/OTP docs after 27 you will get the markdown directly from beam file using code:get_doc/1. There will of course always remain applications out there that uses edoc, so being able to handle both formats is a good thing.

Nice. Like you said the translation to markdown will need to stay until support for OTP 26 is dropped but that's not going to happen soon (ElixirLS currently still supports OTP 22)