Better code blocks #17
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds some new functionality to code blocks. Firstly
it allows arbitrary delimiters meaning that code containing the
usual code block terminator can now be parsed correctly. The
syntax for this is:
{delimiter@ocaml[ ... ]delimiter}
The delimiter can contain the chars `[ a-z A-Z 0-9 _ - ]`, the
same as the language tag that comes after the '@' symbol. Note
that there's no way to have a delimited code block without a
language tag.
The second piece of functionality is that code blocks can now have
associated output:
{@ocaml[
... ocaml code ...
][
... odoc formatted output ...
]}
This syntax also supports the delimiters as above. The delimiters
only encode the _code_ block, not the output:
{delim@ocaml[
... ocaml code containing ][ or ]} ...
]delim[
... odoc formatted output ...
]}
The idea is that the odoc formatted output should be well formed and thus
any escaping is done in the usual way.
The output can then contain, for example, error blocks produced by mdx:
{delim@ocaml[
let x = "]}
]delim[
{err@mdx-error[
Line 1, characters 9-10:
Error: String literal not terminated]err}
]}
In addition this also allows the possibility of code blocks to produce
rich output - ie., allowing marked-up such as tables, headings,
images and so on, in such a way that they are associated with the code
block, and hence can be manipulated by a 'test-promote' workflow.
|
Review comment from group review in Cambridge: Could we perhaps make the delimiting generic, e.g.: |
|
While I think having delimiters elsewhere is a worthy goal, we can do that in another PR. |
Julow
reviewed
Jun 1, 2023
| | In_explicit_list -> (List.rev acc, next_token, where_in_line) | ||
| | In_tag -> (List.rev acc, next_token, where_in_line) | ||
| | In_table_cell -> (List.rev acc, next_token, where_in_line) | ||
| | In_code_results -> (List.rev acc, next_token, where_in_line)) |
There was a problem hiding this comment.
Most cases have the same type and could perhaps be written with an or-pattern ?
There was a problem hiding this comment.
Annoyingly not (the return types are different in each case). See the comment immediately below.
| @@ -219,18 +217,19 @@ let emit_verbatim input start_offset buffer = | |||
| let t = trim_trailing_blank_lines t in | |||
| emit input (`Verbatim t) ~start_offset | |||
|
|
|||
| let emit_code_block ~start_offset input metadata c = | |||
| let c = trim_trailing_blank_lines c in | |||
| let emit_code_block ~start_offset content_offset input metadata delim terminator c has_results = | |||
There was a problem hiding this comment.
The location calculations in this function are quite complicated and would deserve some comments.
Why is content_offset needed ?
There was a problem hiding this comment.
I've added a comment.
Julow
approved these changes
Jun 2, 2023
src/lexer.mll
Outdated
| @@ -267,6 +267,9 @@ let raw_markup_target = | |||
| let language_tag_char = | |||
| ['a'-'z' 'A'-'Z' '0'-'9' '_' '-' ] | |||
|
|
|||
| let delim_char = | |||
| ['a'-'z' 'A'-'Z' '0'-'9' '_' '-' ] | |||
There was a problem hiding this comment.
Without - ? This might interfere with the {- ...} syntax with a different parser engine.
There was a problem hiding this comment.
Huh, I thought I had removed that, as per your previous comment. Will fix, thanks!
| @@ -219,18 +217,19 @@ let emit_verbatim input start_offset buffer = | |||
| let t = trim_trailing_blank_lines t in | |||
| emit input (`Verbatim t) ~start_offset | |||
|
|
|||
| let emit_code_block ~start_offset input metadata c = | |||
| let c = trim_trailing_blank_lines c in | |||
| let emit_code_block ~start_offset content_offset input metadata delim terminator c has_results = | |||
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit adds some new functionality to code blocks. Firstly it allows arbitrary delimiters meaning that code containing the usual code block terminator can now be parsed correctly. The syntax for this is:
The delimiter can contain the chars
[ a-z A-Z 0-9 _ - ], the same as the language tag that comes after the '@' symbol. Note that there's no way to have a delimited code block without a language tag.The second piece of functionality is that code blocks can now have associated output:
This syntax also supports the delimiters as above. The delimiters only encode the code block, not the output:
The idea is that the odoc formatted output should be well formed and thus any escaping is done in the usual way.
The output can then contain, for example, error blocks produced by mdx:
In addition this also allows the possibility of code blocks to produce rich output - ie., allowing marked-up such as tables, headings, images and so on, in such a way that they are associated with the code block, and hence can be manipulated by a 'test-promote' workflow.