feat: use output from new doc-ci with odoc 3 features in the package documentation area (#3124)

* Get docs from new docs CI

Co-authored-by: Jon Ludlam <jon@recoil.org>

* Docs: Update to new odoc3 json format

Co-authored-by: Jon Ludlam <jon@recoil.org>

* Docs: use new `status.json` from odoc 3

Co-authored-by: Jon Ludlam <jon@recoil.org>

* Docs: use header field

It was added in ocaml/odoc#1314 and needs not be ignored

Co-authored-by: Jon Ludlam <jon@recoil.org>

* Docs: fix search script location change with odoc_driver

Co-authored-by: Jon Ludlam <jon@recoil.org>

* Docs: fix search results' url

Co-authored-by: Jon Ludlam <jon@recoil.org>

* Docs: compute sidebar from toc json

Co-authored-by: Jon Ludlam <jon@recoil.org>

* Docs: do not reload global toc on toc click

Co-authored-by: Jon Ludlam <jon@recoil.org>

* Docs: uniformize toc look

Co-authored-by: Jon Ludlam <jon@recoil.org>

* Docs: small refactor of breadcrumbs

Co-authored-by: Jon Ludlam <jon@recoil.org>

* Docs: allow breadcrumbs without href

Odoc 3 allows to have breadcrumbs without href. Currently, the breadcrumbs are
still computed by ocaml.org, but soon it will be taken from odoc3's json output.

Co-authored-by: Jon Ludlam <jon@recoil.org>

* Docs breadcrumbs: Adding `page` kind

With odoc 3 and hierarchical documentation, breadcrumbs will include pages.

They are rendered with `/` between them (while other components are rendered
with `.` between them). So We can have `library1/Module.Submodule`.

Pages cannot yet happen since we still compute breadcrumbs ourselves.

Co-authored-by: Jon Ludlam <jon@recoil.org>

* rename `library_path` to `path`: pages also have/are in breadcrumbs

Co-authored-by: Jon Ludlam <jon@recoil.org>

* Docs breadcrumbs: use the one computed by odoc without processing

Co-authored-by: Jon Ludlam <jon@recoil.org>

* Add dependency on ppx_deriving_yojson

* Docs: hide toc until alpine has loaded

From https://alpinejs.dev/directives/cloak :

> Sometimes, when you're using AlpineJS for a part of your template, there is a
> "blip" where you might see your uninitialized template after the page loads,
> but before Alpine loads.
>
> `x-cloak` addresses this scenario by hiding the element it's attached to until
> Alpine is fully loaded on the page.
>
> For `x-cloak` to work however, you must add the following CSS to the page.
>
> ```
> [x-cloak] { display: none !important; }
> ```

* Add source pages to docs

* Render html in breadcrumbs

* Add style for rendered source code

* Handle redirection

Since odoc 3, and the replacement of voodoo by odoc_driver as the odoc driver,
the layout for docs has changed. Since we do not want to break previously
existing links, we redirect the old layout to the new one.

* Sidebar: open sub-toc when clicking on a page from the sidebar

* Doc: handle asset loading

Documentation assets are not present as json files. Therefore, if we do not find
a .json target, we try to load it as an asset.

This is not ideal: assets always require two requests to the documentation
server. 404 pages also trigger two requests...

* Fix package files' links

* Fix package files appearing in global toc

* Fix theme for source rendering

* Revert keeping the sidebar on new page loading

* Finally implement avoid reloading sidebar

* Docs: Improve source code target highlighting

* Improve css of source rendering line numbers

* Improve links to source style

* Docs: only show right sidebar when it exists

* Improve error handling for status.json

* Fix enter on search result

Odoc 3's search has already a leading `/`.

* Fix redirection on edge cases

`old_path` and `new_path` do not include the prefix, which makes the detection
of redirection harder.

Previously, the url `/p/odoc/3.0.0/doc/odoc.odoc/Odoc_odoc/Sidebar/index.html`
would trigger the rewriting:
`doc/Odoc_odoc/Sidebar/index.html`
->
`doc/odoc.odoc/Odoc_odoc/Sidebar/index.html`

which is a bug. Adding a / in the rewriting rules solves it

* Put documentation-related code in a documentation file

* Consider non-html files as assets

* Turn sidebar into Navmap earlier

* Increment cache version and update comments around it

---------

Co-authored-by: Jon Ludlam <jon@recoil.org>

Author: panglesd

asset/css/doc.css

@@ -1,5 +1,7 @@
+[x-cloak] { display: none !important; }
+
 div.odoc {
-  max-width: 56rem /* 896px */;
+  max-width: 67rem;
   position: relative;
 }
 
@@ -186,12 +188,12 @@ div.odoc span.at-tag {
   font-weight: bold;
 }
 
-div.odoc a {
+div.odoc a:not(.source_container a):not(.source_link) {
   font-weight: bold;
   color: #cc4e0c;
 }
 
-.dark div.odoc a {
+.dark div.odoc a:not(.source_container a):not(.source_link) {
   text-decoration: underline;
   color:white;
 }
@@ -314,6 +316,24 @@ div.odoc .comment-delim {
   border-color: rgb(32, 68, 165);
 }
 
+.navmap-tag.page-tag::after {
+  content: "P";
+}
+.page-tag{
+  color: rgb(32, 68, 165);
+  background-color: rgb(32, 68, 165);
+  border-color: rgb(32, 68, 165);
+}
+
+.navmap-tag.source-tag::after {
+  content: "S";
+}
+.source-tag{
+  color: rgb(97, 8, 138);
+  background-color: rgb(97, 8, 138);
+  border-color: rgb(97, 8, 138);
+}
+
 span.icon-expand > .navmap-tag,
 span.no-expand > .navmap-tag {
   color: white;
@@ -344,10 +364,12 @@ span.arrow-expand.open {
 }
 
 span.sign-expand::before {
-  content: " \002B";
+  content: "\002B";
   display: flex;
+  justify-content: center;
   align-items: center;
   font-size: 1.25rem;
+  width: 1.25rem;
   margin-top: -0.25rem;
 }
 
@@ -359,3 +381,279 @@ span.sign-expand.open::before {
 /* Lists of modules */
 
 .modules { list-style-type: none; padding-left:0; }
+
+/* Taken from odoc's css sheet */
+
+.source_container {
+
+  /* light gruvbox theme colors */
+  --bg_h: #f9f5d7;
+  --bg:   #f6f8fa; /*#fbf1c7;*/
+  --bg_s: #f2e5bc;
+  --bg1:  #ebdbb2;
+  --bg2:  #d5c4a1;
+  --bg3:  #bdae93;
+  --bg4:  #a89984;
+  --bg_highlighted: #fff3a0;
+
+  --fg:  #282828;
+  --fg1: #3c3836;
+  --fg2: #504945;
+  --fg3: #665c54;
+  --fg4: #7c6f64;
+
+  --red:    #9d0006;
+  --green:  #79740e;
+  --yellow: #b57614;
+  --blue:   #076678;
+  --purple: #8f3f71;
+  --aqua:   #427b58;
+  --orange: #af3a03;
+  --gray:   #928374;
+
+  --red-dim:    #cc2412;
+  --green-dim:  #98971a;
+  --yellow-dim: #d79921;
+  --blue-dim:   #458598;
+  --purple-dim: #b16286;
+  --aqua-dim:   #689d6a;
+  --orange-dim: #d65d0e;
+  --gray-dim:   #7c6f64;
+
+  /* odoc colors */
+  --odoc-blue: #5c9cf5;
+  --odoc-bg: #FFFFFF;
+  --odoc-bg1: #f6f8fa;
+  --odoc-fg: #333333;
+  --odoc-fg1: #1F2D3D;
+
+}
+
+body.dark .source_container {
+  /* dark gruvbox theme colors */
+  --bg_h: #1d2021;
+  --bg:   #282828;
+  --bg_s: #32302f;
+  --bg1:  #3c3836;
+  --bg2:  #504945;
+  --bg3:  #665c54;
+  --bg4:  #7c6f64;
+  --bg_highlighted: #3c3836;
+
+  --fg:  #fbf1c7;
+  --fg1: #ebdbb2;
+  --fg2: #d5c4a1;
+  --fg3: #bdae93;
+  --fg4: #a89984;
+
+  --red:    #fb4934;
+  --green:  #b8bb26;
+  --yellow: #fabd2f;
+  --blue:   #83a598;
+  --purple: #d3869b;
+  --aqua:   #8ec07c;
+  --gray:   #928374;
+  --orange: #fe8019;
+
+  --red-dim:    #cc2412;
+  --green-dim:  #98971a;
+  --yellow-dim: #d79921;
+  --blue-dim:   #458588;
+  --purple-dim: #b16286;
+  --aqua-dim:   #689d6a;
+  --gray-dim:   #a89984;
+  --orange-dim: #d65d0e;
+
+  /* odoc colors */
+  --odoc-blue: #5c9cf5;
+  --odoc-bg: #202020;
+  --odoc-bg1: #252525;
+  --odoc-fg: #bebebe;
+  --odoc-fg1: #777;
+}
+
+.source_container .source_line_column a {
+  text-decoration: none;
+  color:var(--fg4);
+}
+
+
+.source_container {
+  --code-color: var(--fg);
+  --code-background: var(--bg);
+  --code-highligthed-background: var(--bg_highlighted);
+
+  --source-link-color: var(--fg4);
+  --source-line-column: var(--fg3);
+  --source-line-column-bg: var(--bg_h);
+
+  --source-code-comment: var(--gray);
+  --source-code-docstring: var(--green-dim);
+  --source-code-lident: var(--fg1);
+  --source-code-uident: var(--blue);
+  --source-code-literal: var(--yellow);
+  --source-code-keyword: var(--red);
+  --source-code-underscore: var(--fg3);
+  --source-code-operator: var(--purple);
+  --source-code-parens: var(--orange-dim);
+  --source-code-separator: var(--orange-dim);
+}
+
+.source_container .source_code a:not(:hover) {
+  text-decoration: inherit;
+}
+
+/* Source links float inside preformated text or headings. */
+.odoc a.source_link {
+  float: right;
+  color: gray;
+  font-size: initial;
+  text-decoration: none;
+}
+
+.odoc a.source_link:hover {
+  color: unset;
+}
+
+.source_container {
+  display: flex;
+  padding: 0;
+}
+
+.source_line_column {
+  padding-right: 0.5em;
+  text-align: right;
+  color: var(--source-line-column);
+  background: var(--source-line-column-bg);
+}
+
+.source_line {
+  padding: 0 1em;
+}
+
+.source_code {
+  flex-grow: 1;
+  background: var(--code-background);
+  padding: 0 0.3em;
+  color: var(--code-color);
+}
+
+
+/* Linked highlight */
+.source_code *:target {
+  box-decoration-break: clone;
+  -webkit-box-decoration-break: clone; /* for Safari/Chrome */
+  background-color: var(--code-highligthed-background);
+  padding: 3px 0 3px;
+  /* Extend the background without expanding the element's size. Without expansion
+   of background, lines background don't touch each other and make things look
+   weird on multilines targets */
+}
+
+/* Keywords */
+.source_code :is(.AND, .ANDOP, .AS, .ASSERT,
+.BAR, .BEGIN,
+.CLASS, .CONSTRAINT,
+.DO, .DONE, .DOWNTO,
+.ELSE, .END, .EXCEPTION, .EXTERNAL,
+.FOR, .FUN, .FUNCTION, .FUNCTOR,
+.IF, .IN, .INCLUDE, .INHERIT, .INITIALIZER,
+.LAZY, .LESSMINUS, .LET, .LETOP,
+.MATCH, .METHOD, .MINUSGREATER, .MODULE, .MUTABLE,
+.NEW, .NONREC,
+.OBJECT, .OF, .OPEN,
+.PERCENT, .PRIVATE,
+.REC,
+.SEMISEMI, .SIG, .STRUCT,
+.THEN, .TO, .TRY, .TYPE,
+.VAL, .VIRTUAL,
+.WHEN, .WITH, .WHILE)
+{
+  color: var(--source-code-keyword);;
+}
+
+/* Separators */
+.source_code :is(.COMMA, .COLON, .COLONGREATER, .SEMI) {
+  color: var(--source-code-separator);
+}
+
+/* Parens
+   `begin` and `end ` are excluded because `end` is used in other, more
+   keyword-y contexts*/
+.source_code :is(.BARRBRACKET,
+.LBRACE,
+.LBRACELESS,
+.LBRACKET,
+.LBRACKETAT,
+.LBRACKETATAT,
+.LBRACKETATATAT,
+.LBRACKETBAR,
+.LBRACKETGREATER,
+.LBRACKETLESS,
+.LBRACKETPERCENT,
+.LBRACKETPERCENTPERCENT,
+.LPAREN,
+.RBRACE,
+.RBRACKET,
+.RPAREN)
+{
+  color: var(--source-code-parens);
+}
+
+/* Prefix operators */
+.source_code :is(.ASSERT, .BANG, .PREFIXOP,
+/* Infix operators.
+   A choice had to be made for equal `=` which is both a keyword and an operator.
+   It looked better having it as an operator, because when it is a keyword,
+   there are already loads of keyword around.
+   It would look even nicer if there was a way to distinguish between these
+   two cases.*/
+.INFIXOP0, .INFIXOP1, .INFIXOP2, .INFIXOP3, .INFIXOP4,
+.BARBAR, .PLUS, .STAR, .AMPERAMPER, .AMPERAND, .COLONEQUAL, .GREATER, .LESS,
+.MINUS, .MINUSDOT, .MINUSGREATER, .OR, .PLUSDOT, .PLUSEQ, .EQUAL)
+{
+  color: var(--source-code-operator);
+}
+
+/* Upper case ident
+   `true` and `false` are considered uident here, because you can bind them in a
+   constructor defintion :
+   ```ocaml
+   type my_bool =
+     | true of string
+     | false
+     | Other of int
+   ```
+*/
+.source_code :is(.UIDENT, .COLONCOLON, .TRUE, .FALSE) {
+  color: var(--source-code-uident);
+
+}
+
+/* Lower case idents.
+   Quotes are here because of `type 'a t = 'a list`,
+   and question mark and tildes because of
+   ```ocaml
+   let f ~a ?b () = Option.map a b
+   ```
+*/
+.source_code :is(.LIDENT, .QUESTION, .QUOTE, .TILDE) {
+  color: var(--source-code-lident);
+}
+
+/* Litterals */
+.source_code :is( .STRING, .CHAR, .INT, .FLOAT, .QUOTED_STRING_EXPR, .QUOTED_STRING_ITEM) {
+  color: var(--source-code-literal);
+}
+
+.source_code :is(.UNDERSCORE) {
+  color: var(--source-code-underscore);
+}
+
+.source_code :is(.DOCSTRING) {
+  color: var(--source-code-docstring);
+}
+
+.source_code :is(.COMMENT) {
+  color: var(--source-code-comment);
+}

dune-project

@@ -72,6 +72,7 @@
   ezjsonm
   lambdasoup
   ptime
+  ppx_deriving_yojson
   (cmdliner
    (>= 1.1.0))
   xmlm

ocamlorg.opam

@@ -49,6 +49,7 @@ depends: [
   "ezjsonm"
   "lambdasoup"
   "ptime"
+  "ppx_deriving_yojson"
   "cmdliner" {>= "1.1.0"}
   "xmlm"
   "uri"