Add searchbox to generated documentation

[f-f]

We should be able to generate documentation for all the libraries in the package set (and/or just the libraries used in the project), in a similar way to what stack haddock does for Haskell projects.

The use-cases for this are multiple and useful:

• get specialized documentation for your project without having to rely on Pursuit (because you might have dependencies not from the package set and/or on Pursuit)
• more generally, allow an easy way to have an offline Pursuit
• generate a "static Pursuit" for every package set in Spacchetti and host that alongside the package set, which is effectively the Stackage approach.
  Note that this would obsolete spago publish command #27, as it would entirely remove the "where do I upload my library" problem → you would just add it to a Spacchetti package-set, and have it "published" in the next Spacchetti release

How to achieve this:

• it is possible to generate html docs for the whole package-set by running a purs docs command.
  Justin had a good start with acme-spago. He included a spago project in there, but if this is integrated in spago itself, we wouldn't need it (as we would just generate that info on the fly)
• then we'd need a way to search in the existing docs. Since the whole thing needs to be static files (i.e. no backend), we should have a frontend-side search). A simple way to do this would be to generate markdown docs and then index them with something like Lunr (@justinwoo suggested scraping the externs JSON files, but I'm not sure what that would entail, so probably worth detailing more)

[justinwoo]

Going through externs is probably a lot of extra work. Easier to just scrape the docs output.

If you use "markdown" output, it doesn't give you links:

Markdown:

#### `toUnfoldable`

``` purescript
toUnfoldable :: forall f. Unfoldable f => List ~> f
```

Convert a list into any unfoldable structure.

Running time: `O(n)`

HTML: (note the anchor links)

<div class="decl" id="v:toUnfoldable">
  <h3 class="decl__title clearfix">
    <a class="decl__anchor" href="#v:toUnfoldable">#</a
    ><span>toUnfoldable</span>
  </h3>
  <div class="decl__body">
    <pre class="decl__signature">
        <code>
            
            <a href="Data.List.html#v:toUnfoldable" title="Data.List.toUnfoldable">
                <span class="ident">toUnfoldable</span>
            </a>
            
            <span class="syntax">::</span>
            <span class="keyword">forall</span> f<span class="syntax">.</span>
            
            <a href="Data.Unfoldable.html#t:Unfoldable" title="Data.Unfoldable.Unfoldable">
                <span class="ctor">Unfoldable</span>
            </a>
            
            f <span class="syntax">=&gt;</span>
            
            <a href="Data.List.Types.html#t:List" title="Data.List.Types.List">
                <span class="ctor">List</span>
            </a>
            
            <a href="Data.NaturalTransformation.html#t:type (~&gt;)" title="Data.NaturalTransformation.type (~&gt;)">
                <span class="ident">~&gt;</span>
            </a>
            
            f
        </code>
    </pre>
    <p>Convert a list into any unfoldable structure.</p>
    <p>Running time: <code>O(n)</code></p>
  </div>
</div>

[justinwoo]

I'd just like some suggestions on what all people use to search and browse HTML files though, with directory searching instead of just in-page searching

[f-f]

This is the implementation of Pursuit's search: it imports the PureScript compiler to parse the source of the packages and weigh the tokens to rank them in the results. It might be useful to use this information (so it's easier to implement the search), so to avoid reimplementing that stuff it could be useful if this information could be output by purs docs with some flag.

[hdgarrood]

Pursuit doesn’t do any parsing; the information is all available via the JSON which has been generated by purs publish and uploaded to pursuit.

[hdgarrood]

By the way I’m interested in supporting use cases like this in the compiler: see also purescript/purescript#3528 (comment)

[f-f]

Thanks @hdgarrood, that would be indeed wonderful! 👏

It looks like with a json export for Module we'd have all the data Pursuit has from purs publish, so it would be possible to create the same SearchIndex as in there.

The only thing missing at this point would be a frontend way to parse the input query. For the MVP (Minimum Viable Pursuit, clearly) I wanted to try shipping purescript-cst through GHCJS. But since that's going to be integrated in the compiler (doing weird stuff for MVP is ok, but I don't want to ship the whole compiler over GHCJS), I guess the next best approach would be to write a small parser for signatures in PureScript. I wonder if someone has tried a PureScript parser in PureScript?

[hdgarrood]

They have indeed :) https://github.com/purescript/purescript-in-purescript there was an effort to make the compiler self-hosting a while ago, but it was put on hold because of performance issues.

[f-f]

Cool, thanks for the pointer. I had memory of this, just didn't look in the right place :)

It looks like it would be possible to port some of the parsing code (though the project is quite outdated)

[f-f]

@hdgarrood I think the compiler's feature that would allow this is now being tracked in purescript/purescript#3503 right?