-
Notifications
You must be signed in to change notification settings - Fork 132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add searchbox to generated documentation #89
Comments
Going through externs is probably a lot of extra work. Easier to just scrape the docs output. If you use "markdown" output, it doesn't give you links: Markdown: #### `toUnfoldable`
``` purescript
toUnfoldable :: forall f. Unfoldable f => List ~> f
```
Convert a list into any unfoldable structure.
Running time: `O(n)` HTML: (note the anchor links) <div class="decl" id="v:toUnfoldable">
<h3 class="decl__title clearfix">
<a class="decl__anchor" href="#v:toUnfoldable">#</a
><span>toUnfoldable</span>
</h3>
<div class="decl__body">
<pre class="decl__signature">
<code>
<a href="Data.List.html#v:toUnfoldable" title="Data.List.toUnfoldable">
<span class="ident">toUnfoldable</span>
</a>
<span class="syntax">::</span>
<span class="keyword">forall</span> f<span class="syntax">.</span>
<a href="Data.Unfoldable.html#t:Unfoldable" title="Data.Unfoldable.Unfoldable">
<span class="ctor">Unfoldable</span>
</a>
f <span class="syntax">=></span>
<a href="Data.List.Types.html#t:List" title="Data.List.Types.List">
<span class="ctor">List</span>
</a>
<a href="Data.NaturalTransformation.html#t:type (~>)" title="Data.NaturalTransformation.type (~>)">
<span class="ident">~></span>
</a>
f
</code>
</pre>
<p>Convert a list into any unfoldable structure.</p>
<p>Running time: <code>O(n)</code></p>
</div>
</div> |
I'd just like some suggestions on what all people use to search and browse HTML files though, with directory searching instead of just in-page searching |
This is the implementation of Pursuit's search: it imports the PureScript compiler to parse the source of the packages and weigh the tokens to rank them in the results. It might be useful to use this information (so it's easier to implement the search), so to avoid reimplementing that stuff it could be useful if this information could be output by |
Pursuit doesn’t do any parsing; the information is all available via the JSON which has been generated by |
By the way I’m interested in supporting use cases like this in the compiler: see also purescript/purescript#3528 (comment) |
Thanks @hdgarrood, that would be indeed wonderful! 👏 It looks like with a json export for The only thing missing at this point would be a frontend way to parse the input query. For the MVP (Minimum Viable Pursuit, clearly) I wanted to try shipping |
They have indeed :) https://github.com/purescript/purescript-in-purescript there was an effort to make the compiler self-hosting a while ago, but it was put on hold because of performance issues. |
Cool, thanks for the pointer. I had memory of this, just didn't look in the right place :) It looks like it would be possible to port some of the parsing code (though the project is quite outdated) |
@hdgarrood I think the compiler's feature that would allow this is now being tracked in purescript/purescript#3503 right? |
I guess so, since there isn't a dedicated issue for it. We probably should have a dedicated issue for it, though. |
Actually no, sorry, I don't think a dedicated issue makes sense, because the design I have in mind currently for using externs files while generating docs is very closely tied to this hypothetical new |
Wonderful, thanks! |
Now that #127 went in, let's repurpose this issue to focus on the search functionality, since it's the only thing missing for this |
@hdgarrood I was wondering if I could help in any way with purescript/purescript#3503? |
That would be great! I think the first step should be to implement a |
Update: The goal is to make the docs produced by
|
I checked the code, and from what I saw there I conclude that the most easy way is to add purescript as a dependency and reuse the Regarding the interface, I think that CLI is a must have. The obvious problem will then be speed - when I was tackling with pursuit, its index rebuild was taking quite a lot of time. I think that building the index once and dumping the Regarding search function in the web interface, converting a Trie to JS will probably result in having a too large bundle. It may be possible to use files on the system as Trie nodes, so that new parts of index will be loading into the browser on demand as the user types. |
Apparently, pursuit is depending on an old version of purescript, so maybe it isn't a good idea to copy its approach. The parser is completely new, but pursuit is still using old version to parse types. |
@klntsky thanks for looking into this!
As far as I understand you'd like to create the index on the Haskell side. The problem with that is that we'd need to duplicate types ( Assuming we'd go for the all-PureScript route, this is what I think it should happen when running
I think it's very nice to have, but I'd say the biggest priority here is to replicate the search behavior on Pursuit (in fact that's also the title of this issue)
As I described above, parsing the
I'd say we could try going for a single bing bundle in the beginning, and then benchmark how bad it is, and then eventually split it in parts to be loaded on demand? |
Do we want to generate index for all packages in a set, or for listed dependencies only? If first, maybe it is better to generate it once and distribute with package set somehow? |
@klntsky since we'd generate the Btw I think |
Nevermind, I checked it now and realized that I was wrong. Also, looks like the only available purescript implementation of tries I could find is not suitable for storing search index. |
I'm working on it here: https://github.com/klntsky/spago-search |
@klntsky great work so far! 👏
How about "$ProjectName Docs"? In any case the searchbar going on a newline is perfectly fine, Pursuit does it too on mobile:
I pictured having the results in the same place as they are right now in Pursuit (i.e. let's literally replace the contents of the page with them): I'm not sure how they'd jump since you're just typing in the header so that should not move? |
I figured out it is possible to shrink the width of the container of that "Index" link, at the same time making the search field narrower. In practice, most search queries will fit into the field. And it looks way better than with a line break.
OK. I think we should also restore the main contents when the search field is empty. Another idea unrelated to UI is that we can populate the trie by camel-cased parts of names, i.e. |
The sad thing is that we seemingly can't split the index to load it on demand and have type search ability at the same time, because type searching requires comparing with all available types (if we go the pursuit way). Splitting the index is absolutely required: as for now it is 6.4 MB (for modules used by the project itself), and loading it freezes the page for ~8s on my machine. |
@klntsky I think we can detect when type search is being used right? Then we could split the index and load all the parts when we get a type query. If it's split in parts then it should also be possible to keep the page responsive (and show a loading spinner) while we load the parts |
This is very straightforward, but I'd say I dislike this approach. We can't improve the total time to load the index by loading it in parts: the only thing that we can do is suppressing the "tab is not responding" message. Anyway, last night I brainstormed the problem and found a solution. We can split all types in the index by their shapes. A "type shape" is a lossy encoding of types:
For example, we encode "Type shapes" preserve the crucial info about the type, but don't store the unnecessary details. They can be hashed and used as keys in a Map, which can be loaded on demand. Unfortunately, by using them we are giving up on the ability to find subtypes (e.g. I implemented the search, and IMO the quality level of results is OK (as for now they are only matched by shape, and not sorted by distance, like done in pursuit, but I'll implement that sorting too). |
The This info is also needed to map modules to package names: this is a temporary solution I'd like replace with something less hacky.
File names in |
|
From PS. Initially I thought that this should be done on the spago side, because I completely forgot that there are
OK, I'll have a "documentation day" after I'm done.
Yes, definitely.
I think that the artifacts (a nodejs script and a webapp) can be just downloaded at runtime from github releases. In this case they should also be pinned by hash. This will make offline builds possible while not making spago self-dependent.
Yeah, pursuit has the luxury of extracting the Prim definitions from the compiler. We may write a simple app that generates these jsons later. Also, I think that we may want to put the |
@klntsky sounds great! I invited you to the |
Maybe it's time to choose a better name for that search thing, @f-f? |
@klntsky sounds great! |
OK. One more question then. If you agree to distribute the precompiled app, then it is no more a requirement to avoid JS dependencies, right?
This requires adding |
The docs.json files don’t include re-exports currently, by the way (this is as a result of various implementation details). Perhaps I should have made this clearer before, but the intended way of consuming them is to use the |
I know, this is absolutely fine for our purposes (it's even better this way, cause no need to deal with duplicate index entries).
How stable is the format of these files? Should I expect my ad hoc |
Backwards-incompatible changes are pretty rare (they're a pain for us because they necessitate regenerating all of Pursuit's docs) but forwards-incompatible changes are not that uncommon. These files also aren't part of the compiler's public API, so e.g. they could potentially change in a backwards-incompatible way without a major compiler release. |
I think it's OK to depend on the format of the
@klntsky yep, I'm fine with adding JS deps |
We should be able to generate documentation for all the libraries in the package set (and/or just the libraries used in the project), in a similar way to what
stack haddock
does for Haskell projects.The use-cases for this are multiple and useful:
Note that this would obsolete
spago publish
command #27, as it would entirely remove the "where do I upload my library" problem → you would just add it to a Spacchetti package-set, and have it "published" in the next Spacchetti releaseHow to achieve this:
purs docs
command.Justin had a good start with acme-spago. He included a spago project in there, but if this is integrated in spago itself, we wouldn't need it (as we would just generate that info on the fly)
The text was updated successfully, but these errors were encountered: