Allow boolean operators when searching Dataverse guides

[dlmurphy]

Looking through recent search logs for our guides, I noticed a lot of attempts at using quotations to search for exact phrases. Unfortunately, the search tool for our guides is very basic and does not allow this kind (or any other kind) of boolean operator.

One user searched for "terms of use" in quotations. This is what happens when you attempt this query in our current search:

As you can see, this currently returns a useless list of all pages that use the words "use", "user", "used", etc. However, there are plenty of relevant sections about Terms of Use on our guides that are being drowned out. When I search "terms of use" using quotes on my local copy of the guides using SublimeText, I get these useful results:

In order to help users find what they're looking for in the guides, I propose we rework the Guides Search feature to allow standard boolean search operators, including:

• quotes
• AND
• OR
• NOT
• parentheses

[oscardssmith]

it would be easier to just do the quotes and not parts, as the other 3 require equation parsing (and are needed less often)

[mheppler]

Here is the Sphinx project site to search for more info on features and config options.

https://sphinx.readthedocs.io/en/latest/index.html

[matthew-a-dunlap]

This issue may be a good place to dig into for more details on search. It discusses the sphinx-doc stemmer(s) and how to generate one's own sphinx-doc/sphinx#1784 .

[dlmurphy]

Here's an issue on Sphinx's repo where someone is asking for exact phrase searching with quotation marks:
sphinx-doc/sphinx#3301

It appears that Sphinx does not currently support this because of they way it handles indexing. I'll leave a comment in that issue, and I also plan on asking about this and boolean operators in their google group.

[dlmurphy]

Created a thread in their google group to ask about this: https://groups.google.com/forum/#!topic/sphinx-users/kLX5b9vMWck

[pdurbin]

@dlmurphy what if you download the PDF from http://guides.dataverse.org/en/latest/Dataverse.pdf and do the search from whatever PDF reader you like? Here's how a search for "terms of use" looks on Mac for me:

Related: #2148

[pdurbin]

@dlmurphy as I mentioned you might want to ask in #sphinx at https://writethedocs.slack.com

[dlmurphy]

Good point that the PDF is more searchable. We can't expect our users to find the PDF, download it, and search it whenever they need to find something in the guides. But it is good that there's a publicly accessible workaround like this.

Good idea about Write the Docs, I'm going to look into them and see if they can give us some advice.

[pdurbin]

I don't think we're going to get this out of Sphinx.

It might take moving to Docusaurus: 🦖

• Feature Request/Idea: switching from Sphinx docs to using Docusaurus #8905

Vote to close.

[kuhlaid]

@pdurbin The PDF search with Sphinx is sufficient for most cases in my opinion but I believe this issue can be improved by advertising the PDF option within the guides. So above the Sphinx search box there could be a link to download the PDF version of the documentation (and a note saying for a more advanced search of the documentation, search within this PDF or similar). Currently I don't know where the PDF version of the documentation resides if I simply visit https://guides.dataverse.org/, but it needs to be prominently displayed and accessible. Also a link to the PDF guide should be in the Sphinx footer (shown below) and the top of the guide to make it clear a downloadable guide is available. Just my 2 cents.

[pdurbin]

@kuhlaid not a bad idea. Please feel free to create an issue for this. I'm glad you think PDF search is sufficient! Closing.