-
Notifications
You must be signed in to change notification settings - Fork 229
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
full-text search using Lunr.js #1726
Conversation
I think you're getting the right search position, but applying it to the wrong documents! E.g. if I search There's an indexing issue somewhere in your code, where you're picking the wrong lessons from the search results and then trying to applying the highlighting to the text bodies and thus getting bizarre results - haven't pinpointed it yet, but that's where I'd look. |
Thanks for figuring this out @mdlincoln !!! Definitely appreciate the help!! I wonder if it the search corpus and index are mismatched somehow 🤔 Otherwise it would likely be the results from the idx.search or where I'm creating the var docs 🕵️♀️ |
When I inspect the results from Where is the portion of the code where you try to match that |
huh, so in the e.g. at position 34 in |
Thanks for the feedback @mdlincoln !! I updated the reference to the search index and I think fixed the issue. You were totally right about the doc.id being the wrong thing. Turns out our Lunr ref was looking for url, so I updated the search-index repo and now everything seems to be working. Gonna add in the logic for multiple languages and then I think we're ready for testing 🎉 |
So a weird bug that I could use your help with @mdlincoln. If you've noticed sometimes it gives a blank answer or doesn't show the updated search results or doesn't erase them when you empty out the input. After lots of testing turns out it does all of that if you click enter at the end of the search query, which I'm pretty sure is an issue with Chrome not updating the new html, but could have to do with List.JS as well... I've tried using jquery trigger, as well as resizing the window, and using setTimeout to hide and show the search results. So far nothing has worked... any thoughts or ideas of what we should try??? |
Sorry for the very similar commits here, for some reason I wasn't seeing my pushes in the Github UI so overdid it a bit. Anyways, pretty much have search fully working before tomorrow's meeting 🎉 You can now click on a topic or activity, search within those lessons, undo your search, and still have those lessons selected. I also added logic for a button or enter press which is pretty much fixing the display bug 98% of the time (sometimes you need a second enter press). Looking forward to hearing how it works for others 😅 |
|
I did some searches in French and this is what I found there:
|
Thanks Zoe I see your commits - hoping to peer through them this weekend and then we can launch 🎷 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we move the lunr stemming code into a js/vendor
subdirectory just to make clear that it's drop-in code that didn't need to be customized for our site?
Thinking ahead, how difficult will it e to add in support for Portuguese stemming? Can we make sure that process is documented in the code comments or elsewhere?
js/lessonfilter.js
Outdated
|
||
/* Function to update lesson-list using featureList and sort direction | ||
- uses URI to generate sort directions | ||
*/ | ||
console.log("applying sort from URI"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove the console log functions now that we're ready to push this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These were there originally (i.e. before I started working on search) but happy to remove them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should I also remove where we console.log the new URI?
@mdlincoln I like the idea of the js/vendor and happy to document (though it is almost entirely just following the lunr docs). One question I've had is where to document everything? Do you have any suggestions? Also for the search-info pop out. Are we planning to keep it? If yes, I can add it to our snippets. Finally should I take out all the console.logs or just the one you highlighted. There's quite a few of them so let me know. Thanks! |
Ditch all the console.log() A high-level overview page similar to the wiki I wrote for the translation links would be very helpful. You don't need to explain every single line of code bc the comments are quite good, but mainly to document the architecture. And that mini-help text is great. Let's add it to snippets |
Alrighty! All console.logs are gone and snippet added 🎉 The example of Twitter and Network in the search info works for Spanish but not French, based on the lessons available. I would recommend for French maybe doing I'll start working on the documentation tutorial this week but seems like we're in the final mile 🙌 |
@rivaquiroga @programminghistorian/spanish-team If you can please help translate the final snippet that provides search documentation for users: https://github.com/programminghistorian/jekyll/pull/1726/files#diff-a407e023d60dfefccac66d5a7959003cR375-R384 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ZoeLeBlanc please put the boilerplate JS code into a /vendor directory as discussed above
@mdlincoln thanks for catching that! Just moved the files to vendor directory and updated the lesson index to only load appropriate stemmer for each language (rather than loading them all on each page). |
ohhh good catch, I didn't even think to check it was loading all that extra JS! |
hmm weirdly isn't showing that I created vendor and moved the files 🤔 Should I try pushing up again?? |
oh shoot - at one point I think It ought to be |
ahh that makes more sense! I'll go ahead and fix that in gitignore, and then push up again |
As an alternative to List.js, we could also use Lunr.js which has more robust search capacity.
This branch has my current attempt to get the two of these working together. It requires pulling down the pre-built indices and corpora, then filtering them for a search term, trying to reuse some of the existing UX by having the search results appear where the lesson abstract usually does.
One issue I'm having is that I'm not getting the right position for the search terms in the documents. @mdlincoln I'm using your code from the gist so if you wouldn't mind taking a look I would really appreciate it!
closes #1018
Checklist
Closes #ISSUENUMBER
to the description aboveIf you are having difficulty fixing Travis errors, first consult https://github.com/programminghistorian/jekyll/wiki/Making-Technical-Contributions carefully, especially "Common Travis Errors". Then contact the technical team if you need further help.