Add support for Fuse.js search format #2507

SIGSTACKFAULT · 2024-05-24T20:08:49Z

Sanity check:

Have you opened a thread on the forum? yes but nobody responded
Have you checked to ensure there aren't other open Pull Requests for the same update/change?
Are you doing the PR on the next branch?
Have you created/updated the relevant documentation page(s)?

the Fuse.js format looks like this (I'm working on Bevy's website:

[
  {
    "title": "2D",
    "body": "\n  TODO\n  \nExplain how 2D graphics in Bevy is actually 3D with a weird camera projection\nDefine the relevant abstractions\n\n\n\n",
    "path": "/learn/book/graphics/2d/"
  },
  {
    "title": "Sprite Sheets",
    "body": "\n  TODO\n  \nDemonstrate how to import and use a sprite sheet for basic animations\n\n\n\n",
    "path": "/learn/book/graphics/2d/sprite-sheets/"
  },
  {
    "title": "Sprites",
    "body": "\n  TODO\n  \nDemonstrate how to import and use a sprite\n\n\n\n",
    "path": "/learn/book/graphics/2d/sprites/"
  }
]

it happens to also be compatible with tinysearch, and almost compatible with stork.

components/search/src/fuse.rs

components/search/Cargo.toml

components/site/src/lib.rs

Keats · 2024-05-24T21:27:01Z

components/site/src/lib.rs

+            match self.config.search.index_format {
+                IndexFormat::ElasticlunrJson | IndexFormat::FuseJson => content,
+                IndexFormat::ElasticlunrJavascript | IndexFormat::FuseJavascript => {
+                    format!("window.searchIndex = {}", content)


This can probably go on the enum as well?

of course it could, but I feel like it would be a weird thing to do? completely subjective.

like at that point i might as well put the entire thing inside IndexFormat

Keats · 2024-05-24T21:27:33Z

Sorry I've seen the message but didn't get to respond yet.

remmeber to commit Cargo.lock

SIGSTACKFAULT · 2024-05-27T00:38:16Z

bump @Keats because i replied to some of your reviews with followups

Keats · 2024-05-27T18:40:35Z

docs/content/documentation/content/search.md

+### Fuse
+
+Compatible with [fuse.js](https://www.fusejs.io/), [tinysearch](https://github.com/tinysearch/tinysearch), and almost
+compatible with [stork](https://stork-search.net/)


What does it mean "almost compatible"? Does it work with stork or not?

stork's format is very similar but the keys are slightly different, it's toml, and it wants a bit of a extra metadata. One could conceivably convert fuse_json to stork with a shell script and a copy of jq

I should probably either write all that in the docs or remove the reference entirely

actually maybe i should have fuse_json have a url key instead of path, since fuse doesn't care and that will be more compatible with stork and tiny.

oooh wait i see why include_path exists

Keats · 2024-05-31T10:29:31Z

Thanks!

Keats · 2024-05-31T10:29:59Z

Note: I haven't tried it myself but I assume you did right?

SIGSTACKFAULT · 2024-05-31T18:02:07Z

@Keats yeah i have a branch of bevy-website working using this. When will it ship?

Keats · 2024-06-03T10:57:24Z

I need to have a look at all PRs and issues but i hope this week or the next one

* inital "just barely works" Fuse.js support * implement FuseJavascript; refactor index_for_lang * support search config * move fuse index building to it's own file * update doc of Search.index_format * update config docs * update search documentation * use &str where possible * use libs::serde_json remmeber to commit Cargo.lock * move extension logic to IndexFormat * move the entire filename logic inside IndexFormat * move elasticlunr to it's own module * only create elasticlunr.min.js if we're actually using elasticlunr * move ELASTICLUNR_JS to elasticlunr.js * hide the details of search's submodules * optionally include path * explain include_path better * remove references to stork * replace if with match * support include_description * specify "permalink" * move body cleaning and truncation to a function * update truncate_content_length docs to specify *code points*

Jieiku · 2024-08-07T04:49:36Z

Just wanted to add that the fuse_json index format helped with implementing pagefind in abridge, here is the demo: https://abridge-pagefind.pages.dev/

I planned to update tinysearch in abridge to use fuse_json, but a while ago I ran into an issue that I have not been able to figure out: tinysearch/tinysearch#175

* inital "just barely works" Fuse.js support * implement FuseJavascript; refactor index_for_lang * support search config * move fuse index building to it's own file * update doc of Search.index_format * update config docs * update search documentation * use &str where possible * use libs::serde_json remmeber to commit Cargo.lock * move extension logic to IndexFormat * move the entire filename logic inside IndexFormat * move elasticlunr to it's own module * only create elasticlunr.min.js if we're actually using elasticlunr * move ELASTICLUNR_JS to elasticlunr.js * hide the details of search's submodules * optionally include path * explain include_path better * remove references to stork * replace if with match * support include_description * specify "permalink" * move body cleaning and truncation to a function * update truncate_content_length docs to specify *code points*

SIGSTACKFAULT added 7 commits May 24, 2024 13:29

inital "just barely works" Fuse.js support

cc11d7a

implement FuseJavascript; refactor index_for_lang

39a1e71

support search config

e291942

move fuse index building to it's own file

5c310c3

update doc of Search.index_format

1939277

update config docs

35bd4e0

update search documentation

4cc9a30

Keats requested changes May 24, 2024

View reviewed changes

SIGSTACKFAULT added 6 commits May 24, 2024 17:56

use &str where possible

1dbddce

use libs::serde_json

b6765e3

remmeber to commit Cargo.lock

move extension logic to IndexFormat

7247d86

move the entire filename logic inside IndexFormat

0c29dc8

move elasticlunr to it's own module

c1c6eb4

only create elasticlunr.min.js if we're actually using elasticlunr

d26b612

SIGSTACKFAULT added 2 commits May 26, 2024 21:02

move ELASTICLUNR_JS to elasticlunr.js

df6faab

hide the details of search's submodules

3ddb394

Keats approved these changes May 27, 2024

View reviewed changes

Keats reviewed May 27, 2024

View reviewed changes

SIGSTACKFAULT added 8 commits May 28, 2024 09:37

optionally include path

2b9c1e3

explain include_path better

2298a29

remove references to stork

b4b2c52

replace if with match

2ecb379

support include_description

16ea90a

specify "permalink"

8c3e79b

move body cleaning and truncation to a function

c0fb3b1

update truncate_content_length docs to specify *code points*

1535965

Keats approved these changes May 31, 2024

View reviewed changes

Keats merged commit 8c56a0e into getzola:next May 31, 2024
5 checks passed

SIGSTACKFAULT mentioned this pull request Jun 24, 2024

fix: treat any data change as a modification #2542

Merged

3 tasks

This was referenced Jun 24, 2024

Zola supports fuse_json index now which should be compatible with tinysearch tinysearch/tinysearch#178

Open

How well does uFuzzy support CJK, stopwords, stemmers? leeoniya/uFuzzy#68

Closed

Fix bug in package_abridge.js Jieiku/abridge#176

Merged

Jieiku mentioned this pull request Jul 1, 2024

Investigate additional Search Engines. Jieiku/abridge#178

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for Fuse.js search format #2507

Add support for Fuse.js search format #2507

SIGSTACKFAULT commented May 24, 2024 •

edited

Loading

Keats May 24, 2024

SIGSTACKFAULT May 24, 2024

SIGSTACKFAULT May 24, 2024

Keats commented May 24, 2024

SIGSTACKFAULT commented May 27, 2024

Keats May 27, 2024

SIGSTACKFAULT May 28, 2024 •

edited

Loading

SIGSTACKFAULT May 28, 2024

SIGSTACKFAULT May 28, 2024

Keats commented May 31, 2024

Keats commented May 31, 2024

SIGSTACKFAULT commented May 31, 2024 •

edited

Loading

Keats commented Jun 3, 2024

Jieiku commented Aug 7, 2024 •

edited

Loading

Add support for Fuse.js search format #2507

Add support for Fuse.js search format #2507

Conversation

SIGSTACKFAULT commented May 24, 2024 • edited Loading

Keats May 24, 2024

Choose a reason for hiding this comment

SIGSTACKFAULT May 24, 2024

Choose a reason for hiding this comment

SIGSTACKFAULT May 24, 2024

Choose a reason for hiding this comment

Keats commented May 24, 2024

SIGSTACKFAULT commented May 27, 2024

Keats May 27, 2024

Choose a reason for hiding this comment

SIGSTACKFAULT May 28, 2024 • edited Loading

Choose a reason for hiding this comment

SIGSTACKFAULT May 28, 2024

Choose a reason for hiding this comment

SIGSTACKFAULT May 28, 2024

Choose a reason for hiding this comment

Keats commented May 31, 2024

Keats commented May 31, 2024

SIGSTACKFAULT commented May 31, 2024 • edited Loading

Keats commented Jun 3, 2024

Jieiku commented Aug 7, 2024 • edited Loading

SIGSTACKFAULT commented May 24, 2024 •

edited

Loading

SIGSTACKFAULT May 28, 2024 •

edited

Loading

SIGSTACKFAULT commented May 31, 2024 •

edited

Loading

Jieiku commented Aug 7, 2024 •

edited

Loading