Query and Search blocks: support for Instant Search

[r-chrzan]

What?

Initial implementation for Instant Search using the Search and Query Loop blocks. Added as a new experiment on the Gutenberg Experiments page.

Related to #63053

How does it work?

1. Insert a Query block anywhere on your site.
2. Make sure to disable Force Page Reload i.e. use "enhanced pagination" in that Query block.
3. Insert a Search block anywhere inside of that Query block.

When the Search block is inside of the Query block using "enhanced pagination", it automatically gets "updated" to an Instant Search block.

Any search input in the Instant Search block gets passed as the ?instant-search=<your-search> query search param in the URL.

Multiple Query & Search blocks

It's possible to have multiple Instant Search blocks in a page/post/template. In such a case, ensuring their functionality is isolated is essential. It's also important to remember that the "query type" in the Query block can be Default or Custom. The Default query is inherited from its respective template. For this reason, the Instant search block uses different URL search params when handling each:

• Default - ?instant-search=<search-term>
• Custom - ?instant-search-<queryId>=<search-term>

Limitations

⚠️ Multiple Instant Search blocks using the Default query on in the same template are currently not supported.
⚠️ The Instant Search block does not yet work correctly when placed in the Search template. See #63147 (review).

Pagination

The instant search functionality intersects with pagination of the Query block:

1. Every time the search is updated, the pagination of its respective query is reset back to page 1.
2. The pagination numbers and next/previous, are updated to reflect the number of results in the search.
3. Clearing the search also resets the pagination back to page 1.

Further work

This is an initial prototype and intentionally does not yet implement all the features that should be expected in the final version. Those include but are not limited to:

• Ensure that it works correctly for inherited queries.
• Pagination - Reset page number for every search.
• The search parameter in non-inherited queries should work correctly (as mentioned in https://github.com/WordPress/gutenberg/pull/63147#issuecomment-2214460127)`
• Should work when there is more than one Query Loop block & Search block on the page.
• Ensure all user input is sanitized / validated.
• Correctly handle multiple Default queries.
• a11y - announce search results. This needs testing but should work out of the box with the a11ySpeak() of the interactivity-router here and here.

Video

output_62afd8.mp4

[github-actions]

👋 Thanks for your first Pull Request and for helping build the future of Gutenberg and WordPress, @r-chrzan! In case you missed it, we'd love to have you join us in our Slack community.

If you want to learn more about WordPress development in general, check out the Core Handbook full of helpful information.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Unlinked Accounts

The following contributors have not linked their GitHub and WordPress.org accounts: @rchrzan@whitelabelcoders.com, @ktmn.

Contributors, please read how to link your accounts to ensure your work is properly credited in WordPress releases.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Unlinked contributors: rchrzan@whitelabelcoders.com, ktmn.

Co-authored-by: luisherranz <luisherranz@git.wordpress.org>
Co-authored-by: michalczaplinski <czapla@git.wordpress.org>
Co-authored-by: sirreal <jonsurrell@git.wordpress.org>
Co-authored-by: cbravobernal <cbravobernal@git.wordpress.org>
Co-authored-by: r-chrzan <rchrzan@git.wordpress.org>
Co-authored-by: gziolo <gziolo@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[luisherranz]

Great work!! 👏👏

Let's see what's next 🙂

1. Modifying the query in inherited queries.

   I don't know if you've noticed, but the query_loop_block_query_vars filter only works when the "Inherit query from template" option is disabled.

   

   We should check if the query is inherited, and if so, apply the modification to the general query so it works in both cases.

2. The search parameter in inherited queries.

   It's interesting that we can't use the search parameter ?s= because, in that case, the page that loads is the Search template.

   I'm going to think about it a bit, but it is true that it might not be possible to simply use ?s=. If we can't use it, then we need to think carefully about what the most suitable parameter is. Maybe ?is= for instant search?

3. The search parameter in non-inherited queries.

   When the "Inherit query from template" option is disabled, the query pagination uses a custom search parameter instead of the regular ?paged=2//page/2 of the inherited queries: ?query-X-page=P where X is the query id ($block->context['queryId']).

   In that case, we should use a similar parameter: ?query-X-search=test.

4. Pagination when searching.

   You commented in Slack:

   sometimes, when we are on e.g. page 3 and typing something, i see prev link button without results

   That's what happens when the new results don't have as many pages as the page you were on just before you started the search.

   I think we should try resetting the page number every time a new search is performed. In other words, if the user starts a search on page 3, we should remove the search param from the URL before navigating.

   • User is in /page/3 or ?query-X-page=3.
   • User types new characters.
   • We add the search parameter, but remove the pagination: /?is=test.

   This is probably going to present problems, as it might not be that easy to remove the pagination from the URL in the client for certain permalinks, but I think it's worth a try and we'll see what happens. If that's the case, one option to investigate would be to generate the URL of the first page on the server and send it to the client via the state.

5. Search button closing itself during navigation.

   You commented in Slack:

   it doesn’t work very well when search is just a button. sometimes toggle closes while typing

   That's because the context has a property called isSearchInputVisible which is set to false on the server. So when a navigation occurs, it gets reset to false.

   The way to solve this is to transform that property into an initial property and have a client-only property so that the server property is only checked on the first page load and then only the client-only property is considered.

   I made a commit to implement this pattern by adding this type of getter:

   get isSearchInputVisible() {
     const ctx = getContext();
     if ( typeof ctx.isSearchInputVisible === 'undefined' ) {
       return ctx.isSearchInputInitiallyVisible;
     }
     return ctx.isSearchInputVisible;
   },

[sirreal]

Pagination for Default queries uses the /page/2/ format when using pretty permalinks. Should it be removed from the URL? Here we have two options:

• Remove it from the URL. Downside is possibly breaking links as we have to parse the URL with a regex and remove the /page/X/ part.
• Keep it in the URL and use the ?paged URL param. This works fine in my tests and redirects the request to the correct page. E.g. /page/2/?paged=3 will redirect to /page/3/. The downside is that it requires a redirect so is slightly slower.

Is the information for how a URL should be constructed for a given site available somehow?

[michalczaplinski]

That’s fair. I’m fine with adding enhancement in steps. What’s the reasoning for adding the global handling for the default query in this PR if it isn’t expected to be functioning in the current shape?

That's a good point. I'm happy to handle the default queries in another PR if that makes is easier to review. I will split the current PR up.

What’s the reason the same query needs to be recomputed for every child block? Can it be shared? What prevents from using a WP filter to modify the vars passed to the existing query in Gutenberg?

The reason is that the query_loop_block_query_vars filter does not run for the global queries (inherited from template). See this explanation.

In the follow-up PR (handling the global/inherited query) I can try to use the pre_get_posts filter instead. That should be the appropriate filter for filtering the global/inherited query.

Is there a way to pass a canonical URL without the pagination applied to use it instead without all the complexity explained?

I'm not aware of one. I took a look, and there is a wp_canonical_url filter, but that only works for posts and will not work for archives, for example.

In general, I don't think it is possible to do it reliably server-side in PHP. A site could also, e.g., sit behind an Nginx proxy, which could be serving on a different URL and a WordPress function would have no way of knowing it.

[michalczaplinski]

Is the information for how a URL should be constructed for a given site available somehow?

@sirreal I think that the closest resource would be https://developer.wordpress.org/themes/basics/template-hierarchy/

The final URL depends on both the permalink settings and the Template hierarchy. Is that what you had in mind?

[sirreal]

Sorry that was unclear, I was wondering if the site can be queried to determine whether a search param should be used or some other form.

For this initial version the redirect seems alright and it can be improved in the future. It may be able to observe the redirect and adjust behavior accordingly.

[michalczaplinski]

Sorry that was unclear, I was wondering if the site can be queried to determine whether a search param should be used or some other form.

We could get the value of get_option( 'permalink_structure' ); and pass it to the router's store on the server I guess. If a site is using "ugly permalinks" the value will be an empty string (reference)

But this would not help us much. We'd still have to update the URL on the client side. And for this, we need to parse the URL on the client side to e.g update mysite.lol/page/2#hash?and=query to mysite.lol/page/3#hash?and=query.

[gziolo]

There is still a lot of code repetition for every core block that can be nested inside the Query block. In the case of this PR, nearly the same changes have to be applied to 5 existing core blocks. Unfortunately, the same logic won't be available to custom blocks. I explored some alternatives, and I was surprised at how powerful it could be to use a filter, for example:

function block_core_query_add_url_filtering( $context ) {
	if ( empty( $context['queryId'] ) ) {
		return $context;
	}
	$search_key  = 'query-' . $context['queryId'] . '-s';
	if ( ! isset( $_GET[ $search_key ] ) ) {
		return $context;
	}

	$context['query']['search'] = sanitize_text_field( $_GET[ $search_key ] );

	return $context;
}
add_filter( 'render_block_context', 'block_core_query_add_url_filtering' );

It's a quick proof of concept. This would execute too many times, as it would be applicable to every block nested inside the Query, but it's quite powerful. It probably could be improved by checking the parent block. It basically enabled filtering by a search term provided in the URL targeting a specific custom query. Example URL:

• http://localhost:8888/page/2?query-31-page=1&query-31-s=hey

In my testing, it turned out the URL structure for custom queries doesn't follow the settings for permalinks. The example I shared uses numeric permalinks for the default query, but not for custom queries.

The open question is what would be the ultimate fix in WordPress core. The challenge is that the context is exposed based on the query attribute on the Query block, which is serialized in the database, so we would have to find a way to change the context based on the params passed in the URL matching the current query id.

[michalczaplinski]

That's interesting, thanks for exploring it @gziolo.

I think that this approach might work. I'm going to try it now - we can definitely make it only run when the an ancestor block is Query Loop and has enhanced pagination enabled, etc.

In my testing, it turned out the URL structure for custom queries doesn't follow the settings for permalinks. The example I shared uses numeric permalinks for the default query, but not for custom queries.

Yup, that's true. Only the default queries use the permalink settings, the custom queries use the query-N-page=X param.

The open question is what would be the ultimate fix in WordPress core. The challenge is that the context is exposed based on the query attribute on the Query block, which is serialized in the database, so we would have to find a way to change the context based on the params passed in the URL matching the current query id.

So this means that the latest query (with search parameter) always gets saved in the database? And the problem is that a request might NOT include any (instant) search param, but the child blocks (query-pagination, query-numbers, etc.) could not distinguish between context['query']['search'] that was added by the render_block_context filter and the one that might come from the database, right?

Regardless, I'll give this approach a try 🙂

[michalczaplinski]

The open question is what would be the ultimate fix in WordPress core. The challenge is that the context is exposed based on the query attribute on the Query block, which is serialized in the database, so we would have to find a way to change the context based on the params passed in the URL matching the current query id.

I need some clarification on how this is a problem. 🤔

We're only overriding the context at runtime when a request is being made. So the "query" context including the new "search" parameter should never be saved in the database. Or were you thinking about the case when there is EXPLICITLY a search parameter saved in the database, which would then be overridden by the param from the URL?

[michalczaplinski]

I've started to explore the alternative in #67013. We can discuss there 🙂 .

[michalczaplinski]

We're only overriding the context at runtime when a request is being made. So the "query" context including the new "search" parameter should never be saved in the database. Or were you thinking about the case when there is EXPLICITLY a search parameter saved in the database, which would then be overridden by the param from the URL?

This case is now handled via baa6188.

The remaining doubt I have is how to handle the URL. Should we (on the fronted) modify the URL to include a query param like instant-search-11=<search-term>? Currently, the URL does not change if we have a search term in the block's attributes:

output_9d447c.mp4

I cherry picked baa6188 to #67013 as well.

[gziolo]

The open question is what would be the ultimate fix in WordPress core. The challenge is that the context is exposed based on the query attribute on the Query block, which is serialized in the database, so we would have to find a way to change the context based on the params passed in the URL matching the current query id.

I said that the search term is configurable, and the default value can be set in the editor through UI:

In effect, the default value is always stored in the database and it doesn't have to be an empty string. Now, the functionality that is being explored has to offer a way to override that search value set in the block context by the Query block when rendering the block. Currently, the context for child block is equal to the value serialized in the database. Overriding on the frontend the context based on the URL param detected seems like the most intuitive approach as otherwise, every block has to check the URL to figure out if it should use that value or the one provided by theme/editor's user.

[michalczaplinski]

I said that the search term is configurable, and the default value can be set in the editor through UI:

Right, that's correct. I think we're talking about the same thing 🙂. I just did not realize that one can update the search term in the UI in addition to simply editing the markup (I did the latter in this example).

The effect is the same either way: the query.search attribute gets updated.

It should now work as of baa6188. Whenever the user updates the search term (via UI or just directly editing the markup), the search term will appear in the Search input on the frontend upon loading the page.

What I was considering here:

The remaining doubt I have is how to handle the URL. Should we (on the fronted) modify the URL to include a query param like instant-search-11=?

is something different. I wondered: What to do when loading a page for the first time, including an Instant Search block with a custom search term. In the video in the previous comment I show that the word Surfing appears in the Search input, but the URL does NOT reflect that. I had thought that on initial load, we could (on the front-end in JS) update the URL to include the search term.

However, I gave it some more thought, and it's probably not a good idea and would generate more problems than solve, I think. We should not just update the links in JS after the page loads based on the page's contents.

I will let other folks to help bring it to the finish line.

[gziolo]

I discussed with @ntsekouras some options for the overall strategy of making the Query block filterable on the front end through URL params on WordPress Slack (https://make.wordpress.org/chat/):
https://wordpress.slack.com/archives/C02QB2JS7/p1731660635692969