Changes regarding Meilisearch v1.2.0

[brunoocasali]

This issue gathers the changes related to the v1.1.0 of Meilisearch that will impact the integrations team.

📅 Release date: June 5th, 2023

The whole milestone of v1.2.0 is here!

⚠️ The SDKs, which should be done by the release date, are only tier #1. Some of the features described here for tier #2 will also be ready for the release day. Check the features below to understand which tier will receive which feature.

🖖 Click here to check the current tiers state of the integrations:

Understand everything about tiers here.

Integration	Tier
Javascript	#1
PHP	#1
Instant Meilisearch	#1
Python	#1
Ruby	#1
Go	#1
Strapi	#2
.NET	#2
Rails	#2
Rust	#2
Symfony	#3
Java	#3
Firebase	#3
docs-searchbar.js	#3
Dart	#3
Swift	#3
Vuepress	#3
Gatsby	#3

⚠️ If no change in the public API is added during the bump pull requests, there is no need for a new release, according to the Integrations team's versioning policy guideline.

---

Delete documents by filter

Related to:

• issue: Delete and get documents by filter meilisearch#3477
• spec: Get and delete documents by filter specifications#236

It allows the user to use a filter expression to remove documents.

Since there is already a method called deleteDocuments(uids)/delete_documents(uids) in the SDKs, and this method is calling the route POST /indexes/:index_uid/documents/delete-batch, the requirements to implement the new way is to create a conditional internally to check the presence of the new parameter filters and then call the new method POST /indexes/:index_uid/documents/delete.
When the developer uses the old ids argument, it should keep calling the old implementation.

This will avoid any silent failures the users may find.

For example, in the Ruby SDK:

# Public: Delete documents from an index
#
# documents_ids - An array with document ids (deprecated, optional)
# filter - A hash containing a filter that should match documents. Available ONLY with Meilisearch v1.2 and newer (optional)
# 
# Returns a Task object.
def delete_documents(documents_ids = nil, filter: nil)
  if documents_ids
    http_post "/indexes/#{@uid}/documents/delete-batch", documents_ids
  else
    http_post "/indexes/#{@uid}/documents/delete", { filter: filter }
  end
end

Extra: Add inline documentation for the method, explaining the availability of the filter syntax only for Meilisearch v1.2 and newer.

Extra: Mark the document_ids parameter as deprecated. eg. @Deprecated('migration') on Java/Dart, Obsolete on C#, etc. Add a library if necessary.

Extra: Add a try/catch to detect the error and give the user a hint about the possibility of a version mismatch between the SDK version and the instance version. Check this PR for an example in JavaScript: meilisearch/meilisearch-js#1492

Extra: General recommendations:

• For languages where method overloading is available, use the overloading.
• For languages where a builder is available, use the builder pattern.

🤩 Affected integrations: All the integrations from tier #1.

---

Retrieve documents by filter

Related to:

• issue: Delete and get documents by filter meilisearch#3477
• spec: Specify the new fetch documents route specifications#234

Gives the user the possibility to use a filter expression to retrieve documents, like in search.

Following the same concept in the previous section, deleting by filters, this Meilisearch version introduces a new way of finding documents by introducing a new route and arguments.

Implement in the getDocuments/get_documents method an internal conditional allowing the user to query the documents with the same method but using a new filter method.

When the user calls the get documents method with a filter argument, request POST /indexes/{index_uid}/documents/fetch using a JSON body containing an int offset, int limit, String[] fields and now String filter or String[] filter.
⚠️ If the method invocation does not contain a filter it should still call the previous implementation.

🤩 Affected integrations: All the integrations from tier #1.

---

IS EMPTY filter operator

Related to:

• issue: IS NULL filter operator meilisearch#3615
• spec: Filter operators - IS NULL and IS EMPTY filter operators specifications#232

⚠️ Dart is the only SDK we maintain which have a DSL when it comes to declaring the filters. So, no need to implement anything in any of the other SDKs.

Add the IS EMPTY new operator to the filterExpression DSL on the Dart SDK.

🤩 Affected integrations: Only meilisearch-dart.

---

IS NULL filter operator

Related to:

• issue: IS NULL filter operator meilisearch#3615
• spec: Filter operators - IS NULL and IS EMPTY filter operators specifications#232

⚠️ Dart is the only SDK we maintain which have a DSL when it comes to declaring the filters. So, no need to implement anything in any of the other SDKs.

Add the IS NULL new operator to the filterExpression DSL on the Dart SDK.

🤩 Affected integrations: Only meilisearch-dart.

[ahmednfwela]

since fetchDocuments is POST, does it make sense to also name its body FetchDocumentsQuery ? why not FetchDocumentsBody ?

[sanders41]

For delete by filter:

When the language support named parameters, move ids to a parameter and add filter alongside it.

This means a user could send both ids and filters in the same request. Based on the spec I think delete by filter has to be a new separate method.

* Introduce a new POST /indexes/:index_uid/documents/delete route that lets you delete documents by filter
* The payload must be an object only containing a filter field, i.e.: { "filter": "doggo = bernese" }
* A new error has been introduced in case the specified filter is wrong or empty: invalid_document_delete_filter

The current delete_document is a delete request with the UID in the URL. Delete by filter is a post route with only the filter in the post body. So I think mixing the two is going to cause issues. Does this make sense or am I missing something?

[sanders41]

I have this ready here if anyone wants ideas for a starting point.

In working on it there is one thing I think is worth mentioning that may catch end users (because it got me for a minute 😃). If you forget to add the fields to the filterable attributes and use them for deletes/gets you don't get an error. It acts as everything worked as expected and gives successful responses, it just doesn't do anything.

Another question I have is why add the fetchDocuments method? getDocuments adds a filter parameter so the only difference between fetchDocuments and getDocuments is one uses the post route while the other uses the get route. But the user of the client never sees this so they would end up with two different methods that do the exact same thing.

[bidoubiwa]

What about using deleteDocuments with POST /documents/delete route and stop using POST /delete-batch?
In the same vain, using getDocuments with the POST /documents/fetch route and stop using GET documents/ like @sanders41 suggests?

Both previous routes seem to be just a lesser performant version of the newly introduced routes.

[sanders41]

This is doable without a breaking change since the end user doesn't see this part. Have you measured performance between the options to see the new ones are more performant? If not I may run some tests to see what the difference would be.

[bidoubiwa]

To answer @sanders41, @meilisearch/engine-team can you tell us if the POST /documents/fetch route more performant than GET /documents/?

This is doable without a breaking change since the end user doesn't see this part.

While we can avoid a breaking change on the getDocuments I don't think we can avoid one with deleteDocuments as most of the function parameters are deleteDocuments(ids: str[]). Like @brunoocasali said, overloads might avoid that in some languages but it might be worth making that breaking change now on the others (and maybe even on the one accepting overloads). I think /delete-batch might be deprecated in v2, can you confirm @gmourier ?

[brunoocasali]

This pre-release is finished. All the remaining work was moved to our own repositories, closing this!

Thanks team you're the best! 🎉
Thanks to our meilistars @sanders41 and @ahmednfwela for dealing with this pre-release in Python & Dart for us, it was incredible helpful, you folks rock! 💪 🎉