/documents/fetch with documents ordering

[panzer-punk]

I have a situation when one of the array values has been changed. I'm sending request to /documents/fetch with this array value id, and changing the array and then making document update. It allows to avoid database calls and new aggregations for this field. But meilisearch reordering changed documents, and I get documents with old data (they are just skipped by offset pagination) . I think it'll be more
convenient if there was a managable ordering. There are 100K documents, with 8 array properties each one is from 1 up to 5 values.

[macraig]

Hi @panzer-punk , thanks for your feedback!

I understand that you're trying to update your documents and are using the /fetch route to get the documents you want to update, is that correct?

Would you be able to list the different steps you are taking including the API calls/SDK methods you are using for each step?

[panzer-punk]

@macraig Hi, yes I'm trying to update documents and avoid database calls.
Steps:
The whole document looks something like this:
id: string
title: string
{propertyName}: array
Each property values are aggregated into document (EAV pattern).
Let's assume we have document:

{
"id": "uuid",
"title": "the exploited",
"genre": [
  {
     "id": 12,
     "name": "ponk"
  },
  {
     "id": 13,
     "name": "rock"
  }
]
}

And when the genre is removed or updated I try to do the next steps:

1. Fetch documents with genre.id /documents/fetch

{
"fields": ["id", "genre"],
"filter": "genre.id = 12",
"limit": 500,
"offset": 0
}

2. Since I know the property id I delete or update property value
3. Just updating documents PUT /indexes/{index_uid}/documents

[
"id": "uuid",
"genre": [
  {
     "id": 12,
     "name": "punk"
  },
  {
     "id": 13,
     "name": "rock"
  }
]
]

And on the next page ordering breaks due to updated documents.

[macraig]

@panzer-punk when a document is updated it gets assigned a new internal id. Because the documents route is ordered by internal id, this could change the order or your results. Is that what you see happening?

I get documents with old data (they are just skipped by offset pagination)

What do you mean by old data here?

[panzer-punk]

@macraig Yes, it is what I see exactly. It would be much better if could specify the field for ordering.

[panzer-punk]

@macraig I can't remeber what I meant by old data, but there was only ordering issue, not a data issue.

[macraig]

@panzer-punk I see. At the moment we don't have a way of sorting using /fetch, you can only sort when running a search.

We have however received similar requests from other users and we're using centralizing the feedback on this discussion: #657. I'll close this one and paste your use case in the other one to maintain a single source of truth.

I also encourage you to upvote the feature on our public roadmap, as your input helps us prioritize our future development efforts.