Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SearchAfter does not support date types, and all types except string types are cast; Will cause pagination exceptions #2678

Closed
heerzhang opened this issue Aug 28, 2023 · 1 comment · Fixed by #2679
Labels
type: bug A general bug

Comments

@heerzhang
Copy link

heerzhang commented Aug 28, 2023

There is an issue with sorting and pagination of date fields. If the date is a null value, the searchAfter pagination will stay in place and will not search for new records. The main issue should be in/org/springframework/data/lastsearch/client/elc/RequestConverter. java: 1378 line:

If (! IsEmpty (query. getSearchAfter())){

 Builder. searchAfter(
       query.getSearchAfter().stream().map(it -> FieldValue.of(it.toString())).collect(Collectors.toList()));

}

The code at this point requires that the SortValue of the date type field also be cast to FieldValue. of (it. toString()).

But I feel like I should use FieldValue of (long value) for the date type{

Return new FieldValue (Kind. Long, value);

}Just replace the logic here.

Because the date field returned by getSortValues () is a long integer in string form called 'epoch_millis'.

Even if I take the epoch myself_ Converting Millis to a string formatted with a date [uuuu MM dd] cannot solve the problem, as epoch is returned when date=null_ Millis is an illegal date.

The code is roughly as follows:

ElasticsearchOperations ElasticsearchOperations;

org.springframework.data.elasticsearch.client.elc.NativeQuery query,

List afterObjs=new ArrayList<>(from below lastsortHit. getSortValues());

Query query. setSearchAfter (afterObjs);

SearchHits

searchHits=ElasticsearchOperations. search (query, myClassType);

List<SearchHit

> hits=searchHits. getSearchHits();

SearchHit lastsortHit=hits. get (Maxs-1);
List afterObjs= lastsortHit.getSortValues();

笔记

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Aug 28, 2023
sothawo added a commit to sothawo/spring-data-elasticsearch that referenced this issue Aug 28, 2023
sothawo added a commit that referenced this issue Aug 28, 2023
@sothawo sothawo added this to the 5.2 M3 (2023.1.0) milestone Aug 28, 2023
@sothawo sothawo added type: bug A general bug and removed status: waiting-for-triage An issue we've not yet triaged labels Aug 28, 2023
@sothawo
Copy link
Collaborator

sothawo commented Aug 28, 2023

You are right, that toString() is not the correct way to handle the value returned in the searchAfter list.

But hardcoding a numeric value is wrong as well, the array can contain any type if data, from numeric, to strings or boolean values or other objects. #2679 fixes this by creating the correct FieldValue for the objects.

sothawo added a commit that referenced this issue Aug 28, 2023
Closes #2678

(cherry picked from commit 9adc4d2)
sothawo added a commit that referenced this issue Aug 28, 2023
Closes #2678

(cherry picked from commit 9adc4d2)
(cherry picked from commit 4614c62)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug A general bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants