DynamoDbEnhancedClient: @QueryEnhancedRequest limit parameter is not limiting the search results as expected

[lakshmi-prameela-lanka-wcar]

Describe the bug

Using aws DynamoDB Java sdk V2, and I would like to limit the number of results that are returned when querying by the partition key (code snippet below), but the code below returns the full set of items.

Expected Behavior

QueryEnhnacedRequest operation should limit the number of items that it reads

Current Behavior

The limit parameter that we set on QueryEnhancedRequest is giving all the results as per the query condition, thought the limit parameter is set.

https://stackoverflow.com/questions/62417472/limit-method-in-queryenhancedrequest-for-dynamodb-java-v2-sdk-doesnt-limit-as-e

Steps to Reproduce

QueryEnhancedRequest.Builder queryEnhancedRequest = QueryEnhancedRequest.builder()
    .queryConditional(QueryConditional.keyEqualTo(Key.builder().partitionValue(partitionKey).build()))
    .limit(2);

Context

Trying to fetch results from Dyanmodb using DynamoDbEnhancedClient with QueryEnhancedRequest

Your Environment

• AWS Java SDK version used: SDK version 2.13.52
• JDK version used:11
• Operating System and version:Linux 5.0.0-37-generic x86_64

[debora-ito]

Hi @lakshmi-prameela-lanka-wcar the limit in the QueryEnhancedRequest defines the limit of items per page. The SDK v2 by default uses pagination methods that fetches all pages automatically.

If you want to define a limit in the fetched results, you can specify it in the Iterable using the stream method:

PageIterable<MyMovie> myMovie = moviesTable.query(queryEnhancedRequest);

myMovie.items()
        .stream()
        .limit(2)
        .forEach(content -> System.out.println(" Movie: %s (%s) \n", content.title, content.year));

Let us know if that helps.

EDIT: there's other ways to limit the number of results, see comment here: #3226 (comment)

[github-actions]

It looks like this issue hasn’t been active in longer than a week. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please add a comment to prevent automatic closure, or if the issue is already closed please feel free to reopen it.

[90lantran]

Hi @debora-ito ,
Supposed I have a Movie table and a class like this

public class Movie {

public String movieId;

@DynamoDbPartitionKey
@DynamoSecondaryPartitionKey(indexNames= {moviesCreationTimeIndex})
public String getMovieId(){return movieId};

public String movieName;

@DynamoDbSortKey
public String getMovieName(){return movieName};

public String creationTime;

@DynamoDbSecondarySortKey(indexNames = {moviesCreationTimeIndex})
public String getCreationTime(){return creationTime};

}

I am using the enhanced request with LSI instead of table. My code looks something like this.

Iterator<Page<Movie>> movieIterator = moviesCreationTimeIndex.query(queryEnhancedRequest).iterator();

Did you mean this query automtically fetch all items in dynamoDB? Right now, I run into performance issue because some movies have like 10k items and requests takes like 10s.

I actually want to limit like just 100 movies in order of the creationTime which is why I created the LSI.

[allenlipeng47]

@debora-ito For code snippet you presented, where do I get the lastEvaluatedKey through PageIterable?

[molBR]

Hey folks!
I just found this issue and as it happens I am in the same boat as @90lantran mentioned.
It is indeed a performance issue and the odd thing is that if you build a raw query such as: Query(hash_key=..., ScanIndexForward=True, limit=1) you get the number of items as limited but if you use the sdk2 library you cannot reproduce the result using the semantically similar function. Shouldn't this be a bug?

Source: https://stackoverflow.com/questions/12809295/nosql-getting-the-latest-values-from-tables-dynamodb-azure-table-storage