More async iterator pagination support

[farski]

Is your feature request related to a problem? Please describe.
Based on this recent blog post, it sounded like pagination using async iterators was going to be available for all, or at least most, paginated operations in the SDK. Having spent a little bit of time with the DDB module, the two most common operations that would require pagination (scan and query) don't seem to offer native async iterator commands.

Describe the solution you'd like
It would be ideal if all operations that can produce paginated results supported async iterators.

[trivikr]

Hi @farski, thank you for creating the issue.

The APIs which receive pagination is decided by the Service team. In smithy interface definition language, it is defined by paginated trait.

The paginated trait is missing for scan and query commands in smithy models:

• Scan

  aws-sdk-js-v3/codegen/sdk-codegen/aws-models/dynamodb.2012-08-10.json

  Lines 7043 to 7071 in 56c3639

  	"com.amazonaws.dynamodb#Scan": {
  	"type": "operation",
  	"input": {
  	"target": "com.amazonaws.dynamodb#ScanInput"
  	},
  	"output": {
  	"target": "com.amazonaws.dynamodb#ScanOutput"
  	},
  	"errors": [
  	{
  	"target": "com.amazonaws.dynamodb#InternalServerError"
  	},
  	{
  	"target": "com.amazonaws.dynamodb#InvalidEndpointException"
  	},
  	{
  	"target": "com.amazonaws.dynamodb#ProvisionedThroughputExceededException"
  	},
  	{
  	"target": "com.amazonaws.dynamodb#RequestLimitExceeded"
  	},
  	{
  	"target": "com.amazonaws.dynamodb#ResourceNotFoundException"
  	}
  	],
  	"traits": {
  	"smithy.api#documentation": "<p>The <code>Scan</code> operation returns one or more items and item attributes by accessing every\n item in a table or a secondary index. To have DynamoDB return fewer items, you can provide a <code>FilterExpression</code> operation.</p>\n <p>If the total number of scanned items exceeds the maximum dataset size limit of 1 MB, the\n scan stops and results are returned to the user as a <code>LastEvaluatedKey</code> value\n to continue the scan in a subsequent operation. The results also include the number of\n items exceeding the limit. A scan can result in no table data meeting the filter\n criteria. </p>\n <p>A single <code>Scan</code> operation reads up to the maximum number of items set (if\n using the <code>Limit</code> parameter) or a maximum of 1 MB of data and then apply any\n filtering to the results using <code>FilterExpression</code>. If\n <code>LastEvaluatedKey</code> is present in the response, you need to paginate the\n result set. For more information, see <a href=\"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Scan.html#Scan.Pagination\">Paginating the\n Results</a> in the <i>Amazon DynamoDB Developer Guide</i>. </p>\n <p>\n <code>Scan</code> operations proceed sequentially; however, for faster performance on\n a large table or secondary index, applications can request a parallel <code>Scan</code>\n operation by providing the <code>Segment</code> and <code>TotalSegments</code>\n parameters. For more information, see <a href=\"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Scan.html#Scan.ParallelScan\">Parallel\n Scan</a> in the <i>Amazon DynamoDB Developer Guide</i>.</p>\n <p>\n <code>Scan</code> uses eventually consistent reads when accessing the data in a\n table; therefore, the result set might not include the changes to data in the table\n immediately before the operation began. If you need a consistent copy of the data, as of\n the time that the <code>Scan</code> begins, you can set the <code>ConsistentRead</code>\n parameter to <code>true</code>.</p>"
  	}
  	},

• Query

  aws-sdk-js-v3/codegen/sdk-codegen/aws-models/dynamodb.2012-08-10.json

  Lines 5766 to 5794 in 56c3639

  	"com.amazonaws.dynamodb#Query": {
  	"type": "operation",
  	"input": {
  	"target": "com.amazonaws.dynamodb#QueryInput"
  	},
  	"output": {
  	"target": "com.amazonaws.dynamodb#QueryOutput"
  	},
  	"errors": [
  	{
  	"target": "com.amazonaws.dynamodb#InternalServerError"
  	},
  	{
  	"target": "com.amazonaws.dynamodb#InvalidEndpointException"
  	},
  	{
  	"target": "com.amazonaws.dynamodb#ProvisionedThroughputExceededException"
  	},
  	{
  	"target": "com.amazonaws.dynamodb#RequestLimitExceeded"
  	},
  	{
  	"target": "com.amazonaws.dynamodb#ResourceNotFoundException"
  	}
  	],
  	"traits": {
  	"smithy.api#documentation": "<p>The <code>Query</code> operation finds items based on primary key values.\n You can query any table or secondary index that has a composite primary key (a partition\n key and a sort key).\n </p>\n <p>Use the <code>KeyConditionExpression</code> parameter to provide a specific value\n for the partition key. The <code>Query</code> operation will return all of the items\n from the table or index with that partition key value. You can optionally narrow the\n scope of the <code>Query</code> operation by specifying a sort key value and a\n comparison operator in <code>KeyConditionExpression</code>. To further refine the <code>Query</code> results,\n you can optionally provide a <code>FilterExpression</code>. A <code>FilterExpression</code> determines which items\n within the results should be returned to you. All of the other results are discarded.\n </p>\n <p>\n A <code>Query</code> operation always returns a result set. If no matching items are found,\n the result set will be empty. Queries that do not return results consume the minimum number of\n read capacity units for that type of read operation.\n </p>\n <note>\n <p>\n DynamoDB calculates the number of read capacity units consumed based on item size,\n not on the amount of data that is returned to an application. The number of capacity\n units consumed will be the same whether you request all of the attributes (the default behavior)\n or just some of them (using a projection expression). The number will also be the same\n whether or not you use a <code>FilterExpression</code>.\n </p>\n </note>\n <p>\n <code>Query</code> results are always sorted by the sort key value. If the data type of the sort key is Number,\n the results are returned in numeric order; otherwise, the results are returned in order of UTF-8 bytes.\n By default, the sort order is ascending. To reverse the order, set the <code>ScanIndexForward</code> parameter\n to false.\n </p>\n <p> A single <code>Query</code> operation will read up to the maximum number of items\n set (if using the <code>Limit</code> parameter) or a maximum of 1 MB of data and then\n apply any filtering to the results using <code>FilterExpression</code>. If\n <code>LastEvaluatedKey</code> is present in the response, you will need to paginate\n the result set. For more information, see <a href=\"https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.Pagination\">Paginating\n the Results</a> in the <i>Amazon DynamoDB Developer Guide</i>. </p>\n <p>\n <code>FilterExpression</code> is applied after a <code>Query</code> finishes, but before\n the results are returned.\n A <code>FilterExpression</code> cannot contain partition key or sort key attributes.\n You need to specify those attributes in the <code>KeyConditionExpression</code>.\n </p>\n <note>\n <p>\n A <code>Query</code> operation can return an empty result set and a <code>LastEvaluatedKey</code>\n if all the items read for the page of results are filtered out.\n </p>\n </note>\n <p>You can query a table, a local secondary index, or a global secondary index. For a\n query on a table or on a local secondary index, you can set the\n <code>ConsistentRead</code> parameter to <code>true</code> and obtain a\n strongly consistent result. Global secondary indexes support eventually consistent reads\n only, so do not specify <code>ConsistentRead</code> when querying a global\n secondary index.</p>"
  	}
  	},

The pagination is present in JS SDK v2 for DynamoDB scan and query commands https://github.com/aws/aws-sdk-js/blob/ffcad209c2b2459bb819a27a18bb7b9e83f7e276/apis/dynamodb-2012-08-10.paginators.json#L23-L34

We're tracking this issue internally to populate required pagination trait.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.