-
Notifications
You must be signed in to change notification settings - Fork 90
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add a query paginator #1530
Conversation
google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java
Show resolved
Hide resolved
google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java
Outdated
Show resolved
Hide resolved
google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java
Outdated
Show resolved
Hide resolved
google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java
Outdated
Show resolved
Hide resolved
google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java
Outdated
Show resolved
Hide resolved
google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java
Outdated
Show resolved
Hide resolved
google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java
Outdated
Show resolved
Hide resolved
this.query = query; | ||
this.chunkSize = chunkSize; | ||
this.prevSplitPoint = ByteString.EMPTY; | ||
this.firstRun = true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do you need to set the limit on the query here? otherwise gtNextQuery will return the wrong thing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm this should be handled by advnace and getNextQuery needs to be called after advance:
Query query = Query.create(...).range("a", "z");
Query.QueryPaginator paginator = query.createQueryPaginator(100);
ByteString lastSeenRowKey = ByteString.EMPTY;
while (paginator.advance(lastSeenRowKey)) {
List<Row> rows = client.readRowsCallable().all().call(paginator.getNextQuery());
for (Row row : rows) {
// do some processing
lastSeenRow = row;
}
}
google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/Query.java
Outdated
Show resolved
Hide resolved
private ByteString prevSplitPoint; | ||
|
||
QueryPaginator(@Nonnull Query query, int pageSize) { | ||
this.hasOverallLimit = query.builder.getRowsLimit() == 0 ? false : true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this.hasOverallLimit = query.builder.getRowsLimit() > 0;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤦♀️
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm after nit
🤖 I have created a release *beep* *boop* --- ## [2.17.0](https://togithub.com/googleapis/java-bigtable/compare/v2.16.0...v2.17.0) (2022-12-07) ### Features * Add a query paginator ([#1530](https://togithub.com/googleapis/java-bigtable/issues/1530)) ([5c8e1f6](https://togithub.com/googleapis/java-bigtable/commit/5c8e1f656b15177ecef4709b9e502cef58cca479)) * Add test proxy to java-bigtable client ([#1498](https://togithub.com/googleapis/java-bigtable/issues/1498)) ([132b4e4](https://togithub.com/googleapis/java-bigtable/commit/132b4e4fe340ca316de8eef2e2133b6dabd9dec3)) * Implement cross-project table restore ([#1536](https://togithub.com/googleapis/java-bigtable/issues/1536)) ([335977c](https://togithub.com/googleapis/java-bigtable/commit/335977c499b1776c8fa861d13195ffc49e468e0a)) * Next release from main branch is 2.17.0 ([#1514](https://togithub.com/googleapis/java-bigtable/issues/1514)) ([4fc6a93](https://togithub.com/googleapis/java-bigtable/commit/4fc6a93a25362df7bc107d48f94e1c00b6bd608d)) ### Dependencies * Update dependency com.google.cloud:google-cloud-monitoring-bom … ([#1531](https://togithub.com/googleapis/java-bigtable/issues/1531)) ([ee98338](https://togithub.com/googleapis/java-bigtable/commit/ee9833835a84cee202b142950b28704db682ac0c)) * Update dependency com.google.cloud:google-cloud-shared-dependencies to v3.1.0 ([#1539](https://togithub.com/googleapis/java-bigtable/issues/1539)) ([3244cef](https://togithub.com/googleapis/java-bigtable/commit/3244cefd4d77a44bab6ccaa8f5c03e3d31575625)) * Update dependency kr.motd.maven:os-maven-plugin to v1.7.1 ([#1518](https://togithub.com/googleapis/java-bigtable/issues/1518)) ([8309681](https://togithub.com/googleapis/java-bigtable/commit/830968109a3754a12bd0bc92674fe42ae529b924)) * Update dependency org.graalvm.buildtools:junit-platform-native to v0.9.19 ([#1537](https://togithub.com/googleapis/java-bigtable/issues/1537)) ([7f7c478](https://togithub.com/googleapis/java-bigtable/commit/7f7c478a2163c46d10ed39ea3c1b046f971d4569)) * Update dependency org.graalvm.buildtools:native-maven-plugin to v0.9.19 ([#1538](https://togithub.com/googleapis/java-bigtable/issues/1538)) ([9d3df57](https://togithub.com/googleapis/java-bigtable/commit/9d3df57d7bfeafd9cbcf56ecd58b52cc1b14ba7b)) --- This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #<issue_number_goes_here> ☕️
If you write sample code, please follow the samples format.