-
Notifications
You must be signed in to change notification settings - Fork 334
Persistence implementations for list pagination #1555
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
1c383cf to
66d20aa
Compare
snazy
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think that the approach implemented here yields correct results. The code assumes strict ordering of integer IDs, which is from general experience w/ relational DBs and in particular looking at org.apache.polaris.extension.persistence.relational.jdbc.IdGenerator not the case.
| String query = QueryGenerator.generateSelectQuery(new ModelEntity(), params); | ||
|
|
||
| if (pageToken instanceof EntityIdPageToken entityIdPageToken) { | ||
| query += String.format(" AND id > %d ORDER BY id ASC", entityIdPageToken.getId()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How would this work with org.apache.polaris.extension.persistence.relational.jdbc.IdGenerator?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you're right that it won't; this logic is copied from EclipseLink where IDs are always increasing but does not work with the current way that the JDBC metastore creates IDs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd propose to change the Page/PageToken contract in a way to push the parameter "as is" down to the persistence layer and let the persistence implementation deal with it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I spoke with @singhpk234 who noted this is probably the same discussion as here on the old PR. With that context, I think we might be OK here.
IMO it's alright that the list ordering you'd get across metastores won't be the same. Other than that difference, seems like everything should work with JDBC's IdGenerator. Although the IDs aren't generated sequentially, pagination only uses the entity ID as an essentially arbitrary consistent ordering.
The key implication here is that if an entity gets created in the middle of a listing operation (e.g. between list calls 2 and 3) it may or may not show up in the next page. An alternative would be to try to filter it out so that the behavior is more obvious & consistent, but I think the simple approach that ultimately gives the user a chance to see these new entities is good.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Losing new entities that are stored after pagination start is fine from my POV. The JDBC persistence does not implement catalog-level versioning, so this is unavoidable, I guess.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed that we will naturally lose some entities, the question is whether we are OK with entities stored after pagination start being lost nondeterministically rather than always. Right now, whether the new entity is lost or not depends on what entity ID it gets. If it gets a high entity ID you might see it in a later page and if it gets a low ID you might not.
My thought on this question is "yes", because it's better to show the entity if we can and it simplifies the code.
But if we feel like this is too unintuitive, we can add a secondary filter on the entity's creation time to try and get rid of these entities (on a best-effort basis, since clocks are not perfect).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think current pagination behaviour wrt concurrent changes is fine.
Making it deterministic would be a great addition to Polaris, but that, I think, has a much broader scope. For example, if an entry is deleted after pagination starts, but a client re-submits a page request using an old token, the new response would still be inconsistent with the old response.
From my POV a complete and deterministic pagination solution implies catalog-level versioning.
On this note, I think it's not true actually. The code assumes that IDs are sortable but it doesn't rely on any kind of semantic meaning behind this comparison. So IDs can be created totally randomly and you can still paginate simply by breaking that random key space into pages of some size. There's no assumption that new entries will appear at the end, either. |
adnanhemani
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Small comments, overall LGTM
| try { | ||
| String[] parts = token.split("/"); | ||
| if (parts.length < 1) { | ||
| throw new IllegalArgumentException("Invalid token format: " + token); | ||
| } else if (parts[0].equals(EntityIdPageToken.PREFIX)) { | ||
| int resolvedPageSize = pageSize == null ? Integer.parseInt(parts[2]) : pageSize; | ||
| return new EntityIdPageToken(Long.parseLong(parts[1]), resolvedPageSize); | ||
| } else { | ||
| throw new IllegalArgumentException("Unrecognized page token: " + token); | ||
| } | ||
| } catch (NumberFormatException | IndexOutOfBoundsException e) { | ||
| LOGGER.debug(e.getMessage()); | ||
| throw new IllegalArgumentException("Invalid token format: " + token); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I, personally, find this fragment a bit more complex than it may need to be. Is there a reason why we cannot defensively check for the right amount array length after splitting it right away? Same for the NumberFormationException?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wanted to structure the code in a way that obviously leaves the door open for other PageToken implementations -- those would have different array length expectations. So we check the prefix first, and then parse the token using the logic appropriate for the PageToken implementation that the prefix corresponds to. Ideally we could even push this parsing logic down into some method in the PageToken.
In the old PR, there were 2 parseable PageToken implementations. I do agree that it looks a little clunky with the single PageToken implementation we have now. If this really is too confusing I can simplify this logic and then we can re-complicate it if/when we add a new PageToken.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this really is too confusing I can simplify this logic and then we can re-complicate it if/when we add a new PageToken.
I'd personally prefer this - but don't care enough if we do this or not, I can understand the reasoning.
...ce/common/src/test/java/org/apache/polaris/service/persistence/pagination/PageTokenTest.java
Show resolved
Hide resolved
...ce/common/src/test/java/org/apache/polaris/service/persistence/pagination/PageTokenTest.java
Show resolved
Hide resolved
Following up on apache#1555 * Refactor pagination code to delineate API-level page tokens and internal "pointers to data" * Requests deal with the "previous" token, user-provided page size (optional) and the previous request's page size. * Concentrate the logic of combining page size requests and previous tokens in PageTokenUtil * PageToken subclasses are no longer necessary. EntityIdPaging handles pagination over ordered result sets with static helper methods. Co-authored-by: Eric Maynard <eric.maynard+oss@snowflake.com>
|
This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
Based on apache#1838, following up on apache#1555 * Allows multiple implementations of `Token` referencing the "next page", encapsulated in `PageToken`. No changes to `polaris-core` needed to add custom `Token` implementations. * Extensible to (later) support (cryptographic) signatures to prevent tampered page-token * Refactor pagination code to delineate API-level page tokens and internal "pointers to data" * Requests deal with the "previous" token, user-provided page size (optional) and the previous request's page size. * Concentrate the logic of combining page size requests and previous tokens in `PageTokenUtil` * `PageToken` subclasses are no longer necessary. * Serialzation of `PageToken` uses Jackson serialization (smile format) Since no (metastore level) implementation handling pagination existed before, no backwards compatibility is needed. Co-authored-by: Dmitri Bourlatchkov <dmitri.bourlatchkov@gmail.com> Co-authored-by: Eric Maynard <eric.maynard+oss@snowflake.com>
Based on #1838, following up on #1555 * Allows multiple implementations of `Token` referencing the "next page", encapsulated in `PageToken`. No changes to `polaris-core` needed to add custom `Token` implementations. * Extensible to (later) support (cryptographic) signatures to prevent tampered page-token * Refactor pagination code to delineate API-level page tokens and internal "pointers to data" * Requests deal with the "previous" token, user-provided page size (optional) and the previous request's page size. * Concentrate the logic of combining page size requests and previous tokens in `PageTokenUtil` * `PageToken` subclasses are no longer necessary. * Serialzation of `PageToken` uses Jackson serialization (smile format) Since no (metastore level) implementation handling pagination existed before, no backwards compatibility is needed. Co-authored-by: Dmitri Bourlatchkov <dmitri.bourlatchkov@gmail.com> Co-authored-by: Eric Maynard <eric.maynard+oss@snowflake.com>
* Ignore regenerate.sh on README.md (apache#1999) * OpenAPI-generate: Omit generation timestamp (apache#2004) The jaxrs-resteasy OpenAPI generator adds the generation timestamp to the generated sources by default. This behavior leads to different code for every generation, leading to unnecessary rebuilds (and re-tests), because the generated `.class` files are different. * Update CatalogEntity::Builder to set default CatalogType as INTERNAL (apache#1998) Encountered the issue while adding additional validations to `ExternalCatalog`. The `CatalogEntity::Builder` checks if the `Catalog::type` is set to `INTERNAL`, if not it defaults to `EXTERNAL`. However this is the opposite of the behavior defined in polaris-management-service.yml where the default is set to `INTERNAL`. This change only affects tests because in other cases the catalog entity is generated from the REST request. Testing: Updated CatalogEntityTest to ensure that the default is set to `INTERNAL`. * Support IMPLICIT authentication type for federated catalogs (apache#1925) Previously, the ConnectionConfigInfo required explicit AuthenticationParameters for every federated catalog. However, certain catalogs types that Polaris federates to (either now or in the future) allow `IMPLICIT` authentication, wherein the authentication parameters are picked from the environment or configuration files. This change enables federating to such catalogs without passing dummy secrets. The `IMPLICIT` option is guarded by the `SUPPORTED_EXTERNAL_CATALOG_AUTHENTICATION_TYPES`. Hence users may create federated catalogs with `IMPLICIT` authentication only when the administrator explicitly enables this feature. * Fix helm doc (apache#2001) * Fix helm doc * Remove persistent ref * Remove persistent ref * Fixes based on feedback * Fixes based on feedback * Fixes based on feedback * Fixes based on feedback * feat(auth): Ability to override active roles provider per realm (apache#2000) * feat(auth): Ability to override active roles provider per realm * deprecate old property * add tests * Introduce an option to add object storage prefix to table locations (apache#1966) ### Problem Currently, Polaris enforces that the physical layout of entities maps to the logical layout: ``` catalog └── ns1 ├── ns2 │ └── table_b └── table_a ``` In the above example, the base locations of `table_a` and `ns2` are expected to be children of `ns1`, and the location of `table_b` is expected to be a child of `ns2`. This behavior is controlled by `ALLOW_UNSTRUCTURED_TABLE_LOCATION` and is the basis for the sibling overlap check when `OPTIMIZED_SIBLING_CHECK` is disabled or persistence cannot support the optimized check. However, some users have reported that this physical organization of data can lead to undesirable performance characteristics when hotspotting occurs across namespaces. If the underlying storage is range partitioned by key, this organization will tend to physically collocate logically-similar entities. ### Solution To solve this problem, this PR introduces a new option `DEFAULT_LOCATION_OBJECT_STORAGE_PREFIX_ENABLED` which alters the behavior of the catalog when creating a table without a user-specified location. With the feature disabled, a table such as `ns1.table_a` will have a path like this: ``` s3://catalog/base/ns1/table_a/ ``` With the feature enabled, a prefix is added before the namespace: ``` s3://catalog/base/0010/0101/0110/10010100/ns1/table_a/ ``` This serves to eliminate the physical collocation of tables in the same namespace (or with similarly-named namespaces or table names). This functionality is similar to Iceberg's `write.object-storage.enabled`, but it applies across tables and namespaces. The two features can and should be combined to achieve the best distribution of data files throughout the key space. ### Configuration & Sibling Overlap Check If an admin doesn't care about the risk of vending credentials with the sibling overlap check disabled, they can enable the feature with these configs: ``` polaris.features.DEFAULT_LOCATION_OBJECT_STORAGE_PREFIX_ENABLED=true polaris.features.ALLOW_UNSTRUCTURED_TABLE_LOCATION=true polaris.features.ALLOW_TABLE_LOCATION_OVERLAP=true polaris.behavior-changes.VALIDATE_VIEW_LOCATION_OVERLAP=false ``` In order to use this feature and to preserve the sibling overlap check, you can configure the service with: ``` polaris.features.DEFAULT_LOCATION_OBJECT_STORAGE_PREFIX_ENABLED=true polaris.features.ALLOW_UNSTRUCTURED_TABLE_LOCATION=true polaris.features.OPTIMIZED_SIBLING_CHECK=true ``` However, note that the `OPTIMIZED_SIBLING_CHECK` comes with some caveats as outlined in its description. Namely, it currently only works with some persistence implementations and it requires all location-based entities to have a recently-introduced field set. These locations are expected to be suffixed with `/`, and locations with many `/` may not be eligible for the optimized check. Older Polaris deployments may not meet these requirements without a migration or backfill. Accordingly combining these two features should be considered experimental for the time being. * Cleanup collaborators in `.asf.yaml` (apache#2008) Some devs were added in the past to `.asf.yaml` to let CI run w/o committer approval. After [INFRA-26985](https://issues.apache.org/jira/browse/INFRA-26985) this is no longer necessary, so the file can be cleaned up. * Fix bunch of OpenAPI generation issues (apache#2005) The current way how OpenAPI Java code is generated suffers from a bunch of issues: * Changes to any of the source spec files requires a Gradle `clean`, otherwise old generated Java source will remain - i.e. "no longer" existing sources are not removed. This is addressed by adding an additional action to `GenerateTask`. * The output of `GenerateTask` was explicitly not cached, this is removed, so the output is cached. * Add explicit inputs to `GenerateTask` to the whole templates and spec folders. * Restructure the download page (apache#2011) * Add 1.0.0-incubating release to the downloads page (apache#2018) * Add 1.0.0 docs to the huge menu (apache#2020) * Improve the bundle jar license and notice remove using exclude (apache#1991) * Remove duplicate MetaStoreManagerFactory mocks (apache#2023) also rename the field for clarity and consistency * Update Makefile for python client with auto setup (apache#1995) Automate python client setup and use a virtual env instead to avoid change an end-users' OS python * Add Helm Chart repo to the downloads page (apache#2025) * Publish helm doc (apache#2014) * Make PolarisConfiguration member variables private (apache#2007) * Make PolarisConfiguration members private * Make methods final * Use the 0.9.0 doc from the versioned-docs branch (apache#2026) * Helm key grouping and test cases (apache#2002) * Helm key grouping and test cases * Update README.md * Added backwards compatible * Fix conflict * Use coalesce instead of if else * Remove kind (apache#2028) * Remove kind * Remove k8 dir from check-md-link.yml * Sync helm doc (apache#2034) * Update release-guide.md for publishing docs (apache#2035) * [Site] Simplify the doc directory structure (apache#2033) * [Site] Update release-guide.md for release dir name (apache#2037) * Fix gralde command for helm image and remove simple-values.yaml (apache#2036) * Using the closer.lua download script (apache#2038) * Fix the LICENSE and NOTICE with the latest dependency updates (apache#1939) * Fix invalid redirect from public page (apache#2041) * Make StorageCredentialCache safe for mutli-realm usage (apache#2021) Injecting the request-scoped `RealmContext` into the application-scoped `StorageCredentialCache` makes things unnecessarily complicated. Similarly `StorageCredentialCacheKey` having a `@Nullable callContext` makes it more difficult to reason about. Instead we can determine all realm-specific values at the time of insertion (from the `PolarisCallContext` param of `getOrGenerateSubScopeCreds`). * feat(ci): Improve Gradle cache in CI (apache#1928) * Introduce RealmConfig (apache#2015) Getting a config value currently requires quite a ceremony: ``` ctx.getPolarisCallContext() .getConfigurationStore() .getConfiguration(ctx.getRealmContext(), "ALLOW_WILDCARD_LOCATION", false)) ``` since a `PolarisConfigurationStore` cant be used without a `RealmContext` it makes sense to add a dedicated interface. this allows removal of verbose code and also moves towards injecting that interface via CDI at a request/realm scope in the future. * Fix CI (apache#2043) The `store-gradle-cache` job in the `gradle.yaml` GitHub workflow is missing a "checkout", this change adds it to fix CI. * Fix CI (no 2) (apache#2044) The newly added `store-gradle-cache` CI job has run some Gradle task to trigger Gradle's automatic cache cleanup. In the source project Nessie we used a simple task `showVersion` to do this. As having this task in Polaris might be useful, adding this task as there's no other suitable task (cheap and not generating much output) seems legit. * Bump Quarkus version to unblock IntelliJ build (apache#1958) Use Quarkus 3.24.3 to fix build issues with `:polaris-server:classes` * Use application-scoped StorageCredentialCache (apache#2022) Since `StorageCredentialCache` is application scoped and after 6ddd148 its constructor no longer uses the `RealmContext` passed into `getOrCreateStorageCredentialCache` we can now let all `PolarisEntityManager` instances share the same `StorageCredentialCache`. * Attempt to make Renovate work again (apache#2052) Looks that I accidentally broke Renovate with apache#1891. This was made under the impression of the [Renovate change to support `baseBranches` in forking-renovate] (renovatebot/renovate#36054). However, a [later Renovate change](renovatebot/renovate#35579) seems to break that. The plan here is to: 1. remove the regex from our `baseBranches` option - if that doesn't work then 2. just use the default branch * main: Update actions/stale digest to 128b2c8 (apache#2053) * main: Update dependency com.azure:azure-sdk-bom to v1.2.36 (apache#2054) * main: Update dependency com.fasterxml.jackson:jackson-bom to v2.19.1 (apache#2055) * main: Update dependency com.google.cloud:google-cloud-storage-bom to v2.53.3 (apache#2057) * main: Update registry.access.redhat.com/ubi9/openjdk-21-runtime Docker tag to v1.22-1.1752066187 (apache#2059) * main: Update dependency com.github.ben-manes.caffeine:caffeine to v3.2.2 (apache#2056) * main: Update dependency gradle to v8.14.3 (main) (apache#2058) * main: Update dependency gradle to v8.14.3 * Adjust Gradle update --------- Co-authored-by: Robert Stupp <snazy@snazy.de> * main: Update dependency io.micrometer:micrometer-bom to v1.15.2 (apache#2063) * main: Update dependency com.diffplug.spotless:spotless-plugin-gradle to v7.1.0 (apache#2067) * main: Update dependency com.nimbusds:nimbus-jose-jwt to v10.3.1 (apache#2062) * main: Update docker.io/prom/prometheus Docker tag to v3.5.0 (apache#2071) * main: Update dependency org.junit:junit-bom to v5.13.3 (apache#2064) * main: Update docker.io/jaegertracing/all-in-one Docker tag to v1.71.0 (apache#2070) * main: Update medyagh/setup-minikube action to v0.0.20 (apache#2066) * main: Update dependency org.apache.commons:commons-lang3 to v3.18.0 (apache#2069) * main: Update log4j2 monorepo to v2.25.1 (apache#2073) * main: Update immutables to v2.11.0 (apache#2072) * main: Update dependency org.testcontainers:testcontainers-bom to v1.21.3 (apache#2065) * main: Update dependency com.google.errorprone:error_prone_core to v2.40.0 (apache#2068) * main: Update dependency io.netty:netty-codec-http2 to v4.2.3.Final (apache#2074) * main: Update dependency io.prometheus:prometheus-metrics-exporter-servlet-jakarta to v1.3.10 (apache#2076) * main: Update dependency net.ltgt.gradle:gradle-errorprone-plugin to v4.3.0 (apache#2079) * main: Update dependency io.projectreactor.netty:reactor-netty-http to v1.2.8 (apache#2075) * main: Update dependency com.gradleup.shadow:shadow-gradle-plugin to v8.3.8 (apache#2061) * main: Update dependency org.eclipse.persistence:eclipselink to v4.0.7 (apache#2078) * Add External Identity Providers page to unreleased documentation (apache#2013) --------- Co-authored-by: Alexandre Dutra <adutra@apache.org> Co-authored-by: Eric Maynard <emaynard@apache.org> * main: Update dependency io.opentelemetry:opentelemetry-bom to v1.52.0 (apache#2082) * main: Update dependency software.amazon.awssdk:bom to v2.31.78 (apache#2080) * main: Update dependency com.adobe.testing:s3mock-testcontainers to v4.6.0 (apache#2081) * main: Update dependency io.smallrye.common:smallrye-common-annotation to v2.13.7 (apache#2083) * Revert PR 2033 (apache#2087) The PR apache#2033 was merged within less than 3 hours, late on a Friday. Since that change does not address an issue that seriously deserves a quick reaction nor is it a "nit", I'm proposing to revert the change. We do have [community best practices](https://polaris.apache.org/community/contributing-guidelines/) stating to give the whole community enough time to review, which did not happen. There are concerns that the PR apache#2033 will interfere with the whole effort to automate releases. Since there was no change to review and raise the concerns, I'd like to revert it to not cause any friction with that bigger effort. Revert "Fix invalid redirect from public page (apache#2041)", commit 493bc8e. Revert "[Site] Simplify the doc directory structure (apache#2033)", commit 2db2f10. * Renovate PRs, branch name + PR subject (apache#2060) Until June, Renovate PRs behaved a little bit different than today. The difference is the branch name. Before it was something like `renovate-bot/renovate/main/org.openapi.generator-7.x`, now it's like `renovate-bot/renovate/main-main/actions-stale-digest` (branch name is duplicated). I also noticed that the branch name is repeated in the PR subject, which started to be that way some longer ago. This change removes both duplications. * Simplify RealmEntityManagerFactory usage in tests (apache#2050) since all ctor params are created in `IcebergCatalogTest.before` we can do the same for `RealmEntityManagerFactory` `PolarisAuthzTestBase.entityManager` is already getting derived from `realmEntityManagerFactory`: https://github.com/apache/polaris/blob/2c2052c28f899aaa85e5f11a9131d9812ec62679/runtime/service/src/test/java/org/apache/polaris/service/quarkus/admin/PolarisAuthzTestBase.java#L247 * Use PolarisImmutable for StorageCredentialCacheKey (apache#2029) * remove unused entityId from StorageCredentialCacheKey * convert StorageCredentialCacheKey to immutables * Disable renovatebot on release branches (apache#2085) Per the mailing list thread "[DISCUSS] Disable renovatebot on release branches", we should not do automatic dependency upgrades for release branches. Since it seems `release/1.0.x` is a release branch, we can remove this regex from renovate's list. * Site: Remove non-OSS query engines from front page (apache#2031) * update query engines list * Add Dremio OSS * fix(deps): update immutables to v2.11.1 (apache#2113) * fix(deps): update dependency boto3 to v1.39.4 (apache#2116) * chore: Avoid deprecated `DefaultCredentialsProvider.create()` (apache#2119) Use `DefaultCredentialsProvider.builder().build()` as suggested by AWS SDK javadoc. * fix(deps): update dependency boto3 to v1.39.6 (apache#2120) * Extensible pagination token implementation (apache#1938) Based on apache#1838, following up on apache#1555 * Allows multiple implementations of `Token` referencing the "next page", encapsulated in `PageToken`. No changes to `polaris-core` needed to add custom `Token` implementations. * Extensible to (later) support (cryptographic) signatures to prevent tampered page-token * Refactor pagination code to delineate API-level page tokens and internal "pointers to data" * Requests deal with the "previous" token, user-provided page size (optional) and the previous request's page size. * Concentrate the logic of combining page size requests and previous tokens in `PageTokenUtil` * `PageToken` subclasses are no longer necessary. * Serialzation of `PageToken` uses Jackson serialization (smile format) Since no (metastore level) implementation handling pagination existed before, no backwards compatibility is needed. Co-authored-by: Dmitri Bourlatchkov <dmitri.bourlatchkov@gmail.com> Co-authored-by: Eric Maynard <eric.maynard+oss@snowflake.com> * Site/dev: allow overriding the podman/docker binaries detection (apache#2051) The scripts in the `bin/` directory are built to work with both Docker and podman. There are nuances in how both behave, especially wrt docker/podman-compose. Some local environment specifics require the use of `podman-compose`, others the use of `docker-compose`. The default behavior is to prefer the `podman` and `podman-compose` binaries, if those exist and fall back to `docker` and `docker-compose`. Some setups using podman however require the use of `docker-compose` even if `podman-compose` is installed. This may manifest in an error message stating that `--userns` and `--pod` cannot be used together. In that case create a file `.user-settings` in the `site/` folder and add these two lines: ```bash DOCKER=docker COMPOSE=docker-compose ``` * NoSQL: build descriptions * NoSQL: README nits * NoSQL: Misc ports * Pagination * Policy fixes * Adoptions to "conflicting" changes * runtime-service test abstractions * Last merged commit d2667e5 --------- Co-authored-by: Yong Zheng <yongzheng0809@gmail.com> Co-authored-by: Pooja Nilangekar <poojan@umd.edu> Co-authored-by: Alexandre Dutra <adutra@users.noreply.github.com> Co-authored-by: Eric Maynard <eric.maynard+oss@snowflake.com> Co-authored-by: Yufei Gu <yufei@apache.org> Co-authored-by: Yun Zou <yunzou.colostate@gmail.com> Co-authored-by: Christopher Lambert <xn137@gmx.de> Co-authored-by: Dongjoon Hyun <dongjoon@apache.org> Co-authored-by: JB Onofré <jbonofre@apache.org> Co-authored-by: Alexandre Dutra <adutra@apache.org> Co-authored-by: Adnan Hemani <adnan.h@berkeley.edu> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Mark Hoerth <47870294+markhoerth@users.noreply.github.com> Co-authored-by: Eric Maynard <emaynard@apache.org> Co-authored-by: Danica Fine <danica.fine@gmail.com> Co-authored-by: Dmitri Bourlatchkov <dmitri.bourlatchkov@gmail.com> Co-authored-by: Honah (Jonas) J. <honahx@apache.org>
In #1528, we introduced the interface changes necessary to paginate requests to listTables, listViews, and listNamespaces. This PR adds the persistence-level logic for pagination and a new PageToken type
EntityIdPageTokenused to paginate requests based on entity ID.