Automate regeneration of Python client (#1675)

When running regtests or the CLI, we should run `./gradlew regeneratePythonClient` in order to run against the most up-to-date Python client.

As part of this change, I've also updated the generator to 7.12 to match what's in the `libs.version.toml` and regenerated the code managed in github.

Once the automatic generation works as part of CI, we can remove the generated code from the repo altogether. In the future, it can be added to .gitignore.

Author: eric-maynard

.github/workflows/python-client.yml

@@ -44,6 +44,15 @@ jobs:
       - name: Checkout Polaris project
         uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
 
+      - name: Set up JDK for Gradle
+        uses: actions/setup-java@v4
+        with:
+          distribution: 'temurin'
+          java-version: '21'
+
+      - name: Run regeneratePythonClient
+        run: ./gradlew regeneratePythonClient
+
       - name: Set up Python ${{ matrix.python-version }}
         uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
         with:

.github/workflows/regtest.yml

@@ -43,6 +43,9 @@ jobs:
       - name: Fix permissions
         run: mkdir -p regtests/output && chmod 777 regtests/output && chmod 777 regtests/t_*/ref/*
 
+      - name: Run regeneratePythonClient
+        run: ./gradlew regeneratePythonClient
+
       - name: Image build
         run: |
           ./gradlew \

.github/workflows/spark_client_regtests.yml

@@ -43,6 +43,9 @@ jobs:
       - name: Fix permissions
         run: mkdir -p regtests/output && chmod 777 regtests/output && chmod 777 regtests/t_*/ref/*
 
+      - name: Run regeneratePythonClient
+        run: ./gradlew regeneratePythonClient
+
       - name: Project build without testing
         run: ./gradlew assemble
 

client/python/.openapi-generator-ignore

@@ -40,3 +40,12 @@
 #docs/*.md
 # Then explicitly reverse the ignore rule for a single file:
 #!docs/README.md
+
+.github/workflows/python.yml
+.gitlab-ci.yml
+pyproject.toml
+requirements.txt
+test-requirements.txt
+setup.py
+
+

client/python/.openapi-generator/FILES

@@ -1,6 +1,4 @@
-.github/workflows/python.yml
 .gitignore
-.gitlab-ci.yml
 .travis.yml
 README.md
 docs/AddPartitionSpecUpdate.md
@@ -274,11 +272,7 @@ polaris/catalog/models/view_update.py
 polaris/catalog/models/view_version.py
 polaris/catalog/py.typed
 polaris/catalog/rest.py
-pyproject.toml
-requirements.txt
 setup.cfg
-setup.py
-test-requirements.txt
 test/__init__.py
 test/test_async_planning_result.py
 test/test_completed_planning_result.py

client/python/.openapi-generator/VERSION

@@ -1 +1 @@
-7.8.0-SNAPSHOT
+7.12.0

client/python/.travis.yml

@@ -20,13 +20,13 @@
 # ref: https://docs.travis-ci.com/user/languages/python
 language: python
 python:
-  - "3.7"
   - "3.8"
   - "3.9"
   - "3.10"
   - "3.11"
+  - "3.12"
   # uncomment the following if needed
-  #- "3.11-dev"  # 3.11 development branch
+  #- "3.12-dev"  # 3.12 development branch
   #- "nightly"  # nightly build
 # command to install dependencies
 install:

client/python/README.md

@@ -25,12 +25,12 @@ This Python package is automatically generated by the [OpenAPI Generator](https:
 
 - API version: 0.0.1
 - Package version: 1.0.0
-- Generator version: 7.8.0-SNAPSHOT
+- Generator version: 7.12.0
 - Build package: org.openapitools.codegen.languages.PythonClientCodegen
 
 ## Requirements.
 
-Python 3.7+
+Python 3.8+
 
 ## Installation & Usage
 ### pip install

client/python/docs/CatalogAPI.md

@@ -238,7 +238,11 @@ Name | Type | Description  | Notes
 
 Create a table in the given namespace
 
-Create a table or start a create transaction, like atomic CTAS.  If `stage-create` is false, the table is created immediately.  If `stage-create` is true, the table is not created, but table metadata is initialized and returned. The service should prepare as needed for a commit to the table commit endpoint to complete the create transaction. The client uses the returned metadata to begin a transaction. To commit the transaction, the client sends all create and subsequent changes to the table commit route. Changes from the table create operation include changes like AddSchemaUpdate and SetCurrentSchemaUpdate that set the initial table state.
+Create a table or start a create transaction, like atomic CTAS.
+
+If `stage-create` is false, the table is created immediately.
+
+If `stage-create` is true, the table is not created, but table metadata is initialized and returned. The service should prepare as needed for a commit to the table commit endpoint to complete the create transaction. The client uses the returned metadata to begin a transaction. To commit the transaction, the client sends all create and subsequent changes to the table commit route. Changes from the table create operation include changes like AddSchemaUpdate and SetCurrentSchemaUpdate that set the initial table state.
 
 ### Example
 
@@ -693,7 +697,7 @@ void (empty response body)
 
 List namespaces, optionally providing a parent namespace to list underneath
 
-List all namespaces at a certain level, optionally starting from a given parent namespace. If table accounting.tax.paid.info exists, using 'SELECT NAMESPACE IN accounting' would translate into `GET /namespaces?parent=accounting` and must return a namespace, [\"accounting\", \"tax\"] only. Using 'SELECT NAMESPACE IN accounting.tax' would translate into `GET /namespaces?parent=accounting%1Ftax` and must return a namespace, [\"accounting\", \"tax\", \"paid\"]. If `parent` is not provided, all top-level namespaces should be listed.
+List all namespaces at a certain level, optionally starting from a given parent namespace. If table accounting.tax.paid.info exists, using 'SELECT NAMESPACE IN accounting' would translate into `GET /namespaces?parent=accounting` and must return a namespace, ["accounting", "tax"] only. Using 'SELECT NAMESPACE IN accounting.tax' would translate into `GET /namespaces?parent=accounting%1Ftax` and must return a namespace, ["accounting", "tax", "paid"]. If `parent` is not provided, all top-level namespaces should be listed.
 
 ### Example
 
@@ -1157,7 +1161,13 @@ Name | Type | Description  | Notes
 
 Load a table from the catalog
 
-Load a table from the catalog.  The response contains both configuration and table metadata. The configuration, if non-empty is used as additional configuration for the table that overrides catalog configuration. For example, this configuration may change the FileIO implementation to be used for the table.  The response also contains the table's full metadata, matching the table metadata JSON file.  The catalog configuration may contain credentials that should be used for subsequent requests for the table. The configuration key \"token\" is used to pass an access token to be used as a bearer token for table requests. Otherwise, a token may be passed using a RFC 8693 token type as a configuration key. For example, \"urn:ietf:params:oauth:token-type:jwt=<JWT-token>\".
+Load a table from the catalog.
+
+The response contains both configuration and table metadata. The configuration, if non-empty is used as additional configuration for the table that overrides catalog configuration. For example, this configuration may change the FileIO implementation to be used for the table.
+
+The response also contains the table's full metadata, matching the table metadata JSON file.
+
+The catalog configuration may contain credentials that should be used for subsequent requests for the table. The configuration key "token" is used to pass an access token to be used as a bearer token for table requests. Otherwise, a token may be passed using a RFC 8693 token type as a configuration key. For example, "urn:ietf:params:oauth:token-type:jwt=<JWT-token>".
 
 ### Example
 
@@ -1256,7 +1266,13 @@ Name | Type | Description  | Notes
 
 Load a view from the catalog
 
-Load a view from the catalog.  The response contains both configuration and view metadata. The configuration, if non-empty is used as additional configuration for the view that overrides catalog configuration.  The response also contains the view's full metadata, matching the view metadata JSON file.  The catalog configuration may contain credentials that should be used for subsequent requests for the view. The configuration key \"token\" is used to pass an access token to be used as a bearer token for view requests. Otherwise, a token may be passed using a RFC 8693 token type as a configuration key. For example, \"urn:ietf:params:oauth:token-type:jwt=<JWT-token>\".
+Load a view from the catalog.
+
+The response contains both configuration and view metadata. The configuration, if non-empty is used as additional configuration for the view that overrides catalog configuration.
+
+The response also contains the view's full metadata, matching the view metadata JSON file.
+
+The catalog configuration may contain credentials that should be used for subsequent requests for the view. The configuration key "token" is used to pass an access token to be used as a bearer token for view requests. Otherwise, a token may be passed using a RFC 8693 token type as a configuration key. For example, "urn:ietf:params:oauth:token-type:jwt=<JWT-token>".
 
 ### Example
 
@@ -2078,7 +2094,9 @@ void (empty response body)
 
 Set or remove properties on a namespace
 
-Set and/or remove properties on a namespace. The request body specifies a list of properties to remove and a map of key value pairs to update. Properties that are not in the request are not modified or removed by this call. Server implementations are not required to support namespace properties.
+Set and/or remove properties on a namespace. The request body specifies a list of properties to remove and a map of key value pairs to update.
+Properties that are not in the request are not modified or removed by this call.
+Server implementations are not required to support namespace properties.
 
 ### Example
 
@@ -2173,7 +2191,13 @@ Name | Type | Description  | Notes
 
 Commit updates to a table
 
-Commit updates to a table.  Commits have two parts, requirements and updates. Requirements are assertions that will be validated before attempting to make and commit changes. For example, `assert-ref-snapshot-id` will check that a named ref's snapshot ID has a certain value. Server implementations are required to fail with a 400 status code if any unknown updates or requirements are received.  Updates are changes to make to table metadata. For example, after asserting that the current main ref is at the expected snapshot, a commit may add a new child snapshot and set the ref to the new snapshot id.  Create table transactions that are started by createTable with `stage-create` set to true are committed using this route. Transactions should include all changes to the table, including table initialization, like AddSchemaUpdate and SetCurrentSchemaUpdate. The `assert-create` requirement is used to ensure that the table was not created concurrently.
+Commit updates to a table.
+
+Commits have two parts, requirements and updates. Requirements are assertions that will be validated before attempting to make and commit changes. For example, `assert-ref-snapshot-id` will check that a named ref's snapshot ID has a certain value. Server implementations are required to fail with a 400 status code if any unknown updates or requirements are received.
+
+Updates are changes to make to table metadata. For example, after asserting that the current main ref is at the expected snapshot, a commit may add a new child snapshot and set the ref to the new snapshot id.
+
+Create table transactions that are started by createTable with `stage-create` set to true are committed using this route. Transactions should include all changes to the table, including table initialization, like AddSchemaUpdate and SetCurrentSchemaUpdate. The `assert-create` requirement is used to ensure that the table was not created concurrently.
 
 ### Example
 

client/python/docs/ConfigurationAPI.md

@@ -32,7 +32,31 @@ Method | HTTP request | Description
 
 List all catalog configuration settings
 
- All REST clients should first call this route to get catalog configuration properties from the server to configure the catalog and its HTTP client. Configuration from the server consists of two sets of key/value pairs. - defaults -  properties that should be used as default configuration; applied before client configuration - overrides - properties that should be used to override client configuration; applied after defaults and client configuration  Catalog configuration is constructed by setting the defaults, then client- provided configuration, and finally overrides. The final property set is then used to configure the catalog.  For example, a default configuration property might set the size of the client pool, which can be replaced with a client-specific setting. An override might be used to set the warehouse location, which is stored on the server rather than in client configuration.  Common catalog configuration settings are documented at https://iceberg.apache.org/docs/latest/configuration/#catalog-properties  The catalog configuration also holds an optional `endpoints` field that contains information about the endpoints supported by the server. If a server does not send the `endpoints` field, a default set of endpoints is assumed: - GET /v1/{prefix}/namespaces - POST /v1/{prefix}/namespaces - GET /v1/{prefix}/namespaces/{namespace} - DELETE /v1/{prefix}/namespaces/{namespace} - POST /v1/{prefix}/namespaces/{namespace}/properties - GET /v1/{prefix}/namespaces/{namespace}/tables - POST /v1/{prefix}/namespaces/{namespace}/tables - GET /v1/{prefix}/namespaces/{namespace}/tables/{table} - POST /v1/{prefix}/namespaces/{namespace}/tables/{table} - DELETE /v1/{prefix}/namespaces/{namespace}/tables/{table} - POST /v1/{prefix}/namespaces/{namespace}/register - POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/metrics - POST /v1/{prefix}/tables/rename - POST /v1/{prefix}/transactions/commit 
+ All REST clients should first call this route to get catalog configuration properties from the server to configure the catalog and its HTTP client. Configuration from the server consists of two sets of key/value pairs.
+- defaults -  properties that should be used as default configuration; applied before client configuration
+- overrides - properties that should be used to override client configuration; applied after defaults and client configuration
+
+Catalog configuration is constructed by setting the defaults, then client- provided configuration, and finally overrides. The final property set is then used to configure the catalog.
+
+For example, a default configuration property might set the size of the client pool, which can be replaced with a client-specific setting. An override might be used to set the warehouse location, which is stored on the server rather than in client configuration.
+
+Common catalog configuration settings are documented at https://iceberg.apache.org/docs/latest/configuration/#catalog-properties
+
+The catalog configuration also holds an optional `endpoints` field that contains information about the endpoints supported by the server. If a server does not send the `endpoints` field, a default set of endpoints is assumed:
+- GET /v1/{prefix}/namespaces
+- POST /v1/{prefix}/namespaces
+- GET /v1/{prefix}/namespaces/{namespace}
+- DELETE /v1/{prefix}/namespaces/{namespace}
+- POST /v1/{prefix}/namespaces/{namespace}/properties
+- GET /v1/{prefix}/namespaces/{namespace}/tables
+- POST /v1/{prefix}/namespaces/{namespace}/tables
+- GET /v1/{prefix}/namespaces/{namespace}/tables/{table}
+- POST /v1/{prefix}/namespaces/{namespace}/tables/{table}
+- DELETE /v1/{prefix}/namespaces/{namespace}/tables/{table}
+- POST /v1/{prefix}/namespaces/{namespace}/register
+- POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/metrics
+- POST /v1/{prefix}/tables/rename
+- POST /v1/{prefix}/transactions/commit 
 
 ### Example