Skip to content

Conversation

@MonkeyCanCode
Copy link
Contributor

With change in #2049, we are not making polaris as a package and remove dependency on the the top level polaris bash script (still useful ATM until we start publishing python CLI which is being asked in #2042). However, this does bring one non-backward compatible change where if the venv used by polaris was created long before, it can be a breaking change #2293. To fix this problem, this PR added a check if polaris command existed. If missing (which indicated older venv), it will then use a sub-process to perform a force repair then loaded the new venv and continue the workflow.

Here is how to reproduce this issue and how this PR fix this issue:

To reproduce this issue, I will setup an venv with older branch:

➜  polaris git:(main) git checkout apache-polaris-0.9.0-incubating
➜  polaris git:(apache-polaris-0.9.0-incubating) ./polaris
...
First time setup complete.
Traceback (most recent call last):
  File "/Users/yong/Desktop/1/polaris/regtests/client/python/cli/polaris_cli.py", line 144, in <module>
    PolarisCli.execute()
    ~~~~~~~~~~~~~~~~~~^^
  File "/Users/yong/Desktop/1/polaris/regtests/client/python/cli/polaris_cli.py", line 50, in execute
    client_builder = PolarisCli._get_client_builder(options)
  File "/Users/yong/Desktop/1/polaris/regtests/client/python/cli/polaris_cli.py", line 105, in _get_client_builder
    raise Exception(f'Please provide credentials via either {Argument.to_flag_name(Arguments.CLIENT_ID)} &'
    ...<3 lines>...
                    f' {CLIENT_SECRET_ENV}.')
Exception: Please provide credentials via either --client-id & --client-secret or --access-token. Alternatively, you may set the environment variables CLIENT_ID & CLIENT_SECRET.
(polaris-venv) ➜  polaris git:(apache-polaris-0.9.0-incubating) which polaris
polaris not found
(polaris-venv) ➜  polaris git:(apache-polaris-0.9.0-incubating) ./polaris
Traceback (most recent call last):
  File "/Users/yong/Desktop/1/polaris/regtests/client/python/cli/polaris_cli.py", line 144, in <module>
    PolarisCli.execute()
    ~~~~~~~~~~~~~~~~~~^^
  File "/Users/yong/Desktop/1/polaris/regtests/client/python/cli/polaris_cli.py", line 50, in execute
    client_builder = PolarisCli._get_client_builder(options)
  File "/Users/yong/Desktop/1/polaris/regtests/client/python/cli/polaris_cli.py", line 105, in _get_client_builder
    raise Exception(f'Please provide credentials via either {Argument.to_flag_name(Arguments.CLIENT_ID)} &'
    ...<3 lines>...
                    f' {CLIENT_SECRET_ENV}.')
Exception: Please provide credentials via either --client-id & --client-secret or --access-token. Alternatively, you may set the environment variables CLIENT_ID & CLIENT_SECRET.

Now to reproduce the issue, i will switch to the main branch:

(polaris-venv) ➜  polaris git:(apache-polaris-0.9.0-incubating) git switch main
(polaris-venv) ➜  polaris git:(main) ✗ export DOCKER=docker
(polaris-venv) ➜  polaris git:(main) ✗ ./polaris
Configuration on demand is an incubating feature.
...
License fix complete
Deletion complete
Regeneration complete

BUILD SUCCESSFUL in 15s
14 actionable tasks: 1 executed, 13 up-to-date
./polaris: line 76: exec: polaris: not found
(polaris-venv) ➜  polaris git:(main) ✗ which polaris
polaris not found

Now the issue is easily reproducible. With the current PR, user will then see the following:

➜  polaris git:(main) ✗ ./polaris
Polaris executable not found. Forcing a repair...
Configuration on demand is an incubating feature.
...
Installing the current project: polaris (1.0.0)
Dependencies repaired.
Traceback (most recent call last):
  File "/Users/yong/Desktop/1/polaris/polaris-venv/bin/polaris", line 6, in <module>
    sys.exit(main())
             ~~~~^^
  File "/Users/yong/Desktop/1/polaris/client/python/cli/polaris_cli.py", line 219, in main
    PolarisCli.execute()
    ~~~~~~~~~~~~~~~~~~^^
  File "/Users/yong/Desktop/1/polaris/client/python/cli/polaris_cli.py", line 70, in execute
    client_builder = PolarisCli._get_client_builder(options)
  File "/Users/yong/Desktop/1/polaris/client/python/cli/polaris_cli.py", line 175, in _get_client_builder
    raise Exception(
    ...<5 lines>...
    )
Exception: Please provide credentials via either --client-id & --client-secret or --access-token. Alternatively, you may set the environment variables CLIENT_ID & CLIENT_SECRET.
➜  polaris git:(main) ✗ source polaris-venv/bin/activate
(polaris-venv) ➜  polaris git:(main) ✗ which polaris
/Users/yong/Desktop/1/polaris/polaris-venv/bin/polaris

@github-project-automation github-project-automation bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Aug 10, 2025
@MonkeyCanCode MonkeyCanCode merged commit f6a2abb into apache:main Aug 10, 2025
12 checks passed
@github-project-automation github-project-automation bot moved this from Ready to merge to Done in Basic Kanban Board Aug 10, 2025
snazy added a commit to snazy/polaris that referenced this pull request Nov 20, 2025
* Helm Chart: remove duplicate line in authenticationOptions template (apache#2297)

* Move python client Makefile into the root level one (apache#2140)

* Move python client Makefile into the root level one

* Update workflow

* add client-lint to pre-commit

* Update README.md to include client

* fix(deps): update dependency boto3 to v1.40.5 (apache#2300)

* fix(deps): update dependency org.assertj:assertj-core to v3.27.4 (apache#2298)

* Fix python CI client-integration-test (apache#2305)

`CONTAINER_TOOL` got renamed to `DOCKER` in
6764a88
but then
eda7074
added more `CONTAINER_TOOL` usage

* fix(docs): update BasePolarisAuthenticator to DefaultAuthenticator (apache#2303)

* Fix Hadoop federation to initialize the configuration prior to catalog initialization (apache#2282)

This PR addresses the gap in Hadoop federation to ensure that we initialize the hadoop configuration prior to initializing the catalog object.

The iceberg library expects the hadoop configuration to be initialized before creating a HadoopCatalog object. This change ensures that Polaris is compatible with the underlying iceberg library. Additionally, since the config initialization is based on the underlying (default) core-site.xml file, the change ensures that the federated catalog was created using `IMPLICIT` authentication mode. 

Testing: 
Due to current limitations in the current test setup, tested the change manually.
[TODO] Add a regtest with hadoop federation once the change is baked into the apache/polaris docker image.

* fix(deps): update dependency boto3 to v1.40.6 (apache#2308)

* fix(deps): update dependency io.opentelemetry:opentelemetry-bom to v1.53.0 (apache#2309)

* Perform force repair when polaris not found (apache#2313)

* fix(deps): update dependency com.github.spotbugs:spotbugs-annotations to v4.9.4 (apache#2312)

* chore(deps): update dependency pre-commit to v4.3.0 (apache#2311)

* fix(deps): update dependency software.amazon.awssdk:bom to v2.32.19 (apache#2314)

* fix(deps): update dependency com.gradleup.shadow:shadow-gradle-plugin to v9.0.1 (apache#2310)

* Separate Cloud Integration Tests (apache#2283)

Currently, cloud integration tests are part of the regular test suite
but require cloud credentials to execute properly. This creates
unnecessary overhead for developers who just want to run local builds
or for CI jobs that don't have cloud access configured.

* chore(deps): update actions/checkout action to v5 (apache#2319)

* CatalogEntity: internal endpoint not considered (apache#2292)

Setting an S3 internal endpoint doesn't work, because the property's not carried over from the OpenAPI model type into `AwsStorageConfigInfo`.

* client/python: loosen boto3 dep (apache#2188)

It's hard to depend on this package when the boto3 dependency is so strict.

* fix(deps): update dependency io.micrometer:micrometer-bom to v1.15.3 (apache#2321)

* NoSQL: test adjustment

* Last merged commit 45e8e02

---------

Co-authored-by: Alexandre Dutra <adutra@apache.org>
Co-authored-by: Yong Zheng <yongzheng0809@gmail.com>
Co-authored-by: Mend Renovate <bot@renovateapp.com>
Co-authored-by: Christopher Lambert <xn137@gmx.de>
Co-authored-by: Yujiang Zhong <42907416+zhongyujiang@users.noreply.github.com>
Co-authored-by: Pooja Nilangekar <poojan@umd.edu>
Co-authored-by: Tamas Mate <50709850+tmater@users.noreply.github.com>
Co-authored-by: Tyler Rockwood <rockwotj@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants