Getting-started/eclipselink example does not work #1865
Description
Describe the bug
Issue 1
When testing 1.0.x, I encountered the following error when starting the eclipselink example
| 2025-06-11 16:50:45,682 INFO [io.qua.htt.access-log] [,POLARIS] [,,,] (executor-thread-1) 172.18.0.4 - - [11/Jun/2025:16:50:45 +0000] "POST /api/catalog/v1/oauth/tokens HTTP/1.1" 400 87
spark-sql-1 | Exception in thread "main" org.apache.iceberg.exceptions.NotAuthorizedException: Not authorized: invalid_client: The Client is invalid
spark-sql-1 | at org.apache.iceberg.rest.ErrorHandlers$OAuthErrorHandler.accept(ErrorHandlers.java:271)
spark-sql-1 | at org.apache.iceberg.rest.ErrorHandlers$OAuthErrorHandler.accept(ErrorHandlers.java:252)
spark-sql-1 | at org.apache.iceberg.rest.HTTPClient.throwFailure(HTTPClient.java:215)
spark-sql-1 | at org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:299)
spark-sql-1 | at org.apache.iceberg.rest.BaseHTTPClient.postForm(BaseHTTPClient.java:111)
spark-sql-1 | at org.apache.iceberg.rest.auth.OAuth2Util.fetchToken(OAuth2Util.java:267)
spark-sql-1 | at org.apache.iceberg.rest.auth.OAuth2Manager.initSession(OAuth2Manager.java:87)
spark-sql-1 | at org.apache.iceberg.rest.auth.OAuth2Manager.initSession(OAuth2Manager.java:40)
spark-sql-1 | at org.apache.iceberg.rest.RESTSessionCatalog.initialize(RESTSessionCatalog.java:201)
spark-sql-1 | at org.apache.iceberg.rest.RESTCatalog.initialize(RESTCatalog.java:82)
spark-sql-1 | at org.apache.iceberg.CatalogUtil.loadCatalog(CatalogUtil.java:277)
spark-sql-1 | at org.apache.iceberg.CatalogUtil.buildIcebergCatalog(CatalogUtil.java:331)
spark-sql-1 | at org.apache.iceberg.spark.SparkCatalog.buildIcebergCatalog(SparkCatalog.java:153)
spark-sql-1 | at org.apache.iceberg.spark.SparkCatalog.initialize(SparkCatalog.java:752)
spark-sql-1 | at org.apache.spark.sql.connector.catalog.Catalogs$.load(Catalogs.scala:65)
spark-sql-1 | at org.apache.spark.sql.connector.catalog.CatalogManager.$anonfun$catalog$1(CatalogManager.scala:54)
spark-sql-1 | at scala.collection.mutable.HashMap.getOrElseUpdate(HashMap.scala:86)
spark-sql-1 | at org.apache.spark.sql.connector.catalog.CatalogManager.catalog(CatalogManager.scala:54)
spark-sql-1 | at org.apache.spark.sql.connector.catalog.CatalogManager.currentCatalog(CatalogManager.scala:135)
spark-sql-1 | at org.apache.spark.sql.connector.catalog.CatalogManager.currentNamespace(CatalogManager.scala:94)
spark-sql-1 | at org.apache.spark.sql.internal.CatalogImpl.currentDatabase(CatalogImpl.scala:68)
spark-sql-1 | at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.currentDB$1(SparkSQLCLIDriver.scala:288)
spark-sql-1 | at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.promptWithCurrentDB$1(SparkSQLCLIDriver.scala:295)
spark-sql-1 | at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:299)
spark-sql-1 | at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
spark-sql-1 | at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
spark-sql-1 | at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
spark-sql-1 | at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
spark-sql-1 | at java.base/java.lang.reflect.Method.invoke(Method.java:569)
spark-sql-1 | at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
spark-sql-1 | at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1034)
spark-sql-1 | at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:199)
spark-sql-1 | at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:222)
spark-sql-1 | at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91)
spark-sql-1 | at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1125)
spark-sql-1 | at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1134)
spark-sql-1 | at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
It seems the USER_CLIENT_ID``USER_CLIENT_SECRET env is never set in the getting-started/eclipse/docker-compose.yml
healthcheck:
test: "curl localhost:4040"
interval: 5s
retries: 15
command: [
/opt/spark/bin/spark-sql,
--packages, "org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.9.0,org.apache.iceberg:iceberg-aws-bundle:1.9.0,org.apache.iceberg:iceberg-gcp-bundle:1.9.0,org.apache.iceberg:iceberg-azure-bundle:1.9.0",
--conf, "spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions",
--conf, "spark.sql.catalog.quickstart_catalog=org.apache.iceberg.spark.SparkCatalog",
--conf, "spark.sql.catalog.quickstart_catalog.type=rest",
--conf, "spark.sql.catalog.quickstart_catalog.warehouse=quickstart_catalog",
--conf, "spark.sql.catalog.quickstart_catalog.uri=http://polaris:8181/api/catalog",
--conf, "spark.sql.catalog.quickstart_catalog.credential=${USER_CLIENT_ID}:${USER_CLIENT_SECRET}",
--conf, "spark.sql.catalog.quickstart_catalog.scope=PRINCIPAL_ROLE:ALL",
--conf, "spark.sql.defaultCatalog=quickstart_catalog",
--conf, "spark.sql.catalogImplementation=in-memory",
--conf, "spark.driver.extraJavaOptions=-Divy.cache.dir=/tmp -Divy.home=/tmp"
]
By changing them to CLIENT_ID and CLIENT_SECRET, everything seems to start like normal
Issue 2
The spark sql example should be updated to
CREATE NAMESPACE quickstart_catalog.ns1;
USE quickstart_catalog.ns1;
CREATE TABLE table1 (id int, name string);
INSERT INTO table1 VALUES (1, 'a');
SELECT * FROM table1;
Issue 3
Step 4 could be simplified as
export POLARIS_TOKEN=$(curl -s http://localhost:8181/api/catalog/v1/oauth/tokens \
--user root:s3cr3t \
-d 'grant_type=client_credentials' \
-d 'scope=PRINCIPAL_ROLE:ALL' | jq -r .access_token)
The resolve part in the original code does not work for me somehow
To Reproduce
No response
Actual Behavior
No response
Expected Behavior
No response
Additional context
No response
System information
No response