listTables error with python 3.11

[pmbrull]

Following the steps from the docs

pyspark --packages io.delta:delta-spark_2.12:3.0.0 --conf "spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension" --conf "spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog"

and running

spark.range(1).createTempView("test_view")
spark.catalog.listTables()

gives us

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/pmbrull/projects/OpenMetadata/venv/lib/python3.11/site-packages/pyspark/sql/catalog.py", line 361, in listTables
    iter = self._jcatalog.listTables(dbName).toLocalIterator()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pmbrull/projects/OpenMetadata/venv/lib/python3.11/site-packages/pyspark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1322, in __call__
  File "/Users/pmbrull/projects/OpenMetadata/venv/lib/python3.11/site-packages/pyspark/errors/exceptions/captured.py", line 186, in deco
    raise converted from None
pyspark.errors.exceptions.captured.ParseException:
[PARSE_SYNTAX_ERROR] Syntax error at or near end of input.(line 1, pos 0)

== SQL ==

^^^

We won't support deltalake in 3.11 until we can fix this. The same code is running with python 3.8 - 3.10

[pmbrull]

we're skipping delta tests for 3.11 here until we can find a solution for this, and keeping the old delta & spark versions

[pmbrull]

Having the same error with:

• Python 3.10.12
• Spark 3.4.2
• delta-spark 2.4.0

running with

pyspark --packages io.delta:delta-core_2.12:2.4.0 --conf "spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension" --conf "spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog" --conf "spark.driver.extraJavaOptions=-Dderby.system.home=/tmp/spark/metastore"