Spark ML vector does not work on Serverless

[karthikt16]

Hello,

Running serverless spark batch

"runtime_config":{
"version": "1.1.11",

I got the below error while saving data to bigquery:

The same code works on dataproc. I was able to store the table.
Connector used: gs://spark-lib/bigquery/spark-bigquery-with-dependencies_2.12-0.24.2.jar

I also tested the same code with the most recent connector "gs://spark-lib/bigquery/spark-bigquery-with-dependencies_2.12-0.31.1.jar" and here again it fails.

Could you please check?

Code to replicate the issue
testvector.txt

regards

[davidrabinowitz]

Fixed in release 0.32.1