You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Seems those are RDD APIs and will generate lots of un-supported messages.
Such as:
! <DeserializeToObjectExec> cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec
! <CreateExternalRow> createexternalrow(prediction#327, label#322, 1.0#400, newInstance(class org.apache.spark.ml.linalg.VectorUDT).deserialize, StructField(prediction,DoubleType,true), StructField(label,DoubleType,true), StructField(1.0,DoubleType,false), StructField(probability,org.apache.spark.ml.linalg.VectorUDT@3bfc3ba7,true)) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow
@Expression <AttributeReference> prediction#327 could run on GPU
@Expression <AttributeReference> label#322 could run on GPU
@Expression <AttributeReference> 1.0#400 could run on GPU
! <Invoke> newInstance(class org.apache.spark.ml.linalg.VectorUDT).deserialize cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.Invoke
! <NewInstance> newInstance(class org.apache.spark.ml.linalg.VectorUDT) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.NewInstance
!Expression <AttributeReference> probability#326 cannot run on GPU because expression AttributeReference probability#326 produces an unsupported type org.apache.spark.ml.linalg.VectorUDT@3bfc3ba7
!Expression <AttributeReference> obj#406 cannot run on GPU because expression AttributeReference obj#406 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row)
!Exec <ProjectExec> cannot run on GPU because unsupported data types in input: org.apache.spark.ml.linalg.VectorUDT@3bfc3ba7 [probability#326]; not all expressions can be replaced; unsupported data types in output: org.apache.spark.ml.linalg.VectorUDT@3bfc3ba7 [probability#326]
@Expression <AttributeReference> prediction#327 could run on GPU
@Expression <AttributeReference> label#322 could run on GPU
@Expression <Alias> 1.0 AS 1.0#400 could run on GPU
@Expression <Literal> 1.0 could run on GPU
!Expression <AttributeReference> probability#326 cannot run on GPU because expression AttributeReference probability#326 produces an unsupported type org.apache.spark.ml.linalg.VectorUDT@3bfc3ba7
!Exec <FileSourceScanExec> cannot run on GPU because unsupported data types org.apache.spark.ml.linalg.VectorUDT@3bfc3ba7 [probability] in read for Parquet; unsupported data types in output: org.apache.spark.ml.linalg.VectorUDT@3bfc3ba7 [probability#326]
The text was updated successfully, but these errors were encountered:
I wish we can support pyspark.ml.evaluation.{BinaryClassificationEvaluator, MulticlassClassificationEvaluator}.
Take the example from https://stackoverflow.com/questions/60772315/how-to-evaluate-a-classifier-with-pyspark-2-4-5 :
Seems those are RDD APIs and will generate lots of un-supported messages.
Such as:
The text was updated successfully, but these errors were encountered: