运行FeatureEngForRecModel.py时报错

[Evan-wyl]

21/02/23 10:54:55 ERROR TaskSetManager: Task 0 in stage 10.0 failed 1 times; aborting job
21/02/23 10:54:55 WARN TaskSetManager: Lost task 0.0 in stage 11.0 (TID 210, localhost, executor driver): TaskKilled (Stage cancelled)
Traceback (most recent call last):
File "D:/code/sparrowrecsys/SparrowRecSys-master/RecPySpark/src/com/sparrowrecsys/offline/pyspark/featureeng/FeatureEngForRecModel.py", line 151, in
samplesWithMovieFeatures = addMovieFeatures(movieSamples, ratingSamplesWithLabel)
File "D:/code/sparrowrecsys/SparrowRecSys-master/RecPySpark/src/com/sparrowrecsys/offline/pyspark/featureeng/FeatureEngForRecModel.py", line 54, in addMovieFeatures
samplesWithMovies4.show(5, truncate=False)
File "D:\ProgramData\Anaconda3\envs\recoenv\lib\site-packages\pyspark\sql\dataframe.py", line 380, in show
print(self._jdf.showString(n, int(truncate), vertical))
File "D:\ProgramData\Anaconda3\envs\recoenv\lib\site-packages\py4j\java_gateway.py", line 1257, in call
answer, self.gateway_client, self.target_id, self.name)
File "D:\ProgramData\Anaconda3\envs\recoenv\lib\site-packages\pyspark\sql\utils.py", line 63, in deco
return f(*a, **kw)
File "D:\ProgramData\Anaconda3\envs\recoenv\lib\site-packages\py4j\protocol.py", line 328, in get_return_value
format(target_id, ".", name), value)
py4j.protocol.Py4JJavaError: An error occurred while calling o132.showString.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 10.0 failed 1 times, most recent failure: Lost task 0.0 in stage 10.0 (TID 209, localhost, executor driver): org.apache.spark.SparkException: Python worker failed to connect back.

[wzhe06]

@bfgf52 可以帮忙看一下这个pyspark问题吗？

[shexuan]

看样子似乎是网络问题？重启试试

[sjb961121]

我也有相同的问题，请问你解决了吗 @Evan-wyl

[bfgf52]

我也有相同的问题，请问你解决了吗 @Evan-wyl

请问你们的spark版本号是多少呢？我的spark版本是2.4.0，测试后都是没问题的

[sjb961121]

我也有相同的问题，请问你解决了吗 @Evan-wyl

请问你们的spark版本号是多少呢？我的spark版本是2.4.0，测试后都是没问题的

之前是3.0的版本，改成2.4后确实解决了，谢谢