[NSE-891] Spark executor lost while DatasetFileWriter failed with speculation

[jackylee-ch]

What changes were proposed in this pull request?

When spark speculation is true, the FinalTask may be killed as another attempt succeeded. In this case, DatasetFileWriter may failed and threw exception, which will cause executor failed.

How was this patch tested?

unit test

[github-actions]

#891

[jackylee-ch]

After fix, we got only InterruptedException.

22/05/07 16:30:14:821 ERROR Executor task launch worker for task 66.1 in stage 57.0 (TID 183239) Utils: Aborting task
java.lang.InterruptedException
	at java.lang.Object.wait(Native Method)
	at java.lang.Thread.join(Thread.java:1252)
	at java.lang.Thread.join(Thread.java:1326)
	at com.intel.oap.spark.sql.ArrowWriteQueue.close(ArrowWriteQueue.scala:71)
	at com.intel.oap.spark.sql.execution.datasources.arrow.ArrowFileFormat$$anon$1$$anon$2.close(ArrowFileFormat.scala:97)
	at org.apache.spark.sql.execution.datasources.FileFormatDataWriter.releaseCurrentWriter(FileFormatDataWriter.scala:64)
	at org.apache.spark.sql.execution.datasources.FileFormatDataWriter.releaseResources(FileFormatDataWriter.scala:75)