[Improvement]: Dispose the current tasks after task deserialization error

[zhongqishang]

Search before asking

• I have searched in the issues and found no similar issues.

What would you like to be improved?

Upgrading the Iceberg version causes deserialization error during task recovery.
Before that, we delete all currently running tasks through sql.

2024-02-04 09:56:21,047 ERROR [main] [com.netease.arctic.server.ArcticServiceContainer] [] - AMS start error
java.lang.IllegalArgumentException: deserialization error 
	at com.netease.arctic.utils.SerializationUtil.simpleDeserialize(SerializationUtil.java:74) ~[classes/:?]
	at com.netease.arctic.server.persistence.TaskFilesPersistence.loadTaskInputs(TaskFilesPersistence.java:52) ~[classes/:?]
	at com.netease.arctic.server.optimizing.OptimizingQueue$TableOptimizingProcess.loadTaskRuntimes(OptimizingQueue.java:640) ~[classes/:?]
	at com.netease.arctic.server.optimizing.OptimizingQueue$TableOptimizingProcess.<init>(OptimizingQueue.java:373) ~[classes/:?]
	at com.netease.arctic.server.optimizing.OptimizingQueue.initTableRuntime(OptimizingQueue.java:109) ~[classes/:?]
	at java.util.ArrayList.forEach(ArrayList.java:1259) ~[?:1.8.0_331]
	at com.netease.arctic.server.optimizing.OptimizingQueue.<init>(OptimizingQueue.java:102) ~[classes/:?]
	at com.netease.arctic.server.DefaultOptimizingService.lambda$loadOptimizingQueues$0(DefaultOptimizingService.java:140) ~[classes/:?]
	at java.util.ArrayList.forEach(ArrayList.java:1259) ~[?:1.8.0_331]
	at com.netease.arctic.server.DefaultOptimizingService.loadOptimizingQueues(DefaultOptimizingService.java:130) ~[classes/:?]
	at com.netease.arctic.server.DefaultOptimizingService.access$300(DefaultOptimizingService.java:86) ~[classes/:?]
	at com.netease.arctic.server.DefaultOptimizingService$TableRuntimeHandlerImpl.initHandler(DefaultOptimizingService.java:453) ~[classes/:?]
	at com.netease.arctic.server.table.RuntimeHandlerChain.initialize(RuntimeHandlerChain.java:59) ~[classes/:?]
	at com.netease.arctic.server.table.DefaultTableService.initialize(DefaultTableService.java:365) ~[classes/:?]
	at com.netease.arctic.server.ArcticServiceContainer.startService(ArcticServiceContainer.java:152) ~[classes/:?]
	at com.netease.arctic.server.ArcticServiceContainer.main(ArcticServiceContainer.java:110) [classes/:?]
Caused by: java.io.InvalidClassException: org.apache.iceberg.BaseFile; local class incompatible: stream classdesc serialVersionUID = 2557541712180305280, local class serialVersionUID = -5355849892016662001
	at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:699) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2028) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1875) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2028) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1875) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2209) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1692) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readArray(ObjectInputStream.java:2142) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1680) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2454) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2378) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2236) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1692) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:508) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:466) ~[?:1.8.0_331]
	at java.util.HashMap.readObject(HashMap.java:1418) ~[?:1.8.0_331]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_331]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_331]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_331]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_331]
	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1185) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2345) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2236) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1692) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:508) ~[?:1.8.0_331]
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:466) ~[?:1.8.0_331]
	at com.netease.arctic.utils.SerializationUtil.simpleDeserialize(SerializationUtil.java:71) ~[classes/:?]
	... 15 more

How should we improve?

Catch the deserialization error , dispose tasks that cannot be restored.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Subtasks

No response

Code of Conduct

• I agree to follow this project's Code of Conduct