[Bug] indexWriter don't close if exception thrown when close dataWriter in HdfsShuffleWriteHandler #346
Description
Code of Conduct
- I agree to follow this project's Code of Conduct
Search before asking
- I have searched in the issues and found no similar issues.
Describe the bug
I found there are many threads in shuffle server as follows:
"DataStreamer for file /uniffle/data/application_1662640300316_231973311668726143398/4/138-138/10.xx.xx.xx-59999_0.index block BP-150738345-10.xx.xx.xx-1553601482353:blk_4236218692_3237686719" #1062624
daemon prio=5 os_prio=0 tid=0x00007fb184030000 nid=0x1b323 in Object.wait() [0x00007faf48d0f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:417)
- locked <0x00007fba52000f90> (a java.util.LinkedList)
Locked ownable synchronizers:
- None
And all DataStreamer is for index file.
Affects Version(s)
master
Uniffle Server Log Output
No response
Uniffle Engine Log Output
No response
Uniffle Server Configurations
No response
Uniffle Engine Configurations
No response
Additional context
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!