Skip to content
This repository has been archived by the owner on Jan 12, 2021. It is now read-only.

There is exception printed out in the console (log) and directory spark-<id> is left #21

Open
alex-ber opened this issue Jun 9, 2016 · 0 comments
Assignees
Labels

Comments

@alex-ber
Copy link

alex-ber commented Jun 9, 2016

There is exception printed out in the console (log) and directory spark- is left
after JVM complete normal shutdown in java.io.tmpdir
(strictly speaking it can be other directory, and it can be configured through the property, so
it should be taken in the account)
As far I investigate the root cause is https://issues.apache.org/jira/browse/SPARK-8333 that is there is open bug in Spark (my system is Windows 10 64 bit).

As a work-arround what I did I;

  1. switch off from the log this message
    <logger name="org.apache.spark.util.Utils" level="off" />
  2. Right some rule that clean up all spark* folder from java.io.tmpdir (or more precise position) on the JVM startup

Maybe it worst to put this to the HiveQLUnit?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants