HIVE-29340: Restore Hive ATSHook to populate the Tez View Queries tab #6213
Conversation
|
Have made few minor custom changes:
|
|
There seems to be a behaviour changes in parsing of fields in tez. For instance, in callerId the User is getting appended. 
|
Aggarwal-Raghav
force-pushed
the
HIVE-29340
branch
from
a272064 to
18b6088
Compare
asf-ci-hive
added
tests pending
tests unstable
and removed
tests unstable
tests pending
labels
This is revert of HIVE-20078
Aggarwal-Raghav
force-pushed
the
HIVE-29340
branch
from
18b6088 to
1ee6c8a
Compare
|
| CLIENT_IP_ADDRESS, HIVE_ADDRESS, HIVE_INSTANCE_TYPE, CONF, PERF, LLAP_APP_ID | ||
| }; | ||
| private enum ExecutionMode { | ||
| MR, TEZ, LLAP, SPARK, NONE |
There was a problem hiding this comment.
SPARK and MR are no longer supported
| BlockingQueue<Runnable> senderQueue = new LinkedBlockingQueue<Runnable>(queueCapacity); | ||
| senderExecutor = new ThreadPoolExecutor(1, 1, 0, TimeUnit.MILLISECONDS, senderQueue, threadFactory); | ||
|
|
||
| YarnConfiguration yarnConf = new YarnConfiguration(); |
There was a problem hiding this comment.
Is that only supported for YARN deployments?
There was a problem hiding this comment.
Tez view uses yarn timeline server to pass the query and plan. Based on the comments in ATS, yes it's for YARN deployments.
Ther was a mail thread on this https://www.mail-archive.com/dev@hive.apache.org/msg145250.html as mentioned by @zhangbutao on JIRA, but there was no response. After testing on my setup, i confirmed tez-view uses ATSHook. But query graphical plan and query string is present in DAG as well. So, the dilemma is wheter to keep ATSHook reverted as majorty info is present in DAG or re-enable it just to populate info in Hive Queries tab
CC @abstractdog
There was a problem hiding this comment.
I'm not sure why this hook was dropped, but I think it is useful.
But my question was more about if we could tweak the YARN stuff to be cloud-native.
3 participants
This is revert of HIVE-20078
What changes were proposed in this pull request?
Check the attachements under HIVE-29340
Why are the changes needed?
Does this PR introduce any user-facing change?
Yes, Users will starts seeing rows/fields in tez-view
How was this patch tested?
On local setup