Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] spark historyserver Show incomplete applications kyuubi query engine ui error #6476

Closed
3 of 4 tasks
felixzh2020 opened this issue Jun 18, 2024 · 7 comments
Closed
3 of 4 tasks

[Bug] spark historyserver Show incomplete applications kyuubi query engine ui error #6476

felixzh2020 opened this issue Jun 18, 2024 · 7 comments
Labels
kind:bug This is a clearly a bug priority:major

Comments

@felixzh2020
Copy link
Contributor

felixzh2020 commented Jun 18, 2024 •
edited by pan3793
Loading

Code of Conduct

Search before asking

  • I have searched in the issues and found no similar issues.

Describe the bug

spark historyserver Show incomplete applications kyuubi query engine ui error

<html>
<body>
<!--StartFragment--><h2 style="color: rgb(0, 0, 0); font-family: &quot;Microsoft YaHei&quot;; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">HTTP ERROR 500 java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long</h2>
URI: | /history/application_1698577744226_0074/kyuubi/
-- | --
500
java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long
org.apache.spark.ui.JettyUtils$$anon$1-1fc4c49c
java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long

<h3 style="color: rgb(0, 0, 0); font-family: &quot;Microsoft YaHei&quot;; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Caused by:</h3><pre style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long
	at scala.runtime.BoxesRunTime.unboxToLong(BoxesRunTime.java:107)
	at org.apache.spark.ui.StatementStatsPagedTable.row(EnginePage.scala:542)
	at org.apache.spark.ui.StatementStatsPagedTable.row(EnginePage.scala:452)
	at org.apache.spark.ui.PagedTable.$anonfun$table$1(PagedTable.scala:127)
	at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
	at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
	at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
	at scala.collection.TraversableLike.map(TraversableLike.scala:286)
	at scala.collection.TraversableLike.map$(TraversableLike.scala:279)</pre><!--EndFragment-->
</body>
</html>

Affects Version(s)

master/1.9.1

Kyuubi Server Log Output

No response

Kyuubi Engine Log Output

No response

Kyuubi Server Configurations

No response

Kyuubi Engine Configurations

No response

Additional context

No response

Are you willing to submit PR?

  • Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
  • No. I cannot submit a PR at this time.
@felixzh2020 felixzh2020 added kind:bug This is a clearly a bug priority:major labels Jun 18, 2024
@pan3793
Copy link
Member

pan3793 commented Jun 18, 2024

What's your Spark app/history server version?

felixzh2020 added a commit to felixzh2020/kyuubi that referenced this issue Jun 18, 2024
@felixzh2020
[KYUUBI apache#6476] spark historyserver Show incomplete applications…
a66163a 
… kyuubi query engine ui error
@felixzh2020 felixzh2020 mentioned this issue Jun 18, 2024
Closed
4 tasks
@felixzh2020
Copy link
Contributor Author

3.3.2

@felixzh2020
Copy link
Contributor Author

reproduce this problem：need to exec some sql.
ApplicationMaster -> Kyuubi Query Engine is ok.
Spark historyserver Show incomplete applications -> Kyuubi Query Engine is error.

@wForget
Copy link
Member

wForget commented Jun 19, 2024

Looks like it's related to FasterXML/jackson-module-scala#62

We can add @JsonDeserialize(contentAs = classOf[java.lang.Long]) annotation to operationRunTime/operationCpuTime

@felixzh2020
Copy link
Contributor Author

felixzh2020 commented Jun 19, 2024 •
edited
Loading

@wForget My modification method is effective in our product, but it doesn't look like it's better than yours.
I will try again using your method

felixzh2020 added a commit to felixzh2020/kyuubi that referenced this issue Jun 19, 2024
@felixzh2020
[KYUUBI apache#6476] spark historyserver Show incomplete applications…
b7b0db2 
… kyuubi query engine ui error
@felixzh2020
Copy link
Contributor Author

@wForget It works. Please review.

felixzh2020 added a commit to felixzh2020/kyuubi that referenced this issue Jun 19, 2024
@felixzh2020
[KYUUBI apache#6476] spark historyserver Show incomplete applications…
034bfe5 
… kyuubi query engine ui error
@felixzh2020
Copy link
Contributor Author

@wForget add ok.

pan3793 pushed a commit that referenced this issue Jun 20, 2024
@felixzh2020 @pan3793
[KYUUBI #6476] Fix incomplete app events deserialization in SHS
8d2edc8 
# 🔍 Description
## Issue References 🔗

This pull request fixes #6476 : spark historyserver -> Show incomplete applications -> kyuubi query engine ui error（java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long）.

The reason:
it's related to FasterXML/jackson-module-scala#62

## Describe Your Solution 🔧

add JsonDeserialize(contentAs = classOf[java.lang.Long]) annotation

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6479 from felixzh2020/issues/6476.

Closes #6476

034bfe5 [felixzh] [KYUUBI #6476] spark historyserver Show incomplete applications kyuubi query engine ui error
b7b0db2 [felixzh] [KYUUBI #6476] spark historyserver Show incomplete applications kyuubi query engine ui error
a66163a [felixzh] [KYUUBI #6476] spark historyserver Show incomplete applications kyuubi query engine ui error

Authored-by: felixzh <felixzh2020@126.com>
Signed-off-by: Cheng Pan <chengpan@apache.org>
(cherry picked from commit c6f2ca2)
Signed-off-by: Cheng Pan <chengpan@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind:bug This is a clearly a bug priority:major
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[KYUUBI #6476] Fix incomplete app events deserialization in SHS felixzh2020/kyuubi
3 participants
@felixzh2020 @wForget @pan3793