Add filter ratio dynamic metric #24
Conversation
spark-ui/src/reducers/SqlReducer.ts
Outdated
| return updatedMetrics; | ||
| } | ||
|
|
||
| const filterRatio = ((totalInputRows - outputRows) / totalInputRows) * 100; |
There was a problem hiding this comment.
use formatPercentage util method
There was a problem hiding this comment.
Also "filter ratio" is the opposite I think, if you filter 1000 from 1M it's 0.1% filtered not 99%. Maybe the naming should be better here like "filtered rows percentage"
There was a problem hiding this comment.
I didn't find formatPercentage util I can use in typescript but found NumberFormat, hope it is what you intended :)
| @@ -399,6 +400,7 @@ export function updateSqlNodeMetrics( | |||
|
|
|||
| const notEffectedSqls = currentStore.sqls.filter((sql) => sql.id !== sqlId); | |||
| const runningSql = runningSqls[0]; | |||
| const graph = generateGraph(runningSql.edges, runningSql.nodes); | |||
There was a problem hiding this comment.
This means we are generating the graph for every metric update cycle. This is not ideal.
We should cache the graph if there is no change.
Please at least add a todo comment to cache the graph.
There was a problem hiding this comment.
Yes, I thought about it, but my bad to forget the todo.
I think it should work with map with sqlId key easily.
But detecting actual changes is something different. We can do a check on node metrics before and after or compare their hashes. What do you think?
were replaced with updateNodeMetrics function.