[SPARK-25488][SQL][TEST] Refactor MiscBenchmark to use main method #22500

wangyum · 2018-09-20T15:44:21Z

What changes were proposed in this pull request?

Refactor MiscBenchmark to use main method.
Generate benchmark result:

SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "sql/test:runMain org.apache.spark.sql.execution.benchmark.MiscBenchmark"

How was this patch tested?

manual tests

wangyum · 2018-09-20T15:47:23Z

sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/MiscBenchmark.scala

+            (i.toString, (i + 1).toString, (i + 2).toString, (i + 3).toString)
+          })))).toDF("col", "arr")
+
+        df.selectExpr("*", "explode(arr) as arr_col")


Function name should be explode.

wangyum · 2018-09-20T15:52:26Z

sql/core/benchmarks/MiscBenchmark-results.txt

+generate stack:                          Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
+------------------------------------------------------------------------------------------------
+generate stack wholestage off               17179 / 17719          1.0        1024.0       1.0X
+generate stack wholestage on                13674 / 14112          1.2         815.0       1.3X


I will do some deep dive, which was 15.5X before.

Performance declines because of this commit: Disable generate codegen since it fails my workload .

@rxin Do we have plan to enable generate codegen?

Although this is not related to this refactoring, ping @rxin and @kiszk because @kiszk seemed to want to fix the root cause of the failure.

@rxin which operation in Generator makes failure of your workload? commit comment.

SparkQA · 2018-09-20T19:37:07Z

Test build #96368 has finished for PR 22500 at commit 6252440.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

# Conflicts: # sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/MiscBenchmark.scala

SparkQA · 2018-10-01T19:26:50Z

Test build #96820 has finished for PR 22500 at commit 58fbdd6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2018-10-02T17:49:35Z

Retest this please.

SparkQA · 2018-10-02T18:03:19Z

Test build #96863 has finished for PR 22500 at commit 58fbdd6.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2018-10-02T23:05:43Z

sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/MiscBenchmark.scala

 */
-class MiscBenchmark extends BenchmarkWithCodegen {
-
-  ignore("filter & aggregate without group") {


This refactoring introduces a long function body at runBenchmarkSuite. In general, it's not a better direction.

Could you map each ignore function into an independent function and make runBenchmarkSuite() invoke a series of those functions?

SparkQA · 2018-10-03T02:41:10Z

Test build #96877 has finished for PR 22500 at commit f7a14e3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2018-10-03T04:20:30Z

Test build #96882 has finished for PR 22500 at commit c791249.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

wangyum · 2018-10-03T05:17:04Z

retest this please

SparkQA · 2018-10-03T07:05:01Z

Test build #96888 has finished for PR 22500 at commit c791249.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

wangyum · 2018-10-03T07:22:12Z

retest this please

SparkQA · 2018-10-03T07:50:41Z

Test build #96889 has finished for PR 22500 at commit c791249.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2018-10-03T11:39:40Z

Retest this please.

SparkQA · 2018-10-03T21:04:22Z

Test build #96903 has finished for PR 22500 at commit c791249.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

wangyum · 2018-10-03T21:40:40Z

Retest this please.

SparkQA · 2018-10-04T01:26:16Z

Test build #96910 has finished for PR 22500 at commit c791249.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

wangyum · 2018-10-05T04:58:29Z

@dongjoon-hyun Is this ready to go?

dongjoon-hyun · 2018-10-06T05:03:36Z

Hi, @wangyum .

Could you review and merge [pull] master from apache:master wangyum/spark#15 ?
Could you add [SQL] to the PR title?

SparkQA · 2018-10-06T07:05:01Z

Test build #97035 has finished for PR 22500 at commit 623998b.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2018-10-06T07:05:01Z

Test build #97032 has finished for PR 22500 at commit 9514aee.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

dilipbiswal · 2018-10-06T07:10:35Z

retest this please

SparkQA · 2018-10-06T10:50:34Z

Test build #97036 has finished for PR 22500 at commit 623998b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun

+1, LGTM. Thank you, @wangyum and @dilipbiswal .

Merged to master.

## What changes were proposed in this pull request? Refactor `MiscBenchmark ` to use main method. Generate benchmark result: ```sh SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "sql/test:runMain org.apache.spark.sql.execution.benchmark.MiscBenchmark" ``` ## How was this patch tested? manual tests Closes apache#22500 from wangyum/SPARK-25488. Lead-authored-by: Yuming Wang <yumwang@ebay.com> Co-authored-by: Yuming Wang <wgyumg@gmail.com> Co-authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>

Refactor MiscBenchmark

6252440

wangyum commented Sep 20, 2018

View reviewed changes

wangyum added 2 commits October 1, 2018 23:16

Merge remote-tracking branch 'upstream/master' into SPARK-25488

8da0113

# Conflicts: # sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/MiscBenchmark.scala

merge master

58fbdd6

wangyum added 2 commits October 3, 2018 06:45

Merge remote-tracking branch 'upstream/master' into SPARK-25488

73dedab

benchmark -> runBenchmarkSuite

f7a14e3

dongjoon-hyun reviewed Oct 2, 2018

View reviewed changes

Address comments

c791249

Update result (#15)

9514aee

wangyum changed the title ~~[SPARK-25488][TEST] Refactor MiscBenchmark to use main method~~ [SPARK-25488][SQL][TEST] Refactor MiscBenchmark to use main method Oct 6, 2018

Fix scala doc

623998b

dongjoon-hyun approved these changes Oct 6, 2018

View reviewed changes

asfgit closed this in edf4286 Oct 6, 2018

[SPARK-25488][SQL][TEST] Refactor MiscBenchmark to use main method #22500

[SPARK-25488][SQL][TEST] Refactor MiscBenchmark to use main method #22500

Uh oh!

Conversation

wangyum commented Sep 20, 2018

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

wangyum Sep 20, 2018

Choose a reason for hiding this comment

Uh oh!

wangyum Sep 20, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wangyum Sep 21, 2018

Choose a reason for hiding this comment

Uh oh!

wangyum Sep 21, 2018

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun Sep 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Sep 20, 2018

Uh oh!

SparkQA commented Oct 1, 2018

Uh oh!

dongjoon-hyun commented Oct 2, 2018

Uh oh!

SparkQA commented Oct 2, 2018

Uh oh!

dongjoon-hyun Oct 2, 2018

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Oct 3, 2018

Uh oh!

SparkQA commented Oct 3, 2018

Uh oh!

wangyum commented Oct 3, 2018

Uh oh!

SparkQA commented Oct 3, 2018

Uh oh!

wangyum commented Oct 3, 2018

Uh oh!

SparkQA commented Oct 3, 2018

Uh oh!

dongjoon-hyun commented Oct 3, 2018

Uh oh!

SparkQA commented Oct 3, 2018

Uh oh!

wangyum commented Oct 3, 2018

Uh oh!

SparkQA commented Oct 4, 2018

Uh oh!

wangyum commented Oct 5, 2018

Uh oh!

dongjoon-hyun commented Oct 6, 2018

Uh oh!

SparkQA commented Oct 6, 2018

Uh oh!

SparkQA commented Oct 6, 2018

Uh oh!

dilipbiswal commented Oct 6, 2018

Uh oh!

SparkQA commented Oct 6, 2018

Uh oh!

dongjoon-hyun left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

wangyum Sep 20, 2018 •

edited

Loading

dongjoon-hyun Sep 26, 2018 •

edited

Loading