[SPARK-24215][PySpark] Implement eager evaluation for DataFrame APIs in PySpark #21370

xuanyuanking · 2018-05-19T12:03:35Z

What changes were proposed in this pull request?

This PR is to add eager execution into the __repr__ and _repr_html_ of the DataFrame APIs in PySpark. When eager evaluation is enabled, _repr_html_ returns a rich HTML version of the top-K rows of the DataFrame output. If _repr_html_ is not called by REPL, _repr_ will return the plain text of the top-K rows.

This PR adds three new external SQL confs for controlling the behavior of eager evaluation:

spark.sql.repl.eagerEval.enabled: Enables eager evaluation or not. When true, the top K rows of Dataset will be displayed if and only if the REPL supports the eager evaluation. Currently, the eager evaluation is only supported in PySpark. For the notebooks like Jupyter, the HTML table (generated by _repr_html_) will be returned. For plain Python REPL, the returned outputs are formatted like dataframe.show().
spark.sql.repl.eagerEval.maxNumRows: The max number of rows that are returned by eager evaluation. This only takes effect when spark.sql.repl.eagerEval.enabled is set to true.
spark.sql.repl.eagerEval.truncate: The max number of characters for each cell that is returned by eager evaluation. This only takes effect when spark.sql.repl.eagerEval.enabled is set to true.

How was this patch tested?

New ut in DataFrameSuite and manual test in jupyter. Some screenshot below.

After:

Before:

xuanyuanking · 2018-05-19T12:07:16Z

Not sure who is the right reviewer, maybe @rdblue @gatorsmile ?
Could you help me check whether it is the right implementation for the discussion in the dev list?

SparkQA · 2018-05-19T12:13:36Z

Test build #90834 has finished for PR 21370 at commit ebc0b11.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

felixcheung · 2018-05-19T23:45:03Z

@HyukjinKwon @holdenk @ueshin

felixcheung · 2018-05-19T23:45:50Z

we will wait for the tests to be fixed first.

@xuanyuanking could you update the PR description to clarify which screenshot is "before" which is "after"?

felixcheung

this will need to escape the values to make sure it is legal html too right?

felixcheung · 2018-05-19T23:47:24Z

sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala

why all these comment changes? could you revert them

Sorry for this, I'll revert the IDE changes.

felixcheung · 2018-05-19T23:49:14Z

python/pyspark/sql/dataframe.py

let's add all these to documentation too

Got it. Do it in next commit. Thanks for reminding.

felixcheung · 2018-05-19T23:50:43Z

sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala

when the output is truncated, does jupyter handle that properly?

or does that depend on the html output on L300?

Yes, the truncated string will be showed in table row and controlled by spark.jupyter.default.truncate

HyukjinKwon · 2018-05-21T02:14:51Z

retest this please

HyukjinKwon · 2018-05-21T02:21:37Z

docs/configuration.md

nit: Open -> Enable

HyukjinKwon · 2018-05-21T02:21:42Z

docs/configuration.md

xuanyuanking · 2018-05-21T02:22:10Z

this will need to escape the values to make sure it is legal html too right?

Yes you're right, thanks for your guidance, the new patch consider the escape and add new UT. @felixcheung

ueshin · 2018-05-21T04:31:19Z

sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala

We don't need .map(_.toString)?

We need this toString, because the appendRowString method including both column names and data in original logic, which call toString in data part.
https://github.com/apache/spark/pull/21370/files/f2bb8f334631734869ddf5d8ef1eca1fa29d334a#diff-7a46f10c3cedbf013cf255564d9483cdL312

I know this is not your change, but the rows is already Seq[Seq[String]] and the row is Seq[String], so I think we can remove it.

I see, the cell.toString has been called here. https://github.com/apache/spark/pull/21370/files/f2bb8f334631734869ddf5d8ef1eca1fa29d334a#diff-7a46f10c3cedbf013cf255564d9483cdR271
Got it, I'll fix this in next commit.

ueshin · 2018-05-21T04:34:16Z

sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala

nit: add \n?

the "\n" added in seperatedLine:https://github.com/apache/spark/pull/21370/files#diff-7a46f10c3cedbf013cf255564d9483cdR300

Hmm, the header looks okay, but the data section will be a long line without \n? How about adding \n here and the data section, and just using "" for the seperatedLine?

Ah, I understand your consideration. I'll add this in next commit.

Thanks, done. feb5f4a.

ueshin · 2018-05-21T04:34:23Z

sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala

ueshin · 2018-05-21T04:39:32Z

docs/configuration.md

spark.jupyter.eagerEval.showRows or something?

change to spark.jupyter.eagerEval.showRows,thanks

ueshin · 2018-05-21T04:39:52Z

docs/configuration.md

spark.jupyter.eagerEval.truncate or something?

Yep, change to spark.jupyter.eagerEval.truncate

ueshin · 2018-05-21T04:42:43Z

python/pyspark/sql/dataframe.py

What will be shown if spark.jupyter.eagerEval.enabled is False? Fallback the original automatically?

We need to return None if self._eager_eval is False.

What will be shown if spark.jupyter.eagerEval.enabled is False? Fallback the original automatically?

Yes, it will fallback to call __repr__.

We need to return None if self._eager_eval is False.

Got it, more clear in code logic.

ueshin · 2018-05-21T05:05:16Z

python/pyspark/sql/dataframe.py

spark.jupyter.default.truncate?

My bad, sorry for this.

ueshin · 2018-05-21T05:05:30Z

python/pyspark/sql/dataframe.py

I guess we shouldn't hold these three values but define as @property or refer each time in _repr_html_. Otherwise, we'll hit unexpected behavior, e.g.:

df = ... spark.conf.set("spark.jupyter.eagerEval.enabled", True) df

won't show the html.

Yep, I'll fix it in next commit.

ueshin · 2018-05-21T05:12:48Z

docs/configuration.md

true instead of yes?

true is better since the default value is false.

Got it, thanks.

SparkQA · 2018-05-21T05:27:36Z

Test build #90872 has finished for PR 21370 at commit f2bb8f3.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2018-05-21T05:56:43Z

Test build #90871 has finished for PR 21370 at commit ebc0b11.

This patch fails SparkR unit tests.
This patch merges cleanly.
This patch adds no public classes.

holdenk · 2018-05-21T06:48:08Z

So one thing we might want to take a look at is application/vnd.dataresource+json for tables in the notebooks (see https://github.com/nteract/improved-spark-viz ).

viirya · 2018-05-21T07:05:18Z

docs/configuration.md

nit: Open -> Enable

viirya · 2018-05-21T07:06:08Z

docs/configuration.md

Is it usually we include a JIRA ticket in user document like this? I think we should just briefly describe this feature here.

Yea, I felt the same thing too but there were the same few instances in this page.

viirya · 2018-05-21T07:08:05Z

docs/configuration.md

true is better since the default value is false.

viirya · 2018-05-21T07:12:47Z

sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala

hmm, should we do this html thing in python side?

We can do this in python side, I implement it in scala side mainly consider to reuse the code and logic of show(), maybe it's more natural in show df as html call showString.

This should not be done in Dataset.scala.

Can we create refactor showString to make it reusable for both show and repr_html?

+1 for reusing relevant parts of show.

Thanks for guidance, I will do this in next commit.

@viirya @gatorsmile @rdblue Sorry for the late commit, the refactor do in 94f3414. I spend some time on testing and implementing the transformation of rows between python and scala.

xuanyuanking · 2018-05-21T13:04:20Z

Thanks all reviewer's comments, I address all comments in this commit. Please have a look.

viirya · 2018-05-21T13:31:28Z

python/pyspark/sql/dataframe.py

Add a simple test for _repr_html too?

No problem, I'll added in SQLTests in next commit.

Thanks, done. feb5f4a.

viirya · 2018-05-21T13:38:12Z

docs/configuration.md

Should we add sql? E.g., spark.sql.jupyter.eagerEval.enabled? Because this is just for SQL Dataset. @HyukjinKwon @ueshin

Oh, yes, I'd prefer to add sql.

Got it, fix it in next commit.

Is this Jupyter specific? Can we make it more general?

btw the config flag isn't jupyter specific.

Maybe rename to spark.sql.repl.eagerEval.enabled instead? It should also be documented that this is a hint. There's no guarantee that the repl you're using supports eager evaluation.

Thanks, done. feb5f4a.

SparkQA · 2018-05-21T16:59:35Z

Test build #90896 has finished for PR 21370 at commit a798cf2.

This patch fails SparkR unit tests.
This patch merges cleanly.
This patch adds no public classes.

gatorsmile · 2018-05-21T17:43:55Z

python/pyspark/sql/dataframe.py

Add the function descriptions above this line,

Thanks, done. feb5f4a.

rxin · 2018-05-21T18:08:34Z

Can we also do something a bit more generic that works for non-Jupyter notebooks as well? For example, in IPython or just plain Python REPL. This is in addition to the repl_html stuff which is really cool and useful.

rdblue · 2018-05-21T22:35:28Z

@rxin, __repr__ is the equivalent for ipython and the python REPL. _repr_html_ is the convention used by jupyter to replicate __repr__ in notebooks with HTML output.

ueshin · 2018-05-23T09:00:53Z

sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala

I'm sorry, but I don't think we need the middle \n, it's okay with only the last one.
"<tr><th>", "</th><th>", "</th></tr>\n"

Got it, I'll change it.

I change the format in python _repr_html_ in 94f3414.

ueshin · 2018-05-23T09:00:59Z

sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala

SparkQA · 2018-05-23T10:46:06Z

Test build #91022 has finished for PR 21370 at commit feb5f4a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

xuanyuanking · 2018-05-23T13:39:16Z

Can we also do something a bit more generic that works for non-Jupyter notebooks as well?

Can we accept spark.sql.repl.eagerEval.enabled to control both __repr__ and _repr_html_ ?
The behavior like below:

If not support repr_html and open eagerEval.enable, just call something like show and trigger take inside.
If support repr_html, use the html output. (Here need a small trick, we should add a var in python dataframe to check whether repr_html called or not, otherwise in this mode _repr_html and repr will both call showString).

I test offline an it can work both python shell and Jupyter, if we agree this way, I'll add this support in next commit together will the refactor of showString in scala Dataset.

viirya · 2018-05-23T13:45:14Z

python/pyspark/sql/dataframe.py

repr -> repl?

I think it works either way. REPL is better in my opinion because these settings should (ideally) apply when using any REPL.

Thanks, change to REPL in 94f3414.

HyukjinKwon · 2018-05-24T07:13:50Z

python/pyspark/sql/tests.py

We don't have to compile the pattern each time here since it's not going to be reused. You could just put this into re.sub I believe.

Thanks! Fix it in 94f3414.

rdblue · 2018-05-24T18:15:16Z

docs/configuration.md

I don't think it is obvious what "showRows" does. I would assume that it is a boolean, but it is a limit instead. What about calling this "limit" or "numRows"?

Thanks, change it in 94f3414.

rdblue · 2018-05-24T18:15:52Z

docs/configuration.md

What is the difference between this and showRows? Why are there two properties?

maybe he wants to follow what dataframe.show does, which truncates num characters within a cell. That's useful for console output, but not so much for notebooks.

Yep, I just want to keep the same behavior of dataframe.show.

That's useful for console output, but not so much for notebooks.

Notebooks aren't afraid for too many characters within a cell, so I just delete this?

rdblue · 2018-05-24T18:18:34Z

python/pyspark/sql/dataframe.py

I agree that it would be better to respect spark.sql.repr.eagerEval.enabled here as well.

Thanks for your reply, this implement in 94f3414.

rxin · 2018-05-25T06:49:02Z

python/pyspark/sql/dataframe.py

use named arguments for boolean flags

Thanks, fix in 94f3414.

SparkQA · 2018-06-04T18:52:01Z

Test build #91449 has finished for PR 21370 at commit 597b8d5.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

HyukjinKwon · 2018-06-05T01:22:54Z

Merged to master

xuanyuanking · 2018-06-05T05:20:51Z

Thanks @HyukjinKwon and all reviewers.

gatorsmile · 2018-06-11T00:22:58Z

docs/configuration.md

+  <td>20</td>
+  <td>
+    Default number of rows in eager evaluation output HTML table generated by <code>_repr_html_</code> or plain text,
+    this only take effect when <code>spark.sql.repl.eagerEval.enabled</code> is set to true.


take -> takes

Got it, thanks.

gatorsmile · 2018-06-11T00:23:11Z

docs/configuration.md

+  <td>20</td>
+  <td>
+    Default number of truncate in eager evaluation output HTML table generated by <code>_repr_html_</code> or
+    plain text, this only take effect when <code>spark.sql.repl.eagerEval.enabled</code> set to true.


take -> takes

Got it, thanks, address in next follow up PR.

gatorsmile · 2018-06-11T00:32:13Z

@xuanyuanking @HyukjinKwon Sorry for the delay. Super busy in the week of Spark summit. Will carefully review this PR today or tomorrow.

gatorsmile · 2018-06-11T00:54:34Z

python/pyspark/sql/tests.py

        finally:
            shutil.rmtree(path)

+    def test_repr_html(self):


This function only covers the most basic positive case. We need also add more test cases. For example, the results when spark.sql.repl.eagerEval.enabled is set to false.

Got it, the follow up pr will enhance this test.

gatorsmile · 2018-06-11T00:57:56Z

sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala

+      truncate: Int,
+      vertical: Boolean): Array[Any] = {
+    EvaluatePython.registerPicklers()
+    val numRows = _numRows.max(0).min(Int.MaxValue - 1)


This should be also part of the conf description.

Got it, will be fixed in another pr.

gatorsmile · 2018-06-11T00:58:46Z

python/pyspark/sql/dataframe.py

+            self._support_repr_html = True
+        if self._eager_eval:
+            max_num_rows = max(self._max_num_rows, 0)
+            vertical = False


Any discussion about this?

Yes, the discussion before linked below:
#21370 (comment)
and
#21370 (comment)
We need a named arguments for boolean flags here, but here limited by the named arguments can't work during python call _jdf func, so we do the work around like this.

gatorsmile · 2018-06-11T01:04:06Z

docs/configuration.md

    from JVM to Python worker for every task.
  </td>
 </tr>
+<tr>


All the SQL configurations should follow what we did in the section of Spark SQL https://spark.apache.org/docs/latest/configuration.html#spark-sql.

These confs are not part of spark.sql("SET -v").show(numRows = 200, truncate = false).

Got it, I'll add there configurations into SQLConf.scala in the follow up pr.

gatorsmile · 2018-06-11T01:08:17Z

python/pyspark/sql/dataframe.py

+        """Returns true if the eager evaluation enabled.
+        """
+        return self.sql_ctx.getConf(
+            "spark.sql.repl.eagerEval.enabled", "false").lower() == "true"


Is that possible we can avoid hard-coding these conf key values? cc @ueshin @HyukjinKwon

Probably, we should access to SQLConf object. 1. Agree with not hardcoding it in general but 2. IMHO I want to avoid Py4J JVM accesses in the test because the test can likely be more flaky up to my knowledge, on the other hand (unlike Scala or Java side).

Maybe we should try to take a look about this hardcoding if we see more occurrences next time

In the ongoing release, a nice-to-have refactoring is to move all the Core Confs into a single file just like what we did in Spark SQL Conf. Default values, boundary checking, types and descriptions. Thus, in PySpark, it would be better to do it starting from now.

gatorsmile · 2018-06-11T01:16:01Z

sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala

    }
  }

+  private[sql] def getRowsToPython(


In DataFrameSuite, we have multiple test cases for showString instead of getRows , which is introduced in this PR.

We also need the unit test cases for getRowsToPython.

Got it, the follow up pr I'm working on will add more test for getRows and getRowsToPython.

gatorsmile · 2018-06-11T01:21:57Z

@xuanyuanking Thanks for your contributions! Test coverage is the most critical when we refactor the existing code and add new features. Hopefully, when you submit new PRs in the future, could you also improve this part?

gatorsmile · 2018-06-11T01:24:54Z

docs/configuration.md

+  <td><code>spark.sql.repl.eagerEval.enabled</code></td>
+  <td>false</td>
+  <td>
+    Enable eager evaluation or not. If true and the REPL you are using supports eager evaluation,


Just a question. When the REPL does not support eager evaluation, could we do anything better instead of silently ignoring the user inputs?

Maybe it's hard to have a better way because we can hardly perceive it in dataset while REPL does not support eager evaluation.

I don't completely follow actually - what makes a "REPL does not support eager evaluation"?
in fact, this "eager evaluation" is just object rendering support build into REPL, notebook etc (like print), it's not really a design to be "eager evaluation"

That is true. We are not adding the eager execution in the our Dataset/DataFrame. We just rely on the REPL to trigger it. We need an update on the parameter description to emphasize it.

gatorsmile · 2018-06-11T04:51:14Z

python/pyspark/sql/dataframe.py

+        return int(self.sql_ctx.getConf(
+            "spark.sql.repl.eagerEval.truncate", "20"))
+
    def __repr__(self):


This PR also changed __repr__. Thus, we need to update the PR title and description. A better PR title should be like Implement eager evaluation for DataFrame APIs in PySpark

Done for this and also the PR description as your comment, sorry for this and I'll pay more attention to that next time when the implementation changes during PR review. Is there any way to change the committed git log? Sorry for this.

xuanyuanking · 2018-06-12T03:08:09Z

Test coverage is the most critical when we refactor the existing code and add new features. Hopefully, when you submit new PRs in the future, could you also improve this part?

Of cause and sorry for the late reply, I'll do this in a follow up PR and answer all question from Xiao this night. Thanks for all your comments.

gatorsmile · 2018-06-12T03:22:37Z

@xuanyuanking Just for your reference, for this PR, the PR description can be improved to something like

This PR is to add eager execution into the __repr__ and _repr_html_ of the DataFrame APIs in PySpark. When eager evaluation is enabled, repr_html returns a rich HTML version of the top-K rows of the DataFrame output. If _repr_html_ is not called by REPL, _repr_ will return the plain text of the top-K rows.

This PR adds three new external SQL confs for controlling the behavior of eager evaluation:

spark.sql.repl.eagerEval.enabled: Enables eager evaluation or not. When true, the top K rows of Dataset will be displayed if and only if the REPL supports the eager evaluation. Currently, the eager evaluation is only supported in PySpark. For the notebooks like Jupyter, the HTML table (generated by repr_html) will be returned. For plain Python REPL, the returned outputs are formatted like dataframe.show().

spark.sql.repl.eagerEval.maxNumRows: The max number of rows that are returned by eager evaluation. This only takes effect when spark.sql.repl.eagerEval.enabled is set to true.

spark.sql.repl.eagerEval.truncate: The max number of characters of each row that is returned by eager evaluation. This only takes effect when spark.sql.repl.eagerEval.enabled is set to true.

gatorsmile · 2018-06-12T06:47:41Z

sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala

+    val toJava: (Any) => Any = EvaluatePython.toJava(_, ArrayType(ArrayType(StringType)))
+    val iter: Iterator[Array[Byte]] = new SerDeUtil.AutoBatchedPickler(
+      rows.iterator.map(toJava))
+    PythonRDD.serveIterator(iter, "serve-GetRows")


PythonRDD.serveIterator(iter, "serve-GetRows") returns Int, but the return type of getRowsToPython is Array[Any]. How does it work? cc @xuanyuanking @HyukjinKwon

I think we return Array[Any] for PythonRDD.serveIterator too.

spark/core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala

Line 400 in 628c7b5

def serveIterator(items: Iterator[_], threadName: String): Array[Any] = {

Did I maybe miss something?

Same answer with @HyukjinKwon about the return type, and actually the exact return type we need here is Array[Array[String]], this defined in toJava func.

The changes of the return types were made in a commit [PYSPARK] Update py4j to version 0.10.7.... No PR was opened...

re the py4j commit - there's a good reason for it @gatorsmile
not sure if the change to return type is required with the py4j change though

I think we better don't talk about this here though.

felixcheung reviewed May 19, 2018

View reviewed changes

HyukjinKwon reviewed May 21, 2018

View reviewed changes

ueshin reviewed May 21, 2018

View reviewed changes

viirya reviewed May 21, 2018

View reviewed changes

gatorsmile reviewed May 21, 2018

View reviewed changes

ueshin reviewed May 23, 2018

View reviewed changes

viirya reviewed May 23, 2018

View reviewed changes

HyukjinKwon reviewed May 24, 2018

View reviewed changes

rdblue reviewed May 24, 2018

View reviewed changes

rxin reviewed May 25, 2018

View reviewed changes

xuanyuanking added 5 commits June 4, 2018 07:14

address comments

c933c56

address comments

68fbceb

address comments

6fe1afd

address comments

b9ac316

bug fix

597b8d5

xuanyuanking force-pushed the SPARK-24215 branch from 5b36604 to 597b8d5 Compare June 4, 2018 14:58

asfgit closed this in dbb4d83 Jun 5, 2018

gatorsmile reviewed Jun 11, 2018

View reviewed changes

gatorsmile reviewed Jun 12, 2018

View reviewed changes

xuanyuanking changed the title ~~[SPARK-24215][PySpark] Implement _repr_html_ for dataframes in PySpark~~ [SPARK-24215][PySpark] Implement eager evaluation for DataFrame APIs in PySpark Jun 12, 2018

xuanyuanking mentioned this pull request Jun 13, 2018

[SPARK-24215][PySpark][Follow Up] Implement eager evaluation for DataFrame APIs in PySpark #21553

Closed

xuanyuanking mentioned this pull request Jun 27, 2018

[SPARK-24665][PySpark] Use SQLConf in PySpark to manage all sql configs #21648

Closed

[SPARK-24215][PySpark] Implement eager evaluation for DataFrame APIs in PySpark #21370

[SPARK-24215][PySpark] Implement eager evaluation for DataFrame APIs in PySpark #21370

Uh oh!

Conversation

xuanyuanking commented May 19, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

xuanyuanking commented May 19, 2018

Uh oh!

SparkQA commented May 19, 2018

Uh oh!

felixcheung commented May 19, 2018

Uh oh!

felixcheung commented May 19, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

felixcheung left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HyukjinKwon commented May 21, 2018

Uh oh!

HyukjinKwon May 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xuanyuanking commented May 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xuanyuanking May 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xuanyuanking commented May 19, 2018 •

edited

Loading

felixcheung commented May 19, 2018 •

edited

Loading

HyukjinKwon May 21, 2018 •

edited

Loading

xuanyuanking commented May 21, 2018 •

edited

Loading

xuanyuanking May 21, 2018 •

edited

Loading

xuanyuanking May 21, 2018 •

edited

Loading

ueshin May 21, 2018 •

edited

Loading