Skip to content

Conversation

@MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Sep 28, 2019

What changes were proposed in this pull request?

In the PR, I propose to clone Spark session per-each expression example. Examples can modify SQL settings, and can influence on each other if they run in the same Spark session in parallel.

Why are the changes needed?

This should fix test failures like this checking of the Like example:

org.apache.spark.sql.AnalysisException: the pattern '\%SystemDrive\%\Users%' is invalid, the escape character is not allowed to precede 'U';
      at org.apache.spark.sql.catalyst.util.StringUtils$.fail$1(StringUtils.scala:48)
      at org.apache.spark.sql.catalyst.util.StringUtils$.escapeLikeRegex(StringUtils.scala:57)
      at org.apache.spark.sql.catalyst.expressions.Like.escape(regexpExpressions.scala:108)

Does this PR introduce any user-facing change?

No

How was this patch tested?

By running check outputs of expression examples in org.apache.spark.sql.SQLQuerySuite

@MaxGekk MaxGekk changed the title [WIP][SPARK-21914][FOLLOWUP] Clone SparkSession per each function example [WIP][SPARK-21914][FOLLOWUP][test-hadoop3.2][test-java11] Clone SparkSession per each function example Sep 28, 2019
@SparkQA
Copy link

SparkQA commented Sep 28, 2019

Test build #111521 has finished for PR 25956 at commit 75fe88d.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 28, 2019

Test build #111522 has finished for PR 25956 at commit 75fe88d.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 28, 2019

Test build #111523 has finished for PR 25956 at commit 75fe88d.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@MaxGekk
Copy link
Member Author

MaxGekk commented Sep 28, 2019

jenkins, retest this, please

@MaxGekk MaxGekk changed the title [WIP][SPARK-21914][FOLLOWUP][test-hadoop3.2][test-java11] Clone SparkSession per each function example [SPARK-21914][FOLLOWUP][test-hadoop3.2][test-java11] Clone SparkSession per each function example Sep 28, 2019
@SparkQA
Copy link

SparkQA commented Sep 28, 2019

Test build #111526 has finished for PR 25956 at commit 75fe88d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@MaxGekk
Copy link
Member Author

MaxGekk commented Sep 28, 2019

jenkins, retest this, please

@SparkQA
Copy link

SparkQA commented Sep 28, 2019

Test build #111528 has finished for PR 25956 at commit 75fe88d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@MaxGekk
Copy link
Member Author

MaxGekk commented Sep 28, 2019

jenkins, retest this, please

@MaxGekk
Copy link
Member Author

MaxGekk commented Sep 28, 2019

@HyukjinKwon @wangyum @dongjoon-hyun Take a look at this PR, please.

@SparkQA
Copy link

SparkQA commented Sep 28, 2019

Test build #111530 has finished for PR 25956 at commit 75fe88d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 28, 2019

Test build #111533 has finished for PR 25956 at commit 75fe88d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member

Merged to master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants