selenium test spark, pyspark and sparkSql

[prabhjyotsingh]

What is this PR for?

Test functionality of spark, pyspark, sparksql

What type of PR is it?

Improvement

Todos

• - Selenium for spark
• - Selenium for pyspark
• - Selenium for sparksql
• - refactor with ZEPPELIN-593 Test Paragraph action Disable run and refactor common methods #619

Is there a relevant Jira issue?

ZEPPELIN-587

How should this be tested?

On macOS

PATH=~/Applications/Firefox.app/Contents/MacOS/:$PATH CI="" \
mvn -Dtest=org.apache.zeppelin.integration.TestSparkParagraph -Denforcer.skip=true \
test -pl zeppelin-server

[prabhjyotsingh]

Ready for review

[bzz]

Looks great, but how do you think, is it possible to improve this part (and alike below) a bit, to something more readable and easier to maintain?

May be extracting variables for Keys.chord(Keys.SHIFT, "9") and giving them a short but descriptive names like enter, or eol, etc could help

[bzz]

@prabhjyotsingh thanks for taking care of reivew in timely manner!

It looks great, how do you think can it be further improved by import static HelperKeys so we get rid of duplicating HelperKeys and just keep constants?

[prabhjyotsingh]

@bzz thank you for the quick review, did the relevant change.

[bzz]

Very nice useage of Enum here!

Probably a knit-piking but may be also SHIFT_ENTER diserves a place there too?

Looks great to me.

I'm just curious, may be you know, why open parentesys is a key, but closing one seems to be ok inside the string literal?

👍 for more tests that help improving stability!

[prabhjyotsingh]

@bzz yes thank you. Did it for SHIFT_ENTER as well.

Some javascript associated with ace editor is causing problem with open parenthesis, and other that are defined in HelperKeys, and hence have to use this (OPEN_PARENTHESIS).

[prabhjyotsingh]

Ready for review.

[felixcheung]

Great!
If any of these fails is there a simple way to know why? I think recently there was one minor UI change and it was somewhat ambiguous when Travis fails. What would you recommend? Should UI test be a separate "build" job in Travis?

[prabhjyotsingh]

Sure @felixcheung, this sounds better, have create a new PR #706 for addressing this.

[prabhjyotsingh]

Merged with #706, and #709. CI green.
Ready for review.

[felixcheung]

would it be possible in the check and LOG.error to specify this is from the sql spark test?

[prabhjyotsingh]

Thanks @felixcheung for the review. Made appropriate changes.

[felixcheung]

I assume this is not running the tests, but only skipping?
(spark-1.6 build)
https://s3.amazonaws.com/archive.travis-ci.org/jobs/109789223/log.txt

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.zeppelin.integration.SparkParagraphIT
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/travis/build/apache/incubator-zeppelin/zeppelin-zengine/target/lib/zeppelin-interpreter-0.6.0-incubating-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/travis/build/apache/incubator-zeppelin/zeppelin-zengine/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/travis/build/apache/incubator-zeppelin/zeppelin-interpreter/target/zeppelin-interpreter-0.6.0-incubating-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/travis/build/apache/incubator-zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/travis/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.361 sec - in org.apache.zeppelin.integration.SparkParagraphIT
Running org.apache.zeppelin.integration.ZeppelinIT
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in org.apache.zeppelin.integration.ZeppelinIT
Running org.apache.zeppelin.integration.ParagraphActionsIT
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 sec - in org.apache.zeppelin.integration.ParagraphActionsIT

Results :

Tests run: 7, Failures: 0, Errors: 0, Skipped: 0

[prabhjyotsingh]

Yes skipping since the env "TEST_SELENIUM" is not set.
Otherwise things like these would have shown up

06:06:10,610  INFO org.apache.zeppelin.AbstractZeppelinIT:47 - Starting sleeping for 1 seconds...
06:06:10,611  INFO org.apache.zeppelin.AbstractZeppelinIT:48 - Caller:      org.apache.zeppelin.AbstractZeppelinIT.deleteTestNotebook(AbstractZeppelinIT.java:130)
06:06:11,611  INFO org.apache.zeppelin.AbstractZeppelinIT:56 - Finished.
06:06:11,879  INFO org.apache.zeppelin.AbstractZeppelinIT:47 - Starting sleeping for 0 seconds...
06:06:11,880  INFO org.apache.zeppelin.AbstractZeppelinIT:48 - Caller: org.apache.zeppelin.AbstractZeppelinIT.deleteTestNotebook(AbstractZeppelinIT.java:133)
06:06:11,980  INFO org.apache.zeppelin.AbstractZeppelinIT:56 - Finished.

[felixcheung]

cool LGTM. merging if no more comment