Create seperate matrix for selenium test case #706

prabhjyotsingh · 2016-02-09T09:07:33Z

What is this PR for?

A lot of time travis CI breaks because of selenium test case, and at that point it becomes trivial to know the exact reason. Hence, in this PR creating a separate matrix for testing selenium test cases.

What type of PR is it?

Improvement

Is there a relevant Jira issue?

N/A

How should this be tested?

Check travis UI. It should have a new matrix added.

prabhjyotsingh · 2016-02-09T17:41:21Z

Build #3100.4 is failing (https://travis-ci.org/apache/incubator-zeppelin/jobs/108074063) for

Results :

Tests in error: 
  CassandraInterpreterTest.org.apache.zeppelin.cassandra.CassandraInterpreterTest » ExceptionInInitializer
  CassandraInterpreterTest.org.apache.zeppelin.cassandra.CassandraInterpreterTest » NoClassDefFound

Which is unrelated.

felixcheung · 2016-02-10T03:20:54Z

@doanduyhai any idea with test failure?

DEBUG [17:21:25,783][] ACHILLES_DDL_SCRIPT@:executeScript       SCRIPT : CREATE INDEX IF NOT EXISTS idx1 ON live_data.complex_table(indexed1); 
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 36.451 sec <<< FAILURE! - in org.apache.zeppelin.cassandra.CassandraInterpreterTest
org.apache.zeppelin.cassandra.CassandraInterpreterTest  Time elapsed: 36.451 sec  <<< ERROR!
java.lang.ExceptionInInitializerError: null
    at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
    at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:37)
    at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
    at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:244)
    at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:55)
    at info.archinnov.achilles.script.ScriptExecutor.executeScript(ScriptExecutor.java:61)
    at info.archinnov.achilles.embedded.AchillesInitializer.executeStartupScripts(AchillesInitializer.java:204)
    at info.archinnov.achilles.embedded.AchillesInitializer.initialize(AchillesInitializer.java:98)
    at info.archinnov.achilles.embedded.AchillesInitializer.initializeFromParameters(AchillesInitializer.java:61)
    at info.archinnov.achilles.embedded.CassandraEmbeddedServer.<init>(CassandraEmbeddedServer.java:76)
    at info.archinnov.achilles.embedded.CassandraEmbeddedServerBuilder.buildNativeSessionOnly(CassandraEmbeddedServerBuilder.java:436)
    at org.apache.zeppelin.cassandra.CassandraInterpreterTest.<clinit>(CassandraInterpreterTest.java:59)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9259 (com.datastax.driver.core.exceptions.OperationTimedOutException: [localhost/127.0.0.1] Timed out waiting for server response))

felixcheung · 2016-02-10T03:23:25Z

zeppelin-server/src/test/java/org/apache/zeppelin/ZeppelinIT.java

-    testDepRemoveBtn.click();
-    driver.findElement(By.xpath("//button[contains(.,'Save')]")).submit();
-    driver.switchTo().alert().accept();
+    if (!endToEndTestEnabled()) {


it looks like this is from PR #654 - could we separate that out for review please?

Yes, its from #673, https://github.com/apache/incubator-zeppelin/pull/673/files#diff-fc9c6a01ec538e646b3e99c8fa19a3deR204

Sure, I've created a new PR #709 as a hot fix.
Will revert this whole file once that is merged. As we will need if (!endToEndTestEnabled()) { or else driver @Before will not initialize driver, resulting in this test case to fail.

felixcheung · 2016-02-10T08:13:22Z

Could you check out #708 ? We should avoid rerunning all tests for TEST_SELENIUM

prabhjyotsingh · 2016-02-11T03:52:53Z

Closing this, as this is been taken care by #708

felixcheung · 2016-02-11T04:20:49Z

actually, I think they are addressing different parts of it. It might still make sense to have a separate matrix for just UI tests - to clarify, I was saying you might want to restrict the project list and test scope to avoid running duplicates. Reopen?

### What is this PR for? While investigating CI failure with PR #706, I noticed that we were running all tests in each of 6 jobs in the test matrix. It seems we have accidentally switched back to running all tests even with older versions of Spark. (from PR #626 Filtering out interpreter tests for these runs help make test jobs faster (~30%) and also reduce the chance that an interpreter test might intermittently fail, thus hopefully improve CI reliability. Also changing Spark 1.2.1 to 1.2.2 (last 1.2.x release) ### What type of PR is it? Improvement ### Todos * [x] - Travis CI yml changes ### Is there a relevant Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-665 ### How should this be tested? Run Travis CI ### Screenshots (if appropriate) N/A ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Felix Cheung <felixcheung_m@hotmail.com> Closes #708 from felixcheung/testprofile and squashes the following commits: ac3bd72 [Felix Cheung] fix project list b0c8b9b [Felix Cheung] add spark tests 27d57fe [Felix Cheung] fix project list cd0cb97 [Felix Cheung] test project list for older spark versions

felixcheung · 2016-02-12T09:15:51Z

So as you can see, it's re-running all interpreter tests when TEST_SELENIUM="true"
https://s3.amazonaws.com/archive.travis-ci.org/jobs/108461599/log.txt

could you add the -pl and -Dtest as in 65b47e5?

doanduyhai · 2016-02-12T14:55:27Z

@felixcheung

The error about the CassandraInterpreterTest is because for unit test, I start an embedded Cassandra server and sends CQL queries to test the interpreter.

The exception stack just says that the interpreter cannot connect to the server. It means that the embedded server failed to start for some reason. Can you give me the full stack trace or give me a link to the logs ?

felixcheung · 2016-02-13T03:42:05Z

@doanduyhai job log is here https://travis-ci.org/apache/incubator-zeppelin/jobs/108074063

doanduyhai · 2016-02-13T15:04:07Z

Thanks @felixcheung

So I had a look into the log file, below is some extract:

DEBUG [17:21:08,728][] ACHILLES_DDL_SCRIPT@:executeScript       SCRIPT : CREATE KEYSPACE IF NOT EXISTS zeppelin WITH REPLICATION = {'class':'SimpleStrategy', 'replication_factor':1} AND DURABLE_WRITES=false; 
DEBUG [17:21:08,751][] ACHILLES_DDL_SCRIPT@:executeScript       SCRIPT : CREATE TABLE IF NOT EXISTS zeppelin.artists( name text PRIMARY KEY, born text, died text, country text, gender text, type text, styles list<text> ); 
DEBUG [17:21:10,068][] ACHILLES_DDL_SCRIPT@:executeScript       SCRIPT : CREATE TABLE IF NOT EXISTS zeppelin.ts( key text PRIMARY KEY, val text ); 
DEBUG [17:21:11,292][] ACHILLES_DDL_SCRIPT@:executeScript       SCRIPT : CREATE TABLE IF NOT EXISTS zeppelin.prepared( key text PRIMARY KEY, val text ); 
DEBUG [17:21:12,929][] ACHILLES_DDL_SCRIPT@:executeScript       SCRIPT : CREATE TYPE IF NOT EXISTS zeppelin.address( street_number int, street_name text, zip_code int, country text, extra_info list<text>, phone_numbers map<text,bigint> ); 
DEBUG [17:21:16,426][] ACHILLES_DDL_SCRIPT@:executeScript       SCRIPT : CREATE TABLE IF NOT EXISTS zeppelin.users( login text, firstname text, lastname text, age int, deceased boolean, last_update timestamp, addresses frozen<address>, location frozen<tuple<text,bigint,text>>, PRIMARY KEY(login) ); 
DEBUG [17:21:17,848][] ACHILLES_DDL_SCRIPT@:executeScript       SCRIPT : CREATE KEYSPACE IF NOT EXISTS samples WITH REPLICATION = {'class':'SimpleStrategy', 'replication_factor':1} AND DURABLE_WRITES=false; 
DEBUG [17:21:18,994][] ACHILLES_DDL_SCRIPT@:executeScript       SCRIPT : CREATE KEYSPACE IF NOT EXISTS live_data WITH REPLICATION = {'class':'SimpleStrategy', 'replication_factor':1} AND DURABLE_WRITES=false; 
DEBUG [17:21:20,127][] ACHILLES_DDL_SCRIPT@:executeScript       SCRIPT : CREATE TYPE IF NOT EXISTS live_data.address ( number int, street text, zip int, city text, country text ); 
DEBUG [17:21:21,903][] ACHILLES_DDL_SCRIPT@:executeScript       SCRIPT : CREATE TABLE IF NOT EXISTS live_data.complex_table( pk1 uuid, pk2 int, my_static1 text static, my_static2 text static, clustering1 timestamp, clustering2 int, clustering3 text, indexed1 text, indexed2 int, simple double, my_list list<text>, my_udt_list frozen<list<address>>, my_udt frozen<address>, my_map map<int,text>, key_indexed_map map<int,text>, entries_indexed_map map<int,text>, PRIMARY KEY((pk1,pk2),clustering1, clustering2, clustering3) ) WITH CLUSTERING ORDER BY (clustering1 DESC, clustering2 ASC, clustering3 DESC); 
DEBUG [17:21:23,138][] ACHILLES_DDL_SCRIPT@:executeScript       SCRIPT : CREATE INDEX IF NOT EXISTS pk2idx ON live_data.complex_table(pk2); 
DEBUG [17:21:24,490][] ACHILLES_DDL_SCRIPT@:executeScript       SCRIPT : CREATE INDEX IF NOT EXISTS clustering2idx ON live_data.complex_table(clustering2); 
DEBUG [17:21:25,783][] ACHILLES_DDL_SCRIPT@:executeScript       SCRIPT : CREATE INDEX IF NOT EXISTS idx1 ON live_data.complex_table(indexed1);

If you look at the timestamp, it took more than 7 secondes to create a few tables in the embedded Cassandra. It means that the embedded Cassandra server was busy because not enough system resources. I guess the Travis CI server was quite busy when we launched the test.

There is really nothing to do other than retry

felixcheung · 2016-02-14T05:16:33Z

@doanduyhai I guess we could check on this - if it has become a frequent problem we could get travis to retry tests. I have not seen this much, on the other hand, selenium test failures seem to be more frequent.

…lenium

prabhjyotsingh · 2016-02-14T08:05:35Z

@felixcheung made relevant changes, merged with 65b47e5, and #709 (master), CI green.

felixcheung · 2016-02-15T00:27:04Z

looks good

felixcheung · 2016-02-15T21:49:20Z

merging today if no more comments

prabhjyotsingh mentioned this pull request Feb 9, 2016

selenium test spark, pyspark and sparkSql #654

Closed

4 tasks

prabhjyotsingh force-pushed the seperateMatrixForSelenium branch 3 times, most recently from fa06141 to f93998c Compare February 9, 2016 15:42

create seperate matrix for selenium test case

c1c883c

prabhjyotsingh force-pushed the seperateMatrixForSelenium branch from f93998c to c1c883c Compare February 9, 2016 17:03

felixcheung reviewed Feb 10, 2016
View reviewed changes

This was referenced Feb 10, 2016

testSparkInterpreterDependencyLoading does not check for endToEndTestEnabled #709

Closed

[ZEPPELIN-665] travis CI change to not rerun interpreter tests #708

Closed

prabhjyotsingh closed this Feb 11, 2016

prabhjyotsingh reopened this Feb 11, 2016

Merge remote-tracking branch 'origin/master' into seperateMatrixForSe…

5fce1b2

…lenium

prabhjyotsingh force-pushed the seperateMatrixForSelenium branch from c8ac068 to 5fce1b2 Compare February 14, 2016 05:45

create different profile for selenium

2d334ff

prabhjyotsingh force-pushed the seperateMatrixForSelenium branch from 31c4a28 to 2d334ff Compare February 14, 2016 07:43

remove TEST_SELENIUM from first profile

bc85814

asfgit closed this in 686921e Feb 16, 2016

prabhjyotsingh deleted the seperateMatrixForSelenium branch February 21, 2016 06:30

Create seperate matrix for selenium test case #706

Create seperate matrix for selenium test case #706

Uh oh!

Conversation

prabhjyotsingh commented Feb 9, 2016

What is this PR for?

What type of PR is it?

Is there a relevant Jira issue?

How should this be tested?

Uh oh!

prabhjyotsingh commented Feb 9, 2016

Uh oh!

felixcheung commented Feb 10, 2016

Uh oh!

felixcheung Feb 10, 2016

Choose a reason for hiding this comment

Uh oh!

prabhjyotsingh Feb 10, 2016

Choose a reason for hiding this comment

Uh oh!

felixcheung commented Feb 10, 2016

Uh oh!

prabhjyotsingh commented Feb 11, 2016

Uh oh!

felixcheung commented Feb 11, 2016

Uh oh!

felixcheung commented Feb 12, 2016

Uh oh!

doanduyhai commented Feb 12, 2016

Uh oh!

felixcheung commented Feb 13, 2016

Uh oh!

doanduyhai commented Feb 13, 2016

Uh oh!

felixcheung commented Feb 14, 2016

Uh oh!

prabhjyotsingh commented Feb 14, 2016

Uh oh!

felixcheung commented Feb 15, 2016

Uh oh!

felixcheung commented Feb 15, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants