[SPARK-3686][STREAMING] Wait for sink to commit the channel before check...

[harishreedharan]

...ing for the channel size.

[SparkQA]

QA tests have started for PR 2531 at commit 6ce9d8b.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2531 at commit 6ce9d8b.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20778/

[harishreedharan]

Jenkins, test this please

Thanks,
Hari

On Wed, Sep 24, 2014 at 9:57 PM, UCB AMPLab notifications@github.com
wrote:

Test PASSed.
Refer to this link for build results (access rights to CI server needed):

https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20778/

Reply to this email directly or view it on GitHub:
#2531 (comment)

[harishreedharan]

Jenkins, test this please

[SparkQA]

QA tests have started for PR 2531 at commit 6ce9d8b.

• This patch merges cleanly.

[pwendell]

Hey @harishreedharan - because the Spark tests take a long time, we don't usually allow sleep statements. Is there a way to use some better synchronization to make this test pass deterministically?

[harishreedharan]

Let me see if I can add a method in the sink that can be used to query if the acks have come.

[pwendell]

That would be great. If you really need to rely on sleeps then it would be good to at least sleep in several smaller intervals rather than 1 second and check multiple times. But we've tried to have all our tests use proper synchronization to avoid this.

[SparkQA]

QA tests have finished for PR 2531 at commit 6ce9d8b.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20780/

[harishreedharan]

This is more deterministic. This counts down when a batch is completed.

[harishreedharan]

Jenkins, test this please

[SparkQA]

QA tests have started for PR 2531 at commit 30393c1.

• This patch merges cleanly.

[SparkQA]

QA tests have started for PR 2531 at commit 30393c1.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2531 at commit 30393c1.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20820/

[SparkQA]

QA tests have finished for PR 2531 at commit 30393c1.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20821/

[harishreedharan]

Jenkins, test this please

[SparkQA]

QA tests have started for PR 2531 at commit 30393c1.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2531 at commit 30393c1.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20823/

[harishreedharan]

Jenkins, test this please

[SparkQA]

QA tests have started for PR 2531 at commit 30393c1.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2531 at commit 30393c1.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20837/

[harishreedharan]

@pwendell This seems to be stable at this point.

[pwendell]

Thanks Hari - I'll pull it in.