Skip to content

[SPARK-7838] [STREAMING] Set scope for kinesis stream #6369

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
tdas wants to merge 2 commits into from
Closed

[SPARK-7838] [STREAMING] Set scope for kinesis stream #6369

tdas wants to merge 2 commits into from

Conversation

@tdas
Copy link
Contributor

@tdas tdas commented May 23, 2015

No description provided.

@tdas
Copy link
Contributor Author

tdas commented May 23, 2015

@andrewor14

andrewor14
andrewor14 reviewed May 23, 2015
View reviewed changes
extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisUtils.scala Outdated
Copy link
Contributor

@andrewor14 andrewor14 May 23, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe it would be good to add a comment explaining why this is necessary (there is no KinesisInputDStream or the like)

@andrewor14
Copy link
Contributor

andrewor14 commented May 23, 2015

LGTM. We should get this into 1.4

@SparkQA
Copy link

SparkQA commented May 23, 2015

Test build #33383 has finished for PR 6369 at commit 37775d8.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@andrewor14
Copy link
Contributor

andrewor14 commented May 23, 2015

Merging into master and 1.4 thanks TD.

@asfgit asfgit closed this in baa8983 May 23, 2015
asfgit pushed a commit that referenced this pull request May 23, 2015
@tdas
[SPARK-7838] [STREAMING] Set scope for kinesis stream
b928db4 
Author: Tathagata Das <tathagata.das1565@gmail.com>

Closes #6369 from tdas/SPARK-7838 and squashes the following commits:

87d1c7f [Tathagata Das] Addressed comment
37775d8 [Tathagata Das] set scope for kinesis stream

(cherry picked from commit baa8983)
Signed-off-by: Andrew Or <andrew@databricks.com>
@SparkQA
Copy link

SparkQA commented May 23, 2015

Test build #33394 has finished for PR 6369 at commit 87d1c7f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cfregly
Copy link
Contributor

cfregly commented May 23, 2015

good catch, TD. didn't even know this concept existed. do kafka or other impls need this? should we add this to the Custom Receiver guide?

@tdas
Copy link
Contributor Author

tdas commented May 23, 2015

Recently added along with the new visualizations in Spark web UI

On Sat, May 23, 2015 at 1:17 PM, Chris Fregly notifications@github.com
wrote:

good catch, TD. didn't even know this concept existed. do kafka or other
impls need this? should we add this to the Custom Receiver guide?

—
Reply to this email directly or view it on GitHub
#6369 (comment).

jeanlyn pushed a commit to jeanlyn/spark that referenced this pull request May 28, 2015
@tdas @jeanlyn
[SPARK-7838] [STREAMING] Set scope for kinesis stream
351896c 
Author: Tathagata Das <tathagata.das1565@gmail.com>

Closes apache#6369 from tdas/SPARK-7838 and squashes the following commits:

87d1c7f [Tathagata Das] Addressed comment
37775d8 [Tathagata Das] set scope for kinesis stream
jeanlyn pushed a commit to jeanlyn/spark that referenced this pull request Jun 12, 2015
@tdas @jeanlyn
[SPARK-7838] [STREAMING] Set scope for kinesis stream
64c7cb0 
Author: Tathagata Das <tathagata.das1565@gmail.com>

Closes apache#6369 from tdas/SPARK-7838 and squashes the following commits:

87d1c7f [Tathagata Das] Addressed comment
37775d8 [Tathagata Das] set scope for kinesis stream
nemccarthy pushed a commit to nemccarthy/spark that referenced this pull request Jun 19, 2015
@tdas @nemccarthy
[SPARK-7838] [STREAMING] Set scope for kinesis stream
d4ede8b 
Author: Tathagata Das <tathagata.das1565@gmail.com>

Closes apache#6369 from tdas/SPARK-7838 and squashes the following commits:

87d1c7f [Tathagata Das] Addressed comment
37775d8 [Tathagata Das] set scope for kinesis stream
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@tdas @andrewor14 @SparkQA @cfregly