[SPARK-1537] [YARN] [WiP] Add history provider for YARN Application Timeline Server#5423

Closed

steveloughran wants to merge 15 commits intoapache:masterfrom

steveloughran:stevel/feature/SPARK-1537-ATS

Commits on Dec 23, 2015

SPARK-1537 SPARK-11315 YarnHistoryService patch applied to latest master (the one with the service API merged in)
steveloughran
committed
SPARK-1537 SPARK-11323 yarn history provider patch applied atop the new SPARK-11315 publisher branch
steveloughran
committed
SPARK-11315 scaladoc and import cleanup (minor)
steveloughran
committed
SPARK-11315 event publisher to add a version counter to every entity request pushed. This is for more reliable polling for changes during integration with YARN-7889
steveloughran
committed
SPARK-11323 yarn provider to use entity version counter to compare and track attempt versions. This is for more reliable polling for changes during integration with YARN-7889
steveloughran
committed
SPARK-11315 publish entity version counter as number; no need to stringify
steveloughran
committed
SPARK-11323 tests to get the version counter marshalling working
steveloughran
committed
SPARK-11323 fixing a regression which turned out to be a typo; lots of improvement in test running in the process. Tests can register "failureActions" for execution on a test failure; closures to d…
steveloughran
committed
SPARK-11323 scalastyle fix
steveloughran
committed
SPARK-11323 chop min refresh interval from 10s to 1s, as testing likes this. In production even 10s is probably too short, so it doesn't make things much worse
steveloughran
committed
SPARK-11315 ignore BlockUpdated and ExecutorMetricsUpdate events
steveloughran
committed
SPARK-11315 add info-level logging of post to ATS with entity ID and ATS URL (as info level wasn't giving any details on whether/when entities were published, or under what); downgrade event drop t…
steveloughran
committed
SPARK-11315 style: methods whose arg list is multiline have newline before first arg
steveloughran
committed
SPARK-11323 style: methods whose arg list is multiline have newline before first arg
steveloughran
committed
SPARK-11323 - some more style checking in the rest/ package
steveloughran
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[SPARK-1537] [YARN] [WiP] Add history provider for YARN Application Timeline Server#5423

[SPARK-1537] [YARN] [WiP] Add history provider for YARN Application Timeline Server#5423
steveloughran wants to merge 15 commits intoapache:masterfrom
steveloughran:stevel/feature/SPARK-1537-ATS

Commits on Dec 23, 2015

SPARK-1537 SPARK-11315 YarnHistoryService patch applied to latest master (the one with the service API merged in)

SPARK-1537 SPARK-11323 yarn history provider patch applied atop the new SPARK-11315 publisher branch

SPARK-11315 scaladoc and import cleanup (minor)

SPARK-11315 event publisher to add a version counter to every entity request pushed. This is for more reliable polling for changes during integration with YARN-7889

SPARK-11323 yarn provider to use entity version counter to compare and track attempt versions. This is for more reliable polling for changes during integration with YARN-7889

SPARK-11315 publish entity version counter as number; no need to stringify

SPARK-11323 tests to get the version counter marshalling working

SPARK-11323 fixing a regression which turned out to be a typo; lots of improvement in test running in the process. Tests can register "failureActions" for execution on a test failure; closures to d…

SPARK-11323 scalastyle fix

SPARK-11323 chop min refresh interval from 10s to 1s, as testing likes this. In production even 10s is probably too short, so it doesn't make things much worse

SPARK-11315 ignore BlockUpdated and ExecutorMetricsUpdate events

SPARK-11315 add info-level logging of post to ATS with entity ID and ATS URL (as info level wasn't giving any details on whether/when entities were published, or under what); downgrade event drop t…

SPARK-11315 style: methods whose arg list is multiline have newline before first arg

SPARK-11323 style: methods whose arg list is multiline have newline before first arg

SPARK-11323 - some more style checking in the rest/ package

[SPARK-1537] [YARN] [WiP] Add history provider for YARN Application Timeline Server#5423steveloughran wants to merge 15 commits intoapache:masterapache/spark:masterfrom steveloughran:stevel/feature/SPARK-1537-ATSsteveloughran/spark:stevel/feature/SPARK-1537-ATSCopy head branch name to clipboard

Commits

Commits on Dec 23, 2015

[SPARK-1537] [YARN] [WiP] Add history provider for YARN Application Timeline Server#5423
steveloughran wants to merge 15 commits intoapache:masterfrom
steveloughran:stevel/feature/SPARK-1537-ATS