[SPARK-19900][core]Remove driver when relaunching.#17888

Closed

lycplus wants to merge 142 commits intoapache:masterfrom

lycplus:SPARK-19900

Commits on May 7, 2017

Commits on May 24, 2017

[SPARK-20518][CORE] Supplement the new blockidsuite unit tests

heary-cao
authored and
lycplus
committed
[SPARK-20484][MLLIB] Add documentation to ALS code

danielyli
authored and
lycplus
committed
[SPARK-7481][BUILD] Add spark-hadoop-cloud module to pull in object store access.

steveloughran
authored and
lycplus
committed
[SPARK-20543][SPARKR][FOLLOWUP] Don't skip tests on AppVeyor

felixcheung
authored and
lycplus
committed
[MINOR][SQL][DOCS] Improve unix_timestamp's scaladoc (and typo hunting)

jaceklaskowski
authored and
lycplus
committed
[SPARK-20550][SPARKR] R wrapper for Dataset.alias

zero323
authored and
lycplus
committed
[SPARK-16931][PYTHON][SQL] Add Python wrapper for bucketBy

zero323
authored and
lycplus
committed
[SPARK-12297][SQL] Hive compatibility for Parquet Timestamps

squito
authored and
lycplus
committed
[SPARK-20626][SPARKR] address date test warning with timezone on windows

felixcheung
authored and
lycplus
committed
[SPARK-20380][SQL] Unable to set/unset table comment property using ALTER TABLE SET/UNSET TBLPROPERTIES ddl

sujith71955
authored and
lycplus
committed
[SPARKR][DOC] fix typo in vignettes

Wayne Zhang
authored and
lycplus
committed
[SPARK-20519][SQL][CORE] Modify to prevent some possible runtime exceptions

10110346
authored and
lycplus
committed
[SPARK-19956][CORE] Optimize a location order of blocks with topology information

ConeyLiu
authored and
lycplus
committed
[SPARK-20596][ML][TEST] Consolidate and improve ALS recommendAll test cases

Nick Pentreath
authored and
lycplus
committed
[SPARK-20621][DEPLOY] Delete deprecated config parameter in 'spark-env.sh'

ConeyLiu
authored and
lycplus
committed
[SPARK-20605][CORE][YARN][MESOS] Deprecate not used AM and executor port configuration

jerryshao
authored and
lycplus
committed
[SPARK-20661][SPARKR][TEST] SparkR tableNames() test fails

falaki
authored and
lycplus
committed
[SPARK-20661][SPARKR][TEST][FOLLOWUP] SparkR tableNames() test fails

felixcheung
authored and
lycplus
committed
[SPARK-11968][MLLIB] Optimize MLLIB ALS recommendForAll

Peng
authored and
lycplus
committed
[SPARK-20587][ML] Improve performance of ML ALS recommendForAll

Nick Pentreath
authored and
lycplus
committed
[SPARK-20615][ML][TEST] SparseVector.argmax throws IndexOutOfBoundsException

Jon McLean
authored and
lycplus
committed
[SPARK-20606][ML] ML 2.2 QA: Remove deprecated methods for ML

yanboliang
authored and
lycplus
committed
[SPARK-20667][SQL][TESTS] Cleanup the cataloged metadata after completing the package of sql/core and sql/hive

gatorsmile
authored and
lycplus
committed
[SPARK-20311][SQL] Support aliases for table value functions

maropu
authored and
lycplus
committed
[SPARK-20355] Add per application spark version on the history server headerpage

Sanket
authored and
lycplus
committed
[SPARK-20548][FLAKY-TEST] share one REPL instance among REPL test cases

cloud-fan
authored and
lycplus
committed
[SPARK-20674][SQL] Support registering UserDefinedFunction as named UDF

rxin
authored and
lycplus
committed
[SPARK-19876][BUILD] Move Trigger.java to java source hierarchy

srowen
authored and
lycplus
committed
[SPARK-20627][PYSPARK] Drop the hadoop distirbution name from the Python version

holdenk
authored and
lycplus
committed
Revert "[SPARK-12297][SQL] Hive compatibility for Parquet Timestamps"

rxin
authored and
lycplus
committed
Revert "[SPARK-20311][SQL] Support aliases for table value functions"

yhuai
authored and
lycplus
committed
[SPARK-20373][SQL][SS] Batch queries with 'Dataset/DataFrame.withWatermark()` does not execute

uncleGen
authored and
lycplus
committed
[SPARK-17685][SQL] Make SortMergeJoinExec's currentVars is null when calling createJoinKey

wangyum
authored and
lycplus
committed
[SPARK-20590][SQL] Use Spark internal datasource if multiples are found for the same shorten name

HyukjinKwon
authored and
lycplus
committed
[SPARK-20686][SQL] PropagateEmptyRelation incorrectly handles aggregate without grouping

JoshRosen
authored and
lycplus
committed
[SPARK-20670][ML] Simplify FPGrowth transform

YY-OnCall
authored and
lycplus
committed
[SPARK-20668][SQL] Modify ScalaUDF to handle nullability.

ueshin
authored and
lycplus
committed
[SPARK-20631][PYTHON][ML] LogisticRegression._checkThresholdConsistency should use values not Params

zero323
authored and
lycplus
committed
[SPARK-20630][WEB UI] Fixed column visibility in Executor Tab

ajbozarth
authored and
lycplus
committed
[SPARK-20637][CORE] Remove mention of old RDD classes from comments

michaelmior
authored and
lycplus
committed
[SPARK-20393][WEBU UI] Strengthen Spark to prevent XSS vulnerabilities

n-marion
authored and
lycplus
committed
[SPARK-20688][SQL] correctly check analysis for scalar sub-queries

cloud-fan
authored and
lycplus
committed
[SPARK-20678][SQL] Ndv for columns not in filter condition should also be updated

wangzhenhua
authored and
lycplus
committed
[MINOR][BUILD] Fix lint-java breaks.

ConeyLiu
authored and
lycplus
committed
[SPARK-19447] Remove remaining references to generated rows metric

ala
authored and
lycplus
committed
[SPARK-20689][PYSPARK] python doctest leaking bucketed table

felixcheung
authored and
lycplus
committed
[SPARK-20685] Fix BatchPythonEvaluation bug in case of single UDF w/ repeated arg.

JoshRosen
authored and
lycplus
committed
[SPARK-20606][ML] Revert "[] ML 2.2 QA: Remove deprecated methods for ML"

yanboliang
authored and
lycplus
committed
[SPARK-17029] make toJSON not go through rdd form but operate on dataset always

Robert Kruszewski
authored and
lycplus
committed
[SPARK-20569][SQL] RuntimeReplaceable functions should not take extra parameters

cloud-fan
authored and
lycplus
committed
[SPARK-20311][SQL] Support aliases for table value functions

maropu
authored and
lycplus
committed
[SPARK-20416][SQL] Print UDF names in EXPLAIN

maropu
authored and
lycplus
committed
[SPARK-20600][SS] KafkaRelation should be pretty printed in web UI

jaceklaskowski
authored and
lycplus
committed
[SPARK-20431][SQL] Specify a schema by using a DDL-formatted string

maropu
authored and
lycplus
committed
[SPARK-20399][SQL] Add a config to fallback string literal parsing consistent with old sql parser behavior

viirya
authored and
lycplus
committed
[SPARK-20665][SQL] Bround" and "Round" function return NULL

10110346
authored and
lycplus
committed
[SPARK-20718][SQL] FileSourceScanExec with different filter orders should be the same after canonicalization

wangzhenhua
authored and
lycplus
committed
[SPARK-20704][SPARKR] change CRAN test to run single thread

felixcheung
authored and
lycplus
committed
[SPARK-20619][ML] StringIndexer supports multiple ways to order label

Wayne Zhang
authored and
lycplus
committed
[SPARK-20639][SQL] Add single argument support for to_timestamp in SQL with documentation improvement

HyukjinKwon
authored and
lycplus
committed
[SPARK-20554][BUILD] Remove usage of scala.language.reflectiveCalls

srowen
authored and
lycplus
committed
[SPARK-17424] Fix unsound substitution bug in ScalaReflection.

rdblue
authored and
lycplus
committed
[SPARK-20718][SQL][FOLLOWUP] Fix canonicalization for HiveTableScanExec

wangzhenhua
authored and
lycplus
committed
[SPARK-20710][SQL] Support aliases in CUBE/ROLLUP/GROUPING SETS

maropu
authored and
lycplus
committed
[SPARK-19951][SQL] Add string concatenate operator || to Spark SQL

maropu
authored and
lycplus
committed
[SPARK-20702][CORE] TaskContextImpl.markTaskCompleted should not hide the original error

zsxwing
authored and
lycplus
committed
[SPARK-20714][SS] Fix match error when watermark is set with timeout = no timeout / processing timeout

tdas
authored and
lycplus
committed
[SPARK-20594][SQL] The staging directory should be a child directory starts with "." to avoid being deleted if we set hive.exec.stagingdir under the table directory.

zuotingbing
authored and
lycplus
committed
[SPARK-20719][SQL] Support LIMIT ALL

gatorsmile
authored and
lycplus
committed
[SPARK-18772][SQL] Avoid unnecessary conversion try for special floats in JSON

HyukjinKwon
authored and
lycplus
committed
[SPARK-20725][SQL] partial aggregate should behave correctly for sameResult

cloud-fan
authored and
lycplus
committed
[DOCS][SPARKR] Use verbose names for family annotations in functions.R

zero323
authored and
lycplus
committed
[SPARK-20726][SPARKR] wrapper for SQL broadcast

zero323
authored and
lycplus
committed
[SPARK-20705][WEB-UI] The sort function can not be used in the master page when you use Firefox or Google Chrome.

guoxiaolong
authored and
lycplus
committed
[SPARK-20720][WEB-UI] Executor Summary' should show the exact number, 'Removed Executors' should display the specific number, in the Application Page

guoxiaolong
authored and
lycplus
committed
[SPARK-20730][SQL] Add an optimizer rule to combine nested Concat

maropu
authored and
lycplus
committed
[SPARK-20669][ML] LoR.family and LDA.optimizer should be case insensitive

zhengruifeng
authored and
lycplus
committed
[SPARK-12837][SPARK-20666][CORE][FOLLOWUP] getting name should not fail if accumulator is garbage collected

cloud-fan
authored and
lycplus
committed
[SPARK-20716][SS] StateStore.abort() should not throw exceptions

tdas
authored and
lycplus
committed
[SPARK-17729][SQL] Enable creating hive bucketed tables

tejasapatil
authored and
lycplus
committed
[SPARK-20717][SS] Minor tweaks to the MapGroupsWithState behavior

tdas
authored and
lycplus
committed
[SPARK-20735][SQL][TEST] Enable cross join in TPCDSQueryBenchmark

dongjoon-hyun
authored and
lycplus
committed
[SPARK-20588][SQL] Cache TimeZone instances.

ueshin
authored and
lycplus
committed
[SPARK-20707][ML] ML deprecated APIs should be removed in major release.

yanboliang
authored and
lycplus
committed
[SPARK-20501][ML] ML 2.2 QA: New Scala APIs, docs

yanboliang
authored and
lycplus
committed
[SPARK-20553][ML][PYSPARK] Update ALS examples with recommend-all methods

Nick Pentreath
authored and
lycplus
committed
[SPARK-20677][MLLIB][ML] Follow-up to ALS recommend-all performance PRs

Nick Pentreath
authored and
lycplus
committed
[SPARK-20529][CORE] Allow worker and master work with a proxy server

zsxwing
authored and
lycplus
committed
[SPARK-19372][SQL] Fix throwing a Java exception at df.fliter() due to 64KB bytecode size limit

kiszk
authored and
lycplus
committed
[SPARK-20140][DSTREAM] Remove hardcoded kinesis retry wait and max retries

yashs360
authored and
lycplus
committed
[SQL][TRIVIAL] Lower parser log level to debug

hvanhovell
authored and
lycplus
committed
[SPARK-20690][SQL] Subqueries in FROM should have alias names

viirya
authored and
lycplus
committed
[SPARK-20776] Fix perf. problems in JobProgressListener caused by TaskMetrics construction

JoshRosen
authored and
lycplus
committed
[SPARK-20769][DOC] Incorrect documentation for using Jupyter notebook

aray
authored and
lycplus
committed
[SPARK-20788][CORE] Fix the Executor task reaper's false alarm warning logs

zsxwing
authored and
lycplus
committed
[SPARK-13747][CORE] Add ThreadUtils.awaitReady and disallow Await.ready

zsxwing
authored and
lycplus
committed
[SPARK-20505][ML] Add docs and examples for ml.stat.Correlation and ml.stat.ChiSquareTest.

yanboliang
authored and
lycplus
committed
[SPARK-20700][SQL] InferFiltersFromConstraints stackoverflows for query (v2)

jiangxb1987
authored and
lycplus
committed
[SPARK-20779][EXAMPLES] The ASF header placed in an incorrect location in some files.

zuotingbing
authored and
lycplus
committed
[SPARK-20796] the location of start-master.sh in spark-standalone.md is wrong

liu-zhaokun
authored and
lycplus
committed
[SPARK-20364][SQL] Disable Parquet predicate pushdown for fields having dots in the names

HyukjinKwon
authored and
lycplus
committed
[DSTREAM][DOC] Add documentation for kinesis retry configurations

yashs360
authored and
lycplus
committed
[SPARK-20798] GenerateUnsafeProjection should check if a value is null before calling the getter

ala
authored and
lycplus
committed
[SPARK-20773][SQL] ParquetWriteSupport.writeFields is quadratic in number of fields

tpoterba
authored and
lycplus
committed
[SPARK-20607][CORE] Add new unit tests to ShuffleSuite

heary-cao
authored and
lycplus
committed
[SPARK-20759] SCALA_VERSION in _config.yml should be consistent with pom.xml

liu-zhaokun
authored and
lycplus
committed
[SPARK-20751][SQL] Add built-in SQL Function - COT

wangyum
authored and
lycplus
committed
[SPARK-20763][SQL] The function of `month` and `day` return the value which is not we expected.

10110346
authored and
lycplus
committed
[SPARKR][DOCS][MINOR] Use consistent names in rollup and cube examples

zero323
authored and
lycplus
committed
[SPARKR] Fix bad examples in DataFrame methods and style issues

Wayne Zhang
authored and
lycplus
committed
[SPARK-20506][DOCS] 2.2 migration guide

Nick Pentreath
authored and
lycplus
committed
[SPARK-20781] the location of Dockerfile in docker.properties.templat is wrong

liu-zhaokun
authored and
lycplus
committed
[SPARK-20806][DEPLOY] Launcher: redundant check for Spark lib dir

srowen
authored and
lycplus
committed
[SPARK-20792][SS] Support same timeout operations in mapGroupsWithState function in batch queries as in streaming queries

tdas
authored and
lycplus
committed
[SPARK-20736][PYTHON] PySpark StringIndexer supports StringOrderType

Wayne Zhang
authored and
lycplus
committed
[SPARK-20786][SQL] Improve ceil and floor handle the value which is not expected

heary-cao
authored and
lycplus
committed
[SPARK-20770][SQL] Improve ColumnStats

kiszk
authored and
lycplus
committed
[SPARK-19089][SQL] Add support for nested sequences

michalsenkyr
authored and
lycplus
committed
[SPARK-20687][MLLIB] mllib.Matrices.fromBreeze may crash when converting from Breeze sparse matrix

ghoto
authored and
lycplus
committed
[SPARK-20506][DOCS] Add HTML links to highlight list in MLlib guide for 2.2

Nick Pentreath
authored and
lycplus
committed
[SPARK-20591][WEB UI] Succeeded tasks num not equal in all jobs page and job detail page on spark web ui when speculative task(s) exist.

fjh100456
authored and
lycplus
committed
[SPARK-20609][CORE] Run the SortShuffleSuite unit tests have residual spark_* system directory

heary-cao
authored and
lycplus
committed
[SPARK-20813][WEB UI] Fixed Web UI executor page tab search by status not working

John Lee
authored and
lycplus
committed
[SPARK-20801] Record accurate size of blocks in MapStatus when it's above threshold.

jinxing
authored and
lycplus
committed
[SPARK-20831][SQL] Fix INSERT OVERWRITE data source tables with IF NOT EXISTS

gatorsmile
authored and
lycplus
committed
[SPARK-20764][ML][PYSPARK] Fix visibility discrepancy with numInstances and degreesOfFreedom in LR and GLR - Python version

Peng
authored and
lycplus
committed
[SPARK-20756][YARN] yarn-shuffle jar references unshaded guava

markgrover
authored and
lycplus
committed
[SPARK-15767][ML][SPARKR] Decision Tree wrapper in SparkR

zhengruifeng
authored and
lycplus
committed
[SPARK-20814][MESOS] Restore support for spark.executor.extraClassPath.

Marcelo Vanzin
authored and
lycplus
committed
[SPARK-20751][SQL][FOLLOWUP] Add cot test in MathExpressionsSuite

wangyum
authored and
lycplus
committed
[SPARK-17410][SPARK-17284] Move Hive-generated Stats Info to HiveClientImpl

gatorsmile
authored and
lycplus
committed
[SPARK-20815][SPARKR] NullPointerException in RPackageUtils#checkManifestForR

jrshust
authored and
lycplus
committed
[SPARK-20727] Skip tests that use Hadoop utils on CRAN Windows

shivaram
authored and
lycplus
committed
[SPARK-20399][SQL][FOLLOW-UP] Add a config to fallback string literal parsing consistent with old sql parser behavior

viirya
authored and
lycplus
committed
[MINOR][SPARKR][ML] Joint coefficients with intercept for SparkR linear SVM summary.

yanboliang
authored and
lycplus
committed
[SPARK-20857][SQL] Generic resolved hint node

rxin
authored and
lycplus
committed
[SPARK-15648][SQL] Add teradataDialect for JDBC connection to Teradata

Kirby Linvill
authored and
lycplus
committed
Remove driver when relaunching.
lycplus
committed
Add some.
lycplus
committed
Add a test case.
lycplus
committed