Spark 5529 backport 1.3#5746

Closed

alexrovner wants to merge 647 commits intoapache:masterfrom

alexrovner:SPARK-5529-backport-1.3

This pull request is big! We're only showing the most recent 250 commits

Commits on Mar 2, 2015

[Minor] Fix doc typo for describing primitiveTerm effectiveness condition

viirya
authored and
marmbrus
committed
[SPARK-6040][SQL] Fix the percent bug in tablesample

watermen
authored and
marmbrus
committed
[SPARK-6050] [yarn] Relax matching of vcore count in received containers.

Marcelo Vanzin
authored and
tgravescs
committed

Commits on Mar 3, 2015

Commits on Mar 4, 2015

[SPARK-6141][MLlib] Upgrade Breeze from 0.10 to 0.11 to fix convergence bug
mengxr
committed
[SPARK-6136] [SQL] Removed JDBC integration tests which depends on docker-client
liancheng
committed
[SPARK-6134][SQL] Fix wrong datatype for casting FloatType and default LongType value in defaultPrimitive

viirya
authored and
liancheng
committed
[SPARK-6144] [core] Fix addFile when source files are on "hdfs:"

Marcelo Vanzin
authored and
Andrew Or
committed

Commits on Mar 5, 2015

Commits on Mar 9, 2015

Commits on Mar 11, 2015

Commits on Mar 12, 2015

[SPARK-6128][Streaming][Documentation] Updates to Spark Streaming Programming Guide
tdas
committed
[SPARK-6296] [SQL] Added equals to Column

vlyubin
authored and
rxin
committed
[SPARK-6294] fix hang when call take() in JVM on PythonRDD

Davies Liu
authored and
mengxr
committed
[mllib] [python] Add LassoModel to __all__ in regression.py

jkbradley
authored and
mengxr
committed

Commits on Mar 13, 2015

[SPARK-5310] [SQL] [DOC] Parquet section for the SQL programming guide
liancheng
committed
[SPARK-6278][MLLIB] Mention the change of objective in linear regression
mengxr
committed
SPARK-4704 [CORE] SparkSubmitDriverBootstrap doesn't flush output
srowen
committed
[SPARK-6194] [SPARK-677] [PySpark] fix memory leak in collect()

Davies Liu
authored and
JoshRosen
committed
SPARK-4300 [CORE] Race condition during SparkWorker shutdown
srowen
committed
SPARK-4044 [CORE] Thriftserver fails to start when JAVA_HOME points to JRE instead of JDK
srowen
committed
[SPARK-6036][CORE] avoid race condition between eventlogListener and akka actor system

liyezhang556520
authored and
srowen
committed
[SPARK-6087][CORE] Provide actionable exception if Kryo buffer is not large enough

levkhomich
authored and
srowen
committed
[SPARK-6132] ContextCleaner race condition across SparkContexts

Andrew Or
authored and
srowen
committed
[SPARK-6132][HOTFIX] ContextCleaner InterruptedException should be quiet

Andrew Or
authored and
srowen
committed
[SPARK-6133] Make sc.stop() idempotent

Andrew Or
authored and
srowen
committed
[SPARK-6275][Documentation]Miss toDF() function in docs/sql-programming-guide.md

zzcclp
authored and
srowen
committed

Commits on Mar 14, 2015

[SPARK-6210] [SQL] use prettyString as column name in agg()

Davies Liu
authored and
rxin
committed

Commits on Mar 15, 2015

[SPARK-3619] Part 2. Upgrade to Mesos 0.21 to work around MESOS-1688

jongyoul
authored and
srowen
committed

Commits on Mar 16, 2015

Commits on Mar 17, 2015

Commits on Mar 18, 2015

[SPARK-6330] [SQL] Add a test case for SPARK-6330

Pei-Lun Lee
authored and
liancheng
committed
[SPARK-6366][SQL] In Python API, the default save mode for save and saveAsTable should be "error" instead of "append".

yhuai
authored and
liancheng
committed
[SPARK-6383][SQL]Fixed compiler and errors in Dataframe examples

Tijo Thomas
authored and
rxin
committed
[SPARK-6247][SQL] Fix resolution of ambiguous joins caused by new aliases
marmbrus
committed
[SPARK-6286][minor] Handle missing Mesos case TASK_ERROR.

dragos
authored and
srowen
committed
[SPARK-6325] [core,yarn] Do not change target executor count when killing executors.

Marcelo Vanzin
authored and
srowen
committed

Commits on Mar 19, 2015

[SPARK-6222][Streaming] Dont delete checkpoint data when doing pre-batch-start checkpoint
tdas
committed

Commits on Mar 20, 2015

[SPARK-6286][Mesos][minor] Handle missing Mesos case TASK_ERROR

jongyoul
authored and
srowen
committed
[SPARK-6421][MLLIB] _regression_train_wrapper does not test initialWeights correctly

Lewuathe
authored and
mengxr
committed

Commits on Mar 21, 2015

[SPARK-5821] [SQL] ParquetRelation2 CTAS should check if delete is successful

yanboliang
authored and
liancheng
committed
[SPARK-6315] [SQL] Also tries the case class string parser while reading Parquet schema
liancheng
committed
[SPARK-5821] [SQL] JSON CTAS command should throw error message when delete path failure

yanboliang
authored and
liancheng
committed
[SPARK-5320][SQL]Add statistics method at NoRelation (override super).

x1-
authored and
marmbrus
committed
[SPARK-5680][SQL] Sum function on all null values, should return zero

gvramana
authored and
marmbrus
committed
[SPARK-6250][SPARK-6146][SPARK-5911][SQL] Types are now reserved words in DDL parser.

yhuai
authored and
marmbrus
committed
[SPARK-6428][SQL] Added explicit type for all public methods in sql/core

rxin
authored and
marmbrus
committed
[SPARK-6428][SQL] Added explicit type for all public methods for Hive module

rxin
authored and
marmbrus
committed

Commits on Mar 22, 2015

[SPARK-6408] [SQL] Fix JDBCRDD filtering string literals

ypcat
authored and
liancheng
committed
SPARK-6454 [DOCS] Fix links to pyspark api

kamilsmuga
authored and
srowen
committed
[SPARK-6337][Documentation, SQL]Spark 1.3 doc fixes

vinodkc
authored and
srowen
committed

Commits on Mar 23, 2015

[SPARK-4985] [SQL] parquet support for date type

adrian-wang
authored and
liancheng
committed
[SPARK-6397][SQL] Check the missingInput simply

watermen
authored and
liancheng
committed

Commits on Mar 24, 2015

Commits on Mar 25, 2015

[SPARK-6469] Improving documentation on YARN local directories usage

Christophe Préaud
authored and
Andrew Or
committed
[DOCUMENTATION]Fixed Missing Type Import in Documentation

bllchmbrs
authored and
rxin
committed
[SPARK-6496] [MLLIB] GeneralizedLinearAlgorithm.run(input, initialWeights) should initialize numFeatures

yanboliang
authored and
srowen
committed
SPARK-6063 MLlib doesn't pass mvn scalastyle check due to UTF chars in LDAModel.scala

msjgriffiths
authored and
srowen
committed
[SPARK-6409][SQL] It is not necessary that avoid old inteface of hive, because this will make some UDAF can not work.

pzzs
authored and
marmbrus
committed

Commits on Mar 26, 2015

[SPARK-6450] [SQL] Fixes metastore Parquet table conversion

liancheng
authored and
marmbrus
committed
[SPARK-6463][SQL] AttributeSet.equal should compare size
marmbrus
committed
[SPARK-6536] [PySpark] Column.inSet() in Python

Davies Liu
authored and
rxin
committed
[SPARK-6465][SQL] Fix serialization of GenericRowWithSchema using kryo

marmbrus
authored and
liancheng
committed
[SQL][SPARK-6471]: Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

Yash Datta
authored and
liancheng
committed
[SPARK-6491] Spark will put the current working dir to the CLASSPATH

guliangliang
authored and
srowen
committed
SPARK-6480 [CORE] histogram() bucket function is wrong in some simple edge cases
srowen
committed
[SPARK-6117] [SQL] add describe function to DataFrame for summary statis...

azagrebin
authored and
rxin
committed
[SPARK-6117] [SQL] Improvements to DataFrame.describe()
rxin
committed
[SPARK-6554] [SQL] Don't push down predicates which reference partition column(s)

liancheng
authored and
marmbrus
committed
[DOCS][SQL] Fix JDBC example
marmbrus
committed

Commits on Mar 27, 2015

Commits on Mar 28, 2015

[SPARK-6538][SQL] Add missing nullable Metastore fields when merging a Parquet schema

Adam Budde
authored and
liancheng
committed

Commits on Mar 29, 2015

Commits on Mar 30, 2015

[spark-sql] a better exception message than "scala.MatchError" for unsupported types in Schema creation

eranation
authored and
rxin
committed
[SPARK-5750][SPARK-3441][SPARK-5836][CORE] Added documentation explaining shuffle

Ilya Ganelin
authored and
srowen
committed
[SPARK-6558] Utils.getCurrentUserName returns the full principal name instead of login name

tgravescs
authored and
srowen
committed
[SPARK-6595][SQL] MetastoreRelation should be a MultiInstanceRelation

marmbrus
authored and
liancheng
committed
[SPARK-6571][MLLIB] use wrapper in MatrixFactorizationModel.load
mengxr
committed
[SPARK-6592][SQL] fix filter for scaladoc to generate API doc for Row class under catalyst dir

CodingCat
authored and
rxin
committed
[SPARK-6603] [PySpark] [SQL] add SQLContext.udf and deprecate inferSchema() and applySchema

Davies Liu
authored and
rxin
committed
[SPARK-6369] [SQL] Uses commit coordinator to help committing Hive and Parquet tables
liancheng
committed

Commits on Mar 31, 2015

Commits on Apr 1, 2015

[Doc] Improve Python DataFrame documentation
rxin
committed
SPARK-6626 [DOCS]: Corrected Scala:TwitterUtils parameters

jayson
authored and
srowen
committed
[SPARK-6651][MLLIB] delegate dense vector arithmetics to the underlying numpy array
mengxr
committed
[SPARK-6657] [Python] [Docs] fixed python doc build warnings

jkbradley
authored and
rxin
committed
[SPARK-6578] [core] Fix thread-safety issue in outbound path of network library.

Marcelo Vanzin
authored and
rxin
committed
[SPARK-6642][MLLIB] use 1.2 lambda scaling and remove addImplicit from NormalEquation
mengxr
committed

Commits on Apr 2, 2015

[SPARK-6553] [pyspark] Support functools.partial as UDF

ksonj
authored and
JoshRosen
committed
[SPARK-6660][MLLIB] pythonToJava doesn't recognize object arrays
mengxr
committed
[SPARK-6578] Small rewrite to make the logic more clear in MessageWithHeader.transferTo.
rxin
committed
[SQL] SPARK-6658: Update DataFrame documentation to refer to correct types

chetmancini
authored and
marmbrus
committed
Revert "[SPARK-6618][SQL] HiveMetastoreCatalog.lookupRelation should use fine-grained lock"
liancheng
committed
[SPARK-6667] [PySpark] remove setReuseAddress

Davies Liu
authored and
JoshRosen
committed
[SPARK-6079] Use index to speed up StatusTracker.getJobIdsForGroup()
JoshRosen
committed
SPARK-6414: Spark driver failed with NPE on job cancelation

Hung Lin
authored and
JoshRosen
committed
[SQL] Throw UnsupportedOperationException instead of NotImplementedError
marmbrus
committed
[SPARK-6655][SQL] We need to read the schema of a data source table stored in spark.sql.sources.schema property

yhuai
authored and
marmbrus
committed
[Minor] [SQL] Follow-up of PR #5210

liancheng
authored and
rxin
committed
[SPARK-6618][SPARK-6669][SQL] Lock Hive metastore client correctly.

yhuai
authored and
marmbrus
committed
[SPARK-6672][SQL] convert row to catalyst in createDataFrame(RDD[Row], ...)

mengxr
authored and
marmbrus
committed

Commits on Apr 3, 2015

[SPARK-6686][SQL] Use resolved output instead of names for toDF rename
marmbrus
committed
[SPARK-6650] [core] Stop ExecutorAllocationManager when context stops.

Marcelo Vanzin
authored and
Andrew Or
committed
[SPARK-6575][SQL] Converted Parquet Metastore tables no longer cache metadata

yhuai
authored and
marmbrus
committed
[CORE] The descriptionof jobHistory config should be spark.history.fs.logDirectory

KaiXinXiaoLei
authored and
Andrew Or
committed
[SPARK-6345][STREAMING][MLLIB] Fix for training with prediction

freeman-lab
authored and
mengxr
committed
[SPARK-6621][Core] Fix the bug that calling EventLoop.stop in EventLoop.onReceive/onError/onStart doesn't call onStop

zsxwing
authored and
JoshRosen
committed
[SPARK-6575][SQL] Converted Parquet Metastore tables no longer cache metadata

yhuai
authored and
liancheng
committed
[SPARK-6688] [core] Always use resolved URIs in EventLoggingListener.

Marcelo Vanzin
authored and
Andrew Or
committed
[SPARK-6700] disable flaky test

Davies Liu
authored and
Andrew Or
committed

Commits on Apr 4, 2015

Commits on Apr 6, 2015

SPARK-6205 [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError
srowen
committed

Commits on Apr 7, 2015

[SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py

Matt Aasted
authored and
JoshRosen
committed
[SPARK-6737] Fix memory leak in OutputCommitCoordinator
JoshRosen
committed

Commits on Apr 8, 2015

Preparing development version 1.3.2-SNAPSHOT
pwendell
committed
Revert "Preparing development version 1.3.2-SNAPSHOT"
pwendell
committed
Revert "Preparing Spark release v1.3.1-rc1"
pwendell
committed
Preparing Spark release v1.3.1-rc2
pwendell
committed
[SPARK-6506] [pyspark] Do not try to retrieve SPARK_HOME when not needed...

Marcelo Vanzin
authored and
JoshRosen
committed
[SPARK-6753] Clone SparkConf in ShuffleSuite tests

kayousterhout
authored and
JoshRosen
committed
[SPARK-6781] [SQL] use sqlContext in python shell

Davies Liu
authored and
rxin
committed
[SPARK-6767][SQL] Fixed Query DSL error in spark sql Readme

Tijo Thomas
authored and
rxin
committed

Commits on Apr 9, 2015

[SPARK-6343] Doc driver-worker network reqs

parente
authored and
srowen
committed

Commits on Apr 10, 2015

Commits on Apr 12, 2015

[SPARK-6677] [SQL] [PySpark] fix cached classes

Davies Liu
authored and
JoshRosen
committed

Commits on Apr 13, 2015

[SPARK-6868][YARN] Fix broken container log link on executor page when HTTPS_ONLY.

deanchen
authored and
srowen
committed

Commits on Apr 14, 2015

SPARK-6878 [CORE] Fix for sum on empty RDD fails with exception

Erik van Oosten
authored and
srowen
committed
[SPARK-6905] Upgrade to snappy-java 1.1.1.7
JoshRosen
committed
[SPARK-6766][Streaming] Fix issue about StreamingListenerBatchSubmitted and StreamingListenerBatchStarted (backport to branch 1.3)

zsxwing
authored and
tdas
committed

Commits on Apr 15, 2015

[SPARK-5634] [core] Show correct message in HS when no incomplete apps f...

Marcelo Vanzin
authored and
Andrew Or
committed
[SPARK-6860][Streaming][WebUI] Fix the possible inconsistency of StreamingPage

zsxwing
authored and
srowen
committed
SPARK-4924 addendum. Minor assembly directory fix in load-spark-env-sh

Raschild
authored and
srowen
committed
[SPARK-6886] [PySpark] fix big closure with shuffle

Davies Liu
authored and
JoshRosen
committed
[SPARK-6800][SQL] Update doc for JDBCRelation's columnPartition

viirya
authored and
marmbrus
committed
[SPARK-6937][MLLIB] Fixed bug in PICExample in which the radius were not being accepted on c...

sboeschhuawei
authored and
mengxr
committed

Commits on Apr 17, 2015

[SPARK-6952] Handle long args when detecting PID reuse

Punya Biswal
authored and
srowen
committed
SPARK-6988 : Fix documentation regarding DataFrames using the Java API

Olivier Girardot
authored and
rxin
committed

Commits on Apr 18, 2015

[SPARK-6975][Yarn] Fix argument validation error

jerryshao
authored and
Andrew Or
committed
SPARK-6992 : Fix documentation example for Spark SQL on StructType

Olivier Girardot
authored and
rxin
committed

Commits on Apr 19, 2015

Fixed doc

gaurav324
authored and
rxin
committed

Commits on Apr 20, 2015

[SPARK-6998][MLlib] Make StreamingKMeans 'Serializable'

zsxwing
authored and
rxin
committed
[doc][mllib] Fix typo of the page title in Isotonic regression documents

dobashim
authored and
srowen
committed
fixed doc

ericchiang
authored and
rxin
committed
[doc][streaming] Fixed broken link in mllib section

BenFradet
authored and
mengxr
committed

Commits on Apr 21, 2015

[SPARK-6985][streaming] Receiver maxRate over 1000 causes a StackOverflowError

David McGuire
authored and
srowen
committed
[SPARK-7036][MLLIB] ALS.train should support DataFrames in PySpark
mengxr
committed
[Minor][MLLIB] Fix a minor formatting bug in toString method in Node.scala

Alain
authored and
mengxr
committed

Commits on Apr 22, 2015

[SPARK-7039][SQL]JDBCRDD: Add support on type NVARCHAR

szheng79
authored and
rxin
committed

Commits on Apr 23, 2015

Commits on Apr 24, 2015

[SPARK-7044][SQL] Fix the deadlock in ScriptTransform(for Spark 1.3)

chenghao-intel
authored and
rxin
committed
Update sql-programming-guide.md

kgeis
authored and
rxin
committed

Commits on Apr 27, 2015

[SPARK-4925] Publish Spark SQL hive-thriftserver maven artifact

chernetsov
authored and
pwendell
committed
[SPARK-7103] Fix crash with SparkContext.union when RDD has no partitioner

stshe
authored and
srowen
committed

Commits on Apr 28, 2015

[SPARK-7187] SerializationDebugger should not crash user code

Andrew Or
authored and
rxin
committed
SPARK-5529 Backported changes to 1.3
alexrovner
committed