Branch 2.0#15184

Closed

paulzwu wants to merge 313 commits intoapache:masterfrom

CK50:branch-2.0

This pull request is big! We're only showing the most recent 250 commits

Commits on May 4, 2016

[SPARK-15115][SQL] Reorganize whole stage codegen benchmark suites
rxin
committed
[SPARK-15103][SQL] Refactored FileCatalog class to allow StreamFileCatalog to infer partitioning
tdas
committed
[SPARK-15126][SQL] RuntimeConfig.set should return Unit

rxin
authored and
Andrew Or
committed
[SPARK-15121] Improve logging of external shuffle handler

Thomas Graves
authored and
Andrew Or
committed
[SPARK-12299][CORE] Remove history serving functionality from Master

BryanCutler
authored and
Andrew Or
committed
[SPARK-15031][EXAMPLE] Use SparkSession in Scala/Python/Java example.

dongjoon-hyun
authored and
Andrew Or
committed
[SPARK-13001][CORE][MESOS] Prevent getting offers when reached max cores

sebastienrainville
authored and
Andrew Or
committed
[SPARK-15116] In REPL we should create SparkSession first and get SparkContext from it

cloud-fan
authored and
Andrew Or
committed
[MINOR][SQL] Fix typo in DataFrameReader csv documentation

sethah
authored and
Andrew Or
committed

Commits on May 5, 2016

[SPARK-14896][SQL] Deprecate HiveContext in python
Andrew Or
committed
[SPARK-6339][SQL] Supports CREATE TEMPORARY VIEW tableIdentifier AS query

clockfly
authored and
yhuai
committed
[SPARK-14993][SQL] Fix Partition Discovery Inconsistency when Input is a Path to Parquet File

gatorsmile
authored and
yhuai
committed
[SPARK-15131][SQL] Shutdown StateStore management thread when SparkContext has been shutdown

tdas
authored and
zsxwing
committed
[MINOR] remove dead code
davies
committed
[SPARK-15132][MINOR][SQL] Debug log for generated code should be printed with proper indentation

sarutak
authored and
rxin
committed
[SPARK-15045] [CORE] Remove dead code in TaskMemoryManager.cleanUpAllAllocatedMemory for pageTable

abhi951990
authored and
davies
committed
[SPARK-15123] upgrade org.json4s to 3.2.11 version

liningalex
authored and
srowen
committed
[SPARK-12154] Upgrade to Jersey 2

mccheah
authored and
srowen
committed
[SPARK-15106][PYSPARK][ML] Add PySpark package doc for ML component & remove "BETA"

holdenk
authored and
srowen
committed
[SPARK-14589][SQL] Enhance DB2 JDBC Dialect docker tests

lresende
authored and
srowen
committed
[SPARK-14915][CORE] Don't re-queue a task if another attempt has already succeeded

jasonmoore2k
authored and
srowen
committed
[SPARK-14139][SQL] RowEncoder should preserve schema nullability

cloud-fan
authored and
liancheng
committed
[SPARK-15148][SQL] Upgrade Univocity library from 2.0.2 to 2.1.0

HyukjinKwon
authored and
rxin
committed
[SPARK-15110] [SPARKR] Implement repartitionByColumn for SparkR DataFrames

NarineK
authored and
davies
committed
[MINOR][BUILD] Adds spark-warehouse/ to .gitignore

liancheng
authored and
Andrew Or
committed
[SPARK-14124][SQL][FOLLOWUP] Implement Database-related DDL Commands

gatorsmile
authored and
Andrew Or
committed
[SPARK-15072][SQL][REPL][EXAMPLES] Remove SparkSession.withHiveSupport

techaddict
authored and
Andrew Or
committed
[SPARK-15135][SQL] Make sure SparkSession thread safe

zsxwing
authored and
Andrew Or
committed
[SPARK-15134][EXAMPLE] Indent SparkSession builder patterns and update binary_classification_metrics_example.py

dongjoon-hyun
authored and
Andrew Or
committed
[SPARK-15158][CORE] downgrade shouldRollover message to debug level

depend
authored and
Andrew Or
committed
[SPARK-9926] Parallelize partition logic in UnionRDD.

rdblue
authored and
Andrew Or
committed
[SPARK-14893][SQL] Re-enable HiveSparkSubmitSuite SPARK-8489 test after HiveContext is removed

dilipbiswal
authored and
Andrew Or
committed
[SPARK-15152][DOC][MINOR] Scaladoc and Code style Improvements

jaceklaskowski
authored and
Andrew Or
committed
[HOTFIX] Fix MLUtils compile
Andrew Or
committed

Commits on May 6, 2016

[SPARK-11395][SPARKR] Support over and window specification in SparkR.

Sun Rui
authored and
shivaram
committed
[SPARK-14738][BUILD] Separate docker integration tests from main build

lresende
authored and
srowen
committed
[SPARK-14512] [DOC] Add python example for QuantileDiscretizer

zhengruifeng
authored and
davies
committed
[SPARK-14962][SQL] Do not push down isnotnull/isnull on unsuportted types in ORC

HyukjinKwon
authored and
liancheng
committed
[SPARK-15108][SQL] Describe Permanent UDTF

gatorsmile
authored and
yhuai
committed
[SPARK-14050][ML] Add multiple languages support and additional methods for Stop Words Remover

burakkose
authored and
mengxr
committed
[SPARK-14997][SQL] Fixed FileCatalog to return correct set of files when there is no partitioning scheme in the given paths

tdas
authored and
yhuai
committed

Commits on May 7, 2016

[SPARK-1239] Improve fetching of map output statuses

Thomas Graves
authored and
davies
committed
[SPARK-15087][MINOR][DOC] Follow Up: Fix the Comments

techaddict
authored and
cloud-fan
committed
[SPARK-15051][SQL] Create a TypedColumn alias

kevinyu98
authored and
cloud-fan
committed
[SPARK-15122] [SQL] Fix TPC-DS 41 - Normalize predicates before pulling them out

hvanhovell
authored and
davies
committed
[MINOR][ML][PYSPARK] ALS example cleanup
Nick Pentreath
committed
[DOC][MINOR] Fixed minor errors in feature.ml user guide doc

BryanCutler
authored and
Nick Pentreath
committed
[SPARK-15178][CORE] Remove LazyFileRegion instead use netty's DefaultFileRegion

techaddict
authored and
rxin
committed

Commits on May 8, 2016

[SPARK-12479][SPARKR] sparkR collect on GroupedData throws R error "missing value where TRUE/FALSE needed"

sun-rui
authored and
shivaram
committed

Commits on May 9, 2016

[SPARK-15185][SQL] InMemoryCatalog: Silent Removal of an Existent Table/Function/Partitions by Rename

gatorsmile
authored and
cloud-fan
committed
[SPARK-15184][SQL] Fix Silent Removal of An Existent Temp Table by Rename Table

gatorsmile
authored and
cloud-fan
committed
[SPARK-15211][SQL] Select features column from LibSVMRelation causes failure

viirya
authored and
liancheng
committed
[SPARK-14814][MLLIB] API: Java compatibility, docs

hhbyyh
authored and
srowen
committed
[SPARK-15136][PYSPARK][DOC] Fix links to sphinx style and add a default param doc note

holdenk
authored and
srowen
committed
[MINOR][TEST][STREAMING] make "testDir" able to be claened after test.

wei-mao-intel
authored and
srowen
committed
[SPARK-14459][SQL] Detect relation partitioning and adjust the logical plan

rdblue
authored and
liancheng
committed
[MINOR] [SPARKR] Update data-manipulation.R to use native csv reader

yanboliang
authored and
davies
committed
[SPARK-15093][SQL] create/delete/rename directory for InMemoryCatalog operations if needed

cloud-fan
authored and
Andrew Or
committed
[SPARK-15199][SQL] Disallow Dropping Build-in Functions

gatorsmile
authored and
Andrew Or
committed
[SPARK-14127][SQL] Makes 'DESC [EXTENDED|FORMATTED] <table>' support data source tables

liancheng
authored and
yhuai
committed
[MINOR][DOCS] Remove remaining sqlContext in documentation at examples

HyukjinKwon
authored and
Andrew Or
committed
[SPARK-15223][DOCS] fix wrongly named config reference

philipphoffmann
authored and
Andrew Or
committed
[SPARK-15225][SQL] Replace SQLContext with SparkSession in Encoder documentation

viirya
authored and
Andrew Or
committed
[SPARK-15067][YARN] YARN executors are launched with fixed perm gen size

srowen
authored and
Andrew Or
committed
[MINOR][SQL] Enhance the exception message if checkpointLocation is not set

jerryshao
authored and
Andrew Or
committed
[SAPRK-15220][UI] add hyperlink to running application and completed application

wei-mao-intel
authored and
Andrew Or
committed
[SPARK-15210][SQL] Add missing @DeveloperApi annotation in sql.types

zhengruifeng
authored and
Andrew Or
committed
[SPARK-15166][SQL] Move some hive-specific code from SparkSession
Andrew Or
committed
[SPARK-10653][CORE] Remove unnecessary things from SparkEnv

ajbozarth
authored and
Andrew Or
committed
[SPARK-15173][SQL] DataFrameWriter.insertInto should work with datasource table stored in hive

cloud-fan
authored and
yhuai
committed
[SPARK-14972] Improve performance of JSON schema inference's compatibleType method

JoshRosen
authored and
davies
committed
[SPARK-15209] Fix display of job descriptions with single quotes in web UI timeline

JoshRosen
authored and
sarutak
committed

Commits on May 10, 2016

[SPARK-15025][SQL] fix duplicate of PATH key in datasource table options

xwu0226
authored and
yhuai
committed
[SPARK-15234][SQL] Fix spark.catalog.listDatabases.show()

Andrew Or
authored and
rxin
committed
[SPARK-15229][SQL] Make case sensitivity setting internal
rxin
committed
[SPARK-15187][SQL] Disallow Dropping Default Database

gatorsmile
authored and
cloud-fan
committed
[SPARK-15215][SQL] Fix Explain Parsing and Output

gatorsmile
authored and
hvanhovell
committed
[SPARK-14542][CORE] PipeRDD should allow configurable buffer size for…

Sital Kedia
authored and
srowen
committed
[SPARK-14127][SQL] "DESC <table>": Extracts schema information from table properties for data source tables

liancheng
authored and
yhuai
committed
[SPARK-15154] [SQL] Change key types to Long in tests

robbinspg
authored and
davies
committed
[SPARK-14773] [SPARK-15179] [SQL] Fix SQL building and enable Hive tests

hvanhovell
authored and
davies
committed
[SPARK-13382][DOCS][PYSPARK] Update pyspark testing notes in build docs

holdenk
authored and
Marcelo Vanzin
committed
[SPARK-13670][LAUNCHER] Propagate error from launcher to shell.
Marcelo Vanzin
committed
[SPARK-11249][LAUNCHER] Throw error if app resource is not provided.
Marcelo Vanzin
committed
[SPARK-12837][CORE] reduce network IO for accumulators

cloud-fan
authored and
Andrew Or
committed
[SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

techaddict
authored and
Andrew Or
committed
[SPARK-14603][SQL] Verification of Metadata Operations by Session Catalog

gatorsmile
authored and
Andrew Or
committed
[SPARK-15037][HOTFIX] Replace `sqlContext` and `sparkSession` with `spark`.

dongjoon-hyun
authored and
Andrew Or
committed
[SPARK-15037][HOTFIX] Don't create 2 SparkSessions in constructor
Andrew Or
committed
[SPARK-15195][PYSPARK][DOCS] Update ml.tuning PyDocs

holdenk
authored and
Nick Pentreath
committed
[SPARK-14642][SQL] import org.apache.spark.sql.expressions._ breaks udf under functions

sbcd90
authored and
zsxwing
committed
[SPARK-14986][SQL] Return correct result for empty LATERAL VIEW OUTER

hvanhovell
authored and
yhuai
committed
[SPARK-6005][TESTS] Fix flaky test: o.a.s.streaming.kafka.DirectKafkaStreamSuite.offset recovery
zsxwing
committed
[SPARK-15249][SQL] Use FunctionResource instead of (String, String) in CreateFunction and CatalogFunction for resource

techaddict
authored and
Andrew Or
committed
[SPARK-14936][BUILD][TESTS] FlumePollingStreamSuite is slow

keypointt
authored and
zsxwing
committed
[SPARK-14837][SQL][STREAMING] Added support in file stream source for reading new files added to subdirs

tdas
authored and
yhuai
committed

Commits on May 11, 2016

[SPARK-14476][SQL] Improve the physical plan visualization by adding meta info like table name and file path for data source.

clockfly
authored and
rxin
committed
[SPARK-15261][SQL] Remove experimental tag from DataFrameReader/Writer
rxin
committed
[SPARK-15250][SQL] Remove deprecated json API in DataFrameReader

HyukjinKwon
authored and
rxin
committed
[SPARK-15265][SQL][MINOR] Fix Union query error message indentation

dongjoon-hyun
authored and
rxin
committed
[SPARK-15255][SQL] limit the length of name for cached DataFrame

Davies Liu
authored and
rxin
committed
[SPARK-15246][SPARK-4452][CORE] Fix code style and improve volatile for

lianhuiwang
authored and
rxin
committed
[SPARK-15235][WEBUI] Corresponding row cannot be highlighted even though cursor is on the job on Web UI's timeline

sarutak
authored and
rxin
committed
[SPARK-15189][PYSPARK][DOCS] Update ml.evaluation PyDoc

holdenk
authored and
Nick Pentreath
committed
[SPARK-15231][SQL] Document the semantic of saveAsTable and insertInto and don't drop columns silently

zsxwing
authored and
yhuai
committed
[SPARK-15141][EXAMPLE][DOC] Update OneVsRest Examples

zhengruifeng
authored and
Nick Pentreath
committed
[SPARK-14340][EXAMPLE][DOC] Update Examples and User Guide for ml.BisectingKMeans

zhengruifeng
authored and
Nick Pentreath
committed
[SPARK-15149][EXAMPLE][DOC] update kmeans example

zhengruifeng
authored and
Nick Pentreath
committed
[SPARK-14976][STREAMING] make StreamingContext.textFileStream support wildcard

wei-mao-intel
authored and
srowen
committed
[SPARK-15238] Clarify supported Python versions

nchammas
authored and
srowen
committed
[SPARK-15150][EXAMPLE][DOC] Update LDA examples

zhengruifeng
authored and
Nick Pentreath
committed
[SPARK-14933][SQL] Failed to create view out of a parquet or orc table

xwu0226
authored and
liancheng
committed
[SPARK-15268][SQL] Make JavaTypeInference work with UDTRegistration

viirya
authored and
mengxr
committed
[SPARK-14933][HOTFIX] Replace `sqlContext` with `spark`.

dongjoon-hyun
authored and
yhuai
committed
[SPARK-15241] [SPARK-15242] [SQL] fix 2 decimal-related issues in RowEncoder

cloud-fan
authored and
davies
committed
[SPARK-15037] [SQL] [MLLIB] Part2: Use SparkSession instead of SQLContext in Python TestSuites

techaddict
authored and
davies
committed
[SPARK-15259] Sort time metric should not include spill and record insertion time

ericl
authored and
davies
committed
[SPARK-15085][STREAMING][KAFKA] Rename streaming-kafka artifact

koeninger
authored and
rxin
committed
[SPARK-15248][SQL] Make MetastoreFileCatalog consider directories from partition specs of a partitioned metastore table

tdas
authored and
yhuai
committed
[SPARK-15260] Atomically resize memory pools

Andrew Or
authored and
davies
committed
[SPARK-12200][SQL] Add __contains__ implementation to Row

Maciej Brynski
authored and
rxin
committed
[SPARK-15262] Synchronize block manager / scheduler executor state

Andrew Or
authored and
zsxwing
committed
[SPARK-15270] [SQL] Use SparkSession Builder to build a session with HiveSupport

techaddict
authored and
davies
committed
[SPARK-15278] [SQL] Remove experimental tag from Python DataFrame

rxin
authored and
davies
committed
[SPARK-15257][SQL] Require CREATE EXTERNAL TABLE to specify LOCATION

Andrew Or
authored and
yhuai
committed
[SPARK-15256] [SQL] [PySpark] Clarify DataFrameReader.jdbc() docstring

nchammas
authored and
davies
committed

Commits on May 12, 2016

[SPARK-15276][SQL] CREATE TABLE with LOCATION should imply EXTERNAL
Andrew Or
committed
[SPARK-15264][SPARK-15274][SQL] CSV Reader Error on Blank Column Names

Bill Chambers
authored and
Andrew Or
committed
[SPARK-15072][SQL][PYSPARK] FollowUp: Remove SparkSession.withHiveSupport in PySpark

techaddict
authored and
Andrew Or
committed
[SPARK-15080][CORE] Break copyAndReset into copy and reset

techaddict
authored and
cloud-fan
committed
[SPARK-14346] SHOW CREATE TABLE for data source tables

liancheng
authored and
yhuai
committed
[SPARK-15072][SQL][PYSPARK][HOT-FIX] Remove SparkSession.withHiveSupport from readwrite.py
yhuai
committed
[SPARK-15031][SPARK-15134][EXAMPLE][DOC] Use SparkSession and update indent in examples

zhengruifeng
authored and
Andrew Or
committed
[SPARK-15160][SQL] support data source table in InMemoryCatalog

cloud-fan
authored and
yhuai
committed
[SPARK-15281][PYSPARK][ML][TRIVIAL] Add impurity param to GBTRegressor & add experimental inside of regression.py

holdenk
authored and
Nick Pentreath
committed
[SPARK-15171][SQL] Deprecate registerTempTable and add dataset.createTempView

clockfly
authored and
liancheng
committed
[HOTFIX] SQL test compilation error from merge conflict
Andrew Or
committed
[SPARK-15094][SPARK-14803][SQL] Remove extra Project added in EliminateSerialization

viirya
authored and
yhuai
committed
[SPARK-14684][SPARK-15277][SQL] Partition Spec Validation in SessionCatalog and Checking Partition Spec Existence Before Dropping

gatorsmile
authored and
Andrew Or
committed
[SPARK-14897][SQL] upgrade to jetty 9.2.16

bomeng
authored and
srowen
committed
[SPARK-14421] Upgrades protobuf dependency to 2.6.1 for the new version of KCL, and…

boneill42
authored and
srowen
committed
[SPARK-10605][SQL] Create native collect_list/collect_set aggregates

hvanhovell
authored and
rxin
committed
[SPARK-13902][SCHEDULER] Make DAGScheduler not to create duplicate stage.

ueshin
authored and
kayousterhout
committed

Commits on May 13, 2016

[SPARK-15202][SPARKR] add dapplyCollect() method for DataFrame in SparkR.

sun-rui
authored and
shivaram
committed
[SPARK-15306][SQL] Move object expressions into expressions.objects package
rxin
committed
[SPARK-14541][SQL] Support IFNULL, NULLIF, NVL and NVL2

rxin
authored and
yhuai
committed
[SPARK-13866] [SQL] Handle decimal type in CSV inference at CSV data source.

HyukjinKwon
authored and
davies
committed
[SPARK-15188] Add missing thresholds param to NaiveBayes in PySpark

holdenk
authored and
MLnick
committed
[MINOR][PYSPARK] update _shared_params_code_gen.py

zhengruifeng
authored and
MLnick
committed
[SPARK-15181][ML][PYSPARK] Python API for GLR summaries.

sethah
authored and
MLnick
committed
[SPARK-13961][ML] spark.ml ChiSqSelector and RFormula should support other numeric types for label

BenFradet
authored and
MLnick
committed
[SPARK-15310][SQL] Rename HiveTypeCoercion -> TypeCoercion
rxin
committed
[SPARK-14900][ML] spark.ml classification metrics should include accuracy

wangmiao1981
authored and
srowen
committed
[SPARK-15061][PYSPARK] Upgrade to Py4J 0.10.1

holdenk
authored and
srowen
committed
[SPARK-12972][CORE] Update org.apache.httpcomponents.httpclient
srowen
committed
Revert "[SPARK-12972][CORE] Update org.apache.httpcomponents.httpclient"
srowen
committed
[SPARK-15267][SQL] Refactor options for JDBC and ORC data sources and change default compression for ORC

HyukjinKwon
authored and
rxin
committed

Commits on May 14, 2016

[TRIVIAL] Add () to SparkSession's builder function

tejasapatil
authored and
rxin
committed
[SPARK-15197][DOCS] Added Scaladoc for countApprox and countByValueApprox parameters

Nicholas Tietz
authored and
srowen
committed
[SPARK-15096][ML] LogisticRegression MultiClassSummarizer numClasses can fail if no valid labels are found

wangmiao1981
authored and
srowen
committed

Commits on May 16, 2016

[SPARK-15253][SQL] Support old table schema config key "spark.sql.sources.schema" for DESCRIBE TABLE

clockfly
authored and
cloud-fan
committed
[SPARK-15305][ML][DOC] spark.ml document Bisectiong k-means has the incorrect format

wangmiao1981
authored and
Nick Pentreath
committed
[SPARK-14979][ML][PYSPARK] Add examples for GeneralizedLinearRegression

yanboliang
authored and
Nick Pentreath
committed
[SPARK-14942][SQL][STREAMING] Reduce delay between batch construction and execution

lw-lin
authored and
zsxwing
committed

Commits on May 17, 2016

[SPARK-14906][ML] Copy linalg in PySpark to new ML package
mengxr
committed
[SPARK-12972][CORE] Update org.apache.httpcomponents.httpclient
srowen
committed
[SPARK-12972][CORE][TEST-MAVEN][TEST-HADOOP2.2] Update org.apache.httpcomponents.httpclient, commons-io
srowen
committed
[SPARK-15290][BUILD] Move annotations, like @Since / @DeveloperApi, into spark-tags
srowen
committed
[SPARK-15351][SQL] RowEncoder should support array as the external type for ArrayType
cloud-fan
committed
[SPARK-14434][ML] User guide doc and examples for GaussianMixture in spark.ml

wangmiao1981
authored and
Nick Pentreath
committed
[SPARK-15333][DOCS] Reorganize building-spark.md; rationalize vs wiki
srowen
committed
[SPARK-15318][ML][EXAMPLE] spark.ml Collaborative Filtering example does not work in spark-shell

wangmiao1981
authored and
srowen
committed
[SPARK-15165] [SQL] Codegen can break because toCommentSafeString is not actually safe

sarutak
authored and
davies
committed
[SPARK-14346][SQL][FOLLOW-UP] add tests for CREAT TABLE USING with partition and bucket

cloud-fan
authored and
yhuai
committed
[SPARK-10216][SQL] Avoid creating empty files during overwriting with group by query

HyukjinKwon
authored and
marmbrus
committed
[SPARK-15182][ML] Copy MLlib doc to ML: ml.feature.tf, idf

hhbyyh
authored and
Nick Pentreath
committed
[MINOR][DOCS] Replace remaining 'sqlContext' in ScalaDoc/JavaDoc.

dongjoon-hyun
authored and
Nick Pentreath
committed
[SPARK-14615][ML] Use the new ML Vector and Matrix in the ML pipeline based algorithms

DB Tsai
authored and
mengxr
committed
[SPARK-15244] [PYTHON] Type of column name created with createDataFrame is not consistent.

dongjoon-hyun
authored and
davies
committed
[SPARK-11735][CORE][SQL] Add a check in the constructor of SQLContext/SparkSession to make sure its SparkContext is not stopped

zsxwing
authored and
yhuai
committed
[SPARK-14346][SQL] Native SHOW CREATE TABLE for Hive tables/views

liancheng
authored and
yhuai
committed

Commits on May 18, 2016

[SPARK-14346] Fix scala-2.10 build
yhuai
committed
[SPARK-15171][SQL] Remove the references to deprecated method dataset.registerTempTable

clockfly
authored and
liancheng
committed
Prepare branch for 2.0.0-preview.
rxin
committed
Preparing Spark release 2.0.0-preview
pwendell
committed
Preparing development version 2.0.0-SNAPSHOT
pwendell
committed
[SPARK-14978][PYSPARK] PySpark TrainValidationSplitModel should support validationMetrics

taku-k
authored and
Nick Pentreath
committed
[SPARK-15334][SQL] HiveClient facade not compatible with Hive 0.12

clockfly
authored and
liancheng
committed
[SPARK-15307][SQL] speed up listing files for data source

Davies Liu
authored and
liancheng
committed
[SPARK-15322][MLLIB][CORE][SQL] update deprecate accumulator usage into accumulatorV2 in spark project

WeichenXu123
authored and
srowen
committed
[SPARK-15334][SQL][HOTFIX] Fixes compilation error for Scala 2.10
liancheng
committed
[SPARK-15346][MLLIB] Reduce duplicate computation in picking initial points

mouendless
authored and
srowen
committed
[SPARK-15357] Cooperative spilling should check consumer memory mode

Davies Liu
authored and
davies
committed
[MINOR][SQL] Remove unused pattern matching variables in Optimizers.

dongjoon-hyun
authored and
srowen
committed
[SPARK-15342] [SQL] [PYSPARK] PySpark test for non ascii column name does not actually test with unicode column name

viirya
authored and
davies
committed
[SPARK-14891][ML] Add schema validation for ALS
Nick Pentreath
committed
[SPARK-15373][WEB UI] Spark UI should show consistent timezones.

dongjoon-hyun
authored and
srowen
committed
[SPARK-15392][SQL] fix default value of size estimation of logical plan

Davies Liu
authored and
rxin
committed
[SPARK-15323][SPARK-14463][SQL] Fix reading of partitioned format=text datasets

jurriaan
authored and
rxin
committed

Commits on May 19, 2016

[SPARK-15192][SQL] null check for SparkSession.createDataFrame

cloud-fan
authored and
yhuai
committed
[SPARK-15297][SQL] Fix Set -V Command

gatorsmile
authored and
cloud-fan
committed
[SPARK-14463][SQL] Document the semantics for read.text
rxin
committed
[SPARK-15395][CORE] Use getHostString to create RpcAddress
zsxwing
committed
[SPARK-15381] [SQL] physical object operator should define reference correctly

cloud-fan
authored and
davies
committed
[SPARK-15031][EXAMPLES][FOLLOW-UP] Make Python param example working with SparkSession

HyukjinKwon
authored and
Nick Pentreath
committed
[SPARK-14939][SQL] Add FoldablePropagation optimizer

dongjoon-hyun
authored and
cloud-fan
committed
[SPARK-15362][ML] Make spark.ml KMeansModel load backwards compatible

yanboliang
authored and
jkbradley
committed
[SPARK-15292][ML] ML 2.0 QA: Scala APIs audit for classification

yanboliang
authored and
jkbradley
committed
[SPARK-14613][ML] Add @Since into the matrix and vector classes in spark-mllib-local

Pravin Gadakh
authored and
DB Tsai
committed
[SPARK-15390] fix broadcast with 100 millions rows

Davies Liu
authored and
davies
committed
[SPARK-14603][SQL][FOLLOWUP] Verification of Metadata Operations by Session Catalog

gatorsmile
authored and
Andrew Or
committed
[SPARK-15300] Fix writer lock conflict when remove a block

Davies Liu
authored and
Andrew Or
committed
[SPARK-15387][SQL] SessionCatalog in SimpleAnalyzer does not need to make database directory.

sarutak
authored and
Andrew Or
committed
[SPARK-15322][SQL][FOLLOW-UP] Update deprecated accumulator usage into accumulatorV2

HyukjinKwon
authored and
Andrew Or
committed
[SPARK-15316][PYSPARK][ML] Add linkPredictionCol to GeneralizedLinearRegression

holdenk
authored and
Nick Pentreath
committed
[SPARK-14346][SQL] Lists unsupported Hive features in SHOW CREATE TABLE output

liancheng
authored and
Andrew Or
committed
[SPARK-15317][CORE] Don't store accumulators for every task in listeners

zsxwing
authored and
Andrew Or
committed
[SPARK-15392][SQL] fix default value of size estimation of logical plan

Davies Liu
authored and
Andrew Or
committed
Fix the compiler error introduced by #13153 for Scala 2.10
zsxwing
committed
[SPARK-15411][ML] Add @since to ml.stat.MultivariateOnlineSummarizer.scala
DB Tsai
committed
[SPARK-15361][ML] ML 2.0 QA: Scala APIs audit for ml.clustering

yanboliang
authored and
jkbradley
committed

Commits on May 20, 2016

[SPARK-15414][MLLIB] Make the mllib,ml linalg type conversion APIs public

techaddict
authored and
mengxr
committed
[SPARK-15375][SQL][STREAMING] Add ConsoleSink to structure streaming

jerryshao
authored and
rxin
committed
[SPARK-15341][DOC][ML] Add documentation for "model.write" to clarify "summary" was not saved

yanboliang
authored and
mengxr
committed
[MINOR][ML][PYSPARK] ml.evaluation Scala and Python API sync

yanboliang
authored and
mengxr
committed
[SPARK-15416][SQL] Display a better message for not finding classes removed in Spark 2.0

zsxwing
authored and
marmbrus
committed
[SPARK-15296][MLLIB] Refactor All Java Tests that use SparkSession

techaddict
authored and
mengxr
committed
[SPARK-15321] Fix bug where Array[Timestamp] cannot be encoded/decoded correctly

smungee
authored and
cloud-fan
committed
[SPARK-11827][SQL] Adding java.math.BigInteger support in Java type inference for POJOs and Java collections

kevinyu98
authored and
cloud-fan
committed
[SPARK-15075][SPARK-15345][SQL] Clean up SparkSession builder and propagate config options to existing sessions if specified
rxin
committed
[SPARK-14990][SQL] Fix checkForSameTypeInputExpr (ignore nullability)
rxin
committed
[SPARK-14261][SQL] Memory leak in Spark Thrift Server

dosoft
authored and
rxin
committed
[SPARK-15057][GRAPHX] Remove stale TODO comment for making `enum` in GraphGenerators

dongjoon-hyun
authored and
rxin
committed
[DOC][MINOR] ml.feature Scala and Python API sync

BryanCutler
authored and
rxin
committed
[MINOR] Fix Typos

zhengruifeng
authored and
rxin
committed
[CORE][MINOR] Remove redundant set master in OutputCommitCoordinatorIntegrationSuite

techaddict
authored and
rxin
committed
[HOTFIX] Test compilation error from 52b967f
Andrew Or
committed
Revert "[HOTFIX] Test compilation error from 52b967f"
rxin
committed
Revert "[SPARK-15392][SQL] fix default value of size estimation of logical plan"

davies
authored and
rxin
committed
[SPARK-15313][SQL] EmbedSerializerInFilter rule should keep exprIds of output of surrounded SerializeFromObject.

ueshin
authored and
rxin
committed
[HOTFIX] Add back intended change from SPARK-15392
Andrew Or
committed
[SPARK-15335][SQL] Implement TRUNCATE TABLE Command

lianhuiwang
authored and
rxin
committed
[SPARK-15172][ML] Explicitly tell user initial coefficients is ignored when size mismatch happened in LogisticRegression

dding3
authored and
mengxr
committed
[SPARK-15363][ML][EXAMPLE] Example code shouldn't use VectorImplicits._, asML/fromML

wangmiao1981
authored and
mengxr
committed
[SPARK-15398][ML] Update the warning message to recommend ML usage

zhengruifeng
authored and
mengxr
committed
[SPARK-15394][ML][DOCS] User guide typos and grammar audit

sethah
authored and
mengxr
committed
[SPARK-15339][ML] ML 2.0 QA: Scala APIs and code audit for regression

yanboliang
authored and
mengxr
committed
[SPARK-15367][SQL] Add refreshTable back

gatorsmile
authored and
cloud-fan
committed
[SPARK-15421][SQL] Validate DDL property values
Andrew Or
committed
[SPARK-15417][SQL][PYTHON] PySpark shell always uses in-memory catalog
Andrew Or
committed