Skip to content

[SPARK-9928][SQL] Removal of LogicalLocalTable #9717

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gatorsmile wants to merge 9 commits into from
Closed

[SPARK-9928][SQL] Removal of LogicalLocalTable #9717

Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
01e4cdf
Merge remote-tracking branch 'upstream/master'
gatorsmile Nov 13, 2015
e25b378
Merge remote-tracking branch 'upstream/master' into LogicalLocalTable
gatorsmile Nov 14, 2015
7555b76
Merge branch 'master' of https://github.com/gatorsmile/spark into Log…
gatorsmile Nov 14, 2015
6835704
Merge remote-tracking branch 'upstream/master'
gatorsmile Nov 14, 2015
3a7d465
Merge remote-tracking branch 'upstream/master' into LogicalLocalTable
gatorsmile Nov 14, 2015
cb49f8c
Merge branch 'master' of https://github.com/gatorsmile/spark into Log…
gatorsmile Nov 14, 2015
9180687
Merge remote-tracking branch 'upstream/master'
gatorsmile Nov 14, 2015
45ef950
Merge branch 'master' of https://github.com/gatorsmile/spark into Log…
gatorsmile Nov 14, 2015
195d176
SPARK-9928
gatorsmile Nov 14, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 0 additions & 22 deletions sql/core/src/main/scala/org/apache/spark/sql/execution/ExistingRDD.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,25 +110,3 @@ private[sql] object PhysicalRDD {
PhysicalRDD(output, rdd, relation.toString, relation.isInstanceOf[HadoopFsRelation])
}
}

/** Logical plan node for scanning data from a local collection. */
private[sql]
case class LogicalLocalTable(output: Seq[Attribute], rows: Seq[InternalRow])(sqlContext: SQLContext)
extends LogicalPlan with MultiInstanceRelation {

override def children: Seq[LogicalPlan] = Nil

override def newInstance(): this.type =
LogicalLocalTable(output.map(_.newInstance()), rows)(sqlContext).asInstanceOf[this.type]

override def sameResult(plan: LogicalPlan): Boolean = plan match {
case LogicalRDD(_, otherRDD) => rows == rows
case _ => false
}

@transient override lazy val statistics: Statistics = Statistics(
// TODO: Improve the statistics estimation.
// This is made small enough so it can be broadcasted.
sizeInBytes = sqlContext.conf.autoBroadcastJoinThreshold - 1
)
}