Skip to content

[SPARK-7850][BUILD] Hive 0.12.0 profile in POM should be removed #6393

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
piaozhexiu wants to merge 3 commits into from
Closed

[SPARK-7850][BUILD] Hive 0.12.0 profile in POM should be removed #6393

piaozhexiu wants to merge 3 commits into from

Conversation

@piaozhexiu
Copy link

@piaozhexiu piaozhexiu commented May 25, 2015

I grep'ed hive-0.12.0 in the source code and removed all the profiles and doc references.

@srowen
Copy link
Member

srowen commented May 25, 2015

ok to test

@srowen
Copy link
Member

srowen commented May 25, 2015

I'm OK with that if this is true and the intent, but then do we need hive-0.13.1 anymore either?

@rxin
Copy link
Contributor

rxin commented May 25, 2015

I think this is the intent - but I will let @marmbrus confirm.

@witgo
Copy link
Contributor

witgo commented May 25, 2015

The code is useless ?
https://github.com/apache/spark/tree/master/sql/hive

@SparkQA
Copy link

SparkQA commented May 25, 2015

Test build #33461 has finished for PR 6393 at commit d5342cd.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@chenghao-intel
Copy link
Contributor

chenghao-intel commented May 26, 2015

We probably also need to get rid of the HiveShim code as well in hive/hive-thriftserver modules.

@piaozhexiu
Copy link
Author

piaozhexiu commented May 26, 2015

@chenghao-intel thank you for pointing it out. I deleted 0.12 shim code.

@SparkQA
Copy link

SparkQA commented May 26, 2015

Test build #33491 has finished for PR 6393 at commit 54d8dee.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@marmbrus
Copy link
Contributor

marmbrus commented May 27, 2015

@srowen, we should get rid of -Phive-0.13.1 too. The build should only ever need to support one version of hive from now on. Another part of this task (would be nice to do now, but does not have to happen in this PR) is to inline the Hive13 shim, so we don't have this extra indirection anymore SPARK-6909.

@piaozhexiu
Copy link
Author

piaozhexiu commented May 27, 2015

@marmbrus thanks for the confirmation. I removed -Phive-0.13.1 now.

Regarding SPARK-6909, can we do it in a separate PR? I'm happy to do it if no one is planning to do it.

@SparkQA
Copy link

SparkQA commented May 27, 2015

Test build #33556 timed out for PR 6393 at commit fb429ce after a configured wait of 150m.

@rxin
Copy link
Contributor

rxin commented May 27, 2015

Jenkins, retest this please.

@rxin
Copy link
Contributor

rxin commented May 27, 2015

@piaozhexiu looks good. I will merge once tests come back positive. It'd be great if you have some cycles to help out with SPARK-6909 to move the Hive 13 stuff into the hive module.

@piaozhexiu
Copy link
Author

piaozhexiu commented May 27, 2015

@rxin thank you! I'll look into SPARK-6909 this week.

@SparkQA
Copy link

SparkQA commented May 27, 2015

Test build #33563 has finished for PR 6393 at commit fb429ce.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@rxin
Copy link
Contributor

rxin commented May 27, 2015

Thanks. I've merged this in master & 1.4.

@asfgit asfgit closed this in 6dd6458 May 27, 2015
asfgit pushed a commit that referenced this pull request May 27, 2015
@rxin
[SPARK-7850][BUILD] Hive 0.12.0 profile in POM should be removed
4e12cec 
I grep'ed hive-0.12.0 in the source code and removed all the profiles and doc references.

Author: Cheolsoo Park <cheolsoop@netflix.com>

Closes #6393 from piaozhexiu/SPARK-7850 and squashes the following commits:

fb429ce [Cheolsoo Park] Remove hive-0.13.1 profile
82bf09a [Cheolsoo Park] Remove hive 0.12.0 shim code
f3722da [Cheolsoo Park] Remove hive-0.12.0 profile and references from POM and build docs

(cherry picked from commit 6dd6458)
Signed-off-by: Reynold Xin <rxin@databricks.com>
@piaozhexiu piaozhexiu mentioned this pull request Jun 3, 2015
Closed
asfgit pushed a commit that referenced this pull request Jun 4, 2015
@rxin
[SPARK-6909][SQL] Remove Hive Shim code
0526fea 
This is a follow-up on #6393. I am removing the following files in this PR.
```
./sql/hive/v0.13.1/src/main/scala/org/apache/spark/sql/hive/Shim13.scala
./sql/hive-thriftserver/v0.13.1/src/main/scala/org/apache/spark/sql/hive/thriftserver/Shim13.scala
```
Basically, I re-factored the shim code as follows-
* Rewrote code directly with Hive 0.13 methods, or
* Converted code into private methods, or
* Extracted code into separate classes

But for leftover code that didn't fit in any of these cases, I created a HiveShim object. For eg, helper functions which wrap Hive 0.13 methods to work around Hive bugs are placed here.

Author: Cheolsoo Park <cheolsoop@netflix.com>

Closes #6604 from piaozhexiu/SPARK-6909 and squashes the following commits:

5dccc20 [Cheolsoo Park] Remove hive shim code
jeanlyn pushed a commit to jeanlyn/spark that referenced this pull request Jun 12, 2015
@jeanlyn
[SPARK-7850][BUILD] Hive 0.12.0 profile in POM should be removed
f84d182 
I grep'ed hive-0.12.0 in the source code and removed all the profiles and doc references.

Author: Cheolsoo Park <cheolsoop@netflix.com>

Closes apache#6393 from piaozhexiu/SPARK-7850 and squashes the following commits:

fb429ce [Cheolsoo Park] Remove hive-0.13.1 profile
82bf09a [Cheolsoo Park] Remove hive 0.12.0 shim code
f3722da [Cheolsoo Park] Remove hive-0.12.0 profile and references from POM and build docs
jeanlyn pushed a commit to jeanlyn/spark that referenced this pull request Jun 12, 2015
@jeanlyn
[SPARK-6909][SQL] Remove Hive Shim code
2519f45 
This is a follow-up on apache#6393. I am removing the following files in this PR.
```
./sql/hive/v0.13.1/src/main/scala/org/apache/spark/sql/hive/Shim13.scala
./sql/hive-thriftserver/v0.13.1/src/main/scala/org/apache/spark/sql/hive/thriftserver/Shim13.scala
```
Basically, I re-factored the shim code as follows-
* Rewrote code directly with Hive 0.13 methods, or
* Converted code into private methods, or
* Extracted code into separate classes

But for leftover code that didn't fit in any of these cases, I created a HiveShim object. For eg, helper functions which wrap Hive 0.13 methods to work around Hive bugs are placed here.

Author: Cheolsoo Park <cheolsoop@netflix.com>

Closes apache#6604 from piaozhexiu/SPARK-6909 and squashes the following commits:

5dccc20 [Cheolsoo Park] Remove hive shim code
nemccarthy pushed a commit to nemccarthy/spark that referenced this pull request Jun 19, 2015
@nemccarthy
[SPARK-7850][BUILD] Hive 0.12.0 profile in POM should be removed
e40ab61 
I grep'ed hive-0.12.0 in the source code and removed all the profiles and doc references.

Author: Cheolsoo Park <cheolsoop@netflix.com>

Closes apache#6393 from piaozhexiu/SPARK-7850 and squashes the following commits:

fb429ce [Cheolsoo Park] Remove hive-0.13.1 profile
82bf09a [Cheolsoo Park] Remove hive 0.12.0 shim code
f3722da [Cheolsoo Park] Remove hive-0.12.0 profile and references from POM and build docs
nemccarthy pushed a commit to nemccarthy/spark that referenced this pull request Jun 19, 2015
@nemccarthy
[SPARK-6909][SQL] Remove Hive Shim code
4e89417 
This is a follow-up on apache#6393. I am removing the following files in this PR.
```
./sql/hive/v0.13.1/src/main/scala/org/apache/spark/sql/hive/Shim13.scala
./sql/hive-thriftserver/v0.13.1/src/main/scala/org/apache/spark/sql/hive/thriftserver/Shim13.scala
```
Basically, I re-factored the shim code as follows-
* Rewrote code directly with Hive 0.13 methods, or
* Converted code into private methods, or
* Extracted code into separate classes

But for leftover code that didn't fit in any of these cases, I created a HiveShim object. For eg, helper functions which wrap Hive 0.13 methods to work around Hive bugs are placed here.

Author: Cheolsoo Park <cheolsoop@netflix.com>

Closes apache#6604 from piaozhexiu/SPARK-6909 and squashes the following commits:

5dccc20 [Cheolsoo Park] Remove hive shim code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@piaozhexiu @srowen @rxin @witgo @SparkQA @chenghao-intel @marmbrus