[SPARK-7481] [build] Add spark-cloud module to pull in object store access, + documentation by steveloughran · Pull Request #12004 · apache/spark

[SPARK-7481] stripped down packaging only module

steveloughran

committed

[SPARK-7481] basic instantiation tests verify that dependency hadoop-azure, hadoop-aws, hadoop-openstack and implicitly their transitive dependencies are resolved. They don't verify all dependency …

steveloughran

committed

[SPARK-7481] tests restricted to instantiation; logging modified appropriately

steveloughran

committed

[SPARK-7481] declare httpcomponents:httpclient explicitly, as downstream tests which pulled in spark-cloud but not spark-hive were ending up with inconsistent versions. Add a test for the missing c…

steveloughran

committed

[SPARK-7481] update docs by culling section on cloud integration tests; link to remaning docs from top level.

steveloughran

committed

[SPARK-7481] updated documentation as per review

steveloughran

committed

[SPARK-7481] SBT will build this now, optionally

steveloughran

committed

[SPARK-7481] cloud POM includes jackson-dataformat-cbor, so that the CP is set up consistently for the later versions of the AWS SDK

steveloughran

committed

[SPARK-7481] rebase with master; Pom had got out of sync

steveloughran

committed

[SPARK-7481] rename spark-cloud module to spark-hadoo-cloud, in POMs and docs

steveloughran

committed

[SPARK-7841] bump up cloud pom to 2.2.0-SNAPSHOT; other minor pom cleanup

steveloughran

committed

[SPARK-7481] builds against Hadoop shaded 3.x clients failing as direct references to AWS classes failing. Cut them and rely on transitive load through FS class instantation to force the load. All …

steveloughran

committed

[SPARK-7481] update 2.7 dependencies to include azure, aws and openstack JARs, transitive dependencies on aws and azure SDKs

steveloughran

committed

[SPARK-7481] add joda time as the dependency. Tested against hadoop branch-2, s3 ireland

steveloughran

committed

SPARK-7481 purge all tests from the cloud module

steveloughran

committed

SPARK-7481 add cloud module to sbt sequence

steveloughran

committed

SPARK-7481 break line of mvn XML declaration

steveloughran

committed

SPARK-7481 cloud pom is still JAR (not pom). works against Hadoop 2.6 as well as 2.7, keeping azure the 2.7.x dependency. All dependencies are scoped @ hadoop.scope

steveloughran

committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[SPARK-7481] [build] Add spark-cloud module to pull in object store access, + documentation#12004

[SPARK-7481] [build] Add spark-cloud module to pull in object store access, + documentation#12004
steveloughran wants to merge 18 commits intoapache:masterfrom
steveloughran:features/SPARK-7481-cloud

Commits on Mar 20, 2017

[SPARK-7481] stripped down packaging only module

[SPARK-7481] basic instantiation tests verify that dependency hadoop-azure, hadoop-aws, hadoop-openstack and implicitly their transitive dependencies are resolved. They don't verify all dependency …

[SPARK-7481] tests restricted to instantiation; logging modified appropriately

[SPARK-7481] declare httpcomponents:httpclient explicitly, as downstream tests which pulled in spark-cloud but not spark-hive were ending up with inconsistent versions. Add a test for the missing c…

[SPARK-7481] update docs by culling section on cloud integration tests; link to remaning docs from top level.

[SPARK-7481] updated documentation as per review

[SPARK-7481] SBT will build this now, optionally

[SPARK-7481] cloud POM includes jackson-dataformat-cbor, so that the CP is set up consistently for the later versions of the AWS SDK

[SPARK-7481] rebase with master; Pom had got out of sync

[SPARK-7481] rename spark-cloud module to spark-hadoo-cloud, in POMs and docs

[SPARK-7841] bump up cloud pom to 2.2.0-SNAPSHOT; other minor pom cleanup

[SPARK-7481] builds against Hadoop shaded 3.x clients failing as direct references to AWS classes failing. Cut them and rely on transitive load through FS class instantation to force the load. All …

[SPARK-7481] update 2.7 dependencies to include azure, aws and openstack JARs, transitive dependencies on aws and azure SDKs

[SPARK-7481] add joda time as the dependency. Tested against hadoop branch-2, s3 ireland

SPARK-7481 purge all tests from the cloud module

SPARK-7481 add cloud module to sbt sequence

SPARK-7481 break line of mvn XML declaration

SPARK-7481 cloud pom is still JAR (not pom). works against Hadoop 2.6 as well as 2.7, keeping azure the 2.7.x dependency. All dependencies are scoped @ hadoop.scope

[SPARK-7481] [build] Add spark-cloud module to pull in object store access, + documentation#12004steveloughran wants to merge 18 commits intoapache:masterapache/spark:masterfrom steveloughran:features/SPARK-7481-cloudsteveloughran/spark:features/SPARK-7481-cloudCopy head branch name to clipboard

Commits

Commits on Mar 20, 2017

[SPARK-7481] [build] Add spark-cloud module to pull in object store access, + documentation#12004
steveloughran wants to merge 18 commits intoapache:masterfrom
steveloughran:features/SPARK-7481-cloud