Skip to content
This repository has been archived by the owner on Jan 9, 2020. It is now read-only.

Develop and build Kubernetes modules in isolation without the other Spark modules #570

Open
wants to merge 7 commits into
base: branch-2.2-kubernetes
Choose a base branch
from

Conversation

echarles
Copy link
Member

What changes were proposed in this pull request?

As a developer, I like to load in my IDE and to go via the command line to the module I am working on without having to deal with the other modules. So far, with Spark K8S, we need to build from the root of the module hierarchy.

I have introduced:

  • A maven pom parent for kubernetes
  • The needed jetty dependencies to allow the build of the core in isolation/
  • The copy of bin, sbin and conf from the root the the integration-tests.

How was this patch tested?

So far, with those changes, I can successfully build and test any module or combination of modules (install, test, integration-test)

@mccheah
Copy link

mccheah commented Dec 14, 2017

Not too familiar with Maven, but I think this makes sense.

@mccheah
Copy link

mccheah commented Dec 14, 2017

Do we have a precedence for this, like in another module perhaps?

@echarles
Copy link
Member Author

A lot of maven projects use that constructs that give 2 advantages:

  1. You can build, test and load in the IDE a subset of your module without the rest (let's say here the modules in resource-manager/kuberunetes).
  2. You can define e.g. build plugins, profiles... that apply only to your sub-modules.

I just realize that the Spark code base don't use that construct where it could, so I am afraid someone would come and say it is not inline with the Spark project habits.

Also, this PR integrate some definitions to run the integration-tests from its directory. As a separate repository is going to be created, this may be less interesting...

@ssuchter
Copy link
Member

ssuchter commented Feb 5, 2018

retest this please

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants