Move opendevstack from current "global" CD OCP namespace to opendevstack/ods namespace

[clemensutschig]

I gave this some thought - and the longer I think about it - the more I believe we should do this ..

1. it gives clarity
2. for a user during upgrade there is no difference between changing an image ref in the a components' jenkinsFile - or the entire path of the image used there (from cd/....:2.x to opendevstack/....3.x) and similar for jenkins / webhook / docgen dc
3. We could flip all project specific service account' pull rights to one rolebinding image puller system:authenticated so no impact for a user on that front
4. In case we ship published images, we have to create image streams pointing to those published images anyway in the global namespace, no matter if it's in the current 'CD' or the new opendevstack one..

On our side the change is massive (but the outcome is really nice, as it could be configurable via ods-configuration env properties) .. so people could actually have multiple namespaces, including -dev/test ones..

Changes needed:

1. quickstarters (jenkins slave references / master imagestream)
2. ods-core (all master image streams, scripts, role bindings for pull rights)
3. prov cd (jenkins image)
4. mro / doc gen service (imagestreams, jenkins slave images)

@michaelsauter @metmajer @gerardcl @sino92 - thoughts?

[michaelsauter]

Awesome. I'm totally with you to rename the cd namespace. I always thought the name is confusing (as it is to similar with *-cd namespaces). As a new name, I'd suggest ods as that is shorter, which makes it a better fit for URLs I think. But - best case we make this configurable and just default to ods or opendevstack :)

3.0 is a great time to do that change as prov-app, jenkins and webhook proxy will be moved into the central namespace, so people need to update URLs anyway.

Apart from that, I do not think the update for the user is that hard:

• change image stream pointers in their *-cd namespace and Jenkinsfiles
• change URL of SonarQube and Nexus in their Jenkins deployment

I believe your issue is actually two-fold:

1. Rename central namespace
2. Allow multiple central namespaces

I would vote to go forward with (1) for 3.0. I also love (2), but I think we could support this in a more general fashion. If all the scripts are parameterised, then users can instantiate central namespaces with whatever name they want. They could have -dev/-test, or namespaces with versions in them. So my suggestion would be to parameterise everything, and once that is in place, see what we want to offer - or if we say: if you want it, go ahead and do it but there is no official support.

FYI @oalyman

[michaelsauter]

FYI Image pulling is already allowed for system:authenticated in https://github.com/opendevstack/ods-core/blob/master/ods-setup/setup-ods-project.sh#L47.

[clemensutschig]

the big question, where to we build ...
Or solution rephrased -> do we then create a -dev / -test / -prod namespace as well - to allow building of non BC / Jenkins driven builds (prov app / doc gen service)

The other option ..
a) Development space: ods-dev
a) QA space: ods-test
a) Prod space: ods

Which would mean that we need BC / IS everywhere ... in this case (more thinking) - we could also just create 3 namespaces, each full fledged .

[michaelsauter]

I think I would still build prov-app and doc-gen in their own namespaces. Any central namespace should pull images either from a local namespace, or from Docker Hub. So that a central ods-test actually reflects the "production" setup.

[stitakis]

Not sure about doc-gen, but after the completion of the work of deploying ProvApp to cd namespace as docker image, I would vote to move the ProvApp deployment script (that provision prov-cd, prov-dev and prov-test) to the ProvApp repository.

[clemensutschig]

ok lets spell it out .. below is my understanding of what it would look like

opendevstack

1. Jenkins Master (bc, image and instance)
2. Jenkins Slaves (bc, images)
3. Provison Application (image and instance)
4. Webhook Proxy (bc, image and instance)
5. Document-Generation Service (image)
6. ODS Jenkins Shared Lib (bc / pipeline)
7. Sonarqube (bc / image / instance)
8. Nexus (bc / image / instance)

and the process for complex build images of ODS is to import image from github or build yourself for the below ones

ODS Provision App - ods-provision-app

1. -cd / -dev / -test
2. bc / image / cm ... & dc

ODS Document Generation - ods-doc-gen-svc

1. -cd / -dev / -test
2. bc / image

yes? If we do it this way - we have to make NICE project names .. please :)

[stitakis]

yes, looks good to me.

[michaelsauter]

Looks good, with the following questions / exceptions:

Point 6: Shared lib -> There is no BC/pipeline, right? Or what are you referring to? The shared lib is purely a Git reference.

For Prov App / Doc Gen, I would keep the current namespace setup. Below using the prov-app as an example, but the same applies for DocGen:

My view would be to have prov-cd, prov-dev, prov-test.

In my eyes, there is no prov-prod, because the production instance is located in the central opendevstack namespace. Also, I think having -cd/-dev/-test is a nice setup as it resembles a typical project setup.

Whether to name them (using the -dev namespace as an example) prov-dev, or prov-app-dev, or ods-prov-app-dev or ods-provisioning-app-dev ... I don't really mind as for me this is pure development :)

[clemensutschig]

Point 6: Shared lib -> There is no BC/pipeline, right? Or what are you referring to? The shared lib is purely a Git reference.

it does (we seed a pipeline for it to test) and once the jira plugin comes it will contain a build pipeline as well (jar)

[michaelsauter]

@clemensutschig Could the Jira plugin be published (like the prov app)? If so, that would be great.

[clemensutschig]

@michaelsauter - sure we can do so ... but nonetheless .. would we then create one more project for the plugin ods-jira-plugin-cd

[clemensutschig]

Ok cool - so I guess this is decided .. I vouch that we do this shortly before the release (ODS 3) in one shot, across all ODS components, ok? (including fixing app: labels, names .. etc)

opendevstack

1. Jenkins Master (bc, image and instance)
2. Jenkins Slaves (bc, images)
3. Provison Application (image and instance) - needs import from git/docker-hub script
4. Webhook Proxy (bc, image and instance)
5. Document-Generation Service (image) - needs import from git/docker-hub script
6. Sonarqube (bc / image / instance)
7. Nexus (bc / image / instance (+db))

and the process for complex build images of ODS / thru ods is to import image from github or build yourself for the below ones

ODS Provision App - ods-provision-app

1. -cd / -dev / -test
2. bc / image / cm ... & dc

ODS Document Generation - ods-doc-gen-svc

1. -cd / -dev / -test
2. bc / image

ODS Jira Plugin - ods-jira-project-plugin

1. -cd and upload to nexus?

ODS Jenkins Shared Library - ods-jenkins-shared-lib

1. -cd

**Open: ** we need to provide a solid path for contributions, no hacks,.. so we need to build a way for people to set up the above projects in case they want to develop... @michaelsauter fyi

[clemensutschig]

@oalyman and me gave them dev environments some thoughts ... current state of the union:

1. admin user provisions a new ods project (whichever name wanted) thru provision app
2. admin user provisions a NEW quickstarter (e.g. ODS Development Provision App)
   2.1 provision app creates the bb repo (as today)
   2.2 quickstarter jenkins job init's repo, adds github as remote, fetches, and creates a local master branch from github master (New quickstarter stage initRepoFromODS ods-jenkins-shared-library#268)
   2.3 quickstarter jenkins job pushes the local master upstream into the bb repo (created by step 2.1) - thru existing pushToRepo stage (quickstarters/pushToRemoteStage should support existing local git repo ods-jenkins-shared-library#265)

There are two overall ones - opendevstack/ods-jenkins-shared-library#267 and opendevstack/ods-jenkins-shared-library#266, ensuring we don't create unnecessary diffs for contribution upstream

And all ODS components that are 'quickstarterable' should ship their ocp config in the respective openshift directory, so we can easily pick it up in the odsQUickstarterCreateOpenshiftResources stage :)

Components:

1. prov app: Ship provision app ocp artifacts w/o project in openshift directory ods-provisioning-app#423
2. doc gen: Doc Gen service should shop its ocp config in openshift directory ods-document-generation-svc#20

Quickstarters.

1. New Quickstarters for ODS component contributions ods-quickstarters#261

And eventually with the mro merge we should also get the deploy/rollout thru tailor into the shared lib.. ;-)
@michaelsauter - comments?

[michaelsauter]

Two quick notes:

• Deploy using Tailor is anyway on my list for 3.0. I believe this is not related to MRO actually - even if you don't use the MRO, you should be able to deploy using your infrastructure-as-code approach
• I realised that the suggested approach has the downside of maintaining the ocp-config of the prov-app in two locations (ods-core for consumers, ods-provisioning-app for developers). I would love to take a little more time to think about this before we commit to early on a specific approach

[clemensutschig]

@michaelsauter - re (2) .. this is what @oalyman is currently working on. re (1) ... it was about having that feature set in hand when we merged (because it's already in mro :))

[stitakis]

@michaelsauter I think it is fine that the ocp-config of the prov-app is in 2 locations. The repo ods-core should include the configuration for consumers. The repo ods-provisioning-app should include the ocp-config for developer that want to deploy in openshift.