Kubeadm should have provision to BUILD and install k8s from git url

[mikedanese]

From @sandys on November 12, 2016 8:17

I would also request that kubeadm have the ability to build and install k8s. At the pace that k8s is changing...we would like to dogfood a lot of upcoming k8s changes. If kubeadm can be pointed to a git URL and have it build that k8s version and use it for install/upgrade...that would be a godsend.

For example, we have been blocking on several changes in ingress controllers (for ssl passthrough) and waiting for packaging takes too long. We would love to be able to build and install in one step.

Please note, I'm not referring to building k8s inside a single docker image. I'm talking about building and installing components of k8s (using kubeadm) by pointing to a git url (branch/tag/version/etc) and having the install and upgrade to work

Copied from original issue: kubernetes/kubernetes#36697

[mikedanese]

From @davidchua on November 13, 2016 8:44

👍

I too would like to be able to do just that. It would help greatly to be able to test upcoming kubernetes versions before hand.

[mikedanese]

From @luxas on November 15, 2016 16:39

This is something we probably are going to do, at least for testing.
Follow along when we have made the kubernetes/kubeadm project, which I hope will happen soon.

Thanks for the issue 👍, I strongly think we need something like this.

[mikedanese]

From @sandys on November 15, 2016 16:42

Thanks for the reply!
A lot of us are not sure on the timelines for this. Will this happen in time for 1.5 ? Or will this be later...in which case would it make sense to have this functionality fairly quickly .

Sorry for being a bother.

[mikedanese]

From @luxas on November 15, 2016 16:47

This will be something we hopefully will do in time for v1.6.
The v1.5 window has already passed some weeks ago.

Please take a look at this document to get an understanding of what we will prioritize in time for v1.6:
https://docs.google.com/document/d/1v4k7eN-eetHbbpTAG7WuioRAMsK_FVGi1yC3VIn4Inw/edit#

[sandys]

@mikedanese @luxas if I could ask a question... is this statement still valid, now that kubeadm is split into its own repository ?

This will be something we hopefully will do in time for v1.6.
The v1.5 window has already passed some weeks ago.

Will kubeadm not have its own release cycle now ... or will you guys only release when k8s core gets released. Because we are really looking forward to this feature (which allow us to scale up our k8s experimentation).

Additionally, this was announced today - https://github.com/Mirantis/kubeadm-dind-cluster (https://groups.google.com/forum/#!topic/kubernetes-sig-cluster-lifecycle/xmRL_vVR_64)

Moreover, it’s now possible to work with remote Docker engine without using any host directories, pulling k8s binaries directly from build data container. Cluster control script also has additional subcommand for running e2e tests. The script also takes care of building k8s binaries necessary for the cluster.

Perhaps this code can be leveraged here...

[zen]

Guys, I have an impression that we are building systemd here. Why not leverage other tools for that, but still keeping code separation. I understand new features are cool, but kubeadm mission originally was not to build software.

[ivan4th]

kubeadm-dind-cluster almost does this, BTW. It builds k8s from local source and starts a Docker-in-Docker cluster using kubeadm and the binaries it just built, so 'pointing to git url' can be achieved with a shell one-liner, or I can add it to the script without noticeable effort. It's not "completely real" cluster, but still a good testing ground for k8s and kubeadm. I'll update the project shortly to use out-of-tree kubeadm.

See also the discussion about moving the project to the incubator.

[sandys]

@zen IMHO building is launching a build container. i dont think we are advocating adding a compiler to kubeadm

[mikedanese]

We publish builds from master hourly which will be usable by kubeadm. I don't think refactoring or build system to build locally on deploys is an effective use of time or in scope for kubeadm. hourly builds solves your dogfood requirement.

[luxas]

@mikedanese Yes

@zen I was never talking about building anything on-host, just creating some scripts that would be usable for our CI that would download and test latest k8s core head against the latest kubeadm version.

[sandys]

this is fine.. if push comes to shove, i can use the same infrastructure to
point to my own builds.

however i would like to reiterate (and given the excitement around the dind
project above) that we as early adopters would love a seamless way to build
k8s and deploy it.

On Nov 23, 2016 9:28 PM, "Mike Danese" notifications@github.com wrote:

We publish builds from master hourly which will be usable by kubeadm. I
don't think refactoring or build system to build locally on deploys is an
effective use of time out in scope for kubeadm. It solves your dogfood
requirement.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#3 (comment),
or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAEsU5S_Kaf706PAKaLfhD5YG3GHzihqks5rBGK7gaJpZM4K5uw7
.

[v1k0d3n]

this is huge! i'm a major fan of this approach. currently we're building our own kube-controller and having to update the manifests after kubeadm for some rdb content in 1.5.0beta1. would love to see this feature added.

[luxas]

I'm using a bash script for this task, it works well:

#!/bin/bash

CI_COMMIT=${CI_COMMIT:-latest}
ARCH=${ARCH:-amd64}

if [[ ${ARCH} == "amd64" ]]; then
	ARCH_EXT=""
else
	ARCH_EXT="-${ARCH}"
fi

OLD_REGISTRY=${OLD_REGISTRY:-gcr.io/google_containers}
NEW_REGISTRY=${NEW_REGISTRY:-${OLD_REGISTRY}}

if [[ ${CI_COMMIT} == "latest" ]]; then
	CI_COMMIT=$(curl -sSL https://storage.googleapis.com/kubernetes-release-dev/ci-cross/latest.txt)
fi

CI_COMMIT_TAG=$(echo ${CI_COMMIT} | sed "s/+/-/")

echo "Using commit: ${CI_COMMIT}"
echo "Tagging to registry: ${NEW_REGISTRY}"

images=(
	kube-apiserver
	kube-scheduler
	kube-controller-manager
	kube-proxy
)

for image in ${images[@]}; do
	curl -sSL https://storage.googleapis.com/kubernetes-release-dev/ci-cross/${CI_COMMIT}/bin/linux/${ARCH}/${image}.tar | docker load
	image_tag=$(curl -sSL https://storage.googleapis.com/kubernetes-release-dev/ci-cross/${CI_COMMIT}/bin/linux/${ARCH}/${image}.md5)
	docker tag ${OLD_REGISTRY}/${image}${ARCH_EXT}:${image_tag} ${NEW_REGISTRY}/${image}-${ARCH}:${CI_COMMIT_TAG}
	docker rmi ${OLD_REGISTRY}/${image}${ARCH_EXT}:${image_tag}
	echo "Image created: ${NEW_REGISTRY}/${image}-${ARCH}:${CI_COMMIT_TAG}"
done

cat <<EOF
Now you can run:
	kubeadm init --use-kubernetes-version ${CI_COMMIT_TAG}
EOF

[luxas]

See also: #101

[luxas]

You can use Mirantis project for this: https://github.com/Mirantis/kubeadm-dind-cluster
Or use my script

Or #237 will fix this...