Skip to content
This repository has been archived by the owner on May 22, 2020. It is now read-only.

sharing infrastruture between deployments #123

Closed
mikedanese opened this issue Jun 20, 2016 · 15 comments
Closed

sharing infrastruture between deployments #123

mikedanese opened this issue Jun 20, 2016 · 15 comments

Comments

@mikedanese
Copy link
Contributor

mikedanese commented Jun 20, 2016

There is a certain amount of tooling that will benefit all deployment automations. We should discuss how to develop this tools in a way that they benefit as many of the maintained deployments as possible. Some work that I can think of that would benefit all deployments:

Obviously, some of these have higher relative priority than others. Let's use this issue to track the deployment shared infrastructure effort. Let me know if there are items on this list that are missing.

cc @luxas @errordeveloper @justinsb

@justinsb
Copy link
Contributor

That list looks like a great place to start :-)

For the standardized node container image, did you mean to link to #40?

@mikedanese
Copy link
Contributor Author

Nope, meant to link to #34, thanks.

@mikedanese
Copy link
Contributor Author

@aaronlevy @philips are CoreOS folks interested in claiming any of these items for v1.4 (that you haven't already claimed). Is there anything you want to add to this list?

@mikedanese
Copy link
Contributor Author

@derekwaynecarr
Copy link
Contributor

I am not sure I follow the goal with #34 , are you proposing rkt is the foundation run-time to bootstrap a node?

@aronchick
Copy link

Can you make it clear - is this the single repo? Or something else? Can we please find one repo, and kill all the rest?

@mikedanese
Copy link
Contributor Author

mikedanese commented Jun 23, 2016

@derekwaynecarr there is no dependency on rkt although rkt can easily run that image. There's actually a minimal implemntation of what rkt fly does here the prepares the chroot to be run with systemd and the RootDirectory directive. The idea is to package kubelet and docker and all the various dependencies to standardize the execution environment across deployments. Obviously it would be optional to use this as the basis of your deployment automation but could help. #34 is a prototype of how to build such an artifact but I'm open to other mechanisms. Also i haven't gotten it to work yet due to a docker bug.

@aaronlevy
Copy link

As far as "easier self-hosted control plane boostraping" -- what does this encompass in your view?

We have been working on https://github.com/coreos/bootkube as a means of demonstrating / running self-hosted control planes -- but I think a lot of this functionality could be removed (or simplified) given a writeable kubelet api (submit pods directly to kubelet which will be rectified with apiserver state/objects once available).

As far as checkpointing we would like to start helping with that work. In the self-hosted scenario this is important for a few reasons:

  • Checkpointing configuration on a self-hosted kubelet means the "on-host" kubelet can track the configuration of the self-hosted copy so they don't become skewed. (e.g. I deploy initial flag sets, but over time these change. I would like to checkpoint this new configuration locally so I can recover state without access to an api-server. This is also true of a (non-selfhosted) kubelet which updates-in-place or eventually uses an api-object for its configuration (initial configuration should be replaced by checkpointed configuration that can be used prior to contacting apiserver).
  • If all api-servers die simultaneously, there is no automated recovery. With checkpointing of pods (in this case an api-server) even if you only have a single api-server -- you can recover from local state.

However, I brought up checkpointing in sig-node and that CoreOS would like to start at least scoping out what this work would look like -- and it didn't seem like there was much interest in this. Maybe @vishh can provide some thoughts?

Alternative to doing this work upstream, we've been working on "user-space" checkpointing as a stop gap (making use of side-car containers / dumping flags to disk / podmaster like functionality).

One other thing that is still on the radar is multi-apiserver support in kubelet + kube-proxy. Lower priority if we can get checkpointing in place, but something still to consider.

@bgrant0607
Copy link

cc @kubernetes/sig-cluster-lifecycle

@bgrant0607
Copy link

@aronchick We're moving towards lots of repos. That's github's reality. However, we are trying to merge redundant efforts, which I assume was your main concern.

@bgrant0607
Copy link

@aronchick
Copy link

@bgrant0607 sorry, yes - you are correct. The latter was my issue - as per the community meeting today, I think we all agree we should just choose one getting started center point for work, and focus our efforts there. I still don't understand why there's both kube-deploy and kube-anywhere, weren't those efforts already merged?

@mikedanese
Copy link
Contributor Author

@aronchick min-turnup and kubernetes-anywhere were merged and min-turnup was removed from kube-deploy. kube-deploy holds other projects. It's the contrib for deployment automation.

@vishh
Copy link
Contributor

vishh commented Jun 24, 2016

@aaronlevy Take a look at kubernetes/kubernetes#27980 . That should hopefully solve the config management issues wrt. self-hosting.

@derekwaynecarr @mikedanese Just to clarify, rkt will be a transient dependency for bootstrapping just the kubelet right? #34 (comment)

@roberthbailey
Copy link
Contributor

@mikedanese - many of the items in your initial list are now under development and I'm not sure this issue is useful any longer. Can we close as obsolete?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants