Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow local filesystem state stores (to aid CI pull-request workflows) #6465

Merged
merged 2 commits into from
Jan 17, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 18 additions & 9 deletions docs/state.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,14 @@ though we have to copy the data from the state store to a file where components

The state store uses kops's VFS implementation, so can in theory be stored anywhere.
As of now the following state stores are supported:

* Amazon AWS S3 (s3://)
* local filesystem (file://)
* Digital Ocean (do://)
* Amazon AWS S3 (`s3://`)
* local filesystem (`file://`) (only for dry-run purposes, see [note](#local-filesystem-state-stores) below)
* Digital Ocean (`do://`)
* MemFS (memfs://)
* Google Cloud (gs://)
* Kubernetes (k8s://)
* OpenStack Swift (swift://)
* AliCloud (oss://)
* Google Cloud (`gs://`)
* Kubernetes (`k8s://`)
* OpenStack Swift (`swift://`)
* AliCloud (`oss://`)

The state store is just files; you can copy the files down and put them into git (or your preferred version control system).

Expand Down Expand Up @@ -47,6 +46,16 @@ There are a few ways to configure your state store. In priority order:
+ config file `$HOME/.kops.yaml`
+ config file `$HOME/.kops/config`

## Local filesystem state stores

The local filesystem state store (`file://`) is **not** functional for running clusters. It is permitted so as to enable review workflows.

For example, in a review workflow, it can be desirable to check a set of untrusted changes before they are applied to real infrastructure. If submitted untrusted changes to configuration files are naively run by `kops replace`, then Kops would overwrite the state store used by production infrastructure with changes which have not yet been approved. This is dangerous.

Instead, a review workflow may download the contents of the state bucket to a local directory (using `aws s3 sync` or similar), set the state store to the local directory (e.g. `--state file:///path/to/state/store`), and then run `kops replace` and `kops update` (but for a dry-run only - _not_ `kops update --yes`). This allows the review process to make changes to a local copy of the state bucket, and check those changes, without touching the production state bucket or production infrastructure.

Trying to use a local filesystem state store for real (i.e. `kops update --yes`) clusters will not work since the Kubernetes nodes in the cluster need to be able to read from the same state bucket, and the local filesystem will not be mounted to all of the Kubernetes nodes. In theory, a cluster administrator could put the state store on a shared NFS volume that is mounted to the same directory on each of the nodes; however, that use case is not supported as of yet.

### Configuration file example:

`$HOME/.kops/config` might look like this:
Expand Down Expand Up @@ -171,4 +180,4 @@ if err != nil {

gcsClient, err := storage.New(httpClient)

```
```
5 changes: 1 addition & 4 deletions util/pkg/vfs/vfs.go
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ func IsClusterReadable(p Path) bool {
}

switch p.(type) {
case *S3Path, *GSPath, *SwiftPath, *OSSPath:
case *S3Path, *GSPath, *SwiftPath, *OSSPath, *FSPath:
return true

case *KubernetesPath:
Expand All @@ -111,9 +111,6 @@ func IsClusterReadable(p Path) bool {
case *SSHPath:
return false

case *FSPath:
return false

case *MemFSPath:
return false

Expand Down