Add kv subcommand and CLI #2360

sethvargo · 2016-09-26T15:17:07Z

This implements the kv subcommand CLI for reading, writing, and deleting keys. This pass does not support the transaction-based keys yet, but that could be implemented in a later pass. The framework is in place for adding new commands and formats pretty easily.

Usage: consul kv <subcommand> [options] [args]

  This command has subcommands for interacting with Consul's key-value
  store. Here are some simple examples, and more detailed examples are
  available in the subcommands or the documentation.

  Create or update the key named "redis/config/connections" with the value "5":

      $ consul kv put redis/config/connections 5

  Read this value back:

      $ consul kv get redis/config/connections

  Or get detailed key information:

      $ consul kv get -detailed redis/config/connections

  Finally, delete the key:

      $ consul kv delete redis/config/connections

  For more examples, ask for subcommand help or view the documentation.

Subcommands:

    delete    Removes data from the KV store
    get       Retrieves or lists data from the KV store
    put       Sets or updates data in the KV store

/cc @slackpad @ryanuber @armon

sethvargo · 2016-09-26T16:25:14Z

The failing test seems unrelated and is also failing on master:

--- FAIL: TestHealthChecksInState_DistanceSort (1.06s)
    health_endpoint_test.go:143: bad: [0xc822823b00 0xc821852630]

slackpad · 2016-09-26T20:24:12Z

api/kv.go

+	// base64 encoded upon transport.
+	Value []byte
+
+	// Session is a string representing the name of the session. Any other


s/name/ID/g

slackpad · 2016-09-26T20:25:25Z

api/kv.go

 	CreateIndex uint64
+
+	// ModifyIndex is used for the Check-And-Set operations.


I'd change this to:

ModifyIndex is used for the Check-And-Set operations and can also be fed back into the WaitIndex of the QueryOptions in order to perform blocking queries.

slackpad · 2016-09-26T20:38:42Z

command/kv_delete.go

+
+  -modify-index=<int>     Unsigned integer representing the ModifyIndex of the
+                          key. This is often combined with the -cas flag, but it
+                          can be specified for any key. The default value is 0.


Consul will ignore this on any other key operations (you can't manually set ModifyIndex) so I'd just change this to:

This is used in combination with the -cas flag.

You might even be able to just do -cas= but that might not be very intuitive.

slackpad · 2016-09-26T20:40:22Z

command/kv_delete.go

+			ModifyIndex: *modifyIndex,
+		}
+
+		// If the user did not supply a -modify-index, but wants a check-and-set,


This should just fail out - for a CAS they really need to supply an index.

slackpad · 2016-09-26T20:54:46Z

command/kv_get_test.go

+	output := ui.OutputWriter.String()
+	for key, value := range keys {
+		if !strings.Contains(output, key+":"+value) {
+			t.Fatalf("bad %#v missing %q", output, key+"adafa")


slackpad · 2016-09-26T20:55:36Z

command/kv_put.go

+Usage: consul kv put [options] KEY [DATA]
+
+  Writes the data to the given path in the key-value store. The data can be of
+  any type, but it will be transported as a base64-encoded string for safe


Technically the PUT doesn't Base64 encode because there's no JSON body.

slackpad · 2016-09-26T20:56:45Z

command/kv_put.go

+                          lock. The session must already exist and be specified
+                          via the -session flag. The default value is false.
+
+  -cas                    Perform a Check-And-Set operation. If this value is


Same as delete, if they CAS they must supply an index, otherwise it should fail.

slackpad · 2016-09-26T20:57:33Z

command/kv_put.go

+
+  -modify-index=<int>     Unsigned integer representing the ModifyIndex of the
+                          key. This is often combined with the -cas flag, but it
+                          can be specified for any key. The default value is 0.


Only makes sense for a CAS, otherwise Consul will ignore it.

slackpad · 2016-09-26T20:58:31Z

command/kv_put.go

+
+	switch {
+	case *cas:
+		// If the user did not supply a -modify-index, but wants a check-and-set,


Should remove this.

slackpad · 2016-09-26T20:59:38Z

command/kv_put.go

+
+      $ consul kv put config/redis/maxconns 5
+
+  The data can also be consumed from a file on disk by prefixing with the "@"


Should also document - for stdin.

slackpad · 2016-09-26T21:02:10Z

command/kv_delete.go

+		key = key[1:]
+	}
+
+	// If the key is empty and we are not doing a recursive delete, this is an


You think an empty recursive delete might be worth a special flag that protects people from fat fingers?

This is a good point. I'm leaning towards "no" for this initial pass, since ideally you are using ACLs which prevent this from happening and/or backing up your Consul data.

slackpad · 2016-09-26T21:02:54Z

website/source/docs/commands/kv/delete.html.markdown.erb

+#### KV Delete Options
+
+* `-cas` - Perform a Check-And-Set operation. If this value is specified without
+  -modify-index, the key will first be fetched and the resulting ModifyIndex


Will need an update.

slackpad

Looks awesome - noted some small things and then I think it's good to merge.

slackpad · 2016-09-26T21:06:00Z

website/source/docs/commands/kv/delete.html.markdown.erb

+  will be used on the next query. The default value is false.
+
+* `-modify-index=<int>` - Unsigned integer representing the ModifyIndex of the
+  key. This is often combined with the -cas flag, but it can be specified for


"any key" part needs an update.

slackpad · 2016-09-26T21:06:50Z

website/source/docs/commands/kv/delete.html.markdown.erb

+```
+
+It is also possible to have Consul fetch the current ModifyIndex before making
+the query, by omitting the `-modify-index` flag. If the data is changed between


Should delete this (since the user can't see the data it won't be safe).

slackpad · 2016-09-26T21:07:17Z

website/source/docs/commands/kv/get.html.markdown.erb

+
+Command: `consul kv get`
+
+The `kv get` command is used to retrieves the value from Consul's key-value


"is used to retrieves"

slackpad · 2016-09-26T21:09:16Z

command/kv_put.go

+		Token:      *token,
+	}
+
+	switch {


If we used /v1/txn under the hood for these operations we could print out the resulting CreateIndex and ModifyIndex on success, which would be kind of neat for trying to do CAS stuff manually.

Hmm. I'm not super familiar with that stuff. Did you want me to do something here or were ya just thinking out loud?

We could add it later - mostly thinking out loud.

slackpad · 2016-09-26T21:09:45Z

website/source/docs/commands/kv/put.html.markdown.erb

+Command: `consul kv put`
+
+The `kv put` command writes the data to the given path in the key-value store.
+The data can be of any type, but it will be transported as a base64-encoded


Technically Base64 is only for the read side.

slackpad · 2016-09-26T21:09:59Z

website/source/docs/commands/kv/put.html.markdown.erb

+  operation will create the key and obtain the lock. The session must already
+  exist and be specified via the -session flag. The default value is false.
+
+* `-cas` - Perform a Check-And-Set operation. If this value is specified without


slackpad · 2016-09-26T21:10:45Z

website/source/docs/commands/kv/put.html.markdown.erb

+Success! Data written to: redis/config/connections
+```
+
+It is also possible to have Consul fetch the current ModifyIndex before making


slackpad · 2016-09-26T21:44:07Z

@sethvargo that failing test is a known flaky one unrelated to this - sorry about that.

slackpad · 2016-09-26T21:55:15Z

api/kv.go

@@ -36,9 +36,9 @@ type KVPair struct {
 	// base64 encoded upon transport.
 	Value []byte

-	// Session is a string representing the name of the session. Any other
+	// Session is a string representing the ID of the session. Any othe


@slackpad

Based on discussions with @slackpad, some of the values in the KVPair are ReadOnly. This commit updates the docs to reflect that.

slackpad reviewed Sep 26, 2016

View reviewed changes

slackpad suggested changes Sep 26, 2016

View reviewed changes

slackpad reviewed Sep 26, 2016

View reviewed changes

sethvargo added 14 commits September 26, 2016 16:06

Add documentation to KVPair

227019e

Based on discussions with @slackpad, some of the values in the KVPair are ReadOnly. This commit updates the docs to reflect that.

Add test function for returning an API client

8c30edc

Update middleman

afd0e58

Add kv command stubs

87ad6d4

Add kv get

c8fdc5c

Add kv put command

82bddd7

Add kv delete command

297a223

Add top-level docs command

a6262c8

Fix styles

de7fc2c

Fix copy-paste comments

6cdf354

Gemfile was wonky

b3c2807

Update ModifyIndex comment

12e4650

Rename session name to session ID

55f1b4a

Change delete CAS behavior to require ModifyIndex

c7496c5

sethvargo added 3 commits September 26, 2016 16:06

Update kv get docs

85350a4

Fix CAS operations for put

39fb70f

Fix typo

40de585

sethvargo force-pushed the sethvargo/kv_cli branch from 31832f0 to 40de585 Compare September 26, 2016 23:07

sethvargo added 5 commits September 26, 2016 16:09

Remove debugging code

f9a7be9

Remove base64 comment entirely

28de64c

Fix up kv put help output

d0ac899

Fix kv put flags

1898d07

Update cas docs

f381d19

slackpad approved these changes Sep 26, 2016

View reviewed changes

sethvargo merged commit be17741 into master Sep 26, 2016

sethvargo deleted the sethvargo/kv_cli branch September 26, 2016 23:51

slackpad added this to the 0.7.1 milestone Oct 25, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add kv subcommand and CLI #2360

Add kv subcommand and CLI #2360

sethvargo commented Sep 26, 2016

sethvargo commented Sep 26, 2016 •

edited

Loading

slackpad Sep 26, 2016

slackpad Sep 26, 2016 •

edited

Loading

slackpad Sep 26, 2016

slackpad Sep 26, 2016

slackpad Sep 26, 2016

slackpad Sep 26, 2016

slackpad Sep 26, 2016

slackpad Sep 26, 2016

slackpad Sep 26, 2016

slackpad Sep 26, 2016

slackpad Sep 26, 2016

sethvargo Sep 26, 2016

slackpad Sep 26, 2016

slackpad left a comment

slackpad Sep 26, 2016

slackpad Sep 26, 2016

slackpad Sep 26, 2016

slackpad Sep 26, 2016

sethvargo Sep 26, 2016

slackpad Sep 26, 2016

slackpad Sep 26, 2016

slackpad Sep 26, 2016

slackpad Sep 26, 2016

slackpad commented Sep 26, 2016

slackpad Sep 26, 2016

		CreateIndex uint64

		// ModifyIndex is used for the Check-And-Set operations.


		$ consul kv put config/redis/maxconns 5

		The data can also be consumed from a file on disk by prefixing with the "@"


		Command: `consul kv get`

		The `kv get` command is used to retrieves the value from Consul's key-value

Add kv subcommand and CLI #2360

Add kv subcommand and CLI #2360

Conversation

sethvargo commented Sep 26, 2016

sethvargo commented Sep 26, 2016 • edited Loading

Choose a reason for hiding this comment

slackpad Sep 26, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

slackpad left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

slackpad commented Sep 26, 2016

Choose a reason for hiding this comment

sethvargo commented Sep 26, 2016 •

edited

Loading

slackpad Sep 26, 2016 •

edited

Loading