Skip to content

Commit

Permalink
Support exploring non-default KV stores
Browse files Browse the repository at this point in the history 
Signed-off-by: Kate Goldenring <kate.goldenring@fermyon.com>
  • Loading branch information
@kate-goldenring
kate-goldenring committed Jan 3, 2024
1 parent 1f3da02 commit d8b6663
Show file tree
Hide file tree
Showing 6 changed files with 100 additions and 114 deletions.
6 changes: 2 additions & 4 deletions explorer/go.mod
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,7 @@
module github.com/golang_explorer

go 1.17
go 1.20

require github.com/fermyon/spin/sdk/go v1.0.0
require github.com/fermyon/spin/sdk/go/v2 v2.0.1

require github.com/julienschmidt/httprouter v1.3.0 // indirect

replace github.com/fermyon/spin/sdk/go v1.0.0 => github.com/radu-matei/spin/sdk/go v0.0.0-20230406224338-9d78631f2c2b
4 changes: 2 additions & 2 deletions explorer/go.sum
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
github.com/fermyon/spin/sdk/go/v2 v2.0.1 h1:cVfGCn68Z0O0zjqsNmIDAp9mTEcjedGx+KU+95zODDM=
github.com/fermyon/spin/sdk/go/v2 v2.0.1/go.mod h1:kfJ+gdf/xIaKrsC6JHCUDYMv2Bzib1ohFIYUzvP+SCw=
github.com/julienschmidt/httprouter v1.3.0 h1:U0609e9tgbseu3rBINet9P48AI/D3oJs4dN7jwJOQ1U=
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
github.com/radu-matei/spin/sdk/go v0.0.0-20230406224338-9d78631f2c2b h1:YjGlkvD2pkytU/DdJI+Bg6Zc73SNLToCcKQjKV3H5pc=
github.com/radu-matei/spin/sdk/go v0.0.0-20230406224338-9d78631f2c2b/go.mod h1:yb8lGesopgj/GwPzLPATxcOeqWZT/HjrzEFfwbztAXE=
52 changes: 41 additions & 11 deletions explorer/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,16 @@ <h1>Key Value Store Explorer</h1>

<div class="row">
<div class="col-md-9">

<div class="table-wrap">
<div class="row mb-2 add-kv">
<div class="col-md-6">
<input id="storeLabel" type="text" class="form-control mt-2" placeholder="Store Label">
</div>
<div class="col-md-2">
<button id="load-store" class="btn btn-outline-primary mt-2" title="Load Store">Load</button>
</div>
</div>
</div>
<div class="table-wrap">
<div class="row mb-2 add-kv">
<div class="col-md-6">
Expand Down Expand Up @@ -252,7 +261,7 @@ <h4>About Key Value Storage</h4>
<code>GET</code> &mdash; read the docs for <a
href="https://developer.fermyon.com/spin/kv-store#storing-and-retrieving-data-from-your-default-keyvalue-store"
target="_blank">Storing &amp; Retrieving Data From Your
Default Key/Value Store</a> for guidance and examples.
Key/Value Stores</a> for guidance and examples.
</p>
</div>
</div>
Expand Down Expand Up @@ -289,13 +298,30 @@ <h5 class="modal-title" id="viewModalLabel">View Value</h5>
integrity="sha512-6UofPqm0QupIL0kzS/UIzekR73/luZdC6i/kXDbWnLOJoqwklBK6519iUnShaYceJ0y4FaiPtX/hRnV/X/xlUQ=="
crossorigin="anonymous" referrerpolicy="no-referrer"></script>
<script>
fetch("{{.SpinRoute}}api/stores/default")
.then((response) => response.json())
.then((data) => {
data.keys.sort().forEach((item) => {
insert(item);
let storeLabel = "";

$("#load-store").click(function() {
removeAllPairs();
storeLabel = $("#storeLabel").val();
if (storeLabel.length == 0) {
alert("Store label must not be empty");
return
}
// Catch an Internal error and alert that store does not exist
fetch(`{{.SpinRoute}}api/stores/${storeLabel}`)
.then((response) => {
if (response.status == 401) {
alert(`Access denied to store with label "${storeLabel}." Ensure the app is linked to the store with this label and that the label has been specified in the Spin manifest (spin.toml).`);
return
}
return response.json()
})
.then((data) => {
data.keys.sort().forEach((item) => {
insert(item);
});
});
});
})

// Use this to generate unique and valid ids for keys
const simpleHash = str => {
Expand All @@ -308,6 +334,10 @@ <h5 class="modal-title" id="viewModalLabel">View Value</h5>
return new Uint32Array([hash])[0].toString(36);
};

function removeAllPairs() {
$("#kv > tbody:first").empty();
}

function insert(key) {
let hashedKey = simpleHash(key);
$("#kv > tbody:first").append(`
Expand All @@ -318,7 +348,7 @@ <h5 class="modal-title" id="viewModalLabel">View Value</h5>

$(`#${hashedKey}View`).click(function () {
var key = $(this).data("key");
fetch(`{{.SpinRoute}}api/stores/default/keys/${encodeURIComponent(key)}`).then((response) => response.json()).then((data) => {
fetch(`{{.SpinRoute}}api/stores/${storeLabel}/keys/${encodeURIComponent(key)}`).then((response) => response.json()).then((data) => {
let value = atob(data.value);
$("#valueContent").text(value);
$("#viewModal").modal("show");
Expand All @@ -328,7 +358,7 @@ <h5 class="modal-title" id="viewModalLabel">View Value</h5>

$(`#${hashedKey}Delete`).click(function () {
var key = $(this).data("key");
fetch(`{{.SpinRoute}}api/stores/default/keys/${encodeURIComponent(key)}`, {
fetch(`{{.SpinRoute}}api/stores/${storeLabel}/keys/${encodeURIComponent(key)}`, {
method: 'DELETE',
})
.then(() => {
Expand All @@ -354,7 +384,7 @@ <h5 class="modal-title" id="viewModalLabel">View Value</h5>
key: key,
value: value
};
fetch("{{.SpinRoute}}api/stores/default", {
fetch(`{{.SpinRoute}}api/stores/${storeLabel}`, {
method: 'POST',
headers: {
'Content-Type': 'application/json'
Expand Down
128 changes: 42 additions & 86 deletions explorer/main.go
View file
Original file line number Diff line number Diff line change
@@ -1,24 +1,23 @@
package main

import (
"crypto/rand"
"crypto/subtle"
_ "embed"
"encoding/json"
"fmt"
"log"
"math/big"
"net/http"
"os"
"path"
"strings"
"time"

spin "github.com/fermyon/spin/sdk/go/http"
kv "github.com/fermyon/spin/sdk/go/key_value"
"github.com/fermyon/spin/sdk/go/v2/variables"

spin "github.com/fermyon/spin/sdk/go/v2/http"
"github.com/fermyon/spin/sdk/go/v2/kv"
)

var KV_STORE_CREDENTIALS_KEY string = "kv-credentials"
var KV_STORE_CREDENTIALS_KEY string = "kv_credentials"
var SKIP_AUTH_ENV string = "SPIN_APP_KV_SKIP_AUTH"

// At build time, read the contents of index.html and embed it in the `Html` variable.
Expand Down Expand Up @@ -72,16 +71,18 @@ func getBasePath(h http.Header) string {
func serve(w http.ResponseWriter, r *http.Request) {
user, pass, err := GetCredentials()
if err != nil {
log.Printf("Error getting credentials from KV store: %v", err)
w.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(w, "KV explorer credentials not configured.\n")
// Log more informative error in cloud logs
fmt.Printf("Error getting KV explorer credentials from config store: %v\n", err)
w.WriteHeader(http.StatusForbidden)
return
}

spinRoute = getBasePath(r.Header)
router := spin.NewRouter()

// Access to the list, get, create, and delete KV pairs endpoints is behind basic auth,
// with the credentials stored in the KV store itself.
// with the credentials stored in the config store.
router.GET(path.Join(spinRoute, "/api/stores/:store"), BasicAuth(ListKeysHandler, user, pass))
router.GET(path.Join(spinRoute, "/api/stores/:store/keys/:key"), BasicAuth(GetKeyHandler, user, pass))
router.DELETE(path.Join(spinRoute, "/api/stores/:store/keys/:key"), BasicAuth(DeleteKeyHandler, user, pass))
Expand All @@ -105,19 +106,19 @@ func UIHandler(w http.ResponseWriter, _ *http.Request, _ spin.Params) {
func ListKeysHandler(w http.ResponseWriter, _ *http.Request, p spin.Params) {
storeName := p.ByName("store")

store, err := kv.Open(storeName)
store, err := kv.OpenStore(storeName)
if err != nil {
log.Printf("ERROR: cannot open store: %v", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
fmt.Printf("ERROR: cannot open store: %v\n", err)
http.Error(w, err.Error(), http.StatusUnauthorized)
return
}
defer kv.Close(store)
defer store.Close()

start := time.Now()
keys, err := kv.GetKeys(store)
log.Printf("LIST operation took: %s", time.Since(start))
keys, err := store.GetKeys()
fmt.Printf("LIST operation took: %s\n", time.Since(start))
if err != nil {
log.Printf("ERROR: cannot list keys: %v", err)
fmt.Printf("ERROR: cannot list keys: %v\n", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
Expand All @@ -131,19 +132,19 @@ func GetKeyHandler(w http.ResponseWriter, _ *http.Request, p spin.Params) {
storeName := p.ByName("store")
key := p.ByName("key")

store, err := kv.Open(storeName)
store, err := kv.OpenStore(storeName)
if err != nil {
log.Printf("ERROR: cannot open store: %v", err)
fmt.Printf("ERROR: cannot open store: %v\n", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
defer kv.Close(store)
defer store.Close()

start := time.Now()
value, err := kv.Get(store, key)
log.Printf("GET operation took %s", time.Since(start))
value, err := store.Get(key)
fmt.Printf("GET operation took %s\n", time.Since(start))
if err != nil {
log.Printf("ERROR: cannot get key: %v", err)
fmt.Printf("ERROR: cannot get key: %v\n", err)
w.WriteHeader(http.StatusNotFound)
}

Expand All @@ -157,19 +158,19 @@ func DeleteKeyHandler(w http.ResponseWriter, _ *http.Request, p spin.Params) {
storeName := p.ByName("store")
key := p.ByName("key")

store, err := kv.Open(storeName)
store, err := kv.OpenStore(storeName)
if err != nil {
log.Printf("ERROR: cannot open store: %v", err)
fmt.Printf("ERROR: cannot open store: %v\n", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
defer kv.Close(store)
defer store.Close()

start := time.Now()
err = kv.Delete(store, key)
log.Printf("DELETE operation took %s", time.Since(start))
err = store.Delete(key)
fmt.Printf("DELETE operation took %s\n", time.Since(start))
if err != nil {
log.Printf("ERROR: cannot delete key: %v", err)
fmt.Printf("ERROR: cannot delete key: %v\n", err)
w.WriteHeader(http.StatusNotFound)
}
}
Expand All @@ -184,19 +185,19 @@ func AddKeyHandler(w http.ResponseWriter, r *http.Request, p spin.Params) {
return
}

store, err := kv.Open(storeName)
store, err := kv.OpenStore(storeName)
if err != nil {
log.Printf("ERROR: cannot open store: %v", err)
fmt.Printf("ERROR: cannot open store: %v\n", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
defer kv.Close(store)
defer store.Close()

start := time.Now()
err = kv.Set(store, input.Key, []byte(input.Value))
log.Printf("SET operation took %s", time.Since(start))
err = store.Set(input.Key, []byte(input.Value))
fmt.Printf("SET operation took %s\n", time.Since(start))
if err != nil {
log.Printf("ERROR: cannot add key: %v", err)
fmt.Printf("ERROR: cannot add key: %v\n", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
Expand All @@ -209,6 +210,7 @@ func BasicAuth(h spin.RouterHandle, requiredUser, requiredPassword string) spin.
// when the environment variable is set.
val, ok := os.LookupEnv(SKIP_AUTH_ENV)
if ok && val == "1" {
fmt.Printf("INFO: Skipping authentication\n")
h(w, r, ps)
return
}
Expand All @@ -220,7 +222,7 @@ func BasicAuth(h spin.RouterHandle, requiredUser, requiredPassword string) spin.
// Delegate request to the given handle
h(w, r, ps)
} else {
log.Printf("ERROR: Unauthenticated request")
fmt.Printf("ERROR: Unauthenticated request\n")
// Request Basic Authentication otherwise
w.Header().Set("WWW-Authenticate", "Basic realm=Restricted")
http.Error(w, http.StatusText(http.StatusUnauthorized), http.StatusUnauthorized)
Expand All @@ -229,64 +231,18 @@ func BasicAuth(h spin.RouterHandle, requiredUser, requiredPassword string) spin.
}

// CredsOrDefault checks the KV store for a `credentials` key, and expects
// the value to be `username:password`. If a value is not found, this function
// will generate a random pair and log it once.
// the value to be `username:password`. If a value is not found, it will error.
func GetCredentials() (string, string, error) {
store, err := kv.Open("default")
if err != nil {
log.Printf("ERROR: cannot open store: %v", err)
return "", "", fmt.Errorf("error opening store: %v", err)
}
creds, err := variables.Get(KV_STORE_CREDENTIALS_KEY)

exists, err := kv.Exists(store, KV_STORE_CREDENTIALS_KEY)
if err != nil {
return "", "", fmt.Errorf("cannot check if credentials exists: %v", err)
}

if !exists {
defaultUser, err := GenerateRandomString(10)
if err != nil {
return "", "", fmt.Errorf("failed to generate random string for user: %v", err)
}
defaultPassword, err := GenerateRandomString(30)
if err != nil {
return "", "", fmt.Errorf("failed to generate random string for password: %v", err)
}

kv.Set(store, KV_STORE_CREDENTIALS_KEY, []byte(defaultUser+":"+defaultPassword))

log.Printf("Default user: %v", defaultUser)
log.Printf("Default password: %v", defaultPassword)
log.Printf("This is a randomly generated username and password pair. To change it, please add a `credentials` key in the default store with the value `username:password`. If you delete the credential pair, the next request will generate a new random set.")

return defaultUser, defaultPassword, nil
}

creds, err := kv.Get(store, KV_STORE_CREDENTIALS_KEY)
if err != nil {
return "", "", fmt.Errorf("cannot get credentials pair from store: %v", err)
fmt.Printf("ERROR: cannot get credentials pair from config store: %v\n", err)
fmt.Printf("Set the 'kv_explorer_user' and 'kv_explorer_password' variables using 'spin cloud variables set'.\n")
return "", "", fmt.Errorf("cannot get credentials pair from config store: %v", err)
}

split := strings.Split(string(creds), ":")
return split[0], split[1], nil
}

// GenerateRandomString returns a securely generated random string.
// It will return an error if the system's secure random
// number generator fails to function correctly, in which
// case the caller should not continue.
func GenerateRandomString(n int) (string, error) {
const chars = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz-!@#$%^&*"
ret := make([]byte, n)
for i := 0; i < n; i++ {
num, err := rand.Int(rand.Reader, big.NewInt(int64(len(chars))))
if err != nil {
return "", err
}
ret[i] = chars[num.Int64()]
}

return string(ret), nil
}

func main() {}
13 changes: 4 additions & 9 deletions readme.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,22 +43,17 @@ You can now access the explorer in your browser at the route `/internal/kv-explo

### Credentials

When running locally, you can skip checking for the crendentials on every request by passing the `SPIN_APP_KV_SKIP_AUTH` environment variable:
When running locally, you can skip checking for the credentials on every request by passing the `SPIN_APP_KV_SKIP_AUTH` environment variable:

```bash
$ spin up --env SPIN_APP_KV_SKIP_AUTH=1
```

When deploying to [Fermyon Cloud](https://fermyon.com/cloud), you can pass the credentials pair together with the `spin deploy` command:
When deploying to [Fermyon Cloud](https://fermyon.com/cloud), you are required to set the username and password for the kv explorer with the `spin deploy` command:

```bash
# change the value to your desired basic authentication credentials
$ export KV_CREDENTIALS="user:password"
$ spin deploy --key-value kv-credentials=$KV_CREDENTIALS
$ spin deploy --variable kv_explorer_user="some-username" --variable kv_explorer_password="some-password"
```

The explorer will use the default store to persist the credentials to access the UI and the API. If no values are set, the first invocation will set a randomly generated pair of username and password under the `kv-credentials` key, with the value following the `user:password` format. On the first run, the values will be printed in the logs, and they can be used to log in and change them (creating a new `credentials` value will override the existing value).

### Known limitations

- the explorer can only be used with Spin's default key/value store. When this will be configurable, this component will support working with custom stores as well.
The explorer will use the config variables store to persist the credentials to access the UI and the API. If no values are set, a forbidden error is returned.
Loading

0 comments on commit d8b6663

Please sign in to comment.