Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Keyring #2557

Merged
merged 24 commits into from
May 10, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions cli/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ var configPaths = []string{
"datastore.badger.path",
"api.pubkeypath",
"api.privkeypath",
"keyring.path",
}

// configFlags is a mapping of config keys to cli flags to bind to.
Expand Down Expand Up @@ -71,6 +72,8 @@ func defaultConfig() *viper.Viper {
cfg.SetConfigType("yaml")

cfg.SetDefault("datastore.badger.path", "data")
cfg.SetDefault("keyring.path", "keys")

cfg.SetDefault("net.pubSubEnabled", true)
cfg.SetDefault("net.relay", false)
cfg.SetDefault("log.caller", false)
Expand Down
127 changes: 127 additions & 0 deletions cli/keyring.go
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: I wonder if all this logic with generating hashes should be scoped to this cli package.

Maybe it makes sense to extract it in another place so other that other packages can use it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that would be nice. Would it make sense to have a crypto package?

Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
// Copyright 2024 Democratized Data Foundation
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.

package cli

import (
"crypto/rand"
"errors"
"syscall"

"github.com/99designs/keyring"
Copy link
Contributor

@AndrewSisley AndrewSisley Apr 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: How confident are we in this dependency? The repo hasn't been updated in more than two years, and it is managed by a company that looks to have a focus way away from the security world and may lack the technical expertise or attention to keep it secure (assuming it and its dependencies were secure 2 years ago).

Some of its' dependencies are very out of date, and a couple are even more stale and suspicious looking than this package (e.g. https://github.com/gsterjov/go-libsecret with 14 stars, presumably responsible for the very old version of https://github.com/godbus/dbus)

Different websites say different things RE its' size, one says it only had 16 employees in 2022 (when the repo was last updated), other top search engine hits estimate 300, and 3777 employees.

99Designs:

Global creative platform for designers and clients

@jsimnz @nasdf /All

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've replaced this dependency with a custom implementation using a newer library.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dependencies and code of the new one look much healthier, thanks Keenan :)

"github.com/libp2p/go-libp2p/core/crypto"
"github.com/spf13/cobra"
"golang.org/x/term"
)

const (
peerKeyName = "peer_key"
badgerEncryptionKeyName = "badger_encryption_key"
)

// openKeyring attempts to open the keyring file from the given directory.
//
// If the directory is an empty string the best option for the current OS will be used.
func openKeyring(cmd *cobra.Command, dir string) (keyring.Keyring, error) {
var allowedBackends []keyring.BackendType
if dir != "" {
// only allow file backend if a directory is specified
allowedBackends = append(allowedBackends, keyring.FileBackend)
}

prompt := keyring.PromptFunc(func(s string) (string, error) {
cmd.Print(s)
pass, err := term.ReadPassword(int(syscall.Stdin))
return string(pass), err
})

return keyring.Open(keyring.Config{
AllowedBackends: allowedBackends,
ServiceName: "defradb",
KeychainName: "defradb",
KeychainPasswordFunc: prompt,
FilePasswordFunc: prompt,
FileDir: dir,
KeyCtlScope: "user",
KeyCtlPerm: 0, // TODO
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo: Please create a task if this is something that needs to be handled later. Otherwise it will linger.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will fix before merging.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up replacing this dependency.

KWalletAppID: "defradb",
KWalletFolder: "defradb",
LibSecretCollectionName: "defradb",
PassPrefix: "defradb",
WinCredPrefix: "defradb",
})
}

// generateAES256 generates a new random AES-256 bit encryption key.
func generateAES256() ([]byte, error) {
data := make([]byte, 32)
_, err := rand.Read(data)
return data, err
}

// loadOrGenerateAES256 attempts to load the AES-256 bit key with the given name.
//
// If the key does not exist a new random key is generated and stored in the keyring.
func loadOrGenerateAES256(kr keyring.Keyring, name string) ([]byte, error) {
item, err := kr.Get(name)
if err == nil {
return item.Data, nil
}
if !errors.Is(err, keyring.ErrKeyNotFound) {
return nil, err
}
key, err := generateAES256()
if err != nil {
return nil, err
}
err = kr.Set(keyring.Item{
Key: name,
Data: key,
})
if err != nil {
return nil, err
}
return key, nil
}

// generateEd25519 generates a new random Ed25519 private key.
func generateEd25519() (crypto.PrivKey, error) {
key, _, err := crypto.GenerateKeyPair(crypto.Ed25519, 0)
return key, err
}

// loadOrGenerateEd25519 attempts to load the Ed25519 private key with the given name.
//
// If the key does not exist a new random key is generated and stored in the keyring.
func loadOrGenerateEd25519(kr keyring.Keyring, name string) (crypto.PrivKey, error) {
item, err := kr.Get(name)
if err == nil {
return crypto.UnmarshalPrivateKey(item.Data)
}
if !errors.Is(err, keyring.ErrKeyNotFound) {
return nil, err
}
key, err := generateEd25519()
if err != nil {
return nil, err
}
data, err := crypto.MarshalPrivateKey(key)
if err != nil {
return nil, err
}
err = kr.Set(keyring.Item{
Key: name,
Data: data,
})
if err != nil {
return nil, err
}
return key, nil
}
30 changes: 19 additions & 11 deletions cli/start.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ import (
"fmt"
"os"
"os/signal"
"path/filepath"
"syscall"

"github.com/libp2p/go-libp2p/core/peer"
Expand Down Expand Up @@ -84,23 +83,32 @@ func MakeStartCommand() *cobra.Command {
}

if cfg.GetString("datastore.store") != configStoreMemory {
// It would be ideal to not have the key path tied to the datastore.
// Running with memory store mode will always generate a random key.
// Adding support for an ephemeral mode and moving the key to the
// config would solve both of these issues.
rootDir := mustGetContextRootDir(cmd)
key, err := loadOrGeneratePrivateKey(filepath.Join(rootDir, "data", "key"))
if err != nil {
return err
}
netOpts = append(netOpts, net.WithPrivateKey(key))

// TODO-ACP: Infuture when we add support for the --no-acp flag when admin signatures are in,
// we can allow starting of db without acp. Currently that can only be done programmatically.
// https://github.com/sourcenetwork/defradb/issues/2271
dbOpts = append(dbOpts, db.WithACP(rootDir))
}

if !cfg.GetBool("keyring.disabled") {
keyring, err := openKeyring(cmd, cfg.GetString("keyring.path"))
if err != nil {
return err
}

peerKey, err := loadOrGenerateEd25519(keyring, peerKeyName)
if err != nil {
return err
}
netOpts = append(netOpts, net.WithPrivateKey(peerKey))

encryptionKey, err := loadOrGenerateAES256(keyring, badgerEncryptionKeyName)
if err != nil {
return err
}
storeOpts = append(storeOpts, node.WithEncryptionKey(encryptionKey))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: This implies that if the keyring is enabled, then we also use use at-rest encryption. Should these technically be separate config options?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! I've added a datastore.encryptionDisabled config item.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: I've made the key optional instead of the config option.

}

opts := []node.NodeOpt{
node.WithPeers(peers...),
node.WithStoreOpts(storeOpts...),
Expand Down
41 changes: 0 additions & 41 deletions cli/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ import (
"os"
"path/filepath"

"github.com/libp2p/go-libp2p/core/crypto"
"github.com/spf13/cobra"
"github.com/spf13/viper"

Expand Down Expand Up @@ -153,46 +152,6 @@ func setContextRootDir(cmd *cobra.Command) error {
return nil
}

// loadOrGeneratePrivateKey loads the private key from the given path
// or generates a new key and writes it to a file at the given path.
func loadOrGeneratePrivateKey(path string) (crypto.PrivKey, error) {
key, err := loadPrivateKey(path)
if err == nil {
return key, nil
}
if os.IsNotExist(err) {
return generatePrivateKey(path)
}
return nil, err
}

// generatePrivateKey generates a new private key and writes it
// to a file at the given path.
func generatePrivateKey(path string) (crypto.PrivKey, error) {
key, _, err := crypto.GenerateKeyPair(crypto.Ed25519, 0)
if err != nil {
return nil, err
}
data, err := crypto.MarshalPrivateKey(key)
if err != nil {
return nil, err
}
err = os.MkdirAll(filepath.Dir(path), 0755)
if err != nil {
return nil, err
}
return key, os.WriteFile(path, data, 0644)
}

// loadPrivateKey reads the private key from the file at the given path.
func loadPrivateKey(path string) (crypto.PrivKey, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, err
}
return crypto.UnmarshalPrivateKey(data)
}

func writeJSON(cmd *cobra.Command, out any) error {
enc := json.NewEncoder(cmd.OutOrStdout())
enc.SetIndent("", " ")
Expand Down
10 changes: 10 additions & 0 deletions docs/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,3 +92,13 @@ Logger config overrides. Format `<name>,<key>=<val>,...;<name>,...`.
## `log.nocolor`

Disable colored log output. Defaults to `false`.

## `keyring.path`

Path to store encrypted key files in. Defaults to `keys`.

When set to an empty string the default OS key management will be used.

## `keyring.disabled`

Disable the keyring and generate ephemeral keys instead. Defaults to `false`.
4 changes: 2 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ module github.com/sourcenetwork/defradb
go 1.21.3

require (
github.com/99designs/keyring v1.2.2
github.com/bits-and-blooms/bitset v1.13.0
github.com/bxcodec/faker v2.0.1+incompatible
github.com/cosmos/gogoproto v1.4.12
Expand Down Expand Up @@ -48,6 +49,7 @@ require (
go.opentelemetry.io/otel/sdk/metric v1.25.0
go.uber.org/zap v1.27.0
golang.org/x/exp v0.0.0-20240222234643-814bf88cf225
golang.org/x/term v0.19.0
google.golang.org/grpc v1.63.2
google.golang.org/protobuf v1.33.0
)
Expand All @@ -65,7 +67,6 @@ require (
cosmossdk.io/x/tx v0.13.1 // indirect
filippo.io/edwards25519 v1.0.0 // indirect
github.com/99designs/go-keychain v0.0.0-20191008050251-8e49817e8af4 // indirect
github.com/99designs/keyring v1.2.1 // indirect
github.com/DataDog/datadog-go v3.2.0+incompatible // indirect
github.com/DataDog/zstd v1.5.5 // indirect
github.com/Jorropo/jsync v1.0.1 // indirect
Expand Down Expand Up @@ -295,7 +296,6 @@ require (
golang.org/x/net v0.23.0 // indirect
golang.org/x/sync v0.6.0 // indirect
golang.org/x/sys v0.19.0 // indirect
golang.org/x/term v0.19.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/tools v0.18.0 // indirect
gonum.org/v1/gonum v0.14.0 // indirect
Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@ filippo.io/edwards25519 v1.0.0/go.mod h1:N1IkdkCkiLB6tki+MYJoSx2JTY9NUlxZE7eHn5E
git.apache.org/thrift.git v0.0.0-20180902110319-2566ecd5d999/go.mod h1:fPE2ZNJGynbRyZ4dJvy6G277gSllfV2HJqblrnkyeyg=
github.com/99designs/go-keychain v0.0.0-20191008050251-8e49817e8af4 h1:/vQbFIOMbk2FiG/kXiLl8BRyzTWDw7gX/Hz7Dd5eDMs=
github.com/99designs/go-keychain v0.0.0-20191008050251-8e49817e8af4/go.mod h1:hN7oaIRCjzsZ2dE+yG5k+rsdt3qcwykqK6HVGcKwsw4=
github.com/99designs/keyring v1.2.1 h1:tYLp1ULvO7i3fI5vE21ReQuj99QFSs7lGm0xWyJo87o=
github.com/99designs/keyring v1.2.1/go.mod h1:fc+wB5KTk9wQ9sDx0kFXB3A0MaeGHM9AwRStKOQ5vOA=
github.com/99designs/keyring v1.2.2 h1:pZd3neh/EmUzWONb35LxQfvuY7kiSXAq3HQd97+XBn0=
github.com/99designs/keyring v1.2.2/go.mod h1:wes/FrByc8j7lFOAGLGSNEg8f/PaI3cgTBqhFkHUrPk=
github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 h1:cTp8I5+VIoKjsnZuH8vjyaysT/ses3EvZeaV/1UkF2M=
github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96/go.mod h1:bOvUY6CB00SOBii9/FifXqc0awNKxLFCL/+pkDPuyl8=
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 h1:L/gRVlceqvL25UVaW/CKtUDjefjrs0SPonmDGUVOYP0=
Expand Down
7 changes: 7 additions & 0 deletions node/store.go
Original file line number Diff line number Diff line change
Expand Up @@ -74,5 +74,12 @@ func NewStore(opts ...StoreOpt) (datastore.RootStore, error) {
badgerOpts.ValueLogFileSize = options.valueLogFileSize
badgerOpts.EncryptionKey = options.encryptionKey

if len(options.encryptionKey) > 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: I prefer if options.encryptionKey != "" as it clearly shows that the type of the field is string, not a slice

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you may have misread the type. encryptionKey is a []byte

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, my bad

// Having a cache improves the performance.
// Otherwise, your reads would be very slow while encryption is enabled.
// https://dgraph.io/docs/badger/get-started/#encryption-mode
badgerOpts.IndexCacheSize = 100 << 20
}

return badger.NewDatastore(options.path, &badgerOpts)
}
Loading