vault-plugin-database-redis

A Vault plugin for Redis

This project uses the database plugin interface introduced in Vault version 0.7.1.

The plugin supports the generation of static and dynamic user roles and root credential rotation on the following Redis installations...

• Single primary server
• Primary server and 1 - N secondary (readonly) replica servers
• Redis Cluster
• Redis Sentinel

The plugin can also be configured to persist the generated credentials using the presistence_mode parameter, either to the servers local ACL file, using the Redis ACL SAVE command or to the Redis configuration file with the Redis CONFIG REWRITE command. The Redis installation must have either the aclsave file configured or a writable config file for this to work.

In addition the plugin has been upgraded to support X509 certificate authentication as by default, Redis uses mutual TLS and requires clients to authenticate with a valid certificate (authenticated against trusted root CAs). It is necessary to set the Redis setting tls-auth-clients no to disable client authentication.

Build

Use make dev to build a development version of this plugin.

Please note: In case of the following errors, while creating Redis connection in Vault, please build this plugin with CGO_ENABLED=0 go build -ldflags='-extldflags=-static' -o vault-plugin-database-redis ./cmd/vault-plugin-database-redis/ command. More details on this error can be found here.

Error writing data to database/config/my-redis: Error making API request.

URL: PUT http://127.0.0.1:8200/v1/database/config/my-redis
Code: 400. Errors:

* error creating database object: invalid database version: 2 errors occurred:
        * fork/exec /config/plugin/vault-plugin-database-redis: no such file or directory
        * fork/exec /config/plugin/vault-plugin-database-redis: no such file or directory

Testing

To run tests, go test will first set up the docker.io/redis:latest database image, then execute a set of basic tests against it. To test against different redis images, for example 5.0-buster, set the environment variable REDIS_VERSION=5.0-buster. If you want to run the tests against a local redis installation or an already running redis containerized installation, set the appropriate environment variables before executing.

• TEST_REDIS_PRIMARY_HOST and TEST_REDIS_PRIMARY_PORT for a standalone server.
• TEST_REDIS_PRIMARY_HOST, TEST_REDIS_PRIMARY_PORT and TEST_REDIS_SECONDARIES for a server with primary and N secondary's.
• TEST_REDIS_CLUSTER for a Redis cluster installation. Note: One server and port combination is enough for testing as the plugin will be able to fetch the clusters topography.
• TEST_REDIS_SENTINELS and TEST_REDIS_SENTINEL_MASTER_NAME for a sentinel installation.

Note: The tests assume that the redis database installation has a default user with the following ACL settings user default on >default-pa55w0rd ~* +@all. If it doesn't, you will need to align the Administrator username and password with the pre-set values in the redis_test.go file. The cluster, sentinel and primary-secondary terraform created Redis test installations populate the Redis servers aclfile /tmp/users.acl with the default user default with the same default-pa55w0rd.

Set VAULT_ACC=1 to execute all of the tests including the acceptance tests, or run just a subset of tests by using a command like go test -run TestDriver/Init for example.

Installation

The Vault plugin system is documented on the Vault documentation site.

You will need to define a plugin directory using the plugin_directory configuration directive, then place the vault-plugin-database-redis executable generated above, into the directory.

Please note: This plugin is incompatible with Vault versions before 1.6.0 due to an update of the database plugin interface. You will be able to register the plugin in the plugins catalog with an older version of Vault but when you try to initialize the plugin to connect to a database instance you will get this error.

Error writing data to database/config/my-redis: Error making API request.

URL: PUT http://127.0.0.1:8200/v1/database/config/my-redis
Code: 400. Errors:

* error creating database object: Incompatible API version with plugin. Plugin version: 5, Client versions: [3 4]

Sample commands for registering and starting to use the plugin:

$ SHA256=$(shasum -a 256 plugins/vault-plugin-database-redis | cut -d' ' -f1)

$ vault secrets enable database

$ vault write sys/plugins/catalog/database/vault-plugin-database-redis sha256=$SHA256 \
        command=vault-plugin-database-redis

At this stage you are now ready to initialize the plugin to connect to the redis db using unencrypted or encrypted communications.

Prior to initializing the plugin, ensure that you have created an administration account. Vault will use the user specified here to create/update/revoke database credentials. That user must have the appropriate rule +@admin to perform actions upon other database users. If you are using a Redis cluster then the user must have these two additional rules +readonly +cluster.

Plugin Initialization

Standalone Redis Server.

$ vault write database/config/my-redis plugin_name="vault-plugin-database-redis" \
        primary_host="localhost" primary_port=6379 username="Administrator" password="password" \
        allowed_roles="my-redis-*-role"

# You should consider rotating the admin password. Note that if you do, the new password will never be made available
# through Vault, so you should create a vault-specific database admin user for this.
$ vault write -force database/rotate-root/my-redis

Primary Redis Server and read only secondary replicas.

CACERT=$(cat $CA_CERT_FILE)
TLSCert=$(cat $TLS_CERT_FILE)
TLSKey=$(cat $TLS_KEY_FILE)

vault write database/config/my-redis plugin_name="vault-plugin-database-redis" \
      primary_host="master-server" primary_port="6379" \
      secondaries="redis-secondary-0:6379,redis-secondary-1:6379," \
      username="default" password="default-pa55w0rd" \
      allowed_roles="*" persistence_mode="REWRITE" \
      tls=true ca_cert="$CACERT" tls_cert="$TLSCert" tls_key="$TLSKey"
#Success! Data written to: database/config/my-redis

Redis Cluster.

vault write database/config/my-redis plugin_name="vault-plugin-database-redis" \
      cluster="node-0:6379,node-1:6379,node-3:6379,node-4:6379" \
      username=default password=default-pa55w0rd \
      allowed_roles="*" persistence_mode="REWRITE"
#Success! Data written to: database/config/my-redis

Redis Sentinel.

CACERT=$(cat $CA_CERT_FILE)
TLSCert=$(cat $TLS_CERT_FILE)
TLSKey=$(cat $TLS_KEY_FILE)

vault write database/config/my-redis plugin_name="vault-plugin-database-redis" \
      sentinels="172.27.0.6:26379,172.27.0.7:26379,172.27.0.5:26379" sentinel_master_name=dear_racer \
      sentinel_username="default" sentinel_password="default-pa55w0rd" \
      username=default password=default-pa55w0rd 'allowed_roles=*' \
      persistence_mode=ACLFILE \
      tls=true ca_cert="$CACERT" tls_cert="$TLSCert" tls_key="$TLSKey" 
#Success! Data written to: database/config/my-redis

Note: A sentinel installation requires credentials for the primary and secondaries as well as credentials for the sentinel servers. In this example they are the same but best practice would be to have a sentinel user with minimal control permissions. For example: ACL SETUSER sentinel-user ON >somepassword allchannels +multi +slaveof +ping +exec +subscribe +config|rewrite +role +publish +info +client|setname +client|kill +script|kill. Also note: the plugin provisions credentials to the Redis servers the sentinels manage at this time, not to the sentinels themselves.

Dynamic Role Creation

When you create roles, you need to provide a JSON string containing the Redis ACL rules which are documented here or in the output of the ACL CAT redis command.

# if a creation_statement is not provided the user account will default to a read only user, '["~*", "+@read"]' that can read any key. 
$ vault write database/roles/my-redis-admin-role db_name=my-redis \
        default_ttl="5m" max_ttl="1h" creation_statements='["+@admin"]'

$ vault write database/roles/my-redis-read-foo-role db_name=my-redis \
        default_ttl="5m" max_ttl="1h" creation_statements='["~foo", "+@read"]'
Success! Data written to: database/roles/my-redis-read-foo-role

Note: Starting from Redis 7.0, ACL rules can also be grouped into multiple distinct sets of rules, called selectors. Selectors are added by wrapping the rules in parentheses and providing them just like any other rule. In order to execute a command, either the root permissions (rules defined outside of parenthesis) or any of the selectors (rules defined inside parenthesis) must match the given command. For example:

ACL SETUSER virginia on +GET allkeys (+SET ~app1*)

This sets a user with two sets of permissions, one defined on the user and one defined with a selector. The root user permissions only allow executing the get command, but can be executed on any keys. The selector then grants a secondary set of permissions: access to the SET command to be executed on any key that starts with app1. Using multiple selectors allows you to grant permissions that are different depending on what keys are being accessed.

vault write database/roles/selector-role db_name=my-redis default_ttl="5m" max_ttl="1h" \
      creation_statements='["~foo*", "+get", "(~bar* +get +set)"]'

To retrieve the credentials for the dynamic accounts

$vault read database/creds/my-redis-admin-role
Key                Value
---                -----
lease_id           database/creds/my-redis-admin-role/OxCTXJcxQ2F4lReWPjbezSnA
lease_duration     5m
lease_renewable    true
password           dACqHsav6-attdv1glGZ
username           V_TOKEN_MY-REDIS-ADMIN-ROLE_YASUQUF3GVVD0ZWTEMK4_1608481717

$ vault read database/creds/my-redis-read-foo-role
Key                Value
---                -----
lease_id           database/creds/my-redis-read-foo-role/Yn99BrX4t0NkLyifm4NmsEUB
lease_duration     5m
lease_renewable    true
password           ZN6gdTKszk7oc9Oztc-o
username           V_TOKEN_MY-REDIS-READ-FOO-ROLE_PUAINND1FC5XQGRC0HIF_1608481734

$ vault read database/creds/selector-role
Key                Value
---                -----
lease_id           database/creds/selector-role/7NltInpVSc7lPTtybJbkT0Dn
lease_duration     5m
lease_renewable    true
password           -65RFBsvOCkWCfBwFIMN
username           V_TOKEN_SELECTOR-ROLE_W2ZYZDCXFKNWS7N43WMV_1717101832

Static Role Creation

In order to use static roles, the user must already exist in the Redis ACL list. The example below assumes that there is an existing user with the name "vault-edu". If the user does not exist you will receive the following error.

Error writing data to database/static-roles/static-account: Error making API request.

URL: PUT http://127.0.0.1:8200/v1/database/static-roles/static-account
Code: 400. Errors:

* cannot update static account username

$ vault write database/static-roles/static-account db_name=insecure-redis \
        username="vault-edu" rotation_period="5m"
Success! Data written to: database/static-roles/static-account

To retrieve the credentials for the vault-edu user

$ vault read database/static-creds/static-account
Key                    Value
---                    -----
last_vault_rotation    2020-12-20T10:39:49.647822-06:00
password               ylKNgqa3NPVAioBf-0S5
rotation_period        5m
ttl                    3m59s
username               vault-edu

Spring Cloud Vault Integration

Tested on spring-cloud-vault:3.1.0

In order to enable integration with Spring Cloud Vault and therefore supply dynamically-generated Redis credentials to Spring applications, we can use org.springframework.cloud:spring-cloud-vault-config-databases with Multiple Databases configuration approach.

Sample application.yml configuration (not-related sections are omitted):

spring:
  cloud:
    vault:
      host: 127.0.0.1
      port: 8200
      authentication: TOKEN
      token: ${VAULT_TOKEN}
      databases:
        redis:
          enabled: true
          role: my-redis-role
          backend: database
          username-property: spring.redis.username
          password-property: spring.redis.password
  config:
    import: vault://

Please note: Spring Cloud Vault does not support max_ttl yet, thus we have to set it up to 0 when creating configurations. More details can be found here.

Developing

A set of make targets are provided for quick and easy iterations when developing. These steps assume there is a Vault server running locally and accessible via the vault CLI. See this documentation on how to get started with Vault.

1. make setup-(env|cluster|sentinel|primary-secondary) will start a Redis docker installation and initialize a test user with the username default and password default-pa55w0rd. The cluster, sentinel and primary-secondary installations use docker bridge networking and will only work on Linux servers. They default to fully encrypted with mutual TLS enabled. To run then in plain text mode, pass the -var=use-tls=false flag on the terraform apply command line.
2. source ./bootstrap/terraform/local_environment_setup.sh will export the necessary environment variables generated from the setup step. For the cluster, sentinel and primary-secondary installations, source the export-(cluster|sentinel|primary-secondary)-vars.sh file
3. make configure will build the plugin, register it in your local Vault server and run sample commands to verify everything is working
4. make testacc will run the acceptance tests against the Redis container created during the environment setup
5. make teardown-(env|cluster|sentinel|primary-secondry) will stop the Redis docker installation with any resources generated alongside it such as network configs

When iterating, you can reload any local code changes with make configure as many times as desired to test the latest modifications via the Vault CLI or API.

Note: The docker bridge networking works with a fixed network subnet address of 192.168.200.0/28. This means that the terraform generated Redis server certificate can be generated with a list of actual IP addresses removing the need to test using the insecure_tls=true in the tests.