Skip to content

Commit

Permalink
backport of commit 5fac327 (#26177)
Browse files Browse the repository at this point in the history
Co-authored-by: Robert <17119716+robmonte@users.noreply.github.com>
  • Loading branch information
1 parent 7193079 commit b1126ec
Show file tree
Hide file tree
Showing 9 changed files with 485 additions and 10 deletions.
103 changes: 103 additions & 0 deletions website/content/docs/commands/operator/import.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
---
layout: docs
page_title: operator import - Command
description: >-
The "operator import" command imports secrets from external systems
in to Vault.
---

# operator import

@include 'alerts/enterprise-only.mdx'

@include 'alerts/alpha.mdx'

The `operator import` command imports secrets from external systems in to Vault.
Secrets with the same name at the same storage path will be overwritten upon import.

<Note title="Imports can be long-running processes">

You can write import plans that read from as many sources as you want. The
amount of data migrated from each source depends on the filters applied and the
dataset available. Be mindful of the time needed to read from each source,
apply any filters, and store the data in Vault.

</Note>

## Examples

Read the config file `import.hcl` to generate a new import plan:

```shell-session
$ vault operator import -config import.hcl plan
```

Output:

<CodeBlockConfig hideClipboard>

-----------
Import plan
-----------
The following namespaces are missing:
* ns-1/

The following mounts are missing:
* ns-1/mount-1

Secrets to be imported to the destination "my-dest-1":
* secret-1
* secret-2

</CodeBlockConfig>

## Configuration

The `operator import` command uses a dedicated configuration file to specify the source,
destination, and mapping rules. To learn more about these types and secrets importing in
general, refer to the [Secrets Import documentation](/vault/docs/import).

```hcl
source_gcp {
name = "my-gcp-source-1"
credentials = "@/path/to/service-account-key.json"
}
destination_vault {
name = "my-dest-1"
address = "http://127.0.0.1:8200/"
token = "root"
namespace = "ns-1"
mount = "mount-1"
}
mapping_passthrough {
name = "my-map-1"
source = "my-gcp-1"
destination = "my-dest-1"
priority = 1
}
```

## Usage

### Arguments

- `plan` - Executes a read-only operation to let operators preview the secrets to import based on the configuration file.

- `apply` - Executes the import operations to read the specified secrets from the source and write them into Vault.
Apply first executes a plan, then asks the user to approve the results before performing the actual import.

### Flags

The `operator import` command accepts the following flags:

- `-config` `(string: "import.hcl")` - Path to the import configuration HCL file. The default path is `import.hcl`.

- `-auto-approve` `(bool: <false>)` - Automatically responds "yes" to all user-input prompts for the `apply` command.

- `-auto-create` `(bool: <false>)` - Automatically creates any missing namespaces and KVv2 mounts when
running the `apply` command.

- `-log-level` ((#\_log_level)) `(string: "info")` - Log verbosity level. Supported values (in
order of descending detail) are `trace`, `debug`, `info`, `warn`, and `error`. You can also set log-level with the `VAULT_LOG_LEVEL` environment variable.
1 change: 1 addition & 0 deletions website/content/docs/commands/operator/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ Usage: vault operator <subcommand> [options] [args]
Subcommands:
generate-root Generates a new root token
import Import secrets from external systems into Vault
init Initializes a server
key-status Provides information about the active encryption key
rekey Generates new unseal keys
Expand Down
52 changes: 52 additions & 0 deletions website/content/docs/import/gcpsm.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
layout: docs
page_title: GCP secret import source
description: The Google Cloud Platform Secret Manager source imports secrets from GCP to Vault.
---

# GCP import source

Use the GCP source to import secret data from GCP Secret Manager into your Vault instance. To use dynamic
credentials with GCP import, ensure the [GCP secrets engine](/vault/docs/secrets/gcp) is
already configured.

## Argument reference

Refer to the [HCL syntax](/vault/docs/import#hcl-syntax-1) for arguments common to all source types.

## Additional arguments

- `credentials` `(string: "")` - The path to the service account key credentials file for the service account
with the [necessary permissions](#permissions). If `credentials` is set, then `vault_mount_path` and
`vault_role_name` must be unset.

- `vault_mount_path` `(string: "")` - The Vault mount path to a pre-configured GCP
secrets engine used to generate dynamic credentials for the importer. If
`vault_mount_path` or `vault_role_name` are set, then `credentials` must be
unset.

- `vault_role_name` `(string: "")` - The Vault role used to generate a dynamic
credential for the importer. The role name must exist in the pre-configured
GCP secrets engine mount. If `vault_role_name` or `vault_mount_path` are set,
then `credentials` must be unset.

## Example

Define and configure the `my-gcp-source-1` GCP source:

```hcl
source_gcp {
name = "my-gcp-source-1"
secrets_engine_mount = "gcp"
secrets_engine_role = "my-gcp-role-1"
}
```

## Permissions

To use GCP import, you must grant the associated GCP identity permission to read secrets:

```shell-session
"secretmanager.secrets.list",
"secretmanager.versions.access",
```
153 changes: 153 additions & 0 deletions website/content/docs/import/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
---
layout: docs
page_title: Secrets import
description: Secrets import allows you to safely onboard secrets from external sources into Vault KV for management.
---


# Secrets import

@include 'alerts/enterprise-only.mdx'

@include 'alerts/alpha.mdx'

Distributing sensitive information across multiple external systems creates
several challenges, including:

- Increased operational overhead.
- Increased exposure risk from data sprawl.
- Increased risk of outdated and out-of-sync information.

Using Vault as a single source of truth (SSOT) for sensitive data increases
security and reduces management overhead, but migrating preexisting data from multiple
and/or varied sources can be complex and costly.

The secrets import process helps you automate and streamline your sensitive data
migration with codified import plans as HCL files. Import plans tell Vault which KVv2 secrets
engine instance to store the expected secret data in, the source system for which data will be
read from, and how to filter this data. Three HCL blocks make this possible:

- The `destination` block defines target KVv2 mounts.
- The `source` block provides credentials for connecting to the external system.
- The `mapping` block defines how Vault should decide which data gets imported before
writing the information to KVv2.

## Destinations

Vault stores imported secrets in a Vault KVv2 secrets engine mount. Destination
blocks start with `destination_vault` and define the desired KVv2 mount path and
an optional namespace. The combination of these represent the exact location in your
Vault instance you want the information stored.

### HCL syntax


```hcl
destination_vault {
name = "my-dest-1"
namespace = "ns-1"
mount = "mount-1"
}
```

- `name` `(string: <required>)` - A unique name for the destination block that can
be referenced in subsequent mapping blocks.

- `mount` `(string: <required>)` - The mount path for the target KVv2 instance.

- `address` `(string)` - Optional network address of the Vault server with the
KVv2 secrets engine enabled. By default, the Vault client's address will be used.

- `token` `(string)` - Optional authentication token for the Vault server at the
specified address. By default, the Vault client's token will be used.

- `namespace` `(string)` - Optional namespace path containing the specified KVv2
mount. By default, Vault looks for the KVv2 mount under the root namespace.



## Sources

Vault can import secrets from the following sources:
- [GCP Secret Manager](/vault/docs/import/gcpsm)

To pull data from a source during import, Vault needs read credentials for the
external system. You can provide credentials directly as part of the import
plan, or use Vault to automatically generate dynamic credentials if you already
have the corresponding secrets engine configured.

### HCL syntax

Source blocks start with `source_<external_system>` and include any connection
information required by the target system or the secrets engine to leverage. For example:

```hcl
source_gcp {
name = "my-gcp-source-1"
credentials = "@/path/to/service-account-key.json"
}
```

- `name` `(string: <required>)` - A unique name for the source block that can be
referenced in subsequent mapping blocks.

- `credentials` `(string: <required>)` - Path to a credential file or token with
read permissions for the target system.

Depending on the source system, additional information may be required. Refer to
the connection documentation for your source system to determine the full set of
required fields for that system type.



## Mappings

Mappings glue the source and destination together and filter the migrated data,
to determine what is imported and what is ignored. Vault currently supports the
following mapping methods:

- [mapping_passthrough](/vault/docs/import/mappings#passthrough)
- [mapping_metadata](/vault/docs/import/mappings#metadata)
- [mapping_regex](/vault/docs/import/mappings#regex)

### HCL syntax

Mapping blocks start with `mapping_<filter_type>` and require a source name,
destination name, an execution priority, and any corresponding transformations
or filters that apply for each mapping type. For example:

```hcl
mapping_regex {
name = "my-map-1"
source = "my-gcp-source-1"
destination = "my-dest-1"
priority = 1
expression = "^database/.*$"
}
```

- `name` `(string: <required>)` - A unique name for the mapping block.

- `source` `(string: <required>)` - The name of a previously-defined source block
**from** which the data should be read.

- `destination` `(string: <required>)` - The name of a previously defined
destination block **to** which the data should be written.

- `priority` `(integer: <required>)` - The order in which Vault should apply the
mapping block during the import process. The lower the number, the higher the
priority. For example, a mapping with priority 1 executes before a mapping
with priority 2.

Depending on the filter type, additional fields may be required or possible. Refer
to the [import mappings documentation](/vault/docs/import/mappings) for the available
supported options and for a list of each mapping's specific fields.

<Tip title="Priority matters">

Vault applies mapping definitions in priority order and a given secret only
matches to the first mapping that applies. Once Vault imports a secret with a
particular mapping, subsequent reads from the same source will ignore that
secret. See the [priority section](/vault/docs/import/mappings#priority) for an example.

</Tip>
Loading

0 comments on commit b1126ec

Please sign in to comment.