diff --git a/website/content/docs/connect/cluster-peering/index.mdx b/website/content/docs/connect/cluster-peering/index.mdx index 9c42bf5dc274..4dc23cb00c2d 100644 --- a/website/content/docs/connect/cluster-peering/index.mdx +++ b/website/content/docs/connect/cluster-peering/index.mdx @@ -41,6 +41,7 @@ Regardless of whether you connect your clusters through WAN federation or cluste | Shares support queries and service endpoints | ✅ | ✅ | | Connects clusters owned by different operators | ❌ | ✅ | | Functions without declaring primary datacenter | ❌ | ✅ | +| Can use sameness groups for identical services | ❌ | ✅ | | Replicates exported services for service discovery | ❌ | ✅ | | Gossip protocol: Requires LAN gossip only | ❌ | ✅ | | Forwards service requests for service discovery | ✅ | ❌ | @@ -60,6 +61,7 @@ The following resources are available to help you use Consul's cluster peering f - [Establish cluster peering connections](/consul/docs/connect/cluster-peering/usage/establish-cluster-peering) - [Manage cluster peering connections](/consul/docs/connect/cluster-peering/usage/manage-connections) - [Manage L7 traffic with cluster peering](/consul/docs/connect/cluster-peering/usage/peering-traffic-management) +- [Create sameness groups](/consul/docs/connect/cluster-peering/usage/create-sameness-groups) ### Kubernetes documentation @@ -67,6 +69,7 @@ The following resources are available to help you use Consul's cluster peering f - [Establish cluster peering connections on Kubernetes](/consul/docs/k8s/connect/cluster-peering/usage/establish-peering) - [Manage cluster peering connections on Kubernetes](/consul/docs/k8s/connect/cluster-peering/usage/manage-peering) - [Manage L7 traffic with cluster peering on Kubernetes](/consul/docs/k8s/connect/cluster-peering/usage/l7-traffic) +- [Create sameness groups on Kubernetes](/consul/docs/k8s/connect/cluster-peering/usage/create-sameness-groups) ### HCP Consul documentation diff --git a/website/content/docs/connect/cluster-peering/usage/create-sameness-groups.mdx b/website/content/docs/connect/cluster-peering/usage/create-sameness-groups.mdx new file mode 100644 index 000000000000..96047cc9c46e --- /dev/null +++ b/website/content/docs/connect/cluster-peering/usage/create-sameness-groups.mdx @@ -0,0 +1,301 @@ +--- +page_title: Create sameness groups +description: |- + Learn how to create sameness groups between partitions and cluster peers so that Consul can identify instances of the same service across partitions and datacenters. +--- + +# Create sameness groups + +This topic describes how to designate groups of services as functionally identical in your network. Sameness groups are the preferred failover strategy for deployments with cluster peering connections. For more information about sameness groups and failover, refer to [Failover overview](/consul/docs/connect/failover). + + + +Sameness groups are a beta feature in this version of Consul. Functionality is subject to change. You should never use the beta release in secure environments or production scenarios. Features in beta may experience performance issues, scaling issues, and limited support. + + + +## Workflow + +Sameness groups are a user-defined set of partitions that Consul uses to identify services in different administrative partitions with the same name and namespace as being the same service. One of the use cases for sameness groups is to create a blanket failover policy for deployments with cluster peering connections. + +To create and use sameness groups in your network, complete the following steps: + +- **Create sameness group configuration entries for each member of the group**. For each partition that you want to include in the sameness group, you must write and apply a sameness group configuration entry that defines the group’s members from that partition’s perspective. Refer to the [sameness group configuration entry reference](/consul/docs/connect/config-entries/sameness-group) for details on configuration hierarchy, default values, and specifications. +- **Export services to members of the sameness group**. You must write and apply an exported services configuration entry that makes the partition’s services available to other members of the group. Refer to [exported services configuration entry reference](/consul/docs/connect/config-entries/exported-services) for additional specification information. +- **Create service intentions for each member of the sameness group**. For each partition that you want to include in the sameness group, you must write and apply service intentions configuration entries to authorize traffic to your services from all members of the group. Refer to the [service intentions configuration entry reference](/consul/docs/connect/config-entries/service-intentions) for additional specification information. + +## Requirements + +- All datacenters where you want to create sameness groups must run Consul v1.16 or later. Refer to [upgrade instructions](/consul/docs/upgrading/instructions) for more information about how to upgrade your deployment. +- A [Consul Enterprise license](/consul/docs/enterprise/license/overview) is required. + +### Before you begin + +Before creating a sameness group, take the following actions to prepare your network: + +#### Check namespace and service naming conventions + +Sameness groups are defined at the partition level. As a result, Consul assumes that namespaces and services in a sameness group that have the same name are identical. This behavior occurs even when two different partitions in the group contain functionally different services that share a name and namespace. This situation would lead to errors, as requests will be sent to the incorrect service. + +To prevent errors, check the names of the services deployed to your network and the namespaces they are deployed in. Pay particular attention to the default namespace to confirm that services have unique names. If different services share a name, you should either change one of the service’s name or deploy one of the services to a different namespace. After adding the partition to the sameness group, differentiating the names of services or namespaces enables Consul to correctly assume which services are identical. + +#### Deploy mesh gateways for each partition + +Mesh gateways are required for cluster peering connections and recommended to secure cross-partition traffic in a single datacenter. Therefore, we recommend securing your network, and especially your production environment, by deploying mesh gateways to each datacenter. Refer to [mesh gateways specifications](/consul/docs/connect/cluster-peering/tech-specs#mesh-gateway-specifications) for more information about configuring mesh gateways. + +#### Establish cluster peering relationships between remote partitions + +You must establish connections with cluster peers before you can create a sameness group that includes them. A cluster peering connection exists between two admin partitions in different datacenters, and each connection between two partitions must be established separately with each peer. Refer to [establish cluster peering connections](/consul/docs/connect/cluster-peering/usage/establish-cluster-peering) for step-by-step instructions. + +To establish cluster peering connections and define a define group as part of the same workflow, follow instructions up to [Export services between clusters](/consul/docs/connect/cluster-peering/usage/establish-cluster-peering#export-services-between-clusters). You can use the same exported services and service intention configuration entries to establish the cluster peering connection and create the sameness group. + +## Create a sameness group + +To create a sameness group, you must write and apply sets of three configuration entries for each partition that is a member of the group: + +- Sameness group configuration entries: Define the sameness group from each partition’s perspective. +- Exported services configuration entries: Make services available to other partitions in the group. +- Service intentions configuration entries: Authorize traffic between services across partitions. + +### Define the sameness group from each partition’s perspective + +To define a sameness group for a partition, create a [sameness group configuration entry](/consul/docs/connect/config-entries/sameness-group) that describes the partitions and cluster peers that are part of the group. Typically, this order follows this pattern: + +1. The local partition +1. Other partitions in the same datacenter +1. Partitions with established cluster peering relationships + +If you want all services to failover to other instances in the sameness group by default, set `DefaultForFailover=true` and list the group members in the order you want to use in a failover scenario. + +After you create the configuration entry, apply it to the Consul server with the following CLI command: + +```shell-session +$ consul config write sameness-group-a.hcl +``` + +Then, repeat the process to create and apply a configuration entry for every partition that is a member of the sameness group. + +Be aware that the sameness group configuration entries are different for each partition. The following example demonstrates how to format three different configuration entries for three partitions that are part of the sameness group `sameness-group-a` when Partition 1 and Partition 2 are in DC1, and the third partition is Partition 1 in DC2: + + + + + +```hcl +Kind = "sameness-group" +Name = "sameness-group-a" +Partition = "partition-1" +DefaultForFailover = true +Members = [ + {Partition = "partition-1"}, + {Partition = "partition-2"}, + {Peer = "dc2-partition-1"} + ] +``` + + + + + +```hcl +Kind = "sameness-group" +Name = "sameness-group-a" +Partition = "partition-2" +DefaultForFailover = true +Members = [ + {Partition = "partition-2"}, + {Partition = "partition-1"}, + {Peer = "dc2-partition-1"} + ] +``` + + + + + +```hcl +Kind = "sameness-group" +Name = "sameness-group-a" +Partition = "partition-1" +DefaultForFailover = true +Members = [ + {Partition = "partition-1"}, + {Peer = "dc1-partition-1"}, + {Peer = "dc1-partition-2"} + ] +``` + + + + + +### Export services to other partitions in the sameness group + +To make services available to other members of the sameness group, you must write and apply an [exported services configuration entry](/consul/docs/connect/config-entries/exported-services) to each partition in the group. This configuration entry exports the local partition's services to the rest of the group members. In each configuration entry, set the sameness group as the `Consumer` for the exported services. You can export multiple services in a single exported services configuration entry. + +Because you are configuring the consumer to reference the sameness group instead of listing out each partition and cluster peer, you do not need to edit this configuration again when you add a partition or peer to the group. + +The following example demonstrates how to format three different `exported-service` configuration entries to make a service named `api` deployed to the `store` namespace of each partition available to all other group members: + + + + + +```hcl +Kind = "exported-services" +Name = "group-a-export" +Partition = "partition-1" +Services = [ + { + Name = "api" + Namespace = "store" + Consumers = [ + {SamenessGroup="sameness-group-a"} + ] + } + ] +``` + + + + + +```hcl +Kind = "exported-services" +Name = "group-a-export" +Partition = "partition-2" +Services = [ + { + Name = "api" + Namespace = "store" + Consumers = [ + {SamenessGroup="sameness-group-a"} + ] + } + ] +``` + + + + + +```hcl +Kind = "exported-services" +Name = "group-a-export" +Partition = "partition-1" +Services = [ + { + Name = "api" + Namespace = "store" + Consumers = [ + {SamenessGroup="sameness-group-a"} + ] + } + ] +``` + + + + + +For more information about exporting services, including examples of configuration entries that export multiple services at the same time, refer to the [exported services configuration entry reference](/consul/docs/connect/config-entries/exported-services). + +After you create each exported services configuration entry, apply it to the Consul server with the following CLI command: + +```shell-session +$ consul config write group-a-export.hcl +``` + +#### Export services for cluster peers and sameness groups at the same time + +It is possible to combine the exported services configuration entry for creating sameness groups with the entry required to [export services when establishing cluster peering connections](/consul/docs/connect/cluster-peering/usage/establish-cluster-peering#export-services-between-clusters). + +In this workflow, configure the `Services[].Consumers` block with the `SamenessGroup` field instead of the `Peer` field. You should write the `sameness-group` configuration entry to Consul before referencing it in the `exported-services` configuration entry. + +### Create service intentions to authorize traffic between group members + +While exporting the service to each member of the sameness group makes the services visible to remote partitions, you must also create service intentions so that local services are authorized to send and receive traffic from a member of the sameness group. + +For each partition that is member of the group, write and apply a [service intentions configuration entry](/consul/docs/connect/config-entries/service-intentions) that defines intentions for the services that are part of the group. In the `Sources` block of the configuration entry, include the service name, its namespace, the sameness group, and grant `allow` permissions. + +Because you are using the sameness group in the `Sources` block rather than listing out each partition and cluster peer, you do not have to make further edits to the service intentions configuration entries when members are added to or removed from the group. + +The following example demonstrates how to format three different `service-intentions` configuration entries to make a service named `api` available to all instances of `payments` deployed in all members of the sameness group including the local partition. In this example, `api` is deployed to the `store` namespace in all three partitions. + + + + + + +```hcl +Kind = "service-intentions" +Name = "api" +Namespace= "store" +Partition="partition-1" +Sources = [ + Name = "api" + Action = "allow" + Namespace = "store" + SamenessGroup = "sameness-group-a" +] +``` + + + + + +```hcl +Kind = "service-intentions" +Name = "api" +Namespace= "store" +Partition= "partition-2" +Sources = [ + Name = "api" + Action = "allow" + Namespace = "store" + SamenessGroup = "sameness-group-a" +] +``` + + + + + +```hcl +Kind = "service-intentions" +Name = "api" +Namespace= "store" +Partition= "partition-1" +Sources = [ + Name = "api" + Action = "allow" + Namespace = "store" + SamenessGroup = "sameness-group-a" +] +``` + + + + + +Refer to [create and manage intentions](/consul/docs/connect/intentions/create-manage-intentions) for more information about how to create and apply service intentions in Consul. + +After you create each service intentions configuration entry, apply it to the Consul server with the following CLI command: + +```shell-session +$ consul config write api.hcl +``` + +#### Create service intentions for cluster peers and sameness groups at the same time + +It is possible to combine the service intentions configuration entry for creating sameness groups with the entry required to [authorize services for peers](/consul/docs/connect/cluster-peering/usage/establish-cluster-peering#authorize-services-for-peers). + +In this workflow, configure the `Sources` block with the `SamenessGroup` field instead of the `Peer` field. You should write the `sameness-group` configuration entry to Consul before referencing it in the `service-intentions` configuration entry. + +## Next steps + +After creating a sameness group, you can use them with static Consul DNS lookups and dynamic DNS lookups (prepared queries) for service discovery use cases. You can also set up failover between services in a sameness group. Refer to the following topics for more details: + +- [Static Consul DNS lookups](/consul/docs/services/discovery/dns-static-lookups) +- [Dynamic Consul DNS lookups](/consul/docs/services/discovery/dns-dynamic-lookups) +- [Failover overview](/consul/docs/connect/failover) \ No newline at end of file diff --git a/website/content/docs/connect/cluster-peering/usage/establish-cluster-peering.mdx b/website/content/docs/connect/cluster-peering/usage/establish-cluster-peering.mdx index b8549c5c3fbe..96e7815d75d9 100644 --- a/website/content/docs/connect/cluster-peering/usage/establish-cluster-peering.mdx +++ b/website/content/docs/connect/cluster-peering/usage/establish-cluster-peering.mdx @@ -14,7 +14,7 @@ This page details the process for establishing a cluster peering connection betw 1. Export services between clusters. 1. Create intentions to authorize services for peers. -Cluster peering between services cannot be established until all four steps are complete. +Cluster peering between services cannot be established until all four steps are complete. If you want to establish cluster peering connections and create sameness groups at the same time, refer to the guidance in [create sameness groups](/consul/docs/connect/cluster-peering/usage/create-sameness-groups). For Kubernetes guidance, refer to [Establish cluster peering connections on Kubernetes](/consul/docs/k8s/connect/cluster-peering/usage/establish-peering). For HCP Consul guidance, refer to [Establish cluster peering connections on HCP Consul](/hcp/docs/consul/usage/cluster-peering/create-connections). diff --git a/website/content/docs/connect/config-entries/exported-services.mdx b/website/content/docs/connect/config-entries/exported-services.mdx index d3a793c891a5..089eeb572314 100644 --- a/website/content/docs/connect/config-entries/exported-services.mdx +++ b/website/content/docs/connect/config-entries/exported-services.mdx @@ -217,6 +217,9 @@ an exported service. Each item in the `Consumers` list must contain exactly one A asterisk wildcard (`*`) cannot be specified as the `Peer`. Added in Consul 1.13.0. - `Partition`: Specifies an admin partition in the datacenter to export the service to. A asterisk wildcard (`*`) cannot be specified as the `Partition`. +- `SamenessGroup`: Specifies as sameness group to export the service to. +A asterisk wildcard (`*`) cannot be specified as the `SamennessGroup`. + ## Examples diff --git a/website/content/docs/connect/config-entries/index.mdx b/website/content/docs/connect/config-entries/index.mdx index de5acd1bce96..a27bca1c5d3f 100644 --- a/website/content/docs/connect/config-entries/index.mdx +++ b/website/content/docs/connect/config-entries/index.mdx @@ -23,6 +23,8 @@ The following configuration entries are supported: - [Proxy Defaults](/consul/docs/connect/config-entries/proxy-defaults) - controls proxy configuration +- [Sameness Group](/consul/docs/connect/config-entries/sameness-group) - defines partitions and cluster peers with identical services + - [Service Defaults](/consul/docs/connect/config-entries/service-defaults) - configures defaults for all the instances of a given service diff --git a/website/content/docs/connect/config-entries/sameness-group.mdx b/website/content/docs/connect/config-entries/sameness-group.mdx new file mode 100644 index 000000000000..8d19b989ff3b --- /dev/null +++ b/website/content/docs/connect/config-entries/sameness-group.mdx @@ -0,0 +1,375 @@ +--- +page_title: Sameness group configuration entry reference +description: |- + Sameness groups enable Consul to associate service instances with the same name deployed to the same namespace as identical services. Learn how to configure a `sameness-group` configuration entry to enable failover between partitions and cluster peers in non-federated networks. +--- + +# Sameness groups configuration entry reference + +This page provides reference information for sameness group configuration entries. Sameness groups associate services with identical names across partitions and cluster peers. + +To learn more about creating a sameness group, refer to [Create sameness groups](/consul/docs/connect/cluster-peering/usage/create-sameness-groups) or [Create sameness groups on Kubernetes](/consul/docs/k8s/connect/cluster-peering/usage/create-sameness-groups). + + + +Sameness groups are a beta feature in this version of Consul. Functionality is subject to change. You should never use the beta release in secure environments or production scenarios. Features in beta may experience performance issues, scaling issues, and limited support. + + + +## Configuration model + +The following list outlines field hierarchy, language-specific data types, and requirements in the sameness group configuration entry. Click on a property name to view additional details, including default values. + + + + + +- [`Kind`](#kind): string | required | must be set to `sameness-group` +- [`Name`](#name): string | required +- [`Partition`](#partition): string | `default` +- [`DefaultForFailover`](#defaultforfailover): boolean | `false` +- [`Members`](#members): list of maps | required + - [`Partition`](#members-partition): string + - [`Peer`](#members-peer): string + + + + + +- [`apiVersion`](#apiversion): string | required | must be set to `consul.hashicorp.com/v1alpha1` +- [`kind`](#kind): string | required | must be set to `SamenessGroup` +- [`metadata`](#metadata): map | required + - [`name`](#metadata-name): string | required +- [`spec`](#spec): map | required + - [`DefaultForFailover`](#spec-defaultforfailover): boolean | `false` + - [`Members`](#spec-members): list of maps | required + - [`Partition`](#spec-members-partition): string + - [`Peer`](#spec-members-peer): string + + + + +## Complete configuration + +When every field is defined, a sameness group configuration entry has the following form: + + + + + +```hcl +Kind = "sameness-group" # required +Name = "" # required +Partition = "" +DefaultForFailover = false +Members = [ # required + { Partition = "" }, + { Peer = "" } +] +``` + + + + + +```json +{ + "Kind": "sameness-group", // required + "Name": "", // required + "Partition": "", + "DefaultForFailover": false, + "Members": [ // required + { + "Partition": "" + }, + { + "Peer": "" + } + ] +} +``` + + + + + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 # required +kind: SamenessGroup # required +metadata: + name: +spec: + defaultForFailover: false + members: # required + partition: + peer: +``` + + + + +## Specifications + +This section provides details about the fields you can configure in the sameness group configuration entry. + + + + + +### `Kind` + +Specifies the type of configuration entry to implement. Must be set to `sameness-group`. + +#### Values + +- Default: None +- This field is required. +- Data type: String value that must be set to `sameness-group`. + +### `Name` + +Specifies a name for the configuration entry that is used to identify the sameness group. To ensure consistency, use descriptive names and make sure that the same name is used when creating configuration entries to add each member to the sameness group. + +#### Values + +- Default: None +- This field is required. +- Data type: String + +### `Partition` + +Specifies the local admin partition that the sameness group applies to. Refer to [admin partitions](/consul/docs/enterprise/admin-partitions) for more information. + +#### Values + +- Default: `default` +- Data type: String + +### `DefaultForFailover` + +Determines whether the sameness group should be used to establish connections to services with the same name during failover scenarios. When this field is set to `true`, DNS queries and upstream requests automatically failover to services in the sameness group according to the order of the members in the `Members` list. + +When this field is set to `false`, you can still use a sameness group for failover by configuring the `Failover` block of a [service resolver configuration entry](/consul/docs/connect/config-entries/service-resolver). + +#### Values + +- Default: `false` +- Data type: Boolean + +### `Members` + +Specifies the partitions and cluster peers that are members of the sameness group from the perspective of the local partition. + +The local partition should be the first member listed. The order of the members determines their precedence during failover scenarios. If a member is listed but Consul cannot connect to it, failover proceeds with the next healthy member in the list. For an example demonstrating how to configure this parameter, refer to [Failover between sameness groups](#failover-between-sameness-groups). + +Each partition can belong to a single sameness group. You cannot associate a partition or cluster peer with multiple sameness groups. + +#### Values + +- Default: None +- This field is required. +- Data type: List that can contain maps of the following parameters: + - [`Partition`](#members-partition) + - [`Peer`](#members-peer) + +### `Members[].Partition` + +Specifies a partition in the local datacenter that is a member of the sameness group. When the value of this field is set to `*`, all local partitions become members of the sameness group. + +#### Values + +- Default: None +- Data type: String + +### `Members[].Peer` + +Specifies the name of a cluster peer that is a member of the sameness group. + +Cluster peering connections must be established before adding a peer to the list of members. Refer to [establish cluster peering connections](/consul/docs/connect/cluster-peering/usage/establish-cluster-peering) for more information. + +#### Values + +- Default: None +- Data type: String + + + + + +### `apiVersion` + +Specifies the version of the Consul API for integrating with Kubernetes. The value must be `consul.hashicorp.com/v1alpha1`. + +#### Values + +- Default: None +- This field is required. +- String value that must be set to `consul.hashicorp.com/v1alpha1`. + +### `kind` + +Specifies the type of configuration entry to implement. Must be set to `SamenessGroup`. + +#### Values + +- Default: None +- This field is required. +- Data type: String value that must be set to `SamenessGroup`. + +## `metadata` + +Map that contains an arbitrary name for the configuration entry and the namespace it applies to. + +#### Values + +- Default: None +- Data type: Map + +### `metadata.name` + +Specifies a name for the configuration entry that is used to identify the sameness group. To ensure consistency, use descriptive names and make sure that the same name is used when creating configuration entries to add each member to the sameness group. + +#### Values + +- Default: None +- This field is required. +- Data type: String + +### `spec` + +Map that contains the details about the `SamenessGroup` configuration entry. The `apiVersion`, `kind`, and `metadata` fields are siblings of the spec field. All other configurations are children. + +#### Values + +- Default: None +- This field is required. +- Data type: Map + +### `spec.defaultForFailover` + +Determines whether the sameness group should be used to establish connections to services with the same name during failover scenarios. When this field is set to `true`, DNS queries and upstream requests automatically failover to services in the sameness group according to the order of the members in the `spec.members` list. + +When this field is set to `false`, you can still use a sameness group for failover by configuring the `Failover` block of a [service resolver configuration entry](/consul/docs/connect/config-entries/service-resolver). + +#### Values + +- Default: `false` +- Data type: Boolean + +### `spec.members` + +Specifies the partitions and cluster peers that are members of the sameness group from the perspective of the local partition. + +The local partition should be the first member listed. The order of the members determines their precedence during failover scenarios. If a member is listed but Consul cannot connect to it, failover proceeds with the next healthy member in the list. For an example demonstrating how to configure this parameter, refer to [Failover between sameness groups](#failover-between-sameness-groups). + +Each partition can belong to a single sameness group. You cannot associate a partition or cluster peer with multiple sameness groups. + +#### Values + +- Default: None +- This field is required. +- Data type: List that can contain maps of the following parameters: + + - [`partition`](#spec-members-partition) + - [`peer`](#spec-members-peer) + +### `spec.members[].partition` + +Specifies a partition in the local datacenter that is a member of the sameness group. When the value of this field is set to `*`, all local partitions become members of the sameness group. + +#### Values + +- Default: None +- Data type: String + +### `spec.members[].peer` + +Specifies the name of a cluster peer that is a member of the sameness group. + +Cluster peering connections must be established before adding a peer to the list of members. Refer to [establish cluster peering connections](/consul/docs/connect/cluster-peering/usage/establish-cluster-peering) for more information. + +#### Values + +- Default: None +- Data type: String + + + + +## Examples + +The following examples demonstrate common sameness group configuration patterns for specific use cases. + +### Failover between members of a sameness group + +In the following example, the configuration entry defines a sameness group named `products-api` that applies to the `store-east` partition in the local datacenter. The sameness group is configured so that when its services’ instances in `store-east` fails, Consul will attempt to establish a failover connection in the following order: + +- Services with the same name in the `store-east` partition +- Services with the same name in the `inventory-east` partition in the same datacenter +- Services with the same name in the `store-west` partition of datacenter `dc2`, which has an established cluster peering connection. +- Services with the same name in the `inventory-west` partition of `dc2`, which has an established cluster peering connection. + + + + + +```hcl +Kind = "sameness-group" +Name = "products-api" +Partition = "store-east" +DefaultForFailover = true +Members = [ + { Partition = "store-east" }, + { Partition = "inventory-east" }, + { Peer = "dc2-store-west" }, + { Peer = "dc2-inventory-west" } +] +``` + + + + + +```json +{ + "Kind": "sameness-group", + "Name": "products-api", + "Partition": "store-east", + "DefaultForFailover": true, + "Members": [ + { + "Partition": "store-east" + }, + { + "Partition": "inventory-east" + }, + { + "Peer": "dc2-store-west" + }, + { + "Peer": "dc2-inventory-west" + } + ] +} +``` + + + + + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +kind: SamenessGroup +metadata: + name: products-api +spec: + defaultForFailover: true + members: + partition: store-east + partition: inventory-east + peer: dc2-store-west + peer: dc2-inventory-west +``` + + + \ No newline at end of file diff --git a/website/content/docs/connect/config-entries/service-intentions.mdx b/website/content/docs/connect/config-entries/service-intentions.mdx index 321b7f160580..1fe1ed6bd487 100644 --- a/website/content/docs/connect/config-entries/service-intentions.mdx +++ b/website/content/docs/connect/config-entries/service-intentions.mdx @@ -25,8 +25,9 @@ The following outline shows how to format the service intentions configuration e - [`Sources`](#sources): list | no default - [`Name`](#sources-name): string | no default - [`Peer`](#sources-peer): string | no default - - [`Namespace`](#sources-namespace): string | no default | - - [`Partition`](#sources-partition): string | no default | + - [`Namespace`](#sources-namespace): string | no default | + - [`Partition`](#sources-partition): string | no default | + - [`SamenessGroup`](#sources-samenessgroup): string | no default | - [`Action`](#sources-action): string | no default | required for L4 intentions - [`Permissions`](#sources-permissions): list | no default - [`Action`](#sources-permissions-action): string | no default | required @@ -68,6 +69,7 @@ The following outline shows how to format the service intentions configuration e - [`peer`](#spec-sources-peer): string | no default - [`namespace`](#spec-sources-namespace): string | no default | - [`partition`](#spec-sources-partition): string | no default | + - [`samenessGroup`](#spec-sources-samenessgroup): string | no default | - [`action`](#spec-sources-action): string | no default | required for L4 intentions - [`permissions`](#spec-sources-permissions): list | no default - [`action`](#spec-sources-permissions-action): string | no default | required @@ -113,6 +115,7 @@ Sources = [ Peer = "" # string Namespace = "" # string Partition = "" # string + SamenessGroup = "" # string Action = "allow" or "deny" # string for L4 intentions Permissions = [ { @@ -181,6 +184,7 @@ spec: peer: namespace: partition: + samenessGroup: action: allow or deny permissions: - action: allow or deny @@ -226,6 +230,7 @@ spec: "Peer":"", "Namespace":"", "Partition":"", + "SamenessGroup":"", "Action":"allow or deny", "Permissions":[ { @@ -353,6 +358,7 @@ List of configurations that define intention sources and the authorization grant - `Peer` - `Namespace` - `Partition` + - `SamenessGroup` - `Action` - `Permissions` - `Precedence` @@ -404,6 +410,16 @@ The `Peer` and `Partition` fields are mutually exclusive. - Default: If [`Peer`](#sources-peer) is unspecified, defaults to the destination [`Partition`](#partition). - Data type: string +### `Sources[].SamenessGroup` + +Specifies the name of a sameness group that the intention allows or denies traffic from. Refer to [create samenes groups](/consul/docs/connect/cluster-peering/usage/create-sameness-groups) for additional information. + +#### Values + +- Default: None +- Data type: string + + ### `Sources[].Action` Specifies the action to take when the source sends traffic to the destination service. The value is either `allow` or `deny`. Do not configure this field to apply L7 intentions to the same source. Configure the [`Permissions`](#sources-permissions) field instead. @@ -644,7 +660,8 @@ List of configurations that define intention sources and the authorization grant - `peer` - `namespace` - `partition` - - `Action` + - `samenessGroup` + - `action` - `permissions` - `type` - `description` @@ -685,6 +702,15 @@ Specifies the name of an admin partition that the intention allows or denies tra - Default: If [`peer`](#sources-peer) is unspecified, defaults to the partition specified in [`spec.destination.partition`](#spec-destination-partition). - Data type: String +### `spec.sources[].samenessGroup` + +Specifies the name of a sameness group that the intention allows or denies traffic from. Refer to [create samenes groups](/consul/docs/k8s/connect/cluster-peering/usage/create-sameness-groups) for additional information. + +#### Values + +- Default: None +- Data type: string + ### `spec.sources[].action` Specifies the action to take when the source sends traffic to the destination service. The value is either `allow` or `deny`. Do not configure this field for L7 intentions. Configure the [`spec.sources.permissions`](#spec-sources-permissions) field instead. diff --git a/website/content/docs/enterprise/index.mdx b/website/content/docs/enterprise/index.mdx index b6805604b1e8..60207b136bb3 100644 --- a/website/content/docs/enterprise/index.mdx +++ b/website/content/docs/enterprise/index.mdx @@ -19,6 +19,7 @@ The following features are [available in several forms of Consul Enterprise](#co ### Multi-Tenancy - [Admin Partitions](/consul/docs/enterprise/admin-partitions): Define administrative boundaries between tenants within a single Consul datacenter - [Namespaces](/consul/docs/enterprise/namespaces): Define resource boundaries within a single admin partition for further organizational flexibility +- [Sameness Groups](/consul/docs/connect/config-entries/samenes-group): Define partitions and cluster peers as members of a group with identical services ### Resiliency - [Automated Backups](/consul/docs/enterprise/backups): Configure the automatic backup of Consul state @@ -83,6 +84,7 @@ Available Enterprise features per Consul form and license include: | [Network Segments](/consul/docs/enterprise/network-segments/network-segments-overview) | No | Yes | With Global Visibility, Routing, and Scale module | | [OIDC Auth Method](/consul/docs/security/acl/auth-methods/oidc) | No | Yes | Yes | | [Redundancy Zones](/consul/docs/enterprise/redundancy) | Not applicable | Yes | With Global Visibility, Routing, and Scale module | +| [Sameness Groups](/consul/docs/connect/config-entries/samenes-group) | No | Yes | N/A | | [Sentinel for KV](/consul/docs/enterprise/sentinel) | All tiers | Yes | With Governance and Policy module | [HashiCorp Cloud Platform (HCP) Consul]: https://cloud.hashicorp.com/products/consul @@ -108,6 +110,7 @@ Consul Enterprise feature availability can change depending on your server and c | [Network Segments](/consul/docs/enterprise/network-segments/network-segments-overview) | ✅ | ❌ | ❌ | | [OIDC Auth Method](/consul/docs/security/acl/auth-methods/oidc) | ✅ | ✅ | ✅ | | [Redundancy Zones](/consul/docs/enterprise/redundancy) | ✅ | ✅ | ✅ | +| [Sameness Groups](/consul/docs/connect/config-entries/samenes-group) | ✅ | ✅ | ✅ | | [Sentinel ](/consul/docs/enterprise/sentinel) | ✅ | ✅ | ✅ | @@ -126,6 +129,7 @@ Consul Enterprise feature availability can change depending on your server and c | [Network Segments](/consul/docs/enterprise/network-segments/network-segments-overview) | ❌ | ❌ | ❌ | | [OIDC Auth Method](/consul/docs/security/acl/auth-methods/oidc) | ✅ | ✅ | ✅ | | [Redundancy Zones](/consul/docs/enterprise/redundancy) | ❌ | ❌ | ❌ | +| [Sameness Groups](/consul/docs/connect/config-entries/samenes-group) | ✅ | ✅ | ✅ | | [Sentinel ](/consul/docs/enterprise/sentinel) | ✅ | ✅ | ✅ | @@ -144,6 +148,7 @@ Consul Enterprise feature availability can change depending on your server and c | [Network Segments](/consul/docs/enterprise/network-segments/network-segments-overview) | ❌ | ❌ | ❌ | | [OIDC Auth Method](/consul/docs/security/acl/auth-methods/oidc) | ❌ | ❌ | ❌ | | [Redundancy Zones](/consul/docs/enterprise/redundancy) | n/a | n/a | n/a | +| [Sameness Groups](/consul/docs/connect/config-entries/samenes-group) | ✅ | ✅ | ✅ | | [Sentinel ](/consul/docs/enterprise/sentinel) | ✅ | ✅ | ✅ | diff --git a/website/content/docs/k8s/connect/cluster-peering/usage/create-sameness-groups.mdx b/website/content/docs/k8s/connect/cluster-peering/usage/create-sameness-groups.mdx new file mode 100644 index 000000000000..24e9e5874be1 --- /dev/null +++ b/website/content/docs/k8s/connect/cluster-peering/usage/create-sameness-groups.mdx @@ -0,0 +1,290 @@ +--- +page_title: Create sameness groups +description: |- + Learn how to create sameness groups between partitions and cluster peers on Kubernetes so that Consul can identify instances of the same service across partitions and datacenters. +--- + +# Create sameness groups on Kubernetes + +This topic describes how to designate groups of services as functionally identical in your network on Consul deployments running Kubernetes. Sameness groups are the preferred failover strategy for deployments with cluster peering connections. For more information about sameness groups and failover, refer to [Failover overview](/consul/docs/connect/failover). + + + +Sameness groups are a beta feature in this version of Consul. Functionality is subject to change. You should never use the beta release in secure environments or production scenarios. Features in beta may experience performance issues, scaling issues, and limited support. + + + +## Overview + +Sameness groups are a user-defined set of partitions that Consul uses to identify services in different administrative partitions with the same name as being the same service. One of the use cases for sameness groups is to create a blanket failover policy for deployments with cluster peering connections. + +To create and use sameness groups in your network, complete the following steps: + +- **Create sameness group configuration entries for each member of the group**. For each partition that you want to include in the sameness group, you must write and apply a sameness group configuration entry that defines the group’s members from that partition’s perspective. Refer to the [sameness group configuration entry reference](/consul/docs/connect/config-entries/sameness-group) for details on configuration hierarchy, default values, and specifications. +- **Export services to members of the sameness group**. You must write and apply an exported services configuration entry that makes the partition’s services available to other members of the group. Refer to [exported services configuration entry reference](/consul/docs/connect/config-entries/exported-services) for additional specification information. +- **Create service intentions for each member of the sameness group**. For each partition that you want to include in the sameness group, you must write and apply service intentions configuration entries to authorize traffic to your services from all members of the group. Refer to the [service intentions configuration entry reference](/consul/docs/connect/config-entries/service-intentions) for additional specification information. + +## Requirements + +- All datacenters where you want to create sameness groups must run Consul v1.16 or later. Refer to [upgrade instructions](/consul/docs/k8s/upgrade) for more information about how to upgrade your deployment. +- A [Consul Enterprise license](/consul/docs/enterprise/license/overview) is required. + +### Before you begin + +Before creating a sameness group, take the following actions to prepare your network: + +#### Check namespace and service naming conventions + +Sameness groups are defined at the partition level. As a result, Consul assumes that namespaces and services in a sameness group that have the same name are identical. This behavior occurs even when two different partitions in the group contain functionally different services that share a name and namespace. This situation would lead to errors, as requests will be sent to the incorrect service. + +To prevent errors, check the names of the services deployed to your network and the namespaces they are deployed in. Pay particular attention to the default namespace to confirm that services have unique names. If different services share a name, you should either change one of the service’s name or deploy one of the services to a different namespace. After adding the partition to the sameness group, differentiating the names of services or namespaces enables Consul to correctly assume which services are identical. + +#### Deploy mesh gateways for each partition + +Mesh gateways are required for cluster peering connections and recommended to secure cross-partition traffic in a single datacenter. Therefore, we recommend securing your network, and especially your production environment, by deploying mesh gateways to each datacenter. Refer to [mesh gateways specifications](/consul/docs/k8s/connect/cluster-peering/tech-specs#mesh-gateway-specifications) for more information about configuring mesh gateways. + +#### Establish cluster peering relationships between remote partitions + +You must establish connections with cluster peers before you can create a sameness group that includes them. A cluster peering connection exists between two admin partitions in different datacenters, and each connection between two partitions must be established separately with each peer. Refer to [establish cluster peering connections](/consul/docs/k8s/connect/cluster-peering/usage/establish-peering) for step-by-step instructions. + +To establish cluster peering connections and define a define group as part of the same workflow, follow instructions up to [Export services between clusters](/consul/docs/k8s/connect/cluster-peering/usage/establish-peering#export-services-between-clusters). You can use the same exported services and service intention configuration entries to establish the cluster peering connection and create the sameness group. + +## Create a sameness group + +To create a sameness group, you must write and apply sets of three configuration entries for each partition that is a member of the group: + +- Sameness group configuration entries: Define the sameness group from each partition’s perspective. +- Exported services configuration entries: Make services available to other partitions in the group. +- Service intentions configuration entries: Authorize traffic between services across partitions. + +### Define the sameness group from each partition’s perspective + +To define a sameness group for a partition, create a [sameness group configuration entry](/consul/docs/connect/config-entries/sameness-group) that describes the partitions and cluster peers that are part of the group. Typically, this order follows this pattern: + +1. The local partition +1. Other partitions in the same datacenter +1. Partitions with established cluster peering relationships + +If you want all services to failover to other instances in the sameness group by default, set `spec.defaultForFailover=true` and list the group members in the order you want to use in a failover scenario. + +After you create the configuration entry, apply it to the Consul server with the following `kubectl` CLI command: + +```shell-session +$ kubectl apply --filename sameness-group-a.yaml +``` + +Then, repeat the process to create and apply a configuration entry for every partition that is a member of the sameness group. + +Be aware that the sameness group configuration entries are different for each partition. The following example demonstrates how to format three different configuration entries for three partitions that are part of the sameness group `sameness-group-a` when Partition 1 and Partition 2 are in DC1, and the third partition is Partition 1 in DC2: + + + + + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +kind: SamenessGroup +metadata: + name: sameness-group-a +spec: + defaultForFailover: true + members: + partition: partition-1 + partition: partition-2 + peer: dc2-partition-1 +``` + + + + + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +kind: SamenessGroup +metadata: + name: sameness-group-a +spec: + defaultForFailover: true + members: + partition: partition-2 + partition: partition-1 + peer: dc2-partition-1 +``` + + + + + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +kind: SamenessGroup +metadata: + name: sameness-group-a +spec: + defaultForFailover: true + members: + partition: partition-1 + peer: dc1-partition-1 + peer: dc2-partition-2 +``` + + + + + +### Export services to other partitions in the sameness group + +To make services available to other members of the sameness group, you must write and apply an [exported services configuration entry](/consul/docs/connect/config-entries/exported-services) to each partition in the group. This configuration entry exports the local partition's services to the rest of the group members. In each configuration entry, set the sameness group as the `Consumer` for the exported services. You can export multiple services in a single exported services configuration entry. + +Because you are configuring the consumer to reference the sameness group instead of listing out each partition and cluster peer, you do not need to edit this configuration again when you add a partition or peer to the group. + +The following example demonstrates how to format three different `exported-service` configuration entries to make a service named `api` deployed to the `store` namespace of each partition available to all other group members: + + + + + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +Kind: ExportedServices +metadata: + name: group-a-export +spec: + services: + - name: api + namespace: store + consumers: + - samenessGroup: sameness-group-a +``` + + + + + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +Kind: ExportedServices +metadata: + name: group-a-export +spec: + services: + - name: api + namespace: store + consumers: + - samenessGroup: sameness-group-a +``` + + + + + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +Kind: ExportedServices +metadata: + name: group-a-export +spec: + services: + - name: api + namespace: store + consumers: + - samenessGroup: sameness-group-a +``` + + + + + +For more information about exporting services, including examples of configuration entries that export multiple services at the same time, refer to the [exported services configuration entry reference](/consul/docs/connect/config-entries/exported-services). + +After you create each exported services configuration entry, apply it to the Consul server with the following CLI command: + +```shell-session +$ kubectl apply --filename group-a-export.yaml +``` + +#### Export services for cluster peers and sameness groups at the same time + +It is possible to combine the exported services configuration entry for creating sameness groups with the entry required to [export services when establishing cluster peering connections](/consul/docs/k8s/connect/cluster-peering/usage/establish-peering#export-services-between-clusters). + +In this workflow, configure the `services[].consumers` block with the `samenessGroup` field instead of the `peer` field. You should write the `sameness-group` configuration entry to Consul before referencing it in the `exported-services` configuration entry. + +### Create service intentions to authorize traffic between group members + +While exporting the service to each member of the sameness group makes the services visible to remote partitions, you must also create service intentions so that local services are authorized to send and receive traffic from a member of the sameness group. + +For each partition that is member of the group, write and apply a [service intentions configuration entry](/consul/docs/connect/config-entries/service-intentions) that defines intentions for the services that are part of the group. In the `sources` block of the configuration entry, include the service name, its namespace, the sameness group and grant `allow` permissions. + +Because you are using the sameness group in the `sources` block rather than listing out each partition and cluster peer, you do not have to make further edits to the service intentions configuration entries when members are added to or removed from the group. + +The following example demonstrates how to format three different `service-intentions` configuration entries to make a service named `api` available to all instances of `payments` deployed in all members of the sameness group including the local partition. In this example, `api` is deployed to the `store` namespace in all three partitions. + + + + + + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +kind: ServiceIntentions +metadata: + name: api +spec: + sources: + - name: api + action: allow + namespace: store + samenessGroup: sameness-group-a +``` + + + + + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +kind: ServiceIntentions +metadata: + name: api +spec: + sources: + - name: api + action: allow + namespace: store + samenessGroup: sameness-group-a +``` + + + + + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +kind: ServiceIntentions +metadata: + name: api +spec: + sources: + - name: api + action: allow + namespace: store + samenessGroup: sameness-group-a +``` + + + + + +Refer to [create and manage intentions](/consul/docs/connect/intentions/create-manage-intentions) for more information about how to create and apply service intentions in Consul. + +After you create each service intentions configuration entry, apply it to the Consul server with the following CLI command: + +```shell-session +$ kubectl apply --filename api.yaml +``` + +#### Create service intentions for cluster peers and sameness groups at the same time + +It is possible to combine the service intentions configuration entry for creating sameness groups with the entry required to [authorize services for peers](/consul/docs/connect/cluster-peering/usage/establish-cluster-peering#authorize-services-for-peers). + +In this workflow, configure the `sources` block with the `samenessGroup` field instead of the `peer` field. You should write the `sameness-group` configuration entry to Consul before referencing it in the `service-intentions` configuration entry. \ No newline at end of file diff --git a/website/content/docs/k8s/connect/cluster-peering/usage/establish-peering.mdx b/website/content/docs/k8s/connect/cluster-peering/usage/establish-peering.mdx index 19e504b95d68..167d4fdceced 100644 --- a/website/content/docs/k8s/connect/cluster-peering/usage/establish-peering.mdx +++ b/website/content/docs/k8s/connect/cluster-peering/usage/establish-peering.mdx @@ -18,6 +18,8 @@ The overall process for establishing a cluster peering connection consists of th Cluster peering between services cannot be established until all four steps are complete. +Cluster peering between services cannot be established until all four steps are complete. If you want to establish cluster peering connections and create sameness groups at the same time, refer to the guidance in [create sameness groups](/consul/docs/k8s/connect/cluster-peering/usage/create-sameness-groups). + For general guidance for establishing cluster peering connections, refer to [Establish cluster peering connections](/consul/docs/connect/cluster-peering/usage/establish-cluster-peering). ## Prerequisites diff --git a/website/content/docs/k8s/crds/index.mdx b/website/content/docs/k8s/crds/index.mdx index 6a68960a04df..c34146c3c35b 100644 --- a/website/content/docs/k8s/crds/index.mdx +++ b/website/content/docs/k8s/crds/index.mdx @@ -19,6 +19,7 @@ You can specify the following values in the `kind` field. Click on a configurati - [`PeeringAcceptor`](/consul/docs/k8s/connect/cluster-peering/tech-specs#peeringacceptor) - [`PeeringDialer`](/consul/docs/k8s/connect/cluster-peering/tech-specs#peeringdialer) - [`ProxyDefaults`](/consul/docs/connect/config-entries/proxy-defaults) +- [`SamenessGroup`](/consul/docs/connect/config-entries/sameness-group) - [`ServiceDefaults`](/consul/docs/connect/config-entries/service-defaults) - [`ServiceSplitter`](/consul/docs/connect/config-entries/service-splitter) - [`ServiceRouter`](/consul/docs/connect/config-entries/service-router) diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index edd16a4eb86f..ee250b9035ad 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -453,7 +453,11 @@ "path": "connect/config-entries/proxy-defaults" }, { - "title": "Service defaults", + "title": "Sameness Group", + "path": "connect/config-entries/sameness-group" + }, + { + "title": "Service Defaults", "path": "connect/config-entries/service-defaults" }, { @@ -752,6 +756,10 @@ { "title": "Manage L7 Traffic With Cluster Peering", "path": "connect/cluster-peering/usage/peering-traffic-management" + }, + { + "title": "Create Sameness Groups", + "path": "connect/cluster-peering/usage/create-sameness-groups" } ] } @@ -1299,6 +1307,10 @@ { "title": "Manage L7 Traffic With Cluster Peering", "path": "k8s/connect/cluster-peering/usage/l7-traffic" + }, + { + "title": "Create Sameness Groups", + "path": "k8s/connect/cluster-peering/usage/create-sameness-groups" } ] }