Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding example for kraft based setup with mtls #316

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 82 additions & 0 deletions security/kraft-mtls-autogenerated/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Deploy KRaft-based setup with mtls and auto-generated certs


- [Set the current tutorial directory](#set-the-current-tutorial-directory)
- [Deploy Confluent for Kubernetes](#deploy-confluent-for-kubernetes)
- [Provide a Certificate Authority](#provide-a-certificate-authority)
- [Set up cluster](#set-up-cluster)
- [Tear down Cluster](#tear-down-cluster)

This playbook explains how to deploy KRaft based setup with mtls and auto-generated certs

Before continuing with the scenario, ensure that you have set up the
[prerequisites](/README.md#prerequisites).


## Set the current tutorial directory

Set the tutorial directory under the directory you downloaded this Github repo:

```
export TUTORIAL_HOME=<Github repo directory>/security/kraft-mtls-autogenerated
```

## Deploy Confluent for Kubernetes

This workflow scenario assumes you are using the namespace `confluent`.

Set up the Helm Chart:

```
helm repo add confluentinc https://packages.confluent.io/helm
```

Install Confluent For Kubernetes using Helm:

```
helm upgrade --install confluent-operator confluentinc/confluent-for-kubernetes -n confluent
```

Check that the Confluent For Kubernetes pod comes up and is running:

```
kubectl get pods -n confluent
```


## Provide a Certificate Authority

Confluent For Kubernetes provides auto-generated certificates for Confluent Platform
components to use for TLS network encryption. A CA is needed to achieve this.

Generate a CA pair to use:

```
openssl genrsa -out $TUTORIAL_HOME/ca/ca-key.pem 2048

openssl req -new -key $TUTORIAL_HOME/ca/ca-key.pem -x509 \
-days 1000 \
-out $TUTORIAL_HOME/ca/ca.pem \
-subj "/C=US/ST=CA/L=MountainView/O=Confluent/OU=Operator/CN=TestCA"
```

Create the the K8s secret

```
kubectl create secret tls ca-pair-sslcerts \
--cert=$TUTORIAL_HOME/ca/ca.pem \
--key=$TUTORIAL_HOME/ca/ca-key.pem -n confluent
```

## Set up cluster

### Deploy CP
```
kubectl apply -f $TUTORIAL_HOME/confluent-platform-mtls.yml -n confluent
```

## Tear down

```
kubectl delete -f $TUTORIAL_HOME/confluent-platform-mtls.yml -n confluent
```
244 changes: 244 additions & 0 deletions security/kraft-mtls-autogenerated/confluent-platform-mtls.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,244 @@
apiVersion: platform.confluent.io/v1beta1
kind: KRaftController
metadata:
name: kraftcontroller
namespace: confluent
spec:
dataVolumeCapacity: 10G
image:
application: docker.io/confluentinc/cp-server:7.7.0
init: confluentinc/confluent-init-container:2.9.0
listeners:
controller:
tls:
enabled: true
authentication:
type: mtls
principalMappingRules:
- RULE:.*CN[\\s]?=[\\s]?([a-zA-Z0-9.]*)?.*/$1/
tls:
autoGeneratedCerts: true
configOverrides:
server:
- authorizer.class.name=org.apache.kafka.metadata.authorizer.StandardAuthorizer
- super.users=User:kafka;User:kraftcontroller
- allow.everyone.if.no.acl.found=true

replicas: 3
---
apiVersion: platform.confluent.io/v1beta1
kind: Kafka
metadata:
name: kafka
namespace: confluent
spec:
image:
application: docker.io/confluentinc/cp-server:7.7.0
init: confluentinc/confluent-init-container:2.9.0
replicas: 3
dataVolumeCapacity: 10G
tls:
autoGeneratedCerts: true
configOverrides:
server:
- authorizer.class.name=org.apache.kafka.metadata.authorizer.StandardAuthorizer
- super.users=User:kafka;User:kraftcontroller
- allow.everyone.if.no.acl.found=true

listeners:
internal:
authentication:
type: mtls
principalMappingRules:
- RULE:.*CN[\\s]?=[\\s]?([a-zA-Z0-9.]*)?.*/$1/
tls:
enabled: true
external:
authentication:
type: mtls
principalMappingRules:
- RULE:.*CN[\\s]?=[\\s]?([a-zA-Z0-9.]*)?.*/$1/
tls:
enabled: true
externalAccess:
type: loadBalancer
loadBalancer:
domain: mydomain.example
brokerPrefix: b
bootstrapPrefix: kafka
dependencies:
kRaftController:
controllerListener:
tls:
enabled: true
authentication:
type: mtls
clusterRef:
name: kraftcontroller
---
apiVersion: platform.confluent.io/v1beta1
kind: Connect
metadata:
name: connect
namespace: confluent
spec:
replicas: 1
image:
application: confluentinc/cp-server-connect:7.7.0
init: confluentinc/confluent-init-container:2.9.0
tls:
autoGeneratedCerts: true
authentication:
type: mtls
externalAccess:
type: loadBalancer
loadBalancer:
domain: mydomain.example
prefix: connect
dependencies:
kafka:
bootstrapEndpoint: kafka.confluent.svc.cluster.local:9071
authentication:
type: mtls
tls:
enabled: true
---
apiVersion: platform.confluent.io/v1beta1
kind: SchemaRegistry
metadata:
name: schemaregistry
namespace: confluent
spec:
replicas: 1
image:
application: confluentinc/cp-schema-registry:7.7.0
init: confluentinc/confluent-init-container:2.9.0
tls:
autoGeneratedCerts: true
authentication:
type: mtls
externalAccess:
type: loadBalancer
loadBalancer:
domain: mydomain.example
prefix: schemaregistry
dependencies:
kafka:
bootstrapEndpoint: kafka.confluent.svc.cluster.local:9071
authentication:
type: mtls
tls:
enabled: true
---
apiVersion: platform.confluent.io/v1beta1
kind: KafkaRestProxy
metadata:
name: kafkarestproxy
namespace: confluent
spec:
replicas: 1
image:
application: confluentinc/cp-kafka-rest:7.7.0
init: confluentinc/confluent-init-container:2.9.0
tls:
autoGeneratedCerts: true
authentication:
type: mtls
externalAccess:
type: loadBalancer
loadBalancer:
domain: mydomain.example
prefix: kafkarestproxy
dependencies:
kafka:
bootstrapEndpoint: kafka.confluent.svc.cluster.local:9071
authentication:
type: mtls
tls:
enabled: true
schemaRegistry:
url: https://schemaregistry.confluent.svc.cluster.local:8081
authentication:
type: mtls
tls:
enabled: true
---
apiVersion: platform.confluent.io/v1beta1
kind: KsqlDB
metadata:
name: ksqldb
namespace: confluent
spec:
replicas: 1
image:
application: confluentinc/cp-ksqldb-server:7.7.0
init: confluentinc/confluent-init-container:2.9.0
dataVolumeCapacity: 10Gi
externalAccess:
type: loadBalancer
loadBalancer:
domain: mydomain.example
prefix: ksqldb
tls:
autoGeneratedCerts: true
authentication:
type: mtls
dependencies:
kafka:
bootstrapEndpoint: kafka.confluent.svc.cluster.local:9071
authentication:
type: mtls
tls:
enabled: true
---
apiVersion: platform.confluent.io/v1beta1
kind: ControlCenter
metadata:
name: controlcenter
namespace: confluent
spec:
replicas: 1
podTemplate:
probe:
liveness:
periodSeconds: 10
failureThreshold: 5
timeoutSeconds: 500
image:
application: confluentinc/cp-enterprise-control-center:7.7.0
init: confluentinc/confluent-init-container:2.9.0
dataVolumeCapacity: 10Gi
externalAccess:
type: loadBalancer
loadBalancer:
domain: mydomain.example
prefix: controlcenter
tls:
autoGeneratedCerts: true
dependencies:
kafka:
bootstrapEndpoint: kafka.confluent.svc.cluster.local:9071
authentication:
type: mtls
tls:
enabled: true
connect:
- name: connect
url: https://connect.confluent.svc.cluster.local:8083
authentication:
type: mtls
tls:
enabled: true
ksqldb:
- name: ksqldb
url: https://ksqldb.confluent.svc.cluster.local:8088
authentication:
type: mtls
tls:
enabled: true
schemaRegistry:
url: https://schemaregistry.confluent.svc.cluster.local:8081
authentication:
type: mtls
tls:
enabled: true