Skip to content

Commit 427fee9

Browse files
committed
Fixes to examples/README-authz.md
Signed-off-by: Marko Strukelj <marko.strukelj@gmail.com>
1 parent 83ad12b commit 427fee9

File tree

1 file changed

+79
-29
lines changed

1 file changed

+79
-29
lines changed

examples/README-authz.md

+79-29
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,13 @@
1-
## Token based authorization with Keycloak Authorization Services
1+
## Token Based Authorization with Keycloak Authorization Services
22

33
Once the Kafka Broker has obtained an access token by using Strimzi Kafka OAuth for authentication, it is possible to use centrally managed authorization rules to enforce access restrictions onto Kafka Clients.
44
For this Strimzi Kafka OAuth supports the use of Keycloak Authorization Services.
55

66
A custom authorizer has to be configured on the Kafka Broker to take advantage of Authorization Services REST endpoints available on Keycloak, which provide a list of granted permissions on resources for authenticated users.
77
The list is fetched once, and enforced locally on the Kafka Broker for each user session in order to provide fast authorization decisions.
88

9-
## Authorization example
9+
10+
## Building the Example Project
1011

1112
Before using the example, we first need to build the project, and prepare resources.
1213

@@ -21,22 +22,30 @@ Now build the project, and prepare resources:
2122

2223
We are now ready to start up the containers and see Keycloak Authorization Services in action.
2324

25+
26+
## Starting Up the Containers
27+
2428
First make sure any existing containers with the same name are removed, otherwise we might use previous configurations:
2529

2630
docker rm keycloak kafka zookeeper
2731

2832
Let's start up all the containers with authorization configured, and we'll then perform any manual step, and explain how everything works.
2933

30-
docker-compose -f compose.yml -f keycloak/compose.yml -f keycloak-import/compose.yml -f kafka-oauth-strimzi/compose-authz.yml up --build
34+
docker-compose -f compose.yml -f keycloak/compose.yml -f keycloak-import/compose.yml \
35+
-f kafka-oauth-strimzi/compose-authz.yml up --build
3136

3237
When everything starts up without errors we should have one instance of `keycloak` listening on localhost:8080.
33-
You can login to the admin console by opening `http://localhost:8080/auth/admin` and using `admin` as both username, and a password.
3438

35-
In upper left corner under Keycloak icon you should see `Master` selected as a current realm.
39+
40+
## Using Keycloak Admin Console to Configure Authorization
41+
42+
You can login to the Admin Console by opening `http://localhost:8080/auth/admin` and using `admin` as both username, and a password.
43+
44+
In the upper left corner under Keycloak icon you should see `Master` selected as a current realm.
3645
Moving the mouse pointer over it should reveal two additional realms - `Demo` and `Kafka-authz`.
3746

3847
For this example we are interested in the `kafka-authz` realm.
39-
Selecting it will open the `Realm Settings` for `kafka-authz` realm.
48+
Selecting it will open the `Realm Settings` for the `kafka-authz` realm.
4049
Next to `Realm Settings` there are other sections we are interested in - `Groups`, `Roles`, `Clients` and `Users`.
4150

4251
Under `Groups` we can see several groups that can be used to mark users as having some permissions.
@@ -61,15 +70,18 @@ Clients `team-a-client`, and `team-b-client` are confidential clients representi
6170
The authorization configuration is defined in `kafka` client under `Authorization` tab.
6271
This tab becomes visible when `Authorization Enabled` is turned on under `Settings` tab.
6372

64-
If we take a look under `Resources` sub-tab we'll see the list of resource definitions.
73+
74+
## Authorization Services - Resources, Authorization Scopes, Policies and Permissions
75+
76+
If we take a look under `Resources` sub-tab of `Authorization` tab, we'll see the list of resource definitions.
6577
These are resource specifiers - patterns in a specific format, that are used to target policies to specific resources.
6678
The format is quite simple. For example:
6779

68-
kafka-cluster:cluster-1,Topic:a_* ... targets only topics in kafka cluster 'cluster-1' with names starting with 'a_'
80+
- `kafka-cluster:cluster-1,Topic:a_*` ... targets only topics in kafka cluster 'cluster-1' with names starting with 'a_'
6981

7082
If `kafka-cluster:XXX` segment is not present, the specifier targets any cluster.
7183

72-
Group:x_* ... targets all consumer groups on any cluster with names starting with 'x_'
84+
- `Group:x_*` ... targets all consumer groups on any cluster with names starting with 'x_'
7385

7486
The possible resource types mirror the Kafka authorization model (Topic, Group, Cluster).
7587

@@ -102,6 +114,7 @@ Let's click on `Dev Team A` policy. We see that it matches all users that have a
102114
Similarly, the `Dev Team B ...` permissions perform matching using `Dev Team B` policy which also uses realm role to match allowed users - in this case those with realm role `Dev Team B`.
103115
The `Dev Team B ...` permissions grant users `Describe` and `Read` on `Topic:x_*`, and `Group:x_*` resources, effectively giving matching users and clients the ability to read from topics, and update the consumed offsets for topics and consumer groups that have names starting with 'x_'.
104116

117+
## Targeting Permissions - Clients and Roles vs. Users and Groups
105118

106119
In Keycloak, confidential clients with 'service accounts' enabled can authenticate to the server in their own name using a clientId and a secret.
107120
This is convenient for microservices which typically act in their own name, and not as agents of a particular user (like a web site would, for example).
@@ -114,6 +127,9 @@ Performing cluster management is usually done interactively - in person - using
114127
It makes sense to require the user to log-in, before using the resulting access token to authenticate to Kafka Broker.
115128
In this case the access token represents the specific user, rather than the client application.
116129

130+
131+
## Authorization in Action Using CLI Clients
132+
117133
Before continuing, there is one setting we need to check.
118134
Due to [a little bug in Keycloak](https://issues.redhat.com/browse/KEYCLOAK-12640) the realm is at this point misconfigured, and we have to fix the configuration manually.
119135
Under `Clients` / `kafka` / `Authorization` / `Settings` make sure the `Decision Strategy` is set to `Affirmative`, and NOT to `Unanimous`. Click `Save` after fixing it.
@@ -149,10 +165,14 @@ Make sure the necessary classes are on the classpath:
149165

150166
export CLASSPATH=/opt/kafka/libs/strimzi/*:$CLASSPATH
151167

152-
Now let's try produce some messages to topic 'my-topic':
168+
169+
### Producing Messages
170+
171+
Let's try produce some messages to topic 'my-topic':
153172

154173
```
155-
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic my-topic --producer.config=$HOME/team-a-client.properties
174+
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic my-topic \
175+
--producer.config=$HOME/team-a-client.properties
156176
First message
157177
```
158178

@@ -164,46 +184,62 @@ Topic named `my-topic` does not fall in either of these two.
164184
Use CTRL-C to exit the CLI application, and let's try to write to topic `a_messages`.
165185

166186
```
167-
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic a_messages --producer.config ~/team-a-client.properties
187+
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic a_messages \
188+
--producer.config ~/team-a-client.properties
168189
First message
169190
Second message
170191
```
171192

172193
We can see some warnings but looking at Kafka container log there is DEBUG level output saying 'Authorization GRANTED'.
173194

174195
Use CTRL-C to exit the CLI application.
175-
196+
197+
198+
### Consuming Messages
199+
176200
Let's now try to consume the messages we have produced.
177201

178-
bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic a_messages --from-beginning --consumer.config ~/team-a-client.properties
202+
bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic a_messages \
203+
--from-beginning --consumer.config ~/team-a-client.properties
179204

180205
This gives us an error like: 'Not authorized to access group: console-consumer-55841'.
181206

182207
The reason is that we have to override the default consumer group name - `Dev Team A` only has access to consumer groups that have names starting with 'a_'.
183208
Let's set custom consumer group name that starts with 'a_'
184209

185-
bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic a_messages --from-beginning --consumer.config ~/team-a-client.properties --group a_consumer_group_1
210+
bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic a_messages \
211+
--from-beginning --consumer.config ~/team-a-client.properties --group a_consumer_group_1
186212

187213
We should now receive all the messages for 'a_messages' topic, after which the client blocks waiting for more to come down the stream.
188214

189-
Use CTRL-C to exit, and let's list the topics:
215+
Use CTRL-C to exit.
216+
217+
218+
### Using Kafka CLI Administration Tools
219+
220+
Let's now list the topics:
190221

191222
bin/kafka-topics.sh --bootstrap-server kafka:9092 --command-config ~/team-a-client.properties --list
192223

193224
We get one topic listed: 'a_messages'.
194225

195226
Let's try and list the consumer groups:
196227

197-
bin/kafka-consumer-groups.sh --bootstrap-server kafka:9092 --command-config ~/team-a-client.properties --list
228+
bin/kafka-consumer-groups.sh --bootstrap-server kafka:9092 \
229+
--command-config ~/team-a-client.properties --list
198230

199231
Similarly to listing topics, we get one consumer group listed: `a_consumer_group_1`.
200232

201233
There are more CLI administrative tools. For example we can try get default cluster configuration:
202234

203-
bin/kafka-configs.sh --bootstrap-server kafka:9092 --command-config ~/team-a-client.properties --entity-type brokers --describe --entity-default
235+
bin/kafka-configs.sh --bootstrap-server kafka:9092 --command-config ~/team-a-client.properties \
236+
--entity-type brokers --describe --entity-default
204237

205238
But that will fail with `Cluster authorization failed.` error, because this operation requires cluster level permissions which `team-a-client` does not have.
206239

240+
241+
### Client with Different Permissions
242+
207243
Let's prepare a configuration for `team-b-client`:
208244

209245
```
@@ -226,14 +262,16 @@ The `Dev Team B` users have full access to topics beginning with 'b_' on Kafka c
226262
Let's try produce some messages to topic 'a_messages' as `team-b-client`:
227263

228264
```
229-
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic a_messages --producer.config ~/team-b-client.properties
265+
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic a_messages \
266+
--producer.config ~/team-b-client.properties
230267
Message 1
231268
```
232269

233270
We get `Not authorized to access topics: [a_messages]` error as we expected. Let's try to produce to topic `b_messages`:
234271

235272
```
236-
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic b_messages --producer.config ~/team-b-client.properties
273+
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic b_messages \
274+
--producer.config ~/team-b-client.properties
237275
Message 1
238276
Message 2
239277
Message 3
@@ -244,15 +282,17 @@ This should work fine.
244282
What about producing to topic `x_messages`. `team-b-client` is only supposed to be able to read from such a topic.
245283

246284
```
247-
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic x_messages --producer.config ~/team-b-client.properties
285+
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic x_messages \
286+
--producer.config ~/team-b-client.properties
248287
Message 1
249288
```
250289

251290
We get a `Not authorized to access topics: [x_messages]` error as we expected.
252291
Client `team-a-client`, on the other hand, should be able to write to such a topic:
253292

254293
```
255-
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic x_messages --producer.config ~/team-a-client.properties
294+
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic x_messages \
295+
--producer.config ~/team-a-client.properties
256296
Message 1
257297
```
258298

@@ -261,6 +301,9 @@ The reason for failure is that while `team-a-client` can write to `x_messages` t
261301

262302
We now need a power user that can create a topic with all the proper settings - like the right number of partitions and replicas.
263303

304+
305+
### Power User Can Do Anything
306+
264307
Let's create a configuration for user `bob` who has full ability to manage everything on Kafka cluster `cluster2`.
265308

266309
First, `bob` will authenticate to Keycloak server with username and password and get a refresh token.
@@ -298,7 +341,8 @@ We can use this, because we authenticate with a token directly (in this case a r
298341

299342
Let's now try to create `x_messages` topic:
300343

301-
bin/kafka-topics.sh --bootstrap-server kafka:9092 --command-config ~/bob.properties --topic x_messages --create --replication-factor 1 --partitions 3
344+
bin/kafka-topics.sh --bootstrap-server kafka:9092 --command-config ~/bob.properties \
345+
--topic x_messages --create --replication-factor 1 --partitions 1
302346

303347
The operation should succeed. We can list the topics:
304348

@@ -314,7 +358,8 @@ Roles `Dev Team A`, and `Dev Team B` both have `Describe` permission on topics t
314358
We can now again try to produce to the topic as `team-a-client`.
315359

316360
```
317-
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic x_messages --producer.config ~/team-a-client.properties
361+
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic x_messages \
362+
--producer.config ~/team-a-client.properties
318363
Message 1
319364
Message 2
320365
Message 3
@@ -325,29 +370,34 @@ This works.
325370
If we try the same as `team-b-client` it should fail.
326371

327372
```
328-
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic x_messages --producer.config ~/team-b-client.properties
373+
bin/kafka-console-producer.sh --broker-list kafka:9092 --topic x_messages \
374+
--producer.config ~/team-b-client.properties
329375
Message 4
330376
Message 5
331377
```
332378

333379
But `team-b-client` should be able to consume messages from the `x_messages` topic:
334380

335-
bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic x_messages --from-beginning --consumer.config ~/team-b-client.properties --group x_consumer_group_b
381+
bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic x_messages \
382+
--from-beginning --consumer.config ~/team-b-client.properties --group x_consumer_group_b
336383

337384
Whereas `team-a-client` does not have permission to read, even though they can write:
338385

339-
bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic x_messages --from-beginning --consumer.config ~/team-a-client.properties --group x_consumer_group_a
386+
bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic x_messages \
387+
--from-beginning --consumer.config ~/team-a-client.properties --group x_consumer_group_a
340388

341389
We get a `Not authorized to access group: x_consumer_group_a` error.
342390
What if we try to use a consumer group name that starts with 'a_'?
343391

344-
bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic x_messages --from-beginning --consumer.config ~/team-a-client.properties --group a_consumer_group_a
392+
bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic x_messages \
393+
--from-beginning --consumer.config ~/team-a-client.properties --group a_consumer_group_a
345394

346395
We now get a different error: `Not authorized to access topics: [x_messages]`
347396

348397
It just won't work - `Dev Team A` has no `Read` access on topics that start with 'x_'.
349398

350399
User `bob` should have no problem reading from or writing to any topic:
351400

352-
bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic x_messages --from-beginning --consumer.config ~/bob.properties
401+
bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic x_messages \
402+
--from-beginning --consumer.config ~/bob.properties
353403

0 commit comments

Comments
 (0)