Skip to content

Commit

Permalink
[admin] KIP-430: Add authorized operations to describe API
Browse files Browse the repository at this point in the history 
commit bee8d6a80b81a8f31c823750e3e504537cd954ac
Author: Milind L <miluthra@confluent.io>
Date:   Wed Sep 27 11:40:35 2023 +0530

    Squashed commit of the following:

    commit 73cf87c
    Author: Emanuele Sabellico <esabellico@confluent.io>
    Date:   Tue Sep 26 14:48:57 2023 +0200

        Fix segfault if assignor state is NULL, (#4381)

        e.g. a destroy happens before the first assignment.
        Only affects the cooperative-sticky assignor

    commit 1cb8009
    Author: Emanuele Sabellico <esabellico@confluent.io>
    Date:   Thu Sep 21 10:04:07 2023 +0200

        Add destroy call when leader epoch is stale (#4429)

        and partition is in state active

    commit 49f180a
    Author: Pranav Rathi <4427674+pranavrth@users.noreply.github.com>
    Date:   Tue Aug 15 23:47:28 2023 +0530

        Added new environment variable to tests - TESTS_SKIP_BEFORE (#4317)

    commit ff1aaf5
    Author: Pranav Rathi <4427674+pranavrth@users.noreply.github.com>
    Date:   Mon Aug 14 18:31:48 2023 +0530

        Fix ListConsumerGroupOffsets not fetching offsets for all the topics in a group with Apache Kafka version below 2.4.0. (#4346)

commit f6f0a62
Author: Milind L <miluthra@confluent.io>
Date:   Mon Sep 25 17:00:05 2023 +0530

    Fix case where broker RPC might not contain authorized operations

commit 63e0748
Author: Milind L <miluthra@confluent.io>
Date:   Mon Sep 25 16:14:47 2023 +0530

    Fix possible initialization issue

commit 7205354
Author: Milind L <miluthra@confluent.io>
Date:   Mon Sep 25 16:02:37 2023 +0530

    Differentiate between 0 and unrequested authorized ops

commit adc0c4f
Author: Milind L <miluthra@confluent.io>
Date:   Mon Sep 25 09:35:36 2023 +0530

    Avoid malloc(0)

commit 4257dc9
Author: Milind L <miluthra@confluent.io>
Date:   Fri Sep 22 10:35:33 2023 +0530

    Fix CI issues

commit 0e05dea
Author: Milind L <miluthra@confluent.io>
Date:   Fri Sep 22 07:45:53 2023 +0530

    Change TopicCollection create method name

commit 74d98dc
Author: Milind L <miluthra@confluent.io>
Date:   Thu Sep 21 15:02:51 2023 +0530

    Fix warning and add rack to cluster nodes

commit fd14d24
Author: Milind L <miluthra@confluent.io>
Date:   Fri Sep 15 12:38:56 2023 +0530

    Retain topic ordering for DescribeTopics

commit d90bb98
Author: Milind L <miluthra@confluent.io>
Date:   Thu Sep 14 12:41:10 2023 +0530

    Change rack_id to rack in Node public API

commit 0233c0f
Author: Milind L <miluthra@confluent.io>
Date:   Wed Sep 6 15:50:37 2023 +0530

    Fix typo in result type

commit e6d4d63
Author: Milind L <miluthra@confluent.io>
Date:   Mon Sep 4 17:26:44 2023 +0530

    Add brokers_sorted to internal metadata

commit 2c7f5e1
Author: Milind L <miluthra@confluent.io>
Date:   Mon Sep 4 16:53:20 2023 +0530

    Address comments: AuthorizedOperations_parse uses array

commit 9a6ef10
Author: Milind L <miluthra@confluent.io>
Date:   Mon Sep 4 15:47:41 2023 +0530

    Address comments: cosmetic and copyright changes

commit beccb0c
Author: Milind L <miluthra@confluent.io>
Date:   Mon Sep 4 15:10:54 2023 +0530

    Address comments: Cosmetic changes

commit 6239372
Author: Milind L <miluthra@confluent.io>
Date:   Mon Sep 4 14:59:20 2023 +0530

    Address comment and RD_IF_FREE fixes

commit bdd8a4b
Author: Milind L <miluthra@confluent.io>
Date:   Mon Sep 4 14:55:26 2023 +0530

    Fix style and refactor MetadataRequest into op/cb

commit 53cd983
Author: Milind L <miluthra@confluent.io>
Date:   Thu Aug 31 17:47:54 2023 +0530

    Address DescribeCluster/DescribeConsumerGroup changes, fix memleak

commit 59ab5bb
Author: Milind L <miluthra@confluent.io>
Date:   Thu Aug 31 13:57:51 2023 +0530

    Address comments for DescribeTopics API

commit 85f7f91
Author: Milind L <miluthra@confluent.io>
Date:   Fri Aug 11 15:08:51 2023 +0530

    Get rid of stray conflict marker

commit bd7f5a7
Author: Milind L <miluthra@confluent.io>
Date:   Wed Aug 9 19:27:39 2023 +0530

    Style fixes

commit ef3f480
Author: Milind L <miluthra@confluent.io>
Date:   Wed Aug 9 19:26:48 2023 +0530

    Fix examples

commit 0ed7250
Merge: 61840ab 07262c4
Author: Milind L <miluthra@confluent.io>
Date:   Wed Aug 9 14:08:09 2023 +0530

    Merge branch 'master' into dev_kip430_cp_review_changes

commit 61840ab
Author: Milind L <miluthra@confluent.io>
Date:   Wed Aug 9 13:53:33 2023 +0530

    Address review comments: Part 2

commit 45780f8
Author: Milind L <miluthra@confluent.io>
Date:   Wed Aug 9 10:13:23 2023 +0530

    Address review comments: Part 1

commit 0bc97b3
Author: Milind L <miluthra@confluent.io>
Date:   Wed Aug 2 11:31:37 2023 +0530

    Break into functions, update Supported KIPS

commit 6f8aa0d
Author: Milind L <miluthra@confluent.io>
Date:   Wed Aug 2 10:40:36 2023 +0530

    Fix 0080 after addressing comments

commit 8d8a473
Author: Milind L <miluthra@confluent.io>
Date:   Tue Aug 1 19:59:39 2023 +0530

    Address review comments: Refactor MetadataRequest into two parts

commit 28615cd
Author: Milind L <miluthra@confluent.io>
Date:   Tue Aug 1 18:10:46 2023 +0530

    Address review comments - Part 2

commit e01ad41
Author: Milind L <miluthra@confluent.io>
Date:   Tue Aug 1 15:09:46 2023 +0530

    Address review comments - Part 1

commit 3215a90
Author: Milind L <miluthra@confluent.io>
Date:   Tue Aug 1 09:50:59 2023 +0530

    Fixed changelog

commit 4a4f503
Author: Milind L <miluthra@confluent.io>
Date:   Mon Jul 31 20:41:57 2023 +0530

    Remove fanout from DescribeTopics

commit 9787d6c
Merge: 8c2b4d4 c07a335
Author: Milind L <miluthra@confluent.io>
Date:   Mon Jul 31 14:47:43 2023 +0530

    Merge branch 'master' into dev_kip430_cp_review_changes

commit 8c2b4d4
Author: Milind L <miluthra@confluent.io>
Date:   Mon Jun 26 11:44:24 2023 +0530

    Fix cluster_id allocation not on tmpabuf

commit 4a4d9df
Author: Milind L <miluthra@confluent.io>
Date:   Mon Jun 26 11:29:26 2023 +0530

    Revert to macro to fix msvc compilation

commit 7fff6eb
Merge: 3a9a340 aa50e52
Author: Milind L <miluthra@confluent.io>
Date:   Mon Jun 26 11:18:39 2023 +0530

    Remove warnings and add CHANGELOG

commit 3a9a340
Author: Milind L <miluthra@confluent.io>
Date:   Sat Jun 24 15:36:22 2023 +0530

    Change API for common stuff

commit 6ddbaba
Merge: f497307 e52aa3b
Author: Milind L <miluthra@confluent.io>
Date:   Fri Jun 23 12:24:15 2023 +0530

    Merge branch 'master' into dev_kip430_cp

commit f497307
Author: Milind L <miluthra@confluent.io>
Date:   Fri Jun 23 12:22:39 2023 +0530

    Fix tests and change API to be similar to existing ones

commit 3fec50a
Author: Milind L <miluthra@confluent.io>
Date:   Thu Jun 15 09:52:33 2023 +0530

    Fix style and docs issues

commit 99a368d
Merge: 7c15746 865cf60
Author: Ruchir Jain <122954065+jainruchir@users.noreply.github.com>
Date:   Wed Jun 14 18:52:29 2023 +0530

    Merge branch 'master' into dev_kip430_cp

commit 7c15746
Author: jainruchir <rujain@confluent.io>
Date:   Wed Jun 14 18:49:32 2023 +0530

    bug fix: request NULL, force_racks in MetadataReq

commit 88707a0
Author: jainruchir <rujain@confluent.io>
Date:   Wed Jun 14 18:11:47 2023 +0530

    bug fixes and include auth ops false default

commit 060d694
Merge: 0e16f98 966b63d
Author: jainruchir <rujain@confluent.io>
Date:   Wed Jun 14 15:54:10 2023 +0530

    Merge 'master' of librdkafka into dev_kip430_cp

commit 0e16f98
Author: jainruchir <rujain@confluent.io>
Date:   Tue Apr 11 10:04:40 2023 +0530

    style-fix

commit b9fea3c
Author: jainruchir <rujain@confluent.io>
Date:   Tue Apr 11 09:33:57 2023 +0530

    ClusterDescription destroy having double free bug removed

commit 7d512bf
Merge: 830d9b7 2c6dd0a
Author: Ruchir Jain <122954065+jainruchir@users.noreply.github.com>
Date:   Tue Apr 11 09:32:39 2023 +0530

    Merge branch 'master' into dev_kip430_cp

commit 830d9b7
Author: jainruchir <rujain@confluent.io>
Date:   Thu Mar 30 14:36:25 2023 +0530

    bug fix for adminoptions

commit 0bfbf67
Author: jainruchir <rujain@confluent.io>
Date:   Fri Mar 17 16:05:22 2023 +0530

    dev_KIP430 branch merged

commit c28152a
Author: jainruchir <rujain@confluent.io>
Date:   Fri Mar 17 15:05:36 2023 +0530

    changes to describeCG tests

commit d2c902b
Author: jainruchir <rujain@confluent.io>
Date:   Tue Feb 21 15:55:50 2023 +0530

    added test for include auth ops in 0081 initial commit

commit 3a79ed6
Author: jainruchir <rujain@confluent.io>
Date:   Fri Feb 17 14:25:37 2023 +0530

    KIP-430 initial changes without checking tests

commit d0e2156
Author: jainruchir <rujain@confluent.io>
Date:   Fri Mar 17 10:01:08 2023 +0530

    bug fix when include authorized ops is 0 desc Topics/Cluster

commit a542f4d
Author: jainruchir <rujain@confluent.io>
Date:   Wed Mar 15 10:25:40 2023 +0530

    integration tests for describecluster

commit 8df6f62
Author: jainruchir <rujain@confluent.io>
Date:   Tue Mar 14 14:04:44 2023 +0530

    describe topics and cluster

commit 069b8e8
Author: jainruchir <rujain@confluent.io>
Date:   Fri Feb 24 14:32:49 2023 +0530

    include authorized topic/cluster ops initial draft
  • Loading branch information
@milindl
milindl committed Sep 27, 2023
1 parent 73cf87c commit efbb966
Show file tree
Hide file tree
Showing 27 changed files with 3,802 additions and 255 deletions.
9 changes: 7 additions & 2 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,14 @@
# librdkafka v2.2.1
# librdkafka v2.3.0

librdkafka v2.2.1 is a maintenance release:
librdkafka v2.3.0 is a feature release:

* Added Topic id to the metadata response which is part of the [KIP-516](https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers)
* Fixed ListConsumerGroupOffsets not fetching offsets for all the topics in a group with Apache Kafka version below 2.4.0.
* Add support for AdminAPI `DescribeCluster()` and `DescribeTopics()`
(#4240, @jainruchir).
* [KIP-430](https://cwiki.apache.org/confluence/display/KAFKA/KIP-430+-+Return+Authorized+Operations+in+Describe+Responses):
Return authorized operations in Describe Responses.
(#4240, @jainruchir).
* Add missing destroy that leads to leaking partition structure memory when there
are partition leader changes and a stale leader epoch is received (#4429).
* Fix a segmentation fault when closing a consumer using the
Expand Down
2 changes: 1 addition & 1 deletion INTRODUCTION.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1926,7 +1926,7 @@ The [Apache Kafka Implementation Proposals (KIPs)](https://cwiki.apache.org/conf
| KIP-412 - AdminAPI: adjust log levels | 2.4.0 | Not supported |
| KIP-421 - Variables in client config files | 2.3.0 | Not applicable (librdkafka, et.al, does not provide a config file interface, and shouldn't) |
| KIP-429 - Consumer: incremental rebalance protocol | 2.4.0 | Supported |
| KIP-430 - AdminAPI: return authorized operations in Describe.. responses | 2.3.0 | Not supported |
| KIP-430 - AdminAPI: return authorized operations in Describe.. responses | 2.3.0 | Supported |
| KIP-436 - Start time in stats | 2.3.0 | Supported |
| KIP-447 - Producer scalability for EOS | 2.5.0 | Supported |
| KIP-455 - AdminAPI: Replica assignment | 2.4.0 (WIP) | Not supported |
Expand Down
2 changes: 2 additions & 0 deletions examples/.gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ rdkafka_performance
transactions
list_consumer_groups
describe_consumer_groups
describe_topics
describe_cluster
list_consumer_group_offsets
alter_consumer_group_offsets
incremental_alter_configs
Expand Down
6 changes: 6 additions & 0 deletions examples/CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,12 @@ target_link_libraries(incremental_alter_configs PUBLIC rdkafka)
add_executable(user_scram user_scram.c ${win32_sources})
target_link_libraries(user_scram PUBLIC rdkafka)

add_executable(describe_topics describe_topics.c ${win32_sources})
target_link_libraries(describe_topics PUBLIC rdkafka)

add_executable(describe_cluster describe_cluster.c ${win32_sources})
target_link_libraries(describe_cluster PUBLIC rdkafka)

# The targets below has Unix include dirs and do not compile on Windows.
if(NOT WIN32)
add_executable(rdkafka_example rdkafka_example.c)
Expand Down
10 changes: 10 additions & 0 deletions examples/Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ EXAMPLES ?= rdkafka_example rdkafka_performance rdkafka_example_cpp \
openssl_engine_example_cpp \
list_consumer_groups \
describe_consumer_groups \
describe_topics \
describe_cluster \
list_consumer_group_offsets \
alter_consumer_group_offsets \
incremental_alter_configs \
Expand Down Expand Up @@ -74,6 +76,14 @@ describe_consumer_groups: ../src/librdkafka.a describe_consumer_groups.c
$(CC) $(CPPFLAGS) $(CFLAGS) $@.c -o $@ $(LDFLAGS) \
../src/librdkafka.a $(LIBS)

describe_topics: ../src/librdkafka.a describe_topics.c
$(CC) $(CPPFLAGS) $(CFLAGS) $@.c -o $@ $(LDFLAGS) \
../src/librdkafka.a $(LIBS)

describe_cluster: ../src/librdkafka.a describe_cluster.c
$(CC) $(CPPFLAGS) $(CFLAGS) $@.c -o $@ $(LDFLAGS) \
../src/librdkafka.a $(LIBS)

list_consumer_group_offsets: ../src/librdkafka.a list_consumer_group_offsets.c
$(CC) $(CPPFLAGS) $(CFLAGS) $@.c -o $@ $(LDFLAGS) \
../src/librdkafka.a $(LIBS)
Expand Down
2 changes: 2 additions & 0 deletions examples/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ For more complex uses, see:
* [delete_records.c](delete_records.c) - Delete records.
* [list_consumer_groups.c](list_consumer_groups.c) - List consumer groups.
* [describe_consumer_groups.c](describe_consumer_groups.c) - Describe consumer groups.
* [describe_topics.c](describe_topics.c) - Describe topics.
* [describe_cluster.c](describe_cluster.c) - Describe cluster.
* [list_consumer_group_offsets.c](list_consumer_group_offsets.c) - List offsets of a consumer group.
* [alter_consumer_group_offsets.c](alter_consumer_group_offsets.c) - Alter offsets of a consumer group.
* [incremental_alter_configs.c](incremental_alter_configs.c) - Incrementally alter resource configurations.
Expand Down
Loading

0 comments on commit efbb966

Please sign in to comment.