Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logger with Admin Client #1758

Merged
merged 6 commits into from
Jul 5, 2024
Merged

Logger with Admin Client #1758

merged 6 commits into from
Jul 5, 2024

Conversation

pranavrth
Copy link
Member

  • Fixed logger not working when provided as an argument to AdminClient
  • Updated examples/adminapi.py to include usage of the custom logger with AdminClient

Copy link
Member

@anchitj anchitj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comments

"""
Create a new AdminClient using the provided configuration dictionary.

The AdminClient is a standard Kafka protocol client, supporting
the standard librdkafka configuration properties as specified at
https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md

:param dict config: Configuration properties. At a minimum ``bootstrap.servers`` **should** be set\n"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it be named conf? And also the trailing \n can be removed

"""
Create a new AdminClient using the provided configuration dictionary.

The AdminClient is a standard Kafka protocol client, supporting
the standard librdkafka configuration properties as specified at
https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md

:param dict config: Configuration properties. At a minimum ``bootstrap.servers`` **should** be set\n"
:param Logger logger: Optional Logger instance to use as a custom log messages handler.

At least 'bootstrap.servers' should be configured.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be removed as it's included above now

src/confluent_kafka/admin/__init__.py Outdated Show resolved Hide resolved
Comment on lines 187 to 192
stringBuffer = StringIO()
logger = logging.getLogger('Admin')
logger.setLevel(logging.DEBUG)
handler = logging.StreamHandler(stringBuffer)
handler.setFormatter(logging.Formatter('%(name)s Logger | %(message)s'))
logger.addHandler(handler)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is being repeated here and the above two functions. Can be moved to setup_logger or some helper function

@@ -920,3 +928,6 @@ def example_list_offsets(a, args):
sys.exit(1)

opsmap[operation](a, args)

# Log messages through custom logger if provided
a.poll(0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Newline missing

logMessage = stringBuffer.getvalue().strip()
stringBuffer.close()

assert "Admin Logger | INIT" in logMessage
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Newline missing

examples/adminapi.py Outdated Show resolved Hide resolved
@pranavrth pranavrth force-pushed the dev_fix-logger-arg-admin-client branch from 5479623 to a2be27b Compare June 20, 2024 11:09
src/confluent_kafka/admin/__init__.py Outdated Show resolved Hide resolved
src/confluent_kafka/admin/__init__.py Outdated Show resolved Hide resolved
src/confluent_kafka/admin/__init__.py Outdated Show resolved Hide resolved
@pranavrth
Copy link
Member Author

This is checked later by librdkafka, whenever possible we delegate the validation there

There is inbetween manipulation of the config properties in python binding as well. Due to that part, I have to check here first.

https://github.com/confluentinc/confluent-kafka-python/blob/master/src/confluent_kafka/src/confluent_kafka.c#L2215

pranavrth added 5 commits July 1, 2024 22:14

Unverified

This user has not yet uploaded their public signing key.
* Updated examples/adminapi.py to include usage of the custom logger with AdminClient

Unverified

This user has not yet uploaded their public signing key.

Unverified

This user has not yet uploaded their public signing key.

Unverified

This user has not yet uploaded their public signing key.

Unverified

This user has not yet uploaded their public signing key.
@pranavrth pranavrth force-pushed the dev_fix-logger-arg-admin-client branch from 53aac04 to 5abfee1 Compare July 1, 2024 16:44
@pranavrth pranavrth requested a review from a team as a code owner July 1, 2024 16:44
Copy link
Contributor

@emasab emasab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, just a few comments

CHANGELOG.md Outdated
- Removed usage of `strcpy` to enhance security of the client (#1745)
- Fixed invalid write in `OAUTHBEARER/OIDC` extensions copy (#1745)
- Fixed an issue related to import error of `TopicCollection` and `TopicPartitionInfo` classes when importing through other module like mypy.
- Fixed a segfault when `commit` or `store_offsets` consumer method is called incorrectly with errored Message object
- Fixed `logger` not working when provided as an argument to `AdminClient`
Copy link
Contributor

@emasab emasab Jul 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify it was possible before:

Suggested change
- Fixed `logger` not working when provided as an argument to `AdminClient`
- Fixed `logger` not working when provided as an argument to `AdminClient` instead of a configuration property.

# Create Admin client
a = AdminClient({'bootstrap.servers': broker,
'debug': 'all'},
logger=logger)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
logger=logger)
logger=logger)
# Alternatively, pass the logger as a key.
# When passing it as an argument, it overwrites the key.
#
# a = AdminClient({'bootstrap.servers': broker,
# 'debug': 'all',
# 'logger': logger})

@@ -4,10 +4,12 @@

v2.4.1 is a maintenance release with the following fixes and enhancements:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Version can be changed to 2.5.0 when we merge delete_records

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already done that in delete_records as that is a new feature.

Unverified

This user has not yet uploaded their public signing key.
Copy link
Contributor

@emasab emasab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Pranav!

@pranavrth pranavrth merged commit 858347c into master Jul 5, 2024
2 checks passed
@pranavrth pranavrth deleted the dev_fix-logger-arg-admin-client branch July 5, 2024 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants