Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raise import error on module import failure to avoid segfault #786

Merged
merged 1 commit into from
May 14, 2020

Conversation

overstre
Copy link
Contributor

@overstre overstre commented Feb 25, 2020

A dependency of confluent-kafka, enum34, published a broken version 1.1.8 that caused a segfault in confluent-kafka python library when accessing the module confluent_kafka.admin. For example, the list_topics call will try to import this module in function cfl_PyObject_lookup.

This results in module pointer being nil in the return of this call:
PyObject *module = PyImport_ImportModule(modulename);

and then it gets accessed here
obj = PyObject_GetAttrString(module, typename);
which results in a segfault:


Thread 28 "python" received signal SIGSEGV, Segmentation fault.
--
[Switching to Thread 0x7fff96ffd700 (LWP 8778)]
0x00000000004efa6b in PyObject_GetAttrString ()
(gdb) backtrace
#0  0x00000000004efa6b in PyObject_GetAttrString ()
#1  0x00007ffff433dae2 in cfl_PyObject_lookup (modulename=modulename@entry=0x7ffff434a067 "confluent_kafka.admin",
typename=typename@entry=0x7ffff434a057 "ClusterMetadata") at confluent_kafka/src/confluent_kafka.c:1962
#2  0x00007ffff43418a0 in c_metadata_to_py (self=0x7ffff0127440, metadata=0x7fff84002570) at confluent_kafka/src/Metadata.c:267
#3  list_topics (self=0x7ffff0127440, args=<optimized out>, kwargs=<optimized out>) at confluent_kafka/src/Metadata.c:382

This pull request is to check the value of module to ensure that it's not null otherwise return a runtime exception. This will make debugging this issue in the future.

Without patch

$ python ~/list.py
Segmentation fault

with patch:

$ python ~/list.py
Traceback (most recent call last):
  File "/home/brian/list.py", line 5, in <module>
    print producer.list_topics(topic=topic).topics.values()
RuntimeError: Module confluent_kafka.admin import failed.

@ghost
Copy link

ghost commented Feb 25, 2020

It looks like @overstre hasn't signed our Contributor License Agreement, yet.

The purpose of a CLA is to ensure that the guardian of a project's outputs has the necessary ownership or grants of rights over all contributions to allow them to distribute under the chosen licence.
Wikipedia

You can read and sign our full Contributor License Agreement here.

Once you've signed reply with [clabot:check] to prove it.

Appreciation of efforts,

clabot

@overstre
Copy link
Contributor Author

[clabot:check]

@ghost
Copy link

ghost commented Feb 25, 2020

@confluentinc It looks like @overstre just signed our Contributor License Agreement. 👍

Always at your service,

clabot

@rnpridgeon
Copy link
Contributor

What are your thoughts on removing the enum34 dependency all together. I'm actually not a huge fan of this dependency and we can easily achieve the desired functionality with constants as we do with logical offsets.

@overstre
Copy link
Contributor Author

I'm in favor of reducing dependencies and keeping the style consistent.

@overstre overstre requested a review from edenhill February 25, 2020 21:26
@@ -1959,6 +1959,13 @@ PyObject *cfl_PyObject_lookup (const char *modulename, const char *typename) {
return NULL;
}

if (!module) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for finding this, but I think the initial intent was for the if-statement above to handle this, could you change that one instead to check for module rather than modulename, and remove the first %s and modulename from the argument list.

Also, change it to PyExc_ImportError.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review. I've updated the code.

brian@dev-brian-1:~/code/confluent-kafka-python$ python ~/list.py
Traceback (most recent call last):
  File "/home/brian/list.py", line 5, in <module>
    print producer.list_topics(topic=topic).topics.values()
ImportError: Module not found when looking up confluent_kafka.admin.ClusterMetadata

@overstre overstre changed the title Raise runtime error on module import failure to avoid segfault Raise import error on module import failure to avoid segfault Feb 26, 2020
@overstre overstre requested a review from edenhill February 26, 2020 22:39
@overstre overstre force-pushed the module_import_exception branch from 91e8618 to 22682e0 Compare February 26, 2020 22:45
@overstre overstre force-pushed the module_import_exception branch from 22682e0 to 11dd8e5 Compare February 26, 2020 22:57
@rnpridgeon rnpridgeon merged commit 1fdc89b into confluentinc:master May 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants