Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed couple of "leaks" when gc is disabled #979

Closed
wants to merge 12 commits into from
Closed

Fixed couple of "leaks" when gc is disabled #979

wants to merge 12 commits into from

Conversation

Mephius
Copy link
Contributor

@Mephius Mephius commented Feb 7, 2017

Since I'm using kafka-python in an environment with disabled gc these
"leaks" hit me hard.

Since I'm using kafka-python in an environment with disabled gc these
"leaks" hit me hard.
del X

# Don't del it here, cause with gc disabled this "leaks" to garbage
# del X
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this one is a vendored library, and I really would prefer fixing upstream. otherwise I'm likely to forget and overwrite when I import a newer version. Have you checked if this is fixed already in six?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upstream doesn't have this fix, but I've filed a pull request: benjaminp/six#176

@@ -93,7 +93,8 @@ def __iter__(self):
return iter(self._selector._fd_to_key)


class BaseSelector(six.with_metaclass(ABCMeta)):
@six.add_metaclass(ABCMeta)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same re: vendored library. Can you check with upstream version?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request to vendored library: berkerpeksag/selectors34#7

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upstream merged pull request.

@Mephius
Copy link
Contributor Author

Mephius commented Feb 14, 2017

Sorry for spamming PR with commits.
The trick with this implementation of WeakMethod is that one can't call it via call chaining:Message().encode() cause in this case it looses ref to target object (because of ref_count==0 I believe). So I had to unwind generators and lists comprehensions in order to have ref_count=1 for Message object until it goes out of scope.

Python is magic sometimes.

@dpkp
Copy link
Owner

dpkp commented Mar 3, 2017

I merged the selectors34 code from upstream separately. I am ok merging the rest, though I would prefer to wait for upstream to merge before changing six. That said, given that the change is mostly trivial I would be ok merging.

Can you rebase?

dpkp pushed a commit that referenced this pull request Mar 7, 2017
@dpkp
Copy link
Owner

dpkp commented Mar 7, 2017

Merged manually as 5a0e9715f45b62cfe43e6873b8828f49ab73f710

@dpkp dpkp closed this Mar 7, 2017
@Mephius
Copy link
Contributor Author

Mephius commented Mar 13, 2017

Thank you!
P.S. wasn't able to react because of vacation.

88manpreet pushed a commit to Yelp/kafka-python that referenced this pull request Apr 7, 2017
jeffwidman added a commit that referenced this pull request Oct 22, 2018
Bump `six` to `1.11.0`. Most changes do not affect us, but it's good to
stay up to date. Also, we will likely start vendoring `enum34` in which
case benjaminp/six#178 is needed.

Note that this preserves the `kafka-python` customization from #979
which has been submitted upstream as benjaminp/six#176 but not yet merged.
jeffwidman added a commit that referenced this pull request Oct 22, 2018
Bump `six` to `1.11.0`. Most changes do not affect us, but it's good to
stay up to date. Also, we will likely start vendoring `enum34` in which
case benjaminp/six#178 is needed.

Note that this preserves the `kafka-python` customization from #979
which has been submitted upstream as benjaminp/six#176 but not yet merged.
jeffwidman added a commit that referenced this pull request Oct 22, 2018
Bump `six` to `1.11.0`. Most changes do not affect us, but it's good to
stay up to date. Also, we will likely start vendoring `enum34` in which
case benjaminp/six#178 is needed.

Note that this preserves the `kafka-python` customization from #979
which has been submitted upstream as benjaminp/six#176 but not yet merged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants