Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed issue with invoking _close() on closed event loop #3438

Merged
merged 11 commits into from
Dec 5, 2024

Conversation

vladvildanov
Copy link
Collaborator

@vladvildanov vladvildanov commented Nov 18, 2024

Pull Request check-list

Please make sure to review and check all of these items:

  • Do tests and lints pass with this change?
  • Do the CI tests pass with this change (enable it first in your forked repo and wait for the github action build to finish)?
  • Is the new or changed code fully tested?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
  • Is there an example added to the examples folder (if applicable)?
  • Was the change added to CHANGES file?

NOTE: these things are not required to open a PR and can be done
afterwards / while the PR is open.

Description of change

Closes #3431
Closes #3239

This PR fixes an issue when connection closed by garbage collector after event loop was closed. Also fixed an issue with __repr__ method tried to reference object already removed by garbage collector, by changing weak reference to strong reference.

Copy link

@teodorfn teodorfn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Tested and verified.

When using versions 5.0.2 to 5.2.0 i get RuntimeError: Event loop is closed (which was not an issue in versions <=5.0.1).

With this fix I don't get the error anymore. Thanks!

Comment on lines 120 to 119
self.connection_kwargs["connection_pool"] = weakref.proxy(self)
self.connection_kwargs["connection_pool"] = self
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sort of change has me wonder why it was a weakref in the first place – probably for a reason?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same concerns here as @akx. We need to ensure with a test that it doesn't create a circular reference and prevents garbage collecting unused SentinelConnectionPool objects.

Copy link
Collaborator Author

@vladvildanov vladvildanov Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But isn't it already covered by current tests in test_sentinel.py? I mean, GC is always take in action after the tests been executed. I also had a doubts about this, but I didn't find any reason why weakref was used and current tests works as expected. Do you have a specific test case in mind that is worth to add?

@vladvildanov vladvildanov added the bug Bug label Dec 5, 2024
Copy link
Contributor

@uglide uglide left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd still add some tests to verify, but it looks safer now!

@vladvildanov vladvildanov merged commit 5043b69 into master Dec 5, 2024
33 checks passed
@vladvildanov vladvildanov deleted the vv-3431-fix branch December 5, 2024 12:06
vladvildanov added a commit that referenced this pull request Dec 6, 2024
* Migrate to clients test image (#3415)

* Migrate to client-testing image

- Use clients-testing image for standalone and cluster
- Remove hardcoded TLS certificates and keys
- Remove stunnel
- Remove Cluster docker and configs

* Fix migration bugs

* Create reusable action to run tests

- Reduce copy paste by using reusable action for running tests
- Gain better control of tests matrix

Add missing actions checkout

More fixes in integration workflow

Another attempt to fix matrix

* Reorg test matrix

* Fix jobs names and execution order

* Execute standalone and cluster test simultaneously

* Streamline test execution

- Automatically map Redis version to Redis Stack version and use it for testing module commands
- Remove Graph commands from execution by default
- Include more Redis versions to the test matrix

* More fixes to integration job

* Move python compatibility tests to a separate task

* Improve run-tests action

* Add missing pytest marks for TS tests

* Fix cluster configuration

* Debug cluster tests

* Fix Cluster TLS port

* Move current redis version to env var

* Fix ssl tests

* Show CLUSTER NODES on fail

* Fix integration workflow bugs

* Add workarounds for IPv6 bug in tests

* Use hostname instead of hardcoded IPv4 loopback

* Fix bug in _get_client

* Fix run-tests action

* Fix imports

* Add missing version guards in search tests

* Add compatibility for Redis < 7

* Add missing version guard in search tests

* Fix run-tests

* Add missing tls-auth-clients option

* Skip module tests when Redis < 7 and RESP3 is enabled

* Fix async test_moved_redirection_on_slave_with_default

The test was broken for a while after migrating to all-in-one container with Cluster

* Cleanup test after debugging

* Use correct profile in install_and_test.sh

* Use matrix to execute hiredis<=3.0.0 tests

* Fix hiredis job

* Fix pytest command in install_and_test.sh

* Use 7.4.1 as default version in docker-compose.yml

* Fix uvloop-tests

* Fixed unsecured tempfile.mktemp() command usage (#3446)

* Fixed unsecured tempfile.mktemp() command usage

* Added proper tuple handling

* Fixed bug with SLOWLOG GET response parsing from Redis Enterprise (#3441)

* Fixed issue with invoking _close() on closed event loop (#3438)

* Fixed issue with invoking _close() on closed event loop

* Removed unused import

* Revert weakref changes

* Codestyle fix

* Added test coverage

* Codestyle fixes

* Codestyle fixes

* Removed failure check that fails in 3.12

* Codestyle fixes

* Codestyle fixes

* Fixing randomly failing test (#3437)

* Fixing randomly failing test

* Always rounding up to avoid randomly failing tests

* Always rounding up to avoid randomly failing tests

---------

Co-authored-by: Vladyslav Vildanov <117659936+vladvildanov@users.noreply.github.com>

* Updated package version

---------

Co-authored-by: Igor Malinovskiy <u.glide@gmail.com>
Co-authored-by: Ilian Iliev <ilian@ilian.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug
Projects
None yet
4 participants