Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BaseAdapterRegistry.subscribe/unsubscribe non-symmetric updating _provided, leaves orphan entries #227

Closed
jamadden opened this issue Feb 26, 2021 · 0 comments · Fixed by #228
Labels

Comments

@jamadden
Copy link
Member

While working on #224 I noticed that there were no tests for BaseAdapterRegistry._provided (which keeps a map of {interface: reference count} of interfaces it knows about in order to add them as "extendors" to the AdapterLookup classes).

The register method increments this count and stores it:

n = self._provided.get(provided, 0) + 1
self._provided[provided] = n
if n == 1:
self._v_lookup.add_extendor(provided)

As does the subscribe method:
if provided is not None:
n = self._provided.get(provided, 0) + 1
self._provided[provided] = n
if n == 1:
self._v_lookup.add_extendor(provided)

The inverse of register, unregister, decrements this count and either stores it, or, if it hits 0, removes it:

n = self._provided[provided] - 1
if n == 0:
del self._provided[provided]
self._v_lookup.remove_extendor(provided)
else:
self._provided[provided] = n

However, the inverse of subscribe, unsubscribe, appears to have a bug. It decrements the count, but only if it hits zero is it removed. It never stores it, which means it can never hit 0 just by unsubscribing.

if provided is not None:
n = self._provided[provided] + len(new) - len(old)
if n == 0:
del self._provided[provided]
self._v_lookup.remove_extendor(provided)

Making unsubscribe store the decremented value doesn't break any tests (and fixes the tests I was adding). Because the high-level class Components automatically both registers and subscribes utilities, it's fairly easy to end up with orphaned entries in _provided.

Here, I'll define an interface and class and register it both as the default (unnamed) and a named utility for the same interface.

>>> from zope.interface import Interface, implementer
>>> from zope.interface.registry import Components
>>> class IFoo(Interface):
...     pass
...
>>> @implementer(IFoo)
... class Foo:
...     pass
...
>>> foo = Foo()
>>> foo2 = Foo()
>>> comp = Components()
>>> comp.registerUtility(foo)
None
>>> comp.registerUtility(foo2, name='bar')
None

Now, because IFoo was both provided and subscribed twice, it has a refcount of 4, and the two objects show up once each in _adapters and _subscribers of the adapter registry:

>>> comp.utilities._provided
{<InterfaceClass __main__.IFoo>: 4}
>>> comp.utilities._adapters
[{<InterfaceClass __main__.IFoo>: {'': <__main__.Foo object at 0x10a09dfa0>,
                                   'bar': <__main__.Foo object at 0x10a08e140>}}]
>>> comp.utilities._subscribers
[{<InterfaceClass __main__.IFoo>: {'': (<__main__.Foo object at 0x10a09dfa0>,
                                        <__main__.Foo object at 0x10a08e140>)}}]

Now we can unregister both utilities, and their entries will be gone from _adapters and _subscribers:

>>> comp.unregisterUtility(foo)
True
>>> comp.unregisterUtility(foo2, name='bar')
True
>>> comp.utilities._adapters, comp.utilities._subscribers
([], [])

But IFoo is still in _provided because of this issue:

>>> comp.utilities._provided
{<InterfaceClass __main__.IFoo>: 2}

Because of the way unsubscribe calculates the delta (removing all equal objects) if I register the same identical object (which is if course equal to itself) as both the named and unnamed utility, this appears to work. That is, _provided has no remaining references to IFoo.

When using non-equal objects, this only works (empty _provided) if I make unsubscribe store the decremented value. The equal object case still works then.

@jamadden jamadden added the bug label Feb 26, 2021
jamadden added a commit that referenced this issue Feb 26, 2021
Add extensive tests for this. Fixes #224.

Also adds test for, and fixes #227
jamadden added a commit that referenced this issue Mar 1, 2021
Add extensive tests for this. Fixes #224.

Also adds test for, and fixes #227
jamadden added a commit that referenced this issue Mar 1, 2021
Add extensive tests for this. Fixes #224.

Also adds test for, and fixes #227

Add BAR.rebuild() to fix the refcount issue, and to change datatypes.

It works by using the new methods allRegistrations() and allSubscriptions() to re-create the data in new data structures.

This makes fresh calls to subscribe() and register(). I went this way instead of trying to manually walk the data structures and create them because the logic in those methods is fully tested.
jamadden added a commit that referenced this issue Mar 3, 2021
Add extensive tests for this. Fixes #224.

Also adds test for, and fixes #227

Add BAR.rebuild() to fix the refcount issue, and to change datatypes.

It works by using the new methods allRegistrations() and allSubscriptions() to re-create the data in new data structures.

This makes fresh calls to subscribe() and register(). I went this way instead of trying to manually walk the data structures and create them because the logic in those methods is fully tested.
netbsd-srcmastr referenced this issue in NetBSD/pkgsrc Apr 14, 2021
5.3.0 (2020-03-21)
==================

- No changes from 5.3.0a1


5.3.0a1 (2021-03-18)
====================

- Improve the repr of ``zope.interface.Provides`` to remove ambiguity
  about what is being provided. This is especially helpful diagnosing
  IRO issues.

- Allow subclasses of ``BaseAdapterRegistry`` (including
  ``AdapterRegistry`` and ``VerifyingAdapterRegistry``) to have
  control over the data structures. This allows persistent
  implementations such as those based on ZODB to choose more scalable
  options (e.g., BTrees instead of dicts). See `issue 224
  <https://github.com/zopefoundation/zope.interface/issues/224>`_.

- Fix a reference counting issue in ``BaseAdapterRegistry`` that could
  lead to references to interfaces being kept around even when all
  utilities/adapters/subscribers providing that interface have been
  removed. This is mostly an issue for persistent implementations.
  Note that this only corrects the issue moving forward, it does not
  solve any already corrupted reference counts. See `issue 227
  <https://github.com/zopefoundation/zope.interface/issues/227>`_.

- Add the method ``BaseAdapterRegistry.rebuild()``. This can be used
  to fix the reference counting issue mentioned above, as well as to
  update the data structures when custom data types have changed.

- Add the interface method ``IAdapterRegistry.subscribed()`` and
  implementation ``BaseAdapterRegistry.subscribed()`` for querying
  directly registered subscribers. See `issue 230
  <https://github.com/zopefoundation/zope.interface/issues/230>`_.

- Add the maintenance method
  ``Components.rebuildUtilityRegistryFromLocalCache()``. Most users
  will not need this, but it can be useful if the ``Components.utilities``
  registry is suspected to be out of sync with the ``Components``
  object itself (this might happen to persistent ``Components``
  implementations in the face of bugs).

- Fix the ``Provides`` and ``ClassProvides`` descriptors to stop
  allowing redundant interfaces (those already implemented by the
  underlying class or meta class) to produce an inconsistent
  resolution order. This is similar to the change in ``@implementer``
  in 5.1.0, and resolves inconsistent resolution orders with
  ``zope.proxy`` and ``zope.location``. See `issue 207
  <https://github.com/zopefoundation/zope.interface/issues/207>`_.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant