-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: allow calling Actor.reboot()
from migrating handler, align reboot behavior with JS SDK
#361
Conversation
…, align reboot behavior with JS SDK
src/apify/_actor.py
Outdated
persist_state_listeners = chain.from_iterable( | ||
(self._event_manager._listeners_to_wrappers[Event.PERSIST_STATE] or {}).values() # noqa: SLF001 | ||
) | ||
migrating_listeners = chain.from_iterable( | ||
(self._event_manager._listeners_to_wrappers[Event.MIGRATING] or {}).values() # noqa: SLF001 | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I might be missing something, but the chain.from_iterable
doesn't seem to have a purpose here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It does, _listeners_to_wrappers
is a dict of dicts of lists of wrappers (because technically you can have one listener wrapped multiple times, even though it doesn't make practical sense).
Imagine this:
def my_event_listener(event_data):
print(event_data)
Actor.on(Event.MIGRATING, my_event_listener)
Actor.on(Event.MIGRATING, my_event_listener)
Then _listeners_to_wrappers
looks like this:
{
Event.MIGRATING: {
event_listener: [
wrapper_for_my_event_listener_1,
wrapper_for_my_event_listener_2,
],
}
}
So _listeners_to_wrappers[Event.PERSIST_STATE]).values()
is a list of lists, and with chain.from_iterable()
I flatten it to a single-level list.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Huh, thanks. SDK already depends on Crawlee and that depends on more_itertools
which contain flatten
- could you add that dependency to SDK as well and use that instead? I know it serves no practical purpose other than readability.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, will do.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This fixes several issues with the
Actor.reboot()
behavior:Actor.reboot()
waits for all event handlers to finish, but if itself it was called in an event handler, it would be waiting for itself, getting into a deadlockActor.reboot()
in the JS SDK triggers event handlers for themigrating
andpersistState
events, but in the Python SDK it was triggering only thepersistState
handlersThis aligns the behavior to work like the JS SDK, and prevents reboot getting into an infinite loop by allowing it to be called only once.
Related PR in JS SDK: apify/apify-sdk-js#345