Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(lp_bot): cancel order on stopping mm2 context #1143

Merged
merged 6 commits into from
Nov 24, 2021
Merged

Conversation

Milerius
Copy link

No description provided.

@Milerius Milerius linked an issue Nov 18, 2021 that may be closed by this pull request
@smk762
Copy link

smk762 commented Nov 18, 2021

  • Upon starting mm2, automatically cancel all orders from prior session. To allow for cases where user may not want this to happen, a boolean MM2.json config parameter could define this behavior.
  • When running stop_simple_market_maker_bot, cancel_all_orders (or cancel_order for list of UUIDs / pairs initiated by makerbot) could be initiated to overcome delay while awaiting completion of current loop.
  • When running stop, cancel_all_orders (or cancel_order for list of UUIDs / pairs initiated by makerbot) could be initiated cancel active orders to avoid them appearing on next load.

Which method of mitigation does this implement?

@Milerius
Copy link
Author

  • Upon starting mm2, automatically cancel all orders from prior session. To allow for cases where user may not want this to happen, a boolean MM2.json config parameter could define this behavior.
  • When running stop_simple_market_maker_bot, cancel_all_orders (or cancel_order for list of UUIDs / pairs initiated by makerbot) could be initiated to overcome delay while awaiting completion of current loop.
  • When running stop, cancel_all_orders (or cancel_order for list of UUIDs / pairs initiated by makerbot) could be initiated cancel active orders to avoid them appearing on next load.

Which method of mitigation does this implement?

The last one

@smk762
Copy link

smk762 commented Nov 18, 2021

Confirm orders are cancelled if stopping mm2 gracefully.
I did encounter some delays on myorders response at times tho.

Copy link

@sergeyboyko0791 sergeyboyko0791 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix!
Could you please consider my comments?

.read()
.await
.dispatch_async(ctx.clone(), event.into())
.await;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about adding a static function dispatch_lp_event that will look like:

pub fn dispatch_lp_event(ctx: MmArc, event: LpEvents) -> Result<(), MmError<DispatcherError>> {
  let dispatcher_ctx = DispatcherContext::from_ctx(&ctx).or_mm_error(|| Error::MmCtxNotAvailable)?;
  dispatcher_ctx.dispatcher.read().await.dispatch_async(ctx, event).await;
  Ok()
}

I think we can add it because we're supposed to produce events from a lot different places in the future, and it will reduce amount of code.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like a good idea, will do!

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please replace dispatcher_ctx.dispatcher.read().await.dispatch_async(ctx.clone(), event.into()).await; with dispatch_lp_event().await?

.read()
.await
.dispatch_async(ctx.clone(), event.into())
.await;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We call the MmCtx::stop method from another one place: on the extern function mm2_stop.

I suggest using MmCtx::on_stop to dispatch StopCtxEvent like it does spawn_rpc.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the function MmCtx::on_stop is called before the member of the structure are destroyed? In this case that's probably a good idea!

mm2src/lp_ordermatch/lp_bot.rs Show resolved Hide resolved
@sergeyboyko0791
Copy link

Also I'd say that I think using RwLock is a good decision for the dispatcher.
I tried to imagine a case when it's worth using AsyncMutex instead, and what I realized:

What if an event listener adds another listener when dispatches an event?

async fn process_event_async(&self, ctx: MmArc, event: AnyEvent) {
  let mut dispatcher = dispatcher_ctx.dispatcher.write().await;
  dispatcher.add_listener(simple_market_maker_bot_ctx.clone());
}

But it's more likely to face the deadlock if we use AsyncMutex because event listener may produce an event while dispatching another already.

I think it's out of this PR because I'm afraid that there are other places where we may face the deadlock too.

@Milerius
Copy link
Author

Milerius commented Nov 23, 2021

Also I'd say that I think using RwLock is a good decision for the dispatcher. I tried to imagine a case when it's worth using AsyncMutex instead, and what I realized:

What if an event listener adds another listener when dispatches an event?

async fn process_event_async(&self, ctx: MmArc, event: AnyEvent) {
  let mut dispatcher = dispatcher_ctx.dispatcher.write().await;
  dispatcher.add_listener(simple_market_maker_bot_ctx.clone());
}

But it's more likely to face the deadlock if we use AsyncMutex because event listener may produce an event while dispatching another already.

I think it's out of this PR because I'm afraid that there are other places where we may face the deadlock too.

It's exactly the reason that I decided to use RwLock also I think there is some other place we should use it, for example for my_orders, take the following case:

  • The bot is launched and require my_orders to iterate and update order -> lock the mutex, clone the orders
  • The GUI call my_orders RPC to refresh orders in the GUI -> wait for the lock to be released in order to lock it again

Here I think there is a possible long lock (not a deadlock because the lock will be released). Allowing multiple readers as the same time may be a solution for this kind of use case too.

My advice to avoid deadlock when using the dispatcher:

Add the events that you would like to listen too at the creation of the object / context. Remove them when you shutdown gracefully your app or your context.

- A later refactor of stop/on_stop behaviour of the context will happen in another pull request
@Milerius
Copy link
Author

Summary of the last fixes:

  • Implement dispatch_lp_event
  • Dispatch the stopping event before calling ctx.stop() to ensure the possibility to use the ctx in the listeners
  • Use block_on in the mm2_stop native function when dispatching the event in order to be able to return the status_code afterwards (using spawn will not allow to return anything in the function in this case) - should be ok in this function since it's the stopping of the application

Copy link

@sergeyboyko0791 sergeyboyko0791 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only one thing

@@ -233,6 +235,7 @@ pub extern "C" fn mm2_stop() -> i8 {
},
};

block_on(dispatch_lp_event(ctx.clone(), StopCtxEvent {}.into()));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one question: are we planning to add fields to the StopCtxEvent structure someday? If not, I'd suggest making it as pub struct StopCtxEvent; to simplify it an instance creation: StopCtxEvent.into().

.read()
.await
.dispatch_async(ctx.clone(), event.into())
.await;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please replace dispatcher_ctx.dispatcher.read().await.dispatch_async(ctx.clone(), event.into()).await; with dispatch_lp_event().await?

Copy link

@sergeyboyko0791 sergeyboyko0791 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes! Approve!

@sergeyboyko0791
Copy link

Use block_on in the mm2_stop native function when dispatching the event in order to be able to return the status_code afterwards (using spawn will not allow to return anything in the function in this case) - should be ok in this function since it's the stopping of the application

We decided not to use block_on on the mm2_stop function call since it may lead to the GUI thread blocking. It's enough to check if the mm2 instance is stopping already and spawn async future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improper stopping of mm2 / makerbot could result in zombie orders.
3 participants