Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multistatement queries enter infinite busy loop upon network error #306

Closed
fulara opened this issue Jan 18, 2022 · 3 comments
Closed

multistatement queries enter infinite busy loop upon network error #306

fulara opened this issue Jan 18, 2022 · 3 comments

Comments

@fulara
Copy link
Contributor

fulara commented Jan 18, 2022

I have encountered an issue with multi statement queries.
In my case those queries have multiple inserts to a different tables formatted and send as a string request to database.

If during the execution of the multistatment query a network failure happens, such as target endpoint going down, all writers hang.

version of the rust-mysql-simple: latest master

My understanding is that this is root of the problem
read_packet in handle_result_set from line src/conn/mod.rs:914 returns an error.
handle_next -> query_result:147 matches an error and sets self.state to Errored
self.conn.more_results_exists() will keep on returning true because nothing touches status_flags in this flow.

Callers generally expect the code to return OnBoundary or Done at some point.
death loop is entered via ResultSet drop call.

fn drop(&mut self) {
        while self.next().is_some() {}
    }
fn next()  .. { 
...
Errored(err) => {
  self.handle_next();
  Some(Err(err))
}

In case of network failure let's say the target endpoint goes down, read_packet will be returning error forever, so this will spin forever.

I have made a very silly commit that adds logs to confirm that above is the issue here is the commit:
fulara@186f615

Here are log statements ( from one thread):

Silly log statements
ThreadId(15) Loggin! I am still loopin
ThreadId(15) Loggin! state: in ::iter() Errored(IoError { server disconnected })
ThreadId(15) Loggin! state: OnBoundary index 582637288, conn: Mut(Conn(ConnInner { ... } , last_command: 3, connected: true, has_results: false, local_infile_handler: None })) more result exist?! true
ThreadId(15) Reading a packet....
ThreadId(15) Loggin! Entered err state. now: Errored(IoError { server disconnected }), index b4 incrementing 582637288. err is: IoError { server disconnected }
ThreadId(15) Loggin! I am still loopin
ThreadId(15) Loggin! state: in ::iter() Errored(IoError { server disconnected })
ThreadId(15) Loggin! state: OnBoundary index 582637289, conn: Mut(Conn(ConnInner { ... } , last_command: 3, connected: true, has_results: false, local_infile_handler: None })) more result exist?! true
ThreadId(15) Reading a packet....
ThreadId(15) Loggin! Entered err state. now: Errored(IoError { server disconnected }), index b4 incrementing 582637289. err is: IoError { server disconnected }
ThreadId(15) Loggin! I am still loopin

You can say this is a more involved case of this: #285 Whcih probably does not work in this case because the code bails out earlier.

@blackbeam
Copy link
Owner

Thanks for report. I'll look into it as soon as possible.

@fulara
Copy link
Contributor Author

fulara commented Jan 19, 2022

hey @blackbeam, thanks for response.

this hacky attempt: fulara@67747ad
seems to fix the issue, but no clue how it fits into err handling flow.

@blackbeam
Copy link
Owner

@fulara, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants