Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packets out of sync CodecError when stracing process #244

Closed
Elykz opened this issue Aug 10, 2020 · 1 comment
Closed

Packets out of sync CodecError when stracing process #244

Elykz opened this issue Aug 10, 2020 · 1 comment

Comments

@Elykz
Copy link

Elykz commented Aug 10, 2020

Attaching/Detaching process with strace during a running query might cause a CodecError due to
interrupted systemcall.

Subsequent queries will then return a "Packets out of sync" CodecError

read_timeout needs to be enabled to reproduce the issue (i was not able to reproduce it without it).

Term 1:

$ cargo run --release
   Compiling rust-mysql-outofsync v0.1.0 (/home/endy/rust-mysql-outofsync)
    Finished dev [unoptimized + debuginfo] target(s) in 2.13s
     Running `target/debug/rust-mysql-outofsync`
PID to strace: 10503
Running query that will be interrupted by a strace on the application PID
CodecError { IO error: `Interrupted system call (os error 4)' }
Then running another query
error occured: mysql error: "CodecError { Packets out of sync }"

Term 2:

$ sudo strace -f -p 10503

The program itself:

use mysql::prelude::Queryable;
use std::time::Duration;

use err_derive::Error;

#[derive(Debug, Error)]
pub enum MyError {
    #[error(display = "mysql error: {:?}", _0)]
    MySQLError(String),
}

const USER: &str = "my_user";
const PASS: &str = "my_pass";
const HOST: &str = "my_host";
const DB: &str = "my_db";
const TABLE: &str = "my_table";

fn bug_if_straced(conn: &mut mysql::Conn) -> Result<(), MyError> {
  // my_table contains a few millions entries in order
  // to have time to attach strace to the running process while
  // fetching rows
  let query = format!("SELECT * FROM {}.{}", DB, TABLE);

  let statement = conn.exec_iter(query, ());

     let iter = statement
        .map_err(|e| MyError::MySQLError(format!("{}", e)))?;

     for i in iter {
        if let Err(e) = i {
            println!("{:?}", e);
            break;
        }
     }

  Ok(())
}

fn run_bug_if_straced(conn: &mut mysql::Conn) {
    if let Err(e) = bug_if_straced(conn) {
        println!("error occured: {}", e);
    }
}

fn main() {

  println!("PID to strace: {}", std::process::id());

  // Note: i can only reproduce the issue when read_timeout is specified
  let conn_builder = mysql::OptsBuilder::new()
     .user(Some(USER))
     .pass(Some(PASS))
     .ip_or_hostname(Some(HOST))
     .read_timeout(Some(Duration::from_millis(60000)));

  let mut conn = mysql::Conn::new(conn_builder).unwrap();
 
  println!("Running query that will be interrupted by a strace on the application PID");
  run_bug_if_straced(&mut conn);
  
  println!("Then running another query");
  run_bug_if_straced(&mut conn);
}

The Cargo.toml file:

[package]
name = "rust-mysql-outofsync"
version = "0.1.0"
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
mysql = { version = "18.2.0", features = [] }
err-derive = "0.2"

The issue does not happen 100% of the time but is easy to reproduce while compiled in release mode.
It happens either right after attaching process with strace, or right after detaching.

@blackbeam
Copy link
Owner

This report was very helpful, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants