Dynamic 'params'

[inzanez]

Hi

I tried to generate dynamic params to be used with execute or query, built from a HashMap<String, serde_json::Value>. I know that I could leverage Postgres internal functionality to parse and insert that data into a regular table structure without storing it as JSON/JSONB, but during load tests I realized that this generates additional load on the Postgres backend. So my plan was to do that transformation on the API side, as I can scale that horizontally.

Following is sample of how that transformation is done. This is very simplified though, as it involves different link tables and multiple entries:

let entity = <struct holding HashMap<String, Value>, in this case: { "id": Some(1), "link_id": Some(500) }>

let mut params_prep: Vec<Box<dyn ToSql + Sync>> = Vec::new();
params_prep.push(Box::new(entity.get("id").unwrap().as_i64()));
params_prep.push(Box::new(entity.get("link_id").unwrap().as_i64()));

let sql = "INSERT INTO link_table(id, link_id) VALUES ($1, $2);";
let params: Vec<&(dyn ToSql + Sync)> = params_prep.iter().map(|x| x.as_ref()).collect();
self.db.execute(&stmt, &params).await?;

There's nothing wrong with the above as long as I'm using synchronous postgres. Once I move to the tokio driven version, I get the error that:

let mut params_prep: Vec<Box<dyn ToSql + Sync>> = Vec::new();
has type `std::vec::Vec<std::boxed::Box<dyn postgres::types::ToSql + std::marker::Sync>>` which is not `Send`
...
self.db.execute(&stmt, &params).await?;
await occurs here, with `mut params_prep` maybe used later

I totally agree with that error. The reason I need to pack things into the params_prep vector is so that they are persisted, because they are copies from within the HashMap<String, Value>. If I add Send to the trait object, execute etc. will not accept the params anymore.

I currently see no way around that,...except if I could create the final params vector based on something that is Sync + Send, but using anything else than Vec<Box<dyn ToSql + Sync>> as params_prep did not let me construct the final params vector.

[sfackler]

You should just be able to make a Vec<Box<dyn ToSql + Sync + Send>>.

[inzanez]

@sfackler Almost,...it seems I have to cast the reference to &<dyn ToSql + Sync>.

...
let params: Vec<&(dyn ToSql + Sync)> = params_prep
    .iter()
    .map(|x| x.as_ref() as &(dyn ToSql + Sync))
    .collect();
...

Many thanks! Maybe it helpes someone else too.

[sfackler]

Great!