I had a dream

[Virgiel]

These ideas seem difficult to implement, so it is more of a wish list than a to-do list until someone can show a proof of concept.

Iterator for array params

Currently we use slices. That is good for vectors of copied types, but when they are not, we use a slice of references that cannot be used with a vector without using a temporary vector of references. The most ergonomic solution would be to support iterators, which brings many challenges:

• postgres expects a reference and an iterator need be owned to be consumed, this can be hacked using RefCell.
• we would need a generic ToSql function for iterators.

Hide domains types wrapper

Our generated domain types do not enforce the domain rules and are only used for ser/deser with the postgres protocol. By being exposed to the client in parameters and rows types, they bring noise without any benefit. We should use these types internally but not in the public API. This feature is not really difficult but may require a deep change in the code.

[Virgiel]

Ergonomic array parameters using trait

Let's start with the good part. Let's say we have a query function with an array of strings as parameters. Using a trait and a dynamic dispatch, we can have :

pub fn sql_array(client: &mut impl GenericClient, array: &dyn ArraySql<&str>) {
    client.execute(/* SQL */ , &[array.base()]).unwrap();
}

pub fn usage(client: &mut impl GenericClient) {
    let map: HashMap<String, String> = HashMap::new();

    // Old way with allocation
    let vec: Vec<_> = map.values().map(|s| s.as_str()).collect();
    sql_array(client, &vec.as_slice());
    // A little more ergonomic
    sql_array(client, &vec);
    // Zero allocation
    sql_array(client, &ArrayFn(|| map.values().map(|s| s.as_str())));
}

The idea is to use a trait to allow different types for arrays while restricting to compatible types:

pub trait ArraySql<T>: ToSql + Sync {
    fn base(&self) -> &(dyn ToSql + Sync);
}

impl<T: ToSql + Sync> ArraySql<T> for Vec<T> {
    fn base(&self) -> &(dyn ToSql + Sync) {
        self
    }
}
impl<T: ToSql + Sync> ArraySql<T> for &[T] {
    fn base(&self) -> &(dyn ToSql + Sync) {
        self
    }
}

impl<T: ToSql, I: Iterator<Item = T> + ExactSizeIterator, F: Fn() -> I + Sync> ArraySql<T>
    for ArrayFn<T, I, F>
{
    fn base(&self) -> &(dyn ToSql + Sync) {
        self
    }
}

And now the ugly hack to support iterators. postgres functions use &(dyn ToSql + Sync) and we need to own an iterator to consume it. Using interior mutability with Cell is not possible because of Sync. This is why I chose to use lambda for lazy iterator creation:

struct ArrayFn<T: ToSql, I: Iterator<Item = T> + ExactSizeIterator, F: Fn() -> I + Sync>(F);

impl<T: ToSql, I: Iterator<Item = T> + ExactSizeIterator, F: Fn() -> I + Sync> Debug
    for ArrayFn<T, I, F>
{
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        f.debug_tuple("ArrayFn").finish()
    }
}

// Taken from `postgres`
impl<T: ToSql, I: Iterator<Item = T> + ExactSizeIterator, F: Fn() -> I + Sync> ToSql
    for ArrayFn<T, I, F>
{
    fn to_sql(
        &self,
        ty: &Type,
        w: &mut BytesMut,
    ) -> Result<IsNull, Box<dyn std::error::Error + Sync + Send>> {
        let member_type = match *ty.kind() {
            Kind::Array(ref member) => member,
            _ => panic!("expected array type"),
        };

        let iter = (self.0)();

        let dimension = ArrayDimension {
            len: downcast(iter.len())?,
            lower_bound: 1,
        };

        types::array_to_sql(
            Some(dimension),
            member_type.oid(),
            iter,
            |e, w| match e.to_sql(member_type, w)? {
                IsNull::No => Ok(postgres_protocol::IsNull::No),
                IsNull::Yes => Ok(postgres_protocol::IsNull::Yes),
            },
            w,
        )?;
        Ok(IsNull::No)
    }

    fn accepts(ty: &Type) -> bool {
        match *ty.kind() {
            Kind::Array(ref member) => T::accepts(member),
            _ => false,
        }
    }

    to_sql_checked!();
}

// https://github.com/sfackler/rust-postgres/blob/765395f288861209a644c621bf72172acd482515/postgres-types/src/lib.rs
fn downcast(len: usize) -> Result<i32, Box<dyn std::error::Error + Sync + Send>> {
    if len > i32::max_value() as usize {
        Err("value too large to transmit".into())
    } else {
        Ok(len as i32)
    }
}

The cost

Dynamic dispatch has a runtime cost and prevent some compiler optimisations, but it allow easy codegen:

struct Params<'a> {
    array: &'a dyn ArraySql<&'a str>,
}

impl<'a> Params<'a> {
    pub fn sql_array(self, client: &mut Client) {
        sql_array(client, self.array)
    }
}

Using generic will lower runtime cost, but would make codegen significantly harder:

struct Params<'a, A: ArraySql<&'a str>> {
    array: A,
    data: PhantomData<&'a str>,
}

impl<'a, A: ArraySql<&'a str>> Params<'a, A> {
    pub fn sql_array(self, client: &mut Client) {
        sql_array(client, self.array)
    }
}

pub fn sql_array<'a>(client: &mut impl GenericClient, array: impl ArraySql<&'a str>) {
    client.execute("", &[array.base()]).unwrap();
}

As postgres already use dynamic dispatch, I think it could be an acceptable cost.

Going further than array

While this trick was mandatory to support iterator as array parameters, we could reuse it for other parameters type.

[Virgiel]

Another crazy alternative with less trait and dynamic dispatch:

pub struct Params<'a, I: Iterator<Item = &'a str> + ExactSizeIterator, F: Fn() -> I + Sync> {
    array: &'a Array<'a, &'a str, I, F>,
}

impl<'a, I: Iterator<Item = &'a str> + ExactSizeIterator, F: Fn() -> I + Sync> Params<'a, I, F> {
    pub fn sql_array(self, client: &mut impl GenericClient) {
        sql_array(client, self.array)
    }
}

pub fn sql_array<'a, I: Iterator<Item = &'a str> + ExactSizeIterator, F: Fn() -> I + Sync>(
    client: &mut impl GenericClient,
    array: &'a Array<'a, &'a str, I, F>,
) {
    client.execute("", &[array]).unwrap();
}

pub fn usage(client: &mut impl GenericClient) {
    let map: HashMap<String, String> = HashMap::new();

    // Old way with allocation
    let vec: Vec<_> = map.values().map(|s| s.as_str()).collect();
    sql_array(client, &vec.as_slice().into());
    // A little more ergonomic
    sql_array(client, &(&vec).into());
    // Zero allocation
    sql_array(client, &(|| map.values().map(|s| s.as_str())).into());
}

And all the gory details:

// Zero size iterator
pub struct DumbIter<T> {
    data: PhantomData<T>,
}

impl<T> Iterator for DumbIter<T> {
    type Item = T;

    fn next(&mut self) -> Option<Self::Item> {
        None
    }
}

impl<T> ExactSizeIterator for DumbIter<T> {
    fn len(&self) -> usize {
        0
    }
}

pub enum Array<'a, T: ToSql + Sync, I: Iterator<Item = T> + ExactSizeIterator, F: Fn() -> I + Sync>
{
    Slice(&'a [T]),
    Fn(F),
}

impl<'a, T: ToSql + Sync, I: Iterator<Item = T> + ExactSizeIterator, F: Fn() -> I + Sync> Debug
    for Array<'a, T, I, F>
{
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            Self::Slice(arg0) => f.debug_tuple("Slice").field(arg0).finish(),
            Self::Fn(_) => f.debug_tuple("Fn").field(&"ef").finish(),
        }
    }
}

impl<'a, T: ToSql + Sync, I: Iterator<Item = T> + ExactSizeIterator, F: Fn() -> I + Sync> ToSql
    for Array<'a, T, I, F>
{
    fn to_sql(
        &self,
        ty: &Type,
        out: &mut BytesMut,
    ) -> Result<IsNull, Box<dyn std::error::Error + Sync + Send>>
    where
        Self: Sized,
    {
        match self {
            Array::Slice(it) => it.to_sql(ty, out),
            Array::Fn(it) => {
                let member_type = match *ty.kind() {
                    Kind::Array(ref member) => member,
                    _ => panic!("expected array type"),
                };

                let iter = it();

                let dimension = ArrayDimension {
                    len: downcast(iter.len())?,
                    lower_bound: 1,
                };

                types::array_to_sql(
                    Some(dimension),
                    member_type.oid(),
                    iter,
                    |e, w| match e.to_sql(member_type, w)? {
                        IsNull::No => Ok(postgres_protocol::IsNull::No),
                        IsNull::Yes => Ok(postgres_protocol::IsNull::Yes),
                    },
                    out,
                )?;
                Ok(IsNull::No)
            }
        }
    }
    fn accepts(ty: &Type) -> bool {
        match *ty.kind() {
            Kind::Array(ref member) => T::accepts(member),
            _ => false,
        }
    }

    to_sql_checked!();
}

impl<'a, T: ToSql + Sync> From<&'a [T]> for Array<'a, T, DumbIter<T>, fn() -> DumbIter<T>> {
    fn from(it: &'a [T]) -> Self {
        Self::Slice(it)
    }
}

impl<'a, T: ToSql + Sync> From<&'a Vec<T>> for Array<'a, T, DumbIter<T>, fn() -> DumbIter<T>> {
    fn from(it: &'a Vec<T>) -> Self {
        Self::Slice(it)
    }
}

impl<'a, T: ToSql + Sync, const N: usize> From<&'a [T; N]>
    for Array<'a, T, DumbIter<T>, fn() -> DumbIter<T>>
{
    fn from(it: &'a [T; N]) -> Self {
        Self::Slice(it)
    }
}

impl<'a, T: ToSql + Sync, I: Iterator<Item = T> + ExactSizeIterator, F: Fn() -> I + Sync> From<F>
    for Array<'a, T, I, F>
{
    fn from(it: F) -> Self {
        Self::Fn(it)
    }
}