Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Unpacking Array<[A; N], D> into Array<A, D::Larger> #1466

Open
gekailin opened this issue Dec 20, 2024 · 3 comments
Open

Feature Request: Unpacking Array<[A; N], D> into Array<A, D::Larger> #1466

gekailin opened this issue Dec 20, 2024 · 3 comments

Comments

@gekailin
Copy link

In certain scenarios, it is convenient to create a new dimension by unpacking arrays. For example:

let angles = array![0.1, 0.2, 0.3];
let vectors = angles.mapv(|a| [a.cos(), a.sin()]); 
// Here, `vectors` is of type `Array1<[f64; 2]>`, but an `Array2<f64>` is preferred.

To address this, I have written an extension trait:

pub trait ArrayUnpackExt<A, D, const N: usize>
where
    D: Dimension,
{
    fn unpack(self) -> Array<A, D::Larger>;
}

impl<A, D, const N: usize> ArrayUnpackExt<A, D, N> for Array<[A; N], D>
where
    D: Dimension,
{
    fn unpack(self) -> Array<A, D::Larger> {
        assert!(mem::size_of::<[A; N]>() == mem::size_of::<A>() * N);
        let ndim = self.ndim();
        let mut sh = self.raw_dim().insert_axis(Axis(ndim));
        sh[ndim] = N;
        let mut st = D::zeros(ndim);
        self.strides().iter().enumerate().for_each(|(i, &v)| {
            st[i] = (v * (N as isize)) as usize;
        });
        let st = st.insert_axis(Axis(ndim));
        let (vec, off) = self.into_raw_vec_and_offset();
        match off {
            Some(0) => {
                let vec = {
                    let mut vec = mem::ManuallyDrop::new(vec);
                    let ptr = vec.as_mut_ptr() as *mut A;
                    let len = vec.len() * N;
                    let cap = vec.capacity() * N;
                    unsafe { Vec::from_raw_parts(ptr, len, cap) }
                };
                unsafe { Array::from_shape_vec_unchecked(sh.strides(st), vec) }
            }
            _ => unreachable!(), // Unable to handle non-zero offsets even with unsafe code
        }
    }
}

I believe this functionality would be a valuable addition to the ndarray crate, allowing users to easily transform arrays into higher dimensions when needed. I hope you consider embedding this feature into the library.

Thank you for your attention and consideration.

@gekailin
Copy link
Author

I want to emphasize that the trait above is designed to be zero-copy, meaning it efficiently transforms the array structure without duplicating data. This ensures optimal performance and memory usage, making it an ideal solution for handling large datasets.

@akern40
Copy link
Collaborator

akern40 commented Dec 24, 2024

@gekailin I think this is a cool suggestion. It feels kinda like it falls under the same umbrella as #1463, and its solution feels kind of itertools-esque. It would be great if this worked for both arrays and homogeneous tuples.

A few notes / questions: Is the non-zero offset actually unreachable, or can we just not handle it? If it's the latter, I'd suggest that the method return Result rather than panicking. In addition, I don't think the trait itself needs the const-generic, just the impl.

@gekailin
Copy link
Author

@akern40

  1. Handling non-zero offsets isn't feasible outside the crate. However, within the crate, we can simplify the implementation by directly maintaining the .ptr field.
  2. You're right—the trait should eliminate the const-generic parameters. Thank you for the suggestion.
  3. I'm not familiar with homogeneous tuples. Could you help to complete this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants