Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the Rust API more lazy #138

Open
hhirtz opened this issue May 5, 2022 · 2 comments
Open

Make the Rust API more lazy #138

hhirtz opened this issue May 5, 2022 · 2 comments

Comments

@hhirtz
Copy link
Member

hhirtz commented May 5, 2022

Right now algorithms are run eagerly:

coupe::Rcb { iter_count: 2, ..Default::default() }
    .partition(ids, (points, weights))
    .unwrap();
coupe::KMeans::default()
    .partition(ids, (points, weights))
    .unwrap();
coupe::FiducciaMattheyses { max_imbalance: Some(0.05), ..Default::default() }
    .partition(ids, (adjacency, weights))
    .unwrap();

Could coupe make do with less computations? It would mean we'd need to have a global view of the algorithm chain, something used like iterators?

trait IntoPartitioner {
    fn into_partitioner(&mut self) -> Partitioner<'_>;
}

ids
    .partition()
    .with_vertex_coords(points)
    .with_vertex_weights(weights)
    .with_adjacency(adjacency)
    .rcb(coupe::Rcb { iter_count: 2, ..Default::default() })
    .k_means(Default::default())
    .fidducia_mattheyses(coupe::FiducciaMattheyses {
        max_imbalance: Some(0.05),
        ..Default::default()
    })
    .collect() // ?
    .unwrap()?;
@hhirtz
Copy link
Member Author

hhirtz commented Jun 24, 2022

The partition type also needs to be reworked.

Since even before the trait rewrite in #9, the way to store a partition is a Vec<PartId>, which has the advantage of being easy to use and pass around, but also poses a number of problems:

  • The number of parts can only be guessed (is wrong when a part is empty) and is O(n) to compute. This leads to weird APIs like coupe::imbalance::compute_part_loads and part-info's -n option,
  • Metrics like the imbalance and the edge cut have to be stored separately (or, in current master, recomputed every time an algorithm needs it).

As a middle point between the two suggestions above, we could have a smart Partition type (name TBD) that keeps track of relevant metrics and is updated by algorithms.

@hhirtz
Copy link
Member Author

hhirtz commented Jul 26, 2022

Some difficulties I'm encountering with coupe's current API with the [num-part port]:

  • it's difficult to access a specific metadata from an algorithm run when you accept any algorithm. For example the number of iterations,
  • algorithms can't run in parallel because the Partition trait requires &mut self because some algorithms like Random have internal state that is updated during a run. Although this ended up not really needed since spawning the rayon threadpool on all threads ends up really time consuming compared to the algorithm iterations, so I set RAYON_NUM_THREADS=1 instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant