You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
However, we may have m workers and n partitions, where m>n. Thus, each worker need to scan only part of the partition. Take this case into consideration and refine the trait.
The text was updated successfully, but these errors were encountered:
… trait to better support parallel processing in Runtime (#2744)
<!--
Thanks for your contribution! please review
https://github.com/alibaba/GraphScope/blob/main/CONTRIBUTING.md before
opening an issue.
-->
## What do these changes do?
<!-- Please give a short brief about these changes. -->
Redesign `PartitionInfo`, `ClusterInfo`, and `Router` trait to better
support parallel processing in Runtime, where:
* `PartitionInfo` is used to query the partition information when the
data has been partitioned.
* `ClusterInfo` is used to query the cluster information when the system
is running on a cluster.
* `Router` is used to route the data to the destination worker so that
it can be properly processed, with `PartitionInfo` and `ClusterInfo` as
input.
## Related issue number
<!-- Are there any issues opened that will be resolved by merging this
change? -->
Fixes#2753
Is your feature request related to a problem? Please describe.
Refine
Partitioner
trait to better support parallel scan.Currently,
Partitioner
define theget_worker_partitions
function to specify the partition list that the worker (i.e., threads) is going to process:However, we may have
m
workers andn
partitions, wherem>n
. Thus, each worker need to scan only part of the partition. Take this case into consideration and refine the trait.The text was updated successfully, but these errors were encountered: