Bulk Transfer for 'list' data type

[LouisJenkinsCS]

Given that list is intended to replace 'array-as-vec' operations such as push_back, I feel that it is important that it should be able to provide some of the same features as arrays. One such feature, is the ability to perform a bulk transfer to another locale.

Current Implementation of init= copy initializer

chapel/modules/standard/Lists.chpl

Lines 435 to 449 in c4c5db5

	pragma "no doc"
	inline proc _extendGeneric(collection) {
	//
	// TODO: This could avoid repeated resizes at smaller total capacities
	// if we resized once and then performed repeated moves, rather than
	// calling _append().
	//
	for item in collection {
	// See: https://github.com/chapel-lang/chapel/issues/13225
	pragma "no auto destroy"
	var cpy = item;
	_append_by_ref(cpy);
	}
	}

(Note that this is not only serial iteration over the list, which can result in acquiring the lock once per iteration)

chapel/modules/standard/Lists.chpl

Lines 754 to 773 in c4c5db5

	/*
	Iterate over the elements of this list.
	:yields: A reference to one of the elements contained in this list.
	*/
	iter these() ref {
	//
	// TODO: I'm not even sure of what the best way to WRITE a threadsafe
	// iterator is, _let alone_ whether it should even be threadsafe
	// (I mean, is there even a point when reference/iterator invalidation
	// is still a thing?).
	//
	for i in 1.._size {
	_enter();
	ref result = _getRef(i);
	_leave();
	yield result;
	}
	}

Not only would communication be very fine-grained but also inefficient (no __primitive("chpl_comm_array_get", ...)?) and slow. As is, using push_back may be slower in some ways, but it enables Chapel idioms that the replacement list cannot provide.

Example of Bulk Transfer - Simple Aggregation

// Filled with locale-specific work items
var scatterWork : [LocaleSpace] list(workType);
for work in workToBeSent {
   scatterWork[work.locale.id].append(work);
}
coforall loc in Locales do on loc {
   var ourWork = scatterWork[here.id]; // Bulk Transfer
   doSomethingWith(ourWork);
}

[LouisJenkinsCS]

@e-kayrakli since you seem to be assigned to quite a few list improvement tasks.

[mppf]

I think that this is a reasonable request. I think in order to implement it well, we'd need to get the array bulk transfer code to be more user-facing so that List and other data structure authors can use it.

[e-kayrakli]

This seems thematically related to library for low-level data movements and aggregators. I wonder if all of them can be lumped together in terms of interface/implementation to some extent.

To be clear, I still expect that interface to be used by the List author and not the List user.