Some simplifications

[raphael-proust]

Two relatively simple modifications. (Albeit one is simple but not trivial.)

The first commit implements Lwt_main.yield as a call to Lwt.pause and removes all the support code associated with the previous implementation. This is not trivial and may have subtle effects on the scheduling performed in Lwt_main.run.

The second commit simply avoids some List.rev calls when traversing lists (iter_p, iteri_p, etc.).

[aantron]

The second commit looks good. Thanks!

I want to note for future reference, that it changes the order in which join and internal collect functions visit promises. However, this order was never specified, or, in the case of join, explicitly defined as arbitrary.

The first commit should be fine, but I hesitate to merge it, because it may have some subtle implications. I suggest replacing it with a comment explaining that yield is really the same as pause, but exposed and implemented separately for historical reasons. Perhaps the comment should link to this PR and/or the earlier discussion.

[raphael-proust]

The second commit looks good. Thanks!

I want to note for future reference, that it changes the order in which join and internal collect functions visit promises. However, this order was never specified, or, in the case of join, explicitly defined as arbitrary.

I added a note in the CHANGES file, not sure if that's how you want to keep track of it. If not let me know and I can remove that part.

The first commit should be fine, but I hesitate to merge it, because it may have some subtle implications. I suggest replacing it with a comment explaining that yield is really the same as pause, but exposed and implemented separately for historical reasons. Perhaps the comment should link to this PR and/or the earlier discussion.

I'll make that change. How strongly should I encourage people to use pause rather than yield? Do we want to deprecate yield altogether or just point at pause?

It could be good to ask people to try an run their big project on the yield-is-pause variant and see if anything breaks. A bit like the opam switch thing on the compiler variants.

[raphael-proust]

I just thought of one possible way to keep a fully backwards-compatible semantics for parallel iterators: provide an alternatvie to join that captures the last exception rather than the first. This new version is very similar except for:

-        match !join_result with
-        | Fulfilled () -> join_result := new_result
-        | Rejected _ -> ()
+        join_result := new_result

I don't know if it is worth adding a new function that behaves so similarly to join. In fact, the documentation of join does say that it selects an arbitrary rejections amongst all the available one. So the intended semantics of this new function, its documentation even, would be the same as with vanilla join.

IMO it's not worth it. (IMO we should randomly select either of the join variants at compile time just to actually be arbitrary, but that's a bit mean…)

[aantron]

Thanks!

IMO it's not worth it.

[On adding a new function] I agree.

I added a note in the CHANGES file, not sure if that's how you want to keep track of it. If not let me know and I can remove that part.

I usually write/rewrite the changelog right before release, so this is fine :)

How strongly should I encourage people to use pause rather than yield? Do we want to deprecate yield altogether or just point at pause?

It would probably be too noisy to attach a deprecation annotation to it, so I think the way you have the docs recommend pause in this PR is best.