Windowing Support for the DaskRunner

[alxmrs]

This CL adds basic Windowing support to this runner, including a few tests for side inputs.

Take two of #23913. It looks like some of the issues in CI causing this dask change to fail were resolved on their own with changes in master.

Reviewers: @jrmccluskey

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

[cisaacstern]

A final note to self for now, which is simply that I am now seeing side input mapping probably needs to happen at the pipeline visitor level... where we have access to context like:

 /.../beam/sdks/python/apache_beam/runners/dask/dask_runner.py(150)visit_transform()
-> inputs = list(transform_node.inputs)
(Pdb) transform_node.inputs
(<PCollection[main.None] at 0x131e7ac50>,)
(Pdb) transform_node.side_inputs
[<apache_beam.pvalue._UnpickledSideInput object at 0x131e58130>]
(Pdb) transform_node
AppliedPTransform(Map(mult_by), ParDo)
(Pdb) op
ParDo(applied=AppliedPTransform(Map(mult_by), ParDo))
(Pdb) op_class
<class 'apache_beam.runners.dask.transform_evaluator.ParDo'>
(Pdb) 

Ok... to be continued soon...

[github-actions]

Reminder, please take a look at this pr: @riteshghorse @damccorm

[cisaacstern]

Status:

• Side inputs now work, with implementation based on Ray Runner
• Added additional side inputs test based on Ray Runner tests
• All side inputs tests pass except for 1, which I will take a look at next.

This finally feels like it's getting close. (There is still some upstream work on Dask that needs to go in before this, which I'll push forward as well, as mentioned above.)

[cisaacstern]

I have now gotten this as far as it can go without the above-referenced fix in dask going in, so going to shift focus back there, and will re-ping here once that goes in and this is ready for review.

[github-actions]

Reminder, please take a look at this pr: @riteshghorse @damccorm

[cisaacstern]

After a hiatus, I am working on this again now.

Today I will complete the upstream fix dask/dask#10734 in dask.

Once that goes in, I'll complete this. Thanks all for your patience.

[cisaacstern]

The necessary upstream fix in Dask was merged! 🎉

Once we get a new Dask release that includes this change, I will finish this PR!

[github-actions]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.

[github-actions]

This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.