Add support for naming tasks in @requires #3077
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This change lets a user name the tasks in the requires method like so:
Motivation and Context
When we use the requires decorator, we can save a lot of time and hassle passing parameters around. One issue we have hit is that the run method only sees a list of targets, with no context as to which earlier task generated the results.
Luigi already supports named inputs, by returning a dictionary from the requires function. This change just allows passing in kwargs to the decorator and creates a requires method using those. I added this to the inherits decorator as well,
There wasn't a sensible option in my mind if you try and pass in both, so I made it throw an error in that case.
This doesn't include an update to the documentation but I can add some docs if the approach is OK.
This doesn't handle any fancier nested versions, just a simple dictionary.
Have you tested this? If so, how?
I added some basic unit tests, but can expand these, and have put an example program below.
It's largely taken from demo code we're working on, and is I hope a relatively simple change.
Example: