-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Going deeper on fork orchestration #72
Conversation
Codecov Report
@@ Coverage Diff @@
## dev #72 +/- ##
==========================================
- Coverage 81.88% 81.83% -0.05%
==========================================
Files 37 37
Lines 861 881 +20
Branches 102 107 +5
==========================================
+ Hits 705 721 +16
- Misses 156 160 +4
Continue to review full report at Codecov.
|
lib/orchestrators/join.js
Outdated
// with the joint tasks plus the outermostTask with a new uid | ||
const lineage = [forkee].concat(newUpStreamTasks).concat(newOutermostTasks) | ||
lineageGetter(dispatch, lineage, joinages) | ||
} else { // if a fork is found within another fork |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is the number of nested forks supported "hardcoded" based on these nested if/else blocks?
This PR now also includes new pipelines for testing multiple inputs as well as new uid generation strategy in which upstream tasks are included in uid creation. Check this GSoC17 entry. |
Ok, I think it is ready to merge now, but codecov complains about code that is necessary for some of the tests I ran. Maybe you ( @thejmazz ) can find a way to simplify the code. |
Also added a concurrency pipeline example using bluebird Promise. |
As long as tests are passing I don't mind merging even if code coverage drops. However, if you look at the coverage, it is just that |
This PR intends to at the very least increase the level of forkception, i.e., the ability to had forks inside forks.
For now it just handles one fork inside another fork, but has no capacity to handle a third fork. Also and importantly, fork can proceed a fork, i.e. we can have something like this: