[Relay] Continuation Passing Style #3456
Conversation
|
@junrushao1994 @vinx13 I got it print what you guys want. |
|
turn out system f is not needed. |
|
Nice work! I will check if this is helpful for more complex cases of AD |
|
Is global var not yet supported? |
|
@vinx13 it is not supported, but there is no difficulty (only that I havent write some C++). |
MarisaKirisame
force-pushed
the
cps
branch
3 times, most recently
from
a7ec531 to
0d1ee5e
Compare
src/relay/pass/to_cps.cc
Outdated
| std::function<Expr()> next; | ||
| next = [&]() { | ||
| if (args.size() == op->args.size()) { | ||
| return LetList::Let(CallNode::make(op->op, args), k); |
There was a problem hiding this comment.
CallNode::make(op->op, args, op->attrs, op->type_args) otherwise you lose attributes
|
This is ready to review. |
|
One thing @MarisaKirisame given that we are moving towards new transform.h and Module->Module, would be great to keep everything in the new style |
|
@tqchen I am unable to make UnCPS into the new style as it only work on some of the function. We probably want to discuss design. |
|
I think we might want to have a conversation about where should we put the remaining function->function transformations into. |
* save add me find type checker problem save save lint do lint reset ti add some doc add failed test case add recursion for cps add recursion for cps fix pytest lint save fix test error lint save fix error * fix rebase * fix * fix test * lint * lint * restore rewriteannotationops * do
* save add me find type checker problem save save lint do lint reset ti add some doc add failed test case add recursion for cps add recursion for cps fix pytest lint save fix test error lint save fix error * fix rebase * fix * fix test * lint * lint * restore rewriteannotationops * do
* save add me find type checker problem save save lint do lint reset ti add some doc add failed test case add recursion for cps add recursion for cps fix pytest lint save fix test error lint save fix error * fix rebase * fix * fix test * lint * lint * restore rewriteannotationops * do
The code implement a higher order continuation passing style transform in the spirit of (http://matt.might.net/articles/cps-conversion/, the higher order transform section).
This help training by helping PE which help AD.
Suppose we have some code, of the form
in the partial evaluator, if b is dynamic c and d will be partially evaluated, and their result will be thrown away, so it cannot help e.
However, if we cps the program, it will become
with e inlined, and possible to do further simplification base on the partial evaluator result of c/d.
There are two big design choice this commit to:
We should discuss a bit.
@jroesch @tqchen @nhynes