Relax IRVisitor/IRMuator #10

YuchenJin · 2021-09-23T18:20:41Z

Co-authored-by: ZihengJiang ziheng@apache.org

include/tvm/relax/ir_builder.h

…IRBuilder

altanh · 2021-09-27T23:19:01Z

hey guys I didn't get around to reviewing this since the PR was marked as a "draft", so I'll leave some comments on the code when I have time and we can address in follow ups. Could you ping me next time when it's ready for review? 🙏

YuchenJin · 2021-09-27T23:44:52Z

hey guys I didn't get around to reviewing this since the PR was marked as a "draft", so I'll leave some comments on the code when I have time and we can address in follow ups. Could you ping me next time when it's ready for review? 🙏

Hey Altan, of course! Sorry for merging it so quickly for the sake of pushing on the demo. We will definitely ping you next time and leave longer review time. I think this PR has a lot of places to be improved and discussed, please leave your comments whenever you have time, and we will address them in the following PRs. :) 🙏

* ExprVisitor/ExprMutator for relax nodes. * Update Visitor & Mutator. * Update Mutator. * DataflowMutator interface. * EwiseFMARewriter. * Update fma rewrite and add test. * Update test. * Fix dataflow block dispatching. * Construct new dataflow block with IRBuilder. * VisitBinding return void and mutate internal IRBuilder. * Simplify. * Update emit dataflow output. * Explicit memeory allocation rewrite. * LazyIRBuilder. * Update ExplicitMemMutator. * Overload IRBuilder::Emit to have 3 styles. * Update IRBuilder/IRMutator interfaces and passes. * Add MatchShape binding to IRBuilder. * Improve IRMutator interface; add Normalize and CanProveShapeEqual to IRBuilder * Update EmitMatchShape. Co-authored-by: ZihengJiang <ziheng@apache.org>

* ut except bind params * all complete

* ExprVisitor/ExprMutator for relax nodes. * Update Visitor & Mutator. * Update Mutator. * DataflowMutator interface. * EwiseFMARewriter. * Update fma rewrite and add test. * Update test. * Fix dataflow block dispatching. * Construct new dataflow block with IRBuilder. * VisitBinding return void and mutate internal IRBuilder. * Simplify. * Update emit dataflow output. * Explicit memeory allocation rewrite. * LazyIRBuilder. * Update ExplicitMemMutator. * Overload IRBuilder::Emit to have 3 styles. * Update IRBuilder/IRMutator interfaces and passes. * Add MatchShape binding to IRBuilder. * Improve IRMutator interface; add Normalize and CanProveShapeEqual to IRBuilder * Update EmitMatchShape. Co-authored-by: ZihengJiang <ziheng@apache.org>

* [Op] High level op added * Add LowerToTensorIRPass * Register several high level ops * Register gradients for ops * Unit tests for Ops and gradients [Pass] Add new SimpleAD pass (tlc-pack#3) * Add SimpleAD pass * Write unit tests Co-authored-by: SiriusNEO <1713833595@qq.com> [Relax][AD] Support tuple, modify APIs and move Ops to a new folder (tlc-pack#5) This PR adds support for tuples in the automatic differentiation process of relax. Specifically, two operators, `Tuple()` and `TupleGetItem`, are supported. It also modifies `SimpleAD` api to remove string arguments and use GlobalVar, and adds `gradient` API, a new func to func interface. Besides, we have moved the Ops created by us into a new folder `src/relax/op/training`. [Testing] Relax LSTM demo and numeric gradient test (tlc-pack#6) * lstm test changes * relax training lstm demo Rebase fix (tlc-pack#8) * fix rebase error * rebase fix: AD adapts to new parser * rebase fix: return RuntimeDepShape now Co-authored-by: ubospica <ubospica@gmail.com> [Pass] Polish SimpleAD pass (tlc-pack#9) * SimpleAD now will copy params, which fix the shared Var between functions problem * add error unitests * modify type of require_grads and error test Optimizer API (tlc-pack#7) Co-authored-by: SiriusNEO <1713833595@qq.com> [Relax][Training] Relax Trainer API (tlc-pack#10) * append call pass * add test for append call * successfully train LSTM using this trainer * refractor training structure * add tests for trainer * remove a file * Remove optimizer from this branch * sync with optimizer api * redesign API and add document * fix some problems Co-authored-by: ubospica <ubospica@gmail.com> [FIX][Relax AD] replace ones_like with ones (tlc-pack#11) * ones_like to one * fix rebase Co-authored-by: ubospica <ubospica@gmail.com> Op polish revision * fix ops * debug for ops * prepare for pr * fix pr issues * - Add collapse_sum document - Remove test_relax_ops running part - Add op legalizer test * fix qemu-hack * remove unnecessary imports from test

* ExprVisitor/ExprMutator for relax nodes. * Update Visitor & Mutator. * Update Mutator. * DataflowMutator interface. * EwiseFMARewriter. * Update fma rewrite and add test. * Update test. * Fix dataflow block dispatching. * Construct new dataflow block with IRBuilder. * VisitBinding return void and mutate internal IRBuilder. * Simplify. * Update emit dataflow output. * Explicit memeory allocation rewrite. * LazyIRBuilder. * Update ExplicitMemMutator. * Overload IRBuilder::Emit to have 3 styles. * Update IRBuilder/IRMutator interfaces and passes. * Add MatchShape binding to IRBuilder. * Improve IRMutator interface; add Normalize and CanProveShapeEqual to IRBuilder * Update EmitMatchShape. Co-authored-by: ZihengJiang <ziheng@apache.org>

ZihengJiang reviewed Sep 27, 2021

View reviewed changes

include/tvm/relax/ir_builder.h Outdated Show resolved Hide resolved

YuchenJin and others added 19 commits September 27, 2021 11:35

ExprVisitor/ExprMutator for relax nodes.

e77539a

Update Visitor & Mutator.

c0f3463

Update Mutator.

aad8ef4

DataflowMutator interface.

9afb5e8

EwiseFMARewriter.

cc13cdf

Update fma rewrite and add test.

d9dfc2b

Update test.

8c4b49e

Fix dataflow block dispatching.

37c2652

Construct new dataflow block with IRBuilder.

3dbb335

VisitBinding return void and mutate internal IRBuilder.

f2a1eaf

Simplify.

233ef17

Update emit dataflow output.

a476715

Explicit memeory allocation rewrite.

b842acc

LazyIRBuilder.

ba13fe5

Update ExplicitMemMutator.

23ff9d8

Overload IRBuilder::Emit to have 3 styles.

32b9d70

Update IRBuilder/IRMutator interfaces and passes.

851b560

Add MatchShape binding to IRBuilder.

b4b34b1

Improve IRMutator interface; add Normalize and CanProveShapeEqual to …

d0b648d

…IRBuilder

YuchenJin force-pushed the relax-irvisitor branch from ec1ab88 to c8e6472 Compare September 27, 2021 21:01

Update EmitMatchShape.

60cc239

YuchenJin force-pushed the relax-irvisitor branch from c8e6472 to 60cc239 Compare September 27, 2021 21:40

ZihengJiang approved these changes Sep 27, 2021

View reviewed changes

YuchenJin marked this pull request as ready for review September 27, 2021 22:21

YuchenJin merged commit efe95d2 into relax Sep 27, 2021

YuchenJin deleted the relax-irvisitor branch September 27, 2021 22:59

MasterJH5574 pushed a commit to MasterJH5574/tlc-relax that referenced this pull request Apr 13, 2022

Unittests and Refactor (tlc-pack#10)

1e567d9

* ut except bind params * all complete

jinhongyii added a commit to jinhongyii/relax that referenced this pull request May 7, 2022

Unittests and Refactor (tlc-pack#10)

7c14d07

* ut except bind params * all complete

MasterJH5574 pushed a commit to MasterJH5574/tlc-relax that referenced this pull request Oct 17, 2022

temporarily use i32 for e2e workloads (tlc-pack#10)

9f92718

MasterJH5574 pushed a commit to MasterJH5574/tlc-relax that referenced this pull request Nov 20, 2022

temporarily use i32 for e2e workloads (tlc-pack#10)

35cfadd

jinhongyii pushed a commit to jinhongyii/relax that referenced this pull request Dec 10, 2022

temporarily use i32 for e2e workloads (tlc-pack#10)

6775c6d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Relax IRVisitor/IRMuator #10

Relax IRVisitor/IRMuator #10

YuchenJin commented Sep 23, 2021

altanh commented Sep 27, 2021

YuchenJin commented Sep 27, 2021

Relax IRVisitor/IRMuator #10

Relax IRVisitor/IRMuator #10

Conversation

YuchenJin commented Sep 23, 2021

altanh commented Sep 27, 2021

YuchenJin commented Sep 27, 2021