Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Relay][Frontend][ONNX] operator support: ConstantOfShape #4135

Closed
wants to merge 63 commits into from
Closed

[Relay][Frontend][ONNX] operator support: ConstantOfShape #4135

wants to merge 63 commits into from

Conversation

cchung100m
Copy link
Contributor

Following issue #3544 and discussion at tvm forum, I add the implementation of ConstantOfShape operator for frontend/onnx.py

cchung100m and others added 30 commits October 16, 2019 00:03
@cchung100m
[Relay][Frontend][ONNX] operator support: ConstantOfShape
5742776
@cchung100m
[Relay][Frontend][ONNX] operator support: ConstantOfShape
28aefac
@anijain2305 @zhiics
[QNN] Change default rouning to UPWARD. (apache#4131)
1c0e743
@altanh @vinx13
[Relay][Training] Add and fix gradients (apache#4126)
46fa6ee 
* add and fix gradients

* fix linter issues
@shoubhik @zhiics
Adding support for dequantizing from int32 to float32. (apache#4130)
c106910
@tqchen
Update PULL_REQUEST_TEMPLATE.md
e3fbdc8
@tqchen
[RUNTIME] Refactor object python FFI to new protocol. (apache#4128)
02c1e11 
* [RUNTIME] Refactor object python FFI to new protocol.

This is a pre-req to bring the Node system under object protocol.
Most of the code reflects the current code in the Node system.

- Use new instead of init so subclass can define their own constructors
- Allow register via name, besides type idnex
- Introduce necessary runtime C API functions
- Refactored Tensor and Datatype to directly use constructor.

* address review comments
@weberlo @tmoreau89
[Relay] Improve build error when no lowered funcs are produced (apach…
3185e4a 
…e#4132)

* Improve build error when no lowered funcs

* Switch from fatal to warning
@anijain2305 @tmoreau89
[TOPI][x86] Cascade lake support. (apache#4123)
972f019 
* [TOPI][x86] Cascade lake support.

* Jenkins test debug 1.

* Testing cascade lake alone.
@mshawcroft @tqchen
[DOCKER] Pin torchvision==0.4.1 (apache#4140)
a8a9831 
The existing sequence of pip install commands fetches and installs
torch==1.0.1.post2 then fetches an unpinned version of torchvision,
recent torchvision packages hardwire the specific torch version they
depend on, the overall effect is that we install a pinned torch
version then replace it with whatever version the torchvision package
depends on.

The most recent torchvision==0.4.1 package results in some test case
failures.

This patch pins torchvision back to 0.4.0, the most recent version
that the test suite worked.  Removing the explicit torch install
because it is implied and pinned as dependency of torchvision.

Change-Id: Ib30bf6aed79ff130ea15ef5134fefb0508790574
@lhutton1 @tqchen
[PATCH] Fix undefined __floatdihf in libtvmruntime.so on aarch64. (ap…
cf04697 
…ache#4119)

Arm architecture provides optional FP16 floating point support in two alternative formats, IEEE and an an alternative Arm format.

The ACLE (Arm C Language Extension) defined preprocessor symbol __ARM_FP16_FORMAT_IEEE can be used to distinguish between implementations providing IEEE and the Arm alternative format, but cannot, on its own, be used to determined if FP16 HW support is actually present.

Testing this preprocessor symbol can lead to undefined __floatdihf at runtime on an aarch64 target where no FP16 HW is present.

The relevant preprocessor symbol to determine whether FP16 HW support is present in the target is __ARM_FEATURE_FP16_SCALAR_ARITHMETIC, this symbol implies  __ARM_FP16_FORMAT_IEEE.

The relevant preprocessor symbols are defined by the ACLE standard, section 5.5.21 16-bit floating-point data processing operations, https://static.docs.arm.com/101028/0008/Q2-ACLE_2019Q2_release-0008.pdf
@zhiics @icemelon
[relay][vm] Separate VM runtime with executable (apache#4100)
4052de6 
* [relay][vm] Separate VM runtime with executable

* Address comments

* move ctx back to vm

* make only vm related fields and methods protected

* integrate seriliaztion/deserialization to executable

* create stream
@wweic @zhiics
[Relay][Frontend][TF] Add tensor array ops (apache#3798)
36a9677 
* [Relay][Frontend][TF] Add tensor array ops

* rename

* delete test

* Move utility function

* Refactor

* fix tensor array ops

* fix test

* fix rebase

* Fix serializer bug

* Improve tf convert name lookup to use prelude api

* Fix lint

* Fix test
@gussmith23 @tqchen
Fix typo (apache#4144)
fe418ec
@tqchen
[CI] Pin NNPack pthreadtools version (apache#4152)
e7c88a9
@anijain2305 @zhiics
[QNN][TFLite] Parsing QNN Add op. Adding MobilenetV2. (apache#4142)
fdb01cb
@kevinthesun @zhiics
Add lift_if_then_else pass (apache#3865)
687d4a8 
* Add LiftIfThenElse pass

* Add more comments

* Rename and refactor

* Add description for internal data structure

* Rename a test

* Minor change

* Address comments

* Improve update_for
@tqchen
[CI] Update cpu docker (apache#4153)
3c4b7cc
@wweic @icemelon
[Refactor] Rename Datatype to ADT (apache#4156)
32aad56 
We think it will reduce the confusion with the meaning.

https://discuss.tvm.ai/t/discuss-consider-rename-vm-datatype/4339
@icemelon @tqchen
[Runtime] Enable option to use OpenMP thread pool (apache#4089)
97ea31c
@tqchen
[REFACTOR][NODE][RUNTIME] Move Node to the new Object protocol. (apac…
7895adb 
…he#4161)

* [REFACTOR][NODE][RUNTIME] Move Node to the new Object protocol.

This PR removes the original node system, and make node as a subclass of Object.
This is a major refactor towards a better unified runtime object system.

List of changes in the refactor:

- We now hide data_ field, use Downcast explicitly to get a sub-class object.
- Removed the node system FFI in python.
- Removed the node C API, instead use PackedFunc for list and get attrs.
- Change relay::Op::set_attr_type_key(attr_key_name) to relay::Op::set_attr_type<AttrType>().
  - This change was necessary because of the new Object registration mechanism.
  - Subsequent changes to the op registrations
  - The change revealed a few previous problems that is now fixed.
- Patched up a few missing node type registration.
  - Now we will raise an error if we register object that is not registered.
- The original node.h and container.h are kept in the same location.
- Calling convention: kObjectHandle now equals the old kNodeHandle, kNodeHandle is removed.
- IRFunctor now dispatches on ObjectRef.
- Update to the new type checking API: is_type, derived_from are replaced by IsInstance.
- Removed .hash member function, instead use C++ convention hasher functors.

* Address review comments
@tqchen @zhiics
[CI] Move golang tests to the end (apache#4164)
72dfa60
@ekalda @zhiics
Add support for quantized multiply to Relay (apache#4141)
e583542 
This patch adds multiply operator for quantized tensors.
The details of the quantized multiplication are outlined
in the code.

This builds on pull request 3927 and includes the changes
Animesh mentions in the comments on that request.

Change-Id: I555715b53d0266a91d5c03dc3dfe8fc31e7ce4e1
@monkeyking @tqchen
Fix missspelling (apache#4166)
d660e51 
FIX "After connecting he usb" with "After connecting the usb"
@icemelon @zhiics
[Relay][Pass] Count MAC for BatchMatMul (apache#4157)
e0d286a 
* count MAC for BatchMatMul

* update doc
@zhiics @icemelon
[Relay][QNN] Add unit test for int8 (apache#4159)
6f9d028 
* [bugfix][codegen] fix casting bug in llvm codegen

* update example

* retrigger ci

* check llvm version
@zhiics @jroesch
[relay][vm] Reuse allocated device memory (apache#4170)
5a17707
@altanh @jroesch
add missing gradient check to gradient pass (apache#4169)
c3f02c4
@comaniac @eqy
merge extract_from_program and extract_from_multiple_progam (apache#4173
a21904a 
)
@mbaret @tqchen
[TOPI] Added support for Mali Bifrost target (apache#4047)
ecb0a7e
@Hzfengsy @Laurawly
TensorCore Support using Intrinsic (apache#4136)
324a960 
* add tensor core support

* avoid memory bank conflict

* fix thread sync & better performance

* better performance

* add schedule test for conv2d

* extend into BatchMatMul

* support config fragment shape and layout using intrinsic

* add TensorCore tutorial

* add int support and fix lint

* address comment

* add 32*16*8 TensorCore test

* fix wmma include logic
@jwfromm jwfromm mentioned this pull request Oct 24, 2019
Merged
@jwfromm
Copy link
Contributor

jwfromm commented Oct 24, 2019 •
edited
Loading

Just wanted to note that I implemented this op in parallel and have it included in PR #4197. There are few differences in our implementations and I'm open to removing my take on it if we decide this version is better.

Key differences are I use full instead of tile and I don't special case when initially parsing the nodes to extract inputs. Using full requires a static shape instead of a dynamic one so I use infer_value to convert the input. The trade-off is that my implementation is likely a little slower to convert but will be faster during inference.

tqchen and others added 5 commits October 24, 2019 13:40
@tqchen
[NODE][REFACTOR] Refactor reflection system in node. (apache#4189)
78ca6fc 
* [NODE][REFACTOR] Refactor reflection system in node.

- Removed the old Node, Node is now just an alias of runtime::Object
- Introduce ReflectionVTable, a new columnar dispatcher to support reflection
  - This allows us to remove vtable from most node objects
  - The VisitAttrs are registered via TVM_RESGITER_NODE_TYPE,
    they are no longer virtual.
- Consolidated serialization and reflection features into node.

* Explicit type qualification when calling destructor.

* Fix SPIRV, more comments
@tqchen
hotfix the ci (apache#4199)
decccd6
@anijain2305 @zhiics
[TOPI][x86] Legalize - Support int8xint8 convolution to use VNNI inst…
493c98d 
…ructions. (apache#4196)
@cchung100m
[Relay][Frontend][ONNX] Change test case
bc138a8
@MarisaKirisame @tmoreau89
[Relay] crossentropy_with_logits and its gradient (apache#4075)
1ad6a2a 
* save

* lint
@cchung100m cchung100m changed the title [WIP][Relay][Frontend][ONNX] operator support: ConstantOfShape [Relay][Frontend][ONNX] operator support: ConstantOfShape Oct 25, 2019
@cchung100m
Copy link
Contributor Author

Just wanted to note that I implemented this op in parallel and have it included in PR #4197. There are few differences in our implementations and I'm open to removing my take on it if we decide this version is better.

Key differences are I use full instead of tile and I don't special case when initially parsing the nodes to extract inputs. Using full requires a static shape instead of a dynamic one so I use infer_value to convert the input. The trade-off is that my implementation is likely a little slower to convert but will be faster during inference.

Hi @jwfromm

Thanks for the notification.
I think we might need to invite the reviewers to discuss the differences and trade-off.

zhiics and others added 12 commits October 25, 2019 15:47
@zhiics
[hotfix] missing include headers (apache#4204)
7732873
@altanh @jroesch
[Relay][Training] Add checkpoint annotation for checkpointing memory …
93d610a 
…optimization (apache#4146)

* add checkpoint annotation for checkpointing memory optimization

* add alpha-equivalence checkpoint test and fix gradient type issue

* fix build issues

* ignore checkpoint annotation when checking missing gradients

* refactor, fix checkpoint compute for tuple and add tests
@jroesch
[Relay][Params] Add APIs for storing and retrieving parameters from i…
9cc7874 
…ndividual functions. (apache#4194)

* Add support for attaching params

* Fix types

* Fix test
@soiferj @jroesch
[Relay][Frontend][ONNX] Add support for op Where (apache#4184)
07606e4 
* Add support for op Where

* Update impl version
@BenjaminTu @jroesch
[VTA][Chisel] TSIM VTA Source Refactor (apache#4163)
13b2856 
* app init push

* fix on readme

* change name, add bit serial explanantion

* rm serialLoadMM, change doc

* syntax change for readme

* add parallel test functionality

* fix readme

* add python doc

* syntax

* init commit

* fix empty line

* fix typo
@cchung100m
[Relay][Frontend][ONNX] operator support: ConstantOfShape
66b7190
@cchung100m
[Relay][Frontend][ONNX] operator support: ConstantOfShape
a803bbb
@cchung100m
change test case
44fa066
@cchung100m
[Relay][Frontend][ONNX] Change test case
641c4cd
@cchung100m
[Relay][Frontend][ONNX] operator support: ConstantOfShape
959efb9
@cchung100m
[Relay][Frontend][ONNX] Solve conflict
ae3579f
@cchung100m
[Relay][Frontend][ONNX] Solve conflict
5eea653
@cchung100m
Copy link
Contributor Author

I would like to close this PR due to the miss operation of pushing.

@cchung100m cchung100m closed this Oct 27, 2019
@cchung100m cchung100m deleted the support_operator_ConstantOfShape_in_frontend_onnx branch October 27, 2019 07:46
@cchung100m cchung100m restored the support_operator_ConstantOfShape_in_frontend_onnx branch October 27, 2019 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@soiferj soiferj soiferj requested changes

@Huyuwei Huyuwei Awaiting requested review from Huyuwei

@Laurawly Laurawly Awaiting requested review from Laurawly

@yzhliu yzhliu Awaiting requested review from yzhliu

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.