[microNPU] Allow constants to be given as input to an operator #9515
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lhutton1
requested review from
anijain2305,
areusch,
comaniac,
icemelon,
jroesch,
junrushao,
jwfromm,
MarisaKirisame,
mbrookhart,
merrymercy,
slyubomirsky,
tqchen,
vinx13,
wweic,
yzhliu,
zhiics and
ZihengJiang
as code owners
NicolaLancellotti
requested changes
Nov 16, 2021
ekalda
reviewed
Nov 16, 2021
Comment on lines
+443
to
+456
| def create_relay_graph(): | ||
| inp = relay.var("input", shape=ifm_shape, dtype=dtype) | ||
| scalar = relay.const(np.ones((1, 1, 1, 1), dtype=dtype), dtype=dtype) | ||
| add = relay.qnn.op.add( | ||
| inp, | ||
| scalar, | ||
| relay.const(1.0, dtype="float32"), | ||
| relay.const(0, dtype="int32"), | ||
| relay.const(1.0, dtype="float32"), | ||
| relay.const(0, dtype="int32"), | ||
| relay.const(1.0, dtype="float32"), | ||
| relay.const(0, dtype="int32"), | ||
| ) | ||
| func = relay.Function(relay.analysis.free_vars(add), add) |
There was a problem hiding this comment.
Is there a reason we start from Relay there instead of TFLite?
There was a problem hiding this comment.
I wasn't sure if there was a way to generate similar relay from TFLite, although admittedly I didn't really check. I'll have a look into it.
There was a problem hiding this comment.
Ah, Relay is also needed due to the uint8 restriction so I'll leave this for now
| interpreted as an encoded constant.""" | ||
|
|
||
| def get_graph(): | ||
| dtype = "uint8" |
There was a problem hiding this comment.
Why does the constant need to beuint8? (just asking for enlightenment)
There was a problem hiding this comment.
For now this needs to be uint8 due to https://github.com/apache/tvm/blob/main/python/tvm/relay/backend/contrib/ethosu/tir_to_cs_translator.py#L254. Removing this opens up another can of worms that I'll address in another PR :)
There was a problem hiding this comment.
Yes, makes sense, thanks for clarifying! :)
Currently the expectation is that all constants need to be encoded, however, this is not always the case for scalar inputs. This PR ensures that constants that don't need encoding are not treated like encoded constants by the EncodeConstants pass. Change-Id: I79cf4aa10d01c4ae9ce9cdafb6f21ebb2d028126
Change-Id: I67b61a2d2f67de25c47d2ace0e3a22c59ba8ea15
lhutton1
force-pushed
the
dont-compress-input-constants
branch
from
889bd5f to
e1458c5
Compare
mehrdadh
pushed a commit
to mehrdadh/tvm
that referenced
this pull request
Dec 1, 2021
…e#9515) * [microNPU] Allow constants to be given as input to an operator Currently the expectation is that all constants need to be encoded, however, this is not always the case for scalar inputs. This PR ensures that constants that don't need encoding are not treated like encoded constants by the EncodeConstants pass. Change-Id: I79cf4aa10d01c4ae9ce9cdafb6f21ebb2d028126 * address comments Change-Id: I67b61a2d2f67de25c47d2ace0e3a22c59ba8ea15
mehrdadh
pushed a commit
to mehrdadh/tvm
that referenced
this pull request
Dec 1, 2021
…e#9515) * [microNPU] Allow constants to be given as input to an operator Currently the expectation is that all constants need to be encoded, however, this is not always the case for scalar inputs. This PR ensures that constants that don't need encoding are not treated like encoded constants by the EncodeConstants pass. Change-Id: I79cf4aa10d01c4ae9ce9cdafb6f21ebb2d028126 * address comments Change-Id: I67b61a2d2f67de25c47d2ace0e3a22c59ba8ea15
lhutton1
added a commit
to lhutton1/tvm
that referenced
this pull request
Dec 22, 2021
PR apache#9515 enabled support for scalar constants, but didn't consider the case of a scalar value where the underlying constant data does not have a shape i.e. `constant.shape == []`. See the test case for a visual differece when the scalar value is 1. Change-Id: Id7a238cb5bf999dd5a8428c097202f9fb940a5f0
lhutton1
added a commit
to lhutton1/tvm
that referenced
this pull request
Jan 4, 2022
PR apache#9515 enabled support for scalar constants, but didn't consider the case of a scalar value where the underlying constant data does not have a shape i.e. `constant.shape == []`. See the test case for a visual differece when the scalar value is 1. Change-Id: Id7a238cb5bf999dd5a8428c097202f9fb940a5f0
ylc
pushed a commit
to ylc/tvm
that referenced
this pull request
Jan 7, 2022
…e#9515) * [microNPU] Allow constants to be given as input to an operator Currently the expectation is that all constants need to be encoded, however, this is not always the case for scalar inputs. This PR ensures that constants that don't need encoding are not treated like encoded constants by the EncodeConstants pass. Change-Id: I79cf4aa10d01c4ae9ce9cdafb6f21ebb2d028126 * address comments Change-Id: I67b61a2d2f67de25c47d2ace0e3a22c59ba8ea15
yangulei
pushed a commit
to yangulei/tvm
that referenced
this pull request
Jan 11, 2022
…e#9515) * [microNPU] Allow constants to be given as input to an operator Currently the expectation is that all constants need to be encoded, however, this is not always the case for scalar inputs. This PR ensures that constants that don't need encoding are not treated like encoded constants by the EncodeConstants pass. Change-Id: I79cf4aa10d01c4ae9ce9cdafb6f21ebb2d028126 * address comments Change-Id: I67b61a2d2f67de25c47d2ace0e3a22c59ba8ea15
ylc
pushed a commit
to ylc/tvm
that referenced
this pull request
Jan 13, 2022
…e#9515) * [microNPU] Allow constants to be given as input to an operator Currently the expectation is that all constants need to be encoded, however, this is not always the case for scalar inputs. This PR ensures that constants that don't need encoding are not treated like encoded constants by the EncodeConstants pass. Change-Id: I79cf4aa10d01c4ae9ce9cdafb6f21ebb2d028126 * address comments Change-Id: I67b61a2d2f67de25c47d2ace0e3a22c59ba8ea15
lhutton1
added a commit
to lhutton1/tvm
that referenced
this pull request
Jan 17, 2022
PR apache#9515 enabled support for scalar constants, but didn't consider the case of a scalar value where the underlying constant data does not have a shape i.e. `constant.shape == []`. See the test case for a visual differece when the scalar value is 1. Change-Id: Id7a238cb5bf999dd5a8428c097202f9fb940a5f0
manupak
pushed a commit
that referenced
this pull request
Jan 18, 2022
* [microNPU] Add support for scalar values PR #9515 enabled support for scalar constants, but didn't consider the case of a scalar value where the underlying constant data does not have a shape i.e. `constant.shape == []`. See the test case for a visual differece when the scalar value is 1. Change-Id: Id7a238cb5bf999dd5a8428c097202f9fb940a5f0 * Fix failing test by removing constant Before this PR scalar constants were handled differently so this test was able to pass. Now that scalar constants are handled in the same manner as tensor constants, the test fails since unexpected tir is produced in the compilation pipeline. Since the relay used in this test case is not expected to be produced by higher levels of the compiler, removing this constant for now. Change-Id: I4ea5155778809041339e6faac05af3f72c3e3ea5 * clean up finding tensor from inputs Change-Id: Ideccf84f8c9149148ff23e2406229cf637c982a3
yuanfz98
pushed a commit
to yuanfz98/tvm
that referenced
this pull request
Jan 24, 2022
* [microNPU] Add support for scalar values PR apache#9515 enabled support for scalar constants, but didn't consider the case of a scalar value where the underlying constant data does not have a shape i.e. `constant.shape == []`. See the test case for a visual differece when the scalar value is 1. Change-Id: Id7a238cb5bf999dd5a8428c097202f9fb940a5f0 * Fix failing test by removing constant Before this PR scalar constants were handled differently so this test was able to pass. Now that scalar constants are handled in the same manner as tensor constants, the test fails since unexpected tir is produced in the compilation pipeline. Since the relay used in this test case is not expected to be produced by higher levels of the compiler, removing this constant for now. Change-Id: I4ea5155778809041339e6faac05af3f72c3e3ea5 * clean up finding tensor from inputs Change-Id: Ideccf84f8c9149148ff23e2406229cf637c982a3
crazydemo
pushed a commit
to crazydemo/tvm
that referenced
this pull request
Jan 27, 2022
* [microNPU] Add support for scalar values PR apache#9515 enabled support for scalar constants, but didn't consider the case of a scalar value where the underlying constant data does not have a shape i.e. `constant.shape == []`. See the test case for a visual differece when the scalar value is 1. Change-Id: Id7a238cb5bf999dd5a8428c097202f9fb940a5f0 * Fix failing test by removing constant Before this PR scalar constants were handled differently so this test was able to pass. Now that scalar constants are handled in the same manner as tensor constants, the test fails since unexpected tir is produced in the compilation pipeline. Since the relay used in this test case is not expected to be produced by higher levels of the compiler, removing this constant for now. Change-Id: I4ea5155778809041339e6faac05af3f72c3e3ea5 * clean up finding tensor from inputs Change-Id: Ideccf84f8c9149148ff23e2406229cf637c982a3
ylc
pushed a commit
to ylc/tvm
that referenced
this pull request
Feb 16, 2022
* [microNPU] Add support for scalar values PR apache#9515 enabled support for scalar constants, but didn't consider the case of a scalar value where the underlying constant data does not have a shape i.e. `constant.shape == []`. See the test case for a visual differece when the scalar value is 1. Change-Id: Id7a238cb5bf999dd5a8428c097202f9fb940a5f0 * Fix failing test by removing constant Before this PR scalar constants were handled differently so this test was able to pass. Now that scalar constants are handled in the same manner as tensor constants, the test fails since unexpected tir is produced in the compilation pipeline. Since the relay used in this test case is not expected to be produced by higher levels of the compiler, removing this constant for now. Change-Id: I4ea5155778809041339e6faac05af3f72c3e3ea5 * clean up finding tensor from inputs Change-Id: Ideccf84f8c9149148ff23e2406229cf637c982a3
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently the expectation is that all constants need to be encoded, however, this is not always the case for scalar inputs. This PR ensures that constants that don't need encoding are not treated like encoded constants by the EncodeConstants pass.
cc @mbaret @manupa-arm @ekalda @NicolaLancellotti @dchauhan-arm