-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Target] Add target host field for target specification #7462
Conversation
@comaniac @leandron @junrushao1994 Please review, thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your contribution! It overall looks pretty good. Please find my comments below :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also agree on the comments @junrushao1994 has.
The PR overall looks good to me. It does introduce a breaking change - renaming composite target’s |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In general it looks OK, I'm just a bit worried it creates some ambiguity in declaring the "target host" dependency.
Below my questions and comments.
.add_attr_option<Array<String>>("libs") \ | ||
.add_attr_option<Target>("host") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm just curious on how this plays with existing logic in build()
at build_module.py
, which has a target_host
parameter:
tvm/python/tvm/relay/build_module.py
Line 197 in b7e0cfb
def build(mod, target=None, target_host=None, params=None, mod_name="default"): |
Does this mean we now have two ways of declaring target_host
? If so, are planning to deprecate one of them?
Also, in case the user declares both (doesn't make much sense, but can be done once this PR is meged), which one takes precedece, or, should we show an error? I'm referrin to something like relay.build(target="cuda --host nvidia/jetson-nano", target_host="llvm"...)
cc @manupa-arm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is super important question, so I answered on the thread below: #7462 (comment). Thank you Leandro for bringing this up!
|
||
|
||
def test_target_host_single_string(): | ||
tgt = tvm.target.Target("cuda --host llvm") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(putting this comment here, but this is not comment about this test, just wanted to pin it somewhere with a reference)
I noticed there are some syntax changes in this. If this becomes (with this PR) our preferred way to declare the "target to target host" dependency, should we also update the tutorials to reflect that?
Hey @leandron, thank you for the review! Per the Target RFC, this PR implement the logic that folds
Yes. We can create a This design is a solution to keep perfect backward compatibility. In your example, right now, the signature of the build API is: def build(..., target, target_host...): To be compatible with the existing API without breaking them, our refactoring plan is to create a single
Since our solution keeps perfect backward compatibility, we are still deciding whether we really need deprecation at this time point. Even if we finally decided to deprecate the current API and move to a unified target in the function signature, I think we still need to keep it for 2 release cycles so that the community could really get onboard.
Yeah. We are quite overwhelmed by the upcoming TensorIR upstreaming work. After that, I should really spend some time on doc improvement, etc. |
@@ -46,7 +46,7 @@ class Target(Object): | |||
- :py:func:`tvm.target.intel_graphics` create Intel Graphics target | |||
""" | |||
|
|||
def __init__(self, tag_or_str_or_dict): | |||
def __init__(self, tag_or_str_or_dict, host_tag_or_str_or_dict=None): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for the API, I would almost expect Target(target_host, sub_target_0, sub_target_1, ...)
what would be the future path towards supporting multiple sub-targets? would there be a target_host object which contains all the sub-targets?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In terms of sub-targets, I think it will be better to converge to composite targets instead, where we can specify a list of targets as --devices
: https://github.com/apache/tvm/blob/main/src/target/target_kind.cc#L311-L313.
We can help create a short cut to construct the composite kind, like adding a static function
@staticmethod
def composite(target, devices):
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for the API, I would almost expect
Target(target_host, sub_target_0, sub_target_1, ...)
In terms of API, that would be very good indeed!
If internally we would convert that to represent the required composite target
, with specific dictionary representing all the internal data structures, etc., that's implementation details the we care about, but the end-user, interested in compiling a model shouldn't.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@areusch as a follow-up to this, we've recently updated tvmc
to accept something similar - in terms of user facing syntax to express "targets" for compilation - to what you present here.
Check it out on #7304 for code and https://discuss.tvm.apache.org/t/byoc-partitioning-support-on-tvmc/8901 for discussion.
@junrushao1994 I think we should aim to just have one way to declare Targets. we don't have to remove the other way here, but I think it would be good to indicate the "preferred" way to new users by marking one way deprecated. |
@Areush we can warn-once if |
I would actually go further and disagree, we should strive to eventually only have one way and push to delete everything but the preferred way. I feel like we use lots of extra arguments instead of composite objects and over all feel like we should move towards bundling commonly packaged state into single objects. |
@jroesch It is definitely good that eventually we have only one API in the core system, and to make it happen, here are the steps we planned to take:
After this step, the core system does only have one API.
|
@leandron @areusch @comaniac @jroesch All the tests have passed on this specific PR, please take another look and explicitly approve if it looks good :-) I know there is a lot to discuss about potential future API deprecation, etc, which is beyond the scope of this PR. Let's move the discussion to the Target RFC (https://discuss.tvm.apache.org/t/rfc-tvm-target-specification/6844). Thanks a lot! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM for the scope of this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, great work @zxybazh !
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, as it keeps backwards compatibility, and going forward we can try to converge into a user friendly target API, building upon what's implemented here.
@zxybazh would you mind raising API backward compatibility topic in the Target RFC (https://discuss.tvm.apache.org/t/rfc-tvm-target-specification/6844) thread, so that we can continue our discussion there and more people could see it? Thanks a lot! |
I am going to merge this PR later today if everyone thinks it is good to go :-) |
* Add target host field in Target * Add host as a config field to target * Add target host support for Python api * Add unit tests * Adjust format for cpplint * Remove unnecessary after in Python file * Remove redundancy and add param description * Fix format issue * Fix param description * Add unit test for duplicate target hosts
* Add target host field in Target * Add host as a config field to target * Add target host support for Python api * Add unit tests * Adjust format for cpplint * Remove unnecessary after in Python file * Remove redundancy and add param description * Fix format issue * Fix param description * Add unit test for duplicate target hosts
As mentioned, adding support for target specification with a host (can be two strings, two configs, or a single config with
host
field), liketarget("cuda", "llvm")
. The field is currently calledhost
, creating some conflicts with previous field nametarget_host
incomposite
type of target. While the PR is backward compatible, a lot of legacy code should be changed into the new API for better readbility and simplicity.