Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Target] Add target host field for target specification #7462

Merged
merged 11 commits into from
Feb 23, 2021

Conversation

zxybazh
Copy link
Member

@zxybazh zxybazh commented Feb 17, 2021

As mentioned, adding support for target specification with a host (can be two strings, two configs, or a single config with host field), like target("cuda", "llvm"). The field is currently called host, creating some conflicts with previous field name target_host in composite type of target. While the PR is backward compatible, a lot of legacy code should be changed into the new API for better readbility and simplicity.

@zxybazh
Copy link
Member Author

zxybazh commented Feb 17, 2021

@comaniac @leandron @junrushao1994 Please review, thanks!

Copy link
Member

@junrushao junrushao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution! It overall looks pretty good. Please find my comments below :-)

python/tvm/target/target.py Outdated Show resolved Hide resolved
python/tvm/target/target.py Outdated Show resolved Hide resolved
src/target/target_kind.cc Outdated Show resolved Hide resolved
src/target/target.cc Outdated Show resolved Hide resolved
src/target/target.cc Outdated Show resolved Hide resolved
src/target/target.cc Outdated Show resolved Hide resolved
tests/python/unittest/test_target_target.py Show resolved Hide resolved
Copy link
Contributor

@comaniac comaniac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also agree on the comments @junrushao1994 has.

include/tvm/target/target.h Show resolved Hide resolved
@junrushao
Copy link
Member

The PR overall looks good to me.

It does introduce a breaking change - renaming composite target’s target_host to host for simplification. Therefore, I would love to ask for @leandron to review on the change. Thanks a lot!

Copy link
Contributor

@leandron leandron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general it looks OK, I'm just a bit worried it creates some ambiguity in declaring the "target host" dependency.

Below my questions and comments.

Comment on lines +379 to +380
.add_attr_option<Array<String>>("libs") \
.add_attr_option<Target>("host")
Copy link
Contributor

@leandron leandron Feb 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just curious on how this plays with existing logic in build() at build_module.py, which has a target_host parameter:

def build(mod, target=None, target_host=None, params=None, mod_name="default"):

Does this mean we now have two ways of declaring target_host? If so, are planning to deprecate one of them?

Also, in case the user declares both (doesn't make much sense, but can be done once this PR is meged), which one takes precedece, or, should we show an error? I'm referrin to something like relay.build(target="cuda --host nvidia/jetson-nano", target_host="llvm"...)

cc @manupa-arm

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is super important question, so I answered on the thread below: #7462 (comment). Thank you Leandro for bringing this up!

src/target/target.cc Show resolved Hide resolved


def test_target_host_single_string():
tgt = tvm.target.Target("cuda --host llvm")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(putting this comment here, but this is not comment about this test, just wanted to pin it somewhere with a reference)

I noticed there are some syntax changes in this. If this becomes (with this PR) our preferred way to declare the "target to target host" dependency, should we also update the tutorials to reflect that?

@junrushao
Copy link
Member

junrushao commented Feb 18, 2021

Hey @leandron, thank you for the review!

Per the Target RFC, this PR implement the logic that folds target_host into the target class, so that is why we added a host field in the TargetNode class.

Does this mean we now have two ways of declaring target_host?

Yes. We can create a target with a single dictionary containing the host field (the new canonical), or with two Targets combined (syntactic sugar I), or with a target string like cuda --host=llvm (syntactic sugar II).

This design is a solution to keep perfect backward compatibility. In your example, right now, the signature of the build API is:

def build(..., target, target_host...): 

To be compatible with the existing API without breaking them, our refactoring plan is to create a single target_with_host = Target(target, target_host), and proceed with the subsequent workflow with target_with_host. This way could work with all the 4 possibilities:

  1. target contains the host field, and target_host is None - we use target.host as target_with_host.host
  2. target doesn't contain the host field, and target_host is None - target_with_host.host will be None
  3. target contains the host field, and target_host is not None - we will error out
  4. target doesn't contain the host field, and target_host is not None - target_with_host.host will be target_host

If so, are planning to deprecate one of them?

Since our solution keeps perfect backward compatibility, we are still deciding whether we really need deprecation at this time point. Even if we finally decided to deprecate the current API and move to a unified target in the function signature, I think we still need to keep it for 2 release cycles so that the community could really get onboard.

I noticed there are some syntax changes in this. If this becomes (with this PR) our preferred way to declare the "target to target host" dependency, should we also update the tutorials to reflect that?

Yeah. We are quite overwhelmed by the upcoming TensorIR upstreaming work. After that, I should really spend some time on doc improvement, etc.

@@ -46,7 +46,7 @@ class Target(Object):
- :py:func:`tvm.target.intel_graphics` create Intel Graphics target
"""

def __init__(self, tag_or_str_or_dict):
def __init__(self, tag_or_str_or_dict, host_tag_or_str_or_dict=None):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for the API, I would almost expect Target(target_host, sub_target_0, sub_target_1, ...)

what would be the future path towards supporting multiple sub-targets? would there be a target_host object which contains all the sub-targets?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In terms of sub-targets, I think it will be better to converge to composite targets instead, where we can specify a list of targets as --devices: https://github.com/apache/tvm/blob/main/src/target/target_kind.cc#L311-L313.

We can help create a short cut to construct the composite kind, like adding a static function

@staticmethod
def composite(target, devices):

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for the API, I would almost expect Target(target_host, sub_target_0, sub_target_1, ...)

In terms of API, that would be very good indeed!

If internally we would convert that to represent the required composite target, with specific dictionary representing all the internal data structures, etc., that's implementation details the we care about, but the end-user, interested in compiling a model shouldn't.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@areusch as a follow-up to this, we've recently updated tvmc to accept something similar - in terms of user facing syntax to express "targets" for compilation - to what you present here.

Check it out on #7304 for code and https://discuss.tvm.apache.org/t/byoc-partitioning-support-on-tvmc/8901 for discussion.

include/tvm/target/target.h Show resolved Hide resolved
@areusch
Copy link
Contributor

areusch commented Feb 18, 2021

If so, are planning to deprecate one of them?

Since our solution keeps perfect backward compatibility, we are still deciding whether we really need deprecation at this time point. Even if we finally decided to deprecate the current API and move to a unified target in the function signature, I think we still need to keep it for 2 release cycles so that the community could really get onboard.

@junrushao1994 I think we should aim to just have one way to declare Targets. we don't have to remove the other way here, but I think it would be good to indicate the "preferred" way to new users by marking one way deprecated.

@junrushao
Copy link
Member

@Areush we can warn-once if target_host is not None, but before doing that, perhaps we need consensus from the community

@jroesch
Copy link
Member

jroesch commented Feb 19, 2021

If so, are planning to deprecate one of them?

Since our solution keeps perfect backward compatibility, we are still deciding whether we really need deprecation at this time point. Even if we finally decided to deprecate the current API and move to a unified target in the function signature, I think we still need to keep it for 2 release cycles so that the community could really get onboard.

@junrushao1994 I think we should aim to just have one way to declare Targets. we don't have to remove the other way here, but I think it would be good to indicate the "preferred" way to new users by marking one way deprecated.

I would actually go further and disagree, we should strive to eventually only have one way and push to delete everything but the preferred way. I feel like we use lots of extra arguments instead of composite objects and over all feel like we should move towards bundling commonly packaged state into single objects.

@junrushao
Copy link
Member

junrushao commented Feb 19, 2021

@jroesch It is definitely good that eventually we have only one API in the core system, and to make it happen, here are the steps we planned to take:

  1. in the next PR, we will allow only a single target in the core system, and we are able to make it happen without breaking user-facing API at all. Basically we treat the two-argument case as a syntactic sugar of the single-argument case. See my previous comment for details.

After this step, the core system does only have one API.

  1. If we decided to change user-facing API, we need to send out a deprecation RFC to the community for broader agreement.

  2. In the next release cycle, we keep that API, but prints a deprecation warning in python saying it is going to be removed in the next cycle.

  3. In the next fo next release cycle, we finally remove target_host from the signature. Because the core system has been upgraded in step 1), the effort to remove the argument is minimal

@junrushao
Copy link
Member

@leandron @areusch @comaniac @jroesch

All the tests have passed on this specific PR, please take another look and explicitly approve if it looks good :-)

I know there is a lot to discuss about potential future API deprecation, etc, which is beyond the scope of this PR. Let's move the discussion to the Target RFC (https://discuss.tvm.apache.org/t/rfc-tvm-target-specification/6844). Thanks a lot!

Copy link
Member

@junrushao junrushao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@comaniac comaniac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for the scope of this PR.

Copy link
Contributor

@areusch areusch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, great work @zxybazh !

Copy link
Contributor

@leandron leandron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, as it keeps backwards compatibility, and going forward we can try to converge into a user friendly target API, building upon what's implemented here.

@junrushao
Copy link
Member

@zxybazh would you mind raising API backward compatibility topic in the Target RFC (https://discuss.tvm.apache.org/t/rfc-tvm-target-specification/6844) thread, so that we can continue our discussion there and more people could see it? Thanks a lot!

@junrushao
Copy link
Member

I am going to merge this PR later today if everyone thinks it is good to go :-)

@junrushao junrushao merged commit 794f6c6 into apache:main Feb 23, 2021
@junrushao
Copy link
Member

@zxybazh @leandron @areusch @comaniac @jroesch Thank you guys so much for the inspiring discussion! It is merged!

Lokiiiiii pushed a commit to Lokiiiiii/tvm that referenced this pull request Mar 2, 2021
* Add target host field in Target

* Add host as a config field to target

* Add target host support for Python api

* Add unit tests

* Adjust format for cpplint

* Remove unnecessary  after  in Python file

* Remove redundancy and add param description

* Fix format issue

* Fix param description

* Add unit test for duplicate target hosts
trevor-m pushed a commit to neo-ai/tvm that referenced this pull request Mar 2, 2021
* Add target host field in Target

* Add host as a config field to target

* Add target host support for Python api

* Add unit tests

* Adjust format for cpplint

* Remove unnecessary  after  in Python file

* Remove redundancy and add param description

* Fix format issue

* Fix param description

* Add unit test for duplicate target hosts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants