Enable stateless XNNPACK linear. #35791
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. [ghstack-poisoned]
This was referenced Apr 1, 2020
AshkanAliabadi
pushed a commit
that referenced
this pull request
Apr 1, 2020
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. ghstack-source-id: ee1679bce07882c2c9e94e65e9a0fe353c783cd4 Pull Request resolved: #35791
💊 Build failures summary and remediationsAs of commit 9ec51d7 (more details on the Dr. CI page):
XLA failureJob pytorch_xla_linux_bionic_py3_6_clang9_build is failing. Please create an issue with title prefixed by This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.Please report bugs/suggestions on the GitHub issue tracker. This comment has been revised 126 times. |
AshkanAliabadi
changed the title
|
Need to see a test plan for this. Are we sure this is a win on desktop/server mkldnn is available? |
|
@dreiss Thanks, will limit to mobile only. Test test plan would be passing the CI but the thing is that this would not get much mobile coverage after I limit it to mobile only. How do you suggest I go about testing it more thoroughly? |
|
As long as it's gated to mobile-only, maybe running a convolutional model on the speed benchmark and verifying that it's faster? |
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. [ghstack-poisoned]
AshkanAliabadi
pushed a commit
that referenced
this pull request
Apr 1, 2020
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. ghstack-source-id: 6318260db163e94b4030238b78ef08b42cf9b32e Pull Request resolved: #35791
dreiss
approved these changes
Apr 2, 2020
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. [ghstack-poisoned]
Closed
added 9 commits
April 2, 2020 15:31
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
Closed
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
added 4 commits
April 7, 2020 12:13
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
added 15 commits
April 7, 2020 21:06
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
The optimal solution to use XNNPACK is to separate operator creation from execution - also called prepacking the weights. If we have done our job properly, JIT must have caught and replaced nn.Linear on mobile with the prepacked versions. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm or at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
Optimal usage of the linear operator would require weight prepacking. If we have done our job properly, JIT must have caught and replaced linear operations on mobile with their corresponding prepacked versions, hence enabling said optimal usage. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm and at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
Optimal usage of the linear operator would require weight prepacking. If we have done our job properly, JIT must have caught and replaced linear operations on mobile with their corresponding prepacked versions, hence enabling said optimal usage. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm and at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
Optimal usage of the linear operator would require weight prepacking. If we have done our job properly, JIT must have caught and replaced linear operations on mobile with their corresponding prepacked versions, hence enabling said optimal usage. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm and at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
Optimal usage of the linear operator would require weight prepacking. If we have done our job properly, JIT must have caught and replaced linear operations on mobile with their corresponding prepacked versions, hence enabling said optimal usage. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm and at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
Optimal usage of the linear operator would require weight prepacking. If we have done our job properly, JIT must have caught and replaced linear operations on mobile with their corresponding prepacked versions, hence enabling said optimal usage. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm and at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
Optimal usage of the linear operator would require weight prepacking. If we have done our job properly, JIT must have caught and replaced linear operations on mobile with their corresponding prepacked versions, hence enabling said optimal usage. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm and at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
Optimal usage of the linear operator would require weight prepacking. If we have done our job properly, JIT must have caught and replaced linear operations on mobile with their corresponding prepacked versions, hence enabling said optimal usage. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm and at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
Optimal usage of the linear operator would require weight prepacking. If we have done our job properly, JIT must have caught and replaced linear operations on mobile with their corresponding prepacked versions, hence enabling said optimal usage. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm and at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
Optimal usage of the linear operator would require weight prepacking. If we have done our job properly, JIT must have caught and replaced linear operations on mobile with their corresponding prepacked versions, hence enabling said optimal usage. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm and at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
Optimal usage of the linear operator would require weight prepacking. If we have done our job properly, JIT must have caught and replaced linear operations on mobile with their corresponding prepacked versions, hence enabling said optimal usage. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm and at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
Optimal usage of the linear operator would require weight prepacking. If we have done our job properly, JIT must have caught and replaced linear operations on mobile with their corresponding prepacked versions, hence enabling said optimal usage. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm and at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
Optimal usage of the linear operator would require weight prepacking. If we have done our job properly, JIT must have caught and replaced linear operations on mobile with their corresponding prepacked versions, hence enabling said optimal usage. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm and at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
Optimal usage of the linear operator would require weight prepacking. If we have done our job properly, JIT must have caught and replaced linear operations on mobile with their corresponding prepacked versions, hence enabling said optimal usage. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm and at::matmul. Differential Revision: [D20821863](https://our.internmc.facebook.com/intern/diff/D20821863) [ghstack-poisoned]
|
@AshkanAliabadi merged this pull request in ba3f8d3. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
Optimal usage of the linear operator would require weight prepacking. If we have done our job properly, JIT must have caught and replaced linear operations on mobile with their corresponding prepacked versions, hence enabling said optimal usage. Still, if we somehow end up in at::native::linear for whatever reason, it is still more efficient to go through XNNPACK than the alternatives of at::addmm and at::matmul.
Differential Revision: D20821863