Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Docker] Updating dependency on Mxnet-mkl #4753

Closed
wants to merge 3 commits into from
Closed

[Docker] Updating dependency on Mxnet-mkl #4753

wants to merge 3 commits into from

Conversation

shoubhik
Copy link
Contributor

We need dependency on mxnet mkl package to be able to write tests for end-to-end quantized networks.

@masahi
Copy link
Member

masahi commented Jan 21, 2020

the CI is not updated when you update docker. https://docs.tvm.ai/contribute/pull_request.html#ci-environment

so you might as well add pytorch and torhvision update here too.

@shoubhik
Copy link
Contributor Author

shoubhik commented Jan 21, 2020 •
edited
Loading

the CI is not updated when you update docker. https://docs.tvm.ai/contribute/pull_request.html#ci-environment

@masahi , @tqchen , @yzhliu
I read the steps to update docker image here - https://docs.tvm.ai/contribute/pull_request.html#ci-environment.
I have the following question on building a new version of docker image.
When I look at prebuilt images here which runs the front end tests, it is built on linux/amd64. Any particular reason it is on amd64? the image i build, should it be on amd64 too?

What should be the ec2-instance on which this docker should be built on?

@tqchen
Copy link
Member

tqchen commented Jan 21, 2020

I will merge this PR first, and will report back once we have updated the CI image

@tqchen
Copy link
Member

tqchen commented Jan 21, 2020

The docker can be built on any env. @masahi can you send another PR for pytorch update?

Note that we still need to confirm that all test-cases passes, as there might be problem during CI image update due to breaking dep on upstream. I will be traveling in the incoming week, but will try to keep things updated.

@masahi
Copy link
Member

masahi commented Jan 21, 2020

Sent a torch update to #4756. I confirmed locally that onnx/test_forward.py, the only place we use torch, works with the updated version.

@masahi masahi changed the title Updating dependency on Mxnet-mkl [Docker] Updating dependency on Mxnet-mkl Jan 21, 2020
@shoubhik
Copy link
Contributor Author

I have also updated the tag in Jekins file and the gpu image file.

tqchen
tqchen requested changes Jan 21, 2020
View reviewed changes
docker/Dockerfile.ci_gpu Outdated
@@ -16,7 +16,7 @@
# under the License.

# CI docker GPU env
# tag: v0.56
# tag: v0.57
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we want to avoid updating the tag before build and test the correctness. I will send a separate PR to do so

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it, i'll revert the changes.

@tqchen
Copy link
Member

tqchen commented Jan 21, 2020 •
edited
Loading

NOTE: when building the docker image,

I get ERROR: mxnet_mkl-1.5.1-py2.py3-none-manylinux1_x86_64.whl is not a supported wheel on this platform.
This is an interesting error that might has things to do with wheel not recognized in the current docker setup. We will need to fix this and make sure the docker builds properly before proceed

NOTE: the machine has an AMD CPU. It would be great to check whether mkl works on this setting as our CI machines has AMD CPUs. If the quantization tests has to depends on the availability of Intel CPU, perhaps it is a better idea to find a device agnostic way for validation.

@tqchen
Copy link
Member

tqchen commented Jan 21, 2020

Update: seems was due to latest version of pip, will investigate more

@tqchen
Copy link
Member

tqchen commented Jan 22, 2020

Update: blocked by #4764

@masahi masahi mentioned this pull request Jan 25, 2020
Merged
@tqchen tqchen closed this Feb 3, 2020
@shoubhik shoubhik mentioned this pull request Feb 3, 2020
Merged
shoubhik added a commit to anijain2305/tvm that referenced this pull request Feb 5, 2020
@shoubhik
Reverting the changes made for mxnet-mkldnn test cases. Because of ap…
6fb6a9d 
…ache#4753, mxnet could not be updated to mxnet-mkldnn.
masahi pushed a commit that referenced this pull request Feb 5, 2020
@shoubhik @anijain2305
Mxnet parser for Qnn dialect (#4714)
7d263c3 
* - Additional util methods needed for mxnet frontend for qnn dialect.

* - Fixing call to quantize.

* [QNN] MxNet-MKLDNN parser support for QNN

* [QNN] Relax conv check.

* - Merge from origin

* [QNN] Channel wise changes

* [QNN] Dense changes

* Dense fix for QNN ops.

* - Removed non-mkl code from utils.

- Small refactoring

- Remove "with_sum" from conv

- Simplified code

* - Fixing ring buffer name.

* - Fixing pylint issues.

* - Fixing lint
- Removing redundant commented code.

* - Adding test cases
- Removing unused methods.

* [WIP] end to end test case for mxnet qnn parser

* Changes to parse large CV models.

* Pylint issues.

* Fix Conv2D with sum and quantized pooling.

* Reverting the changes made for mxnet-mkldnn test cases. Because of #4753, mxnet could not be updated to mxnet-mkldnn.

Co-authored-by: Animesh Jain <anijain@umich.edu>
alexwong pushed a commit to alexwong/tvm that referenced this pull request Feb 26, 2020
@shoubhik @anijain2305 @alexwong
Mxnet parser for Qnn dialect (apache#4714)
6c4f3cd 
* - Additional util methods needed for mxnet frontend for qnn dialect.

* - Fixing call to quantize.

* [QNN] MxNet-MKLDNN parser support for QNN

* [QNN] Relax conv check.

* - Merge from origin

* [QNN] Channel wise changes

* [QNN] Dense changes

* Dense fix for QNN ops.

* - Removed non-mkl code from utils.

- Small refactoring

- Remove "with_sum" from conv

- Simplified code

* - Fixing ring buffer name.

* - Fixing pylint issues.

* - Fixing lint
- Removing redundant commented code.

* - Adding test cases
- Removing unused methods.

* [WIP] end to end test case for mxnet qnn parser

* Changes to parse large CV models.

* Pylint issues.

* Fix Conv2D with sum and quantized pooling.

* Reverting the changes made for mxnet-mkldnn test cases. Because of apache#4753, mxnet could not be updated to mxnet-mkldnn.

Co-authored-by: Animesh Jain <anijain@umich.edu>
alexwong pushed a commit to alexwong/tvm that referenced this pull request Feb 28, 2020
@shoubhik @anijain2305 @alexwong
Mxnet parser for Qnn dialect (apache#4714)
ac53f98 
* - Additional util methods needed for mxnet frontend for qnn dialect.

* - Fixing call to quantize.

* [QNN] MxNet-MKLDNN parser support for QNN

* [QNN] Relax conv check.

* - Merge from origin

* [QNN] Channel wise changes

* [QNN] Dense changes

* Dense fix for QNN ops.

* - Removed non-mkl code from utils.

- Small refactoring

- Remove "with_sum" from conv

- Simplified code

* - Fixing ring buffer name.

* - Fixing pylint issues.

* - Fixing lint
- Removing redundant commented code.

* - Adding test cases
- Removing unused methods.

* [WIP] end to end test case for mxnet qnn parser

* Changes to parse large CV models.

* Pylint issues.

* Fix Conv2D with sum and quantized pooling.

* Reverting the changes made for mxnet-mkldnn test cases. Because of apache#4753, mxnet could not be updated to mxnet-mkldnn.

Co-authored-by: Animesh Jain <anijain@umich.edu>
zhiics pushed a commit to neo-ai/tvm that referenced this pull request Mar 2, 2020
@shoubhik @anijain2305 @zhiics
Mxnet parser for Qnn dialect (apache#4714)
fda1f4b 
* - Additional util methods needed for mxnet frontend for qnn dialect.

* - Fixing call to quantize.

* [QNN] MxNet-MKLDNN parser support for QNN

* [QNN] Relax conv check.

* - Merge from origin

* [QNN] Channel wise changes

* [QNN] Dense changes

* Dense fix for QNN ops.

* - Removed non-mkl code from utils.

- Small refactoring

- Remove "with_sum" from conv

- Simplified code

* - Fixing ring buffer name.

* - Fixing pylint issues.

* - Fixing lint
- Removing redundant commented code.

* - Adding test cases
- Removing unused methods.

* [WIP] end to end test case for mxnet qnn parser

* Changes to parse large CV models.

* Pylint issues.

* Fix Conv2D with sum and quantized pooling.

* Reverting the changes made for mxnet-mkldnn test cases. Because of apache#4753, mxnet could not be updated to mxnet-mkldnn.

Co-authored-by: Animesh Jain <anijain@umich.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tqchen tqchen tqchen requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@shoubhik @masahi @tqchen