[BYOC-DNNL] add support for more ops and fusion patterns

[crazydemo]

Enabled OP List:

• Convolution 1D-3D
• Deconvolution 2D-3D
• Pooling 2D-3D (avg / max)
• Elementwise OPs (abs, clip, exp, log, sqrt, round, logsumexp, relu, leaky_relu, tanh, sigmoid, softmax)

[tmoreau89]

CC @apeskov and @mbs-octoml

[crazydemo]

Hi, @apeskov and @mbs-octoml. Could you please take some time to review this PR? Look forward to your comments.

[mbs-octoml]

@mikepapadim can you take a look at a this one, thanks.

[mikepapadim]

@mikepapadim can you take a look at a this one, thanks.

I ll take a look!

[crazydemo]

@mikepapadim can you take a look at a this one, thanks.

I ll take a look!

Hi @mikepapadim, do you have any comments?

[crazydemo]

@tmoreau89 @mbs-octoml @mikepapadim @masahi Could you please help me review this PR?

[masahi]

I can merge this after you pass CI.

[crazydemo]

I can merge this after you pass CI.
Thank you! I will check!

[crazydemo]

Hi @masahi , I wonder which version of OneDNN is used for CI.
I have one build error, which says 'eltwise_logsigmoid' is not a member of 'dnnl::algorithm', and this algo is supported by OneDNNv1.8 or the later version.
Could you help me skip this check?

[masahi]

tvm/docker/install/ubuntu_install_dnnl.sh

Line 24 in 8131364

wget -q https://github.com/oneapi-src/oneDNN/releases/download/v1.5/dnnl_lnx_1.5.0_cpu_gomp.tgz

Probably too old, we should update that first.

[crazydemo]

I will try to update to OneDNN V2.2.

[masahi]

Note that changing the docker script won't immediately update the CI environment (so the build will fail again).You need to send that change separately first like #10179. I can take care of actually updating the ci image.

[crazydemo]

Note that changing the docker script won't immediately update the CI environment (so the build will fail again).You need to send that change separately first like #10179. I can take care of actually updating the ci image.

Do I need to submit an RFC, or just submit an independent PR?

[masahi]

No need for another rfc.

[vpirogov]

The latest oneDNN release is v2.5.2

[crazydemo]

The latest oneDNN release is v2.5.2

Yes, the latest release version is v2.5.2, but only v2.2 has dnnl_lnx_2.2.0_cpu_gomp.tgz

[crazydemo]

The PR, updating CI has been submitted #10266 @masahi

[vpirogov]

@crazydemo, right, we stopped publishing binary releases on Github after v2.2. Binaries for later versions are available on conda-forge.

[crazydemo]

@vpirogov Thank you so much! I will update the CI to the latest version.

[crazydemo]

hi @masahi , how can I restart the ci check?

[masahi]

So this is how it works.

• The new CI image was created 3 hours ago in https://hub.docker.com/r/tlcpackstaging/ci_cpu/tags that contains the new dnnl change. I need to test this image in the staging branch, which I just started at https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/ci-docker-staging/217/pipeline.
• After that CI job passes, I'll open a PR to update Jenkinsfile to point to the new cpu image.
• After that PR is merged, you can resume your work in this PR.

CI update is an involved process. So please wait until at least tomorrow.

[crazydemo]

Got it! Thank you for all your help and explanation!

[masahi]

@crazydemo it's ready now.

[masahi]

LGTM except one minor nit. Can be addressed in a later PR if more is coming.

[masahi]

@crazydemo please look at the CI issues.