Skip to content

Commit 435d3f9

Browse files
committed
Revert "[ML] Upgrade to Pytorch 2.1.2 and zlib 1.2.13 (elastic#2588)"
This reverts commit 089b4d3.
1 parent 089b4d3 commit 435d3f9

File tree

24 files changed

+101
-57
lines changed

24 files changed

+101
-57
lines changed

Diff for: .buildkite/pipelines/build_linux.json.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@
3636
"cpu": "6",
3737
"ephemeralStorage": "20G",
3838
"memory": "64G",
39-
"image": "docker.elastic.co/ml-dev/ml-linux-build:28"
39+
"image": "docker.elastic.co/ml-dev/ml-linux-build:27"
4040
},
4141
"aarch64": {
4242
"provider": "aws",
@@ -100,7 +100,7 @@ def main(args):
100100
"cpu": "6",
101101
"ephemeralStorage": "20G",
102102
"memory": "64G",
103-
"image": "docker.elastic.co/ml-dev/ml-linux-aarch64-cross-build:12"
103+
"image": "docker.elastic.co/ml-dev/ml-linux-aarch64-cross-build:11"
104104
},
105105
"commands": [
106106
".buildkite/scripts/steps/build_and_test.sh"

Diff for: .buildkite/pipelines/build_macos.json.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ def main(args):
8282
"cpu": "6",
8383
"ephemeralStorage": "20G",
8484
"memory": "64G",
85-
"image": "docker.elastic.co/ml-dev/ml-macosx-build:18"
85+
"image": "docker.elastic.co/ml-dev/ml-macosx-build:17"
8686
},
8787
"commands": [
8888
f'if [[ "{args.action}" == "debug" ]]; then export ML_DEBUG=1; fi',

Diff for: 3rd_party/licenses/pytorch-INFO.csv

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
name,version,revision,url,license,copyright,sourceURL
2-
PyTorch,2.1.2,a8e7c98cb95ff97bb30a728c6b2a1ce6bff946eb,https://pytorch.org,BSD-3-Clause,,
2+
PyTorch,1.11.0,bc2c6edaf163b1a1330e37a6e34caf8c553e4755,https://pytorch.org,BSD-3-Clause,,

Diff for: 3rd_party/licenses/zlib-INFO.csv

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
name,version,revision,url,license,copyright,sourceURL
2-
zlib,1.2.13,04f42ceca40f73e2978b50e93806c2a18c1281fc,http://zlib.net,Zlib,Copyright (C) 1995-2022 Jean-loup Gailly and Mark Adler,
2+
zlib,1.2.12,21767c654d31d2dccdde4330529775c6c5fd5389,http://zlib.net,Zlib,Copyright (C) 1995-2022 Jean-loup Gailly and Mark Adler,

Diff for: build-setup/linux.md

+41-3
Original file line numberDiff line numberDiff line change
@@ -319,7 +319,7 @@ Then copy the shared libraries to the system directory:
319319
sudo cp /opt/intel/mkl/lib/intel64/libmkl*.so /usr/local/gcc103/lib
320320
```
321321

322-
### PyTorch 2.1.2
322+
### PyTorch 1.13.1
323323

324324
(This step requires a reasonable amount of memory. It failed on a machine with 8GB of RAM. It succeeded on a 16GB machine. You can specify the number of parallel jobs using environment variable MAX_JOBS. Lower number of jobs will reduce memory usage.)
325325

@@ -338,7 +338,7 @@ sudo /usr/local/gcc103/bin/python3.10 -m pip install install numpy pyyaml setupt
338338
Then obtain the PyTorch code:
339339

340340
```
341-
git clone --depth=1 --branch=v2.1.2 git@github.com:pytorch/pytorch.git
341+
git clone --depth=1 --branch=v1.13.1 git@github.com:pytorch/pytorch.git
342342
cd pytorch
343343
git submodule sync
344344
git submodule update --init --recursive
@@ -366,7 +366,7 @@ export USE_MKLDNN=ON
366366
export USE_QNNPACK=OFF
367367
export USE_PYTORCH_QNNPACK=OFF
368368
[ $(uname -m) = x86_64 ] && export USE_XNNPACK=OFF
369-
export PYTORCH_BUILD_VERSION=2.1.2
369+
export PYTORCH_BUILD_VERSION=1.13.1
370370
export PYTORCH_BUILD_NUMBER=1
371371
/usr/local/gcc103/bin/python3.10 setup.py install
372372
```
@@ -380,6 +380,44 @@ sudo cp torch/lib/libtorch_cpu.so /usr/local/gcc103/lib
380380
sudo cp torch/lib/libc10.so /usr/local/gcc103/lib
381381
```
382382

383+
### Intel Extension for PyTorch
384+
If you are building on x86_64, you can optionally build the Intel Extension for PyTorch (IPEX). This extension provides additional optimizations for Intel CPUs. It is not available for aarch64. IPEX library is required to run PyTorch models quantized using the IPEX backend.
385+
386+
Begin by cloning the IPEX repository:
387+
388+
```bash
389+
git clone https://github.com/intel/intel-extension-for-pytorch.git
390+
cd intel-extension-for-pytorch
391+
git fetch --all
392+
git checkout v1.13.100+cpu
393+
git submodule sync
394+
git submodule update --init --recursive
395+
git config --global --add safe.directory `pwd`
396+
```
397+
398+
IPEX expects that PyTorch build directory contains a file `build-version`, which contains the PyTorch version string. This file is not created by the PyTorch build process, so we need to create it manually:
399+
```bash
400+
echo "1.13.1+cpu\n" > ${PYTORCH_SRC_DIR}/torch/build-version
401+
```
402+
403+
This assumes that you have cloned PyTorch in the directory `${PYTORCH_SRC_DIR}` in the step above. Make sure that this path is correct.
404+
405+
Building IPEX requires a lot of memory. To reduce this requirement, we can patch the IPEX build system to lower the number of parallel processes. We use `sed` to replace the call to `multiprocessing.cpu_count()` with a constant `1` in the file `setup.py`:
406+
```bash
407+
sed -i 's/multiprocessing.cpu_count()/1/g' setup.py
408+
```
409+
410+
Finally, we can build IPEX:
411+
```bash
412+
export CC=/usr/local/gcc103/bin/gcc
413+
export TORCH_VERSION="v1.13.1"
414+
export TORCH_IPEX_VERSION="1.13.100+cpu"
415+
/usr/local/gcc103/bin/python3.10 -m pip install -r requirements.txt
416+
/usr/local/gcc103/bin/python3.10 setup.py clean
417+
/usr/local/gcc103/bin/python3.10 setup.py build_clib ${PYTORCH_SRC_DIR}/torch
418+
cp build/Release/packages/intel_extension_for_pytorch/lib/libintel-ext-pt-cpu.so /usr/local/gcc103/lib
419+
```
420+
383421
### valgrind
384422

385423
`valgrind` is not required to build the code. However, since we build `gcc` ourselves, if you want

Diff for: build-setup/macos.md

+3-15
Original file line numberDiff line numberDiff line change
@@ -130,7 +130,7 @@ Download the graphical installer for Python 3.10.9 from <https://www.python.org/
130130

131131
Install using all the default options. When the installer completes a Finder window pops up. Double click the `Install Certificates.command` file in this folder to install the SSL certificates Python needs.
132132

133-
### PyTorch 2.1.2
133+
### PyTorch 1.13.1
134134

135135
PyTorch requires that certain Python modules are installed. To install them:
136136

@@ -141,7 +141,7 @@ sudo /Library/Frameworks/Python.framework/Versions/3.10/bin/pip3.10 install inst
141141
Then obtain the PyTorch code:
142142

143143
```
144-
git clone --depth=1 --branch=v2.1.2 https://github.com/pytorch/pytorch.git
144+
git clone --depth=1 --branch=v1.13.1 https://github.com/pytorch/pytorch.git
145145
cd pytorch
146146
git submodule sync
147147
git submodule update --init --recursive
@@ -158,18 +158,6 @@ external processes.
158158
Edit `tools/setup_helpers/cmake.py` and add `"DNNL_TARGET_ARCH"` to the list
159159
of environment variables that get passed through to CMake (around line 215).
160160

161-
For compilation on Apple silicon Macs edit `third_party/ideep/include/ideep/utils.hpp`
162-
and add:
163-
164-
```c++
165-
inline void to_bytes(bytestring& bytes, const unsigned long arg) {
166-
auto as_cstring = reinterpret_cast<const char*>(&arg);
167-
bytes.append(as_cstring, sizeof(unsigned long));
168-
}
169-
```
170-
171-
at around line 189. This is necessary to resolve a template specialization issue.
172-
173161
Build as follows:
174162

175163
```
@@ -183,7 +171,7 @@ export USE_MKLDNN=ON
183171
export USE_QNNPACK=OFF
184172
export USE_PYTORCH_QNNPACK=OFF
185173
[ $(uname -m) = x86_64 ] && export USE_XNNPACK=OFF
186-
export PYTORCH_BUILD_VERSION=2.1.2
174+
export PYTORCH_BUILD_VERSION=1.13.1
187175
export PYTORCH_BUILD_NUMBER=1
188176
/Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10 setup.py install
189177
```

Diff for: build-setup/windows.md

+10-10
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ On the "Workloads" page that is displayed after a short while, check "Desktop de
5555

5656
### Git for Windows
5757

58-
Download `Git-2.42.0.2-64-bit.exe` from <https://github.com/git-for-windows/git/releases/download/v2.42.0.windows.2/Git-2.42.0.2-64-bit.exe>.
58+
Download `Git-2.16.3-64-bit.exe` from <https://github.com/git-for-windows/git/releases/download/v2.16.3.windows.1>.
5959

6060
Install it using mainly the default options suggested by the installer, but on the feature selection dialog also check "On the desktop" in the "Additional icons" section.
6161

@@ -65,25 +65,25 @@ As well as providing a Git implementation, Git for Windows comes with Windows po
6565

6666
### CMake
6767

68-
CMake version 3.19.2 is the minimum required to build ml-cpp. Download the MSI installer for version 3.23.3 from <https://github.com/Kitware/CMake/releases/download/v3.23.3/cmake-3.23.3-windows-x86_64.msi> (or get a more recent version).
68+
CMake version 3.19.2 is the minimum required to build ml-cpp. Download the MSI installer for version 3.23.2 from <https://github.com/Kitware/CMake/releases/download/v3.23.2/cmake-3.23.2-win64-x64.msi> (or get a more recent version).
6969

7070
Install it mainly using the default options _except_ on the "Install Options" dialog check "Add CMake to the system PATH for all users".
7171

7272
### zlib
7373

7474
Whilst it is possible to download a pre-built version of `zlib1.dll`, for consistency we want one that links against the Visual Studio 2019 C runtime library. Therefore it is necessary to build zlib from source.
7575

76-
Download the source code from <http://zlib.net/> - the file is called `zlib1213.zip`. Unzip this file under `C:\tools`, so that you end up with a directory called `C:\tools\zlib-1.2.13`.
76+
Download the source code from <http://zlib.net/> - the file is called `zlib1212.zip`. Unzip this file under `C:\tools`, so that you end up with a directory called `C:\tools\zlib-1.2.12`.
7777

7878
To build, start a command prompt using Start Menu -&gt; Apps -&gt; Visual Studio 2019 -&gt; x64 Native Tools Command Prompt for VS 2019, then in it type:
7979

8080
```
81-
cd \tools\zlib-1.2.13
81+
cd \tools\zlib-1.2.12
8282
nmake -f win32/Makefile.msc LOC="-D_WIN32_WINNT=0x0601"
8383
nmake -f win32/Makefile.msc test
8484
```
8585

86-
All the build output will end up in the top level `C:\tools\zlib-1.2.13` directory. Once the build is complete, copy `zlib1.dll` and `minigzip.exe` to `C:\usr\local\bin`. Copy `zlib.lib` and `zdll.lib` to `C:\usr\local\lib`. And copy `zlib.h` and `zconf.h` to `C:\usr\local\include`.
86+
All the build output will end up in the top level `C:\tools\zlib-1.2.12` directory. Once the build is complete, copy `zlib1.dll` and `minigzip.exe` to `C:\usr\local\bin`. Copy `zlib.lib` and `zdll.lib` to `C:\usr\local\lib`. And copy `zlib.h` and `zconf.h` to `C:\usr\local\include`.
8787

8888
### libxml2
8989

@@ -147,8 +147,8 @@ Start a command prompt using Start Menu -&gt; Apps -&gt; Visual Studio 2019 -&gt
147147
```
148148
cd \tools\boost_1_83_0
149149
bootstrap.bat
150-
b2 -j6 --layout=versioned --disable-icu --toolset=msvc-14.2 cxxflags="-std:c++17" linkflags="-std:c++17" --build-type=complete -sZLIB_INCLUDE="C:\tools\zlib-1.2.13" -sZLIB_LIBPATH="C:\tools\zlib-1.2.13" -sZLIB_NAME=zdll --without-context --without-coroutine --without-graph_parallel --without-mpi --without-python architecture=x86 address-model=64 optimization=speed inlining=full define=BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS define=BOOST_LOG_WITHOUT_DEBUG_OUTPUT define=BOOST_LOG_WITHOUT_EVENT_LOG define=BOOST_LOG_WITHOUT_SYSLOG define=BOOST_LOG_WITHOUT_IPC define=_WIN32_WINNT=0x0601
151-
b2 install --prefix=C:\usr\local --layout=versioned --disable-icu --toolset=msvc-14.2 cxxflags="-std:c++17" linkflags="-std:c++17" --build-type=complete -sZLIB_INCLUDE="C:\tools\zlib-1.2.13" -sZLIB_LIBPATH="C:\tools\zlib-1.2.13" -sZLIB_NAME=zdll --without-context --without-coroutine --without-graph_parallel --without-mpi --without-python architecture=x86 address-model=64 optimization=speed inlining=full define=BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS define=BOOST_LOG_WITHOUT_DEBUG_OUTPUT define=BOOST_LOG_WITHOUT_EVENT_LOG define=BOOST_LOG_WITHOUT_SYSLOG define=BOOST_LOG_WITHOUT_IPC define=_WIN32_WINNT=0x0601
150+
b2 -j6 --layout=versioned --disable-icu --toolset=msvc-14.2 cxxflags="-std:c++17" linkflags="-std:c++17" --build-type=complete -sZLIB_INCLUDE="C:\tools\zlib-1.2.12" -sZLIB_LIBPATH="C:\tools\zlib-1.2.12" -sZLIB_NAME=zdll --without-context --without-coroutine --without-graph_parallel --without-mpi --without-python architecture=x86 address-model=64 optimization=speed inlining=full define=BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS define=BOOST_LOG_WITHOUT_DEBUG_OUTPUT define=BOOST_LOG_WITHOUT_EVENT_LOG define=BOOST_LOG_WITHOUT_SYSLOG define=BOOST_LOG_WITHOUT_IPC define=_WIN32_WINNT=0x0601
151+
b2 install --prefix=C:\usr\local --layout=versioned --disable-icu --toolset=msvc-14.2 cxxflags="-std:c++17" linkflags="-std:c++17" --build-type=complete -sZLIB_INCLUDE="C:\tools\zlib-1.2.12" -sZLIB_LIBPATH="C:\tools\zlib-1.2.12" -sZLIB_NAME=zdll --without-context --without-coroutine --without-graph_parallel --without-mpi --without-python architecture=x86 address-model=64 optimization=speed inlining=full define=BOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS define=BOOST_LOG_WITHOUT_DEBUG_OUTPUT define=BOOST_LOG_WITHOUT_EVENT_LOG define=BOOST_LOG_WITHOUT_SYSLOG define=BOOST_LOG_WITHOUT_IPC define=_WIN32_WINNT=0x0601
152152
```
153153

154154
The Boost headers and appropriate libraries should end up in `C:\usr\local\include` and `C:\usr\local\lib` respectively.
@@ -193,7 +193,7 @@ On the "Advanced Options" screen, check "Install for all users" and "Add Python
193193

194194
For the time being, do not take advantage of the option on the final installer screen to reconfigure the machine to allow paths longer than 260 characters. We still support Windows versions that do not have this option.
195195

196-
### PyTorch 2.1.2
196+
### PyTorch 1.13.1
197197

198198
(This step requires a lot of memory. It failed on a machine with 12GB of RAM. It just about fitted on a 20GB machine. 32GB RAM is recommended.)
199199

@@ -209,7 +209,7 @@ Next, in a Git bash shell run:
209209

210210
```
211211
cd /c/tools
212-
git clone --depth=1 --branch=v2.1.2 https://github.com/pytorch/pytorch.git
212+
git clone --depth=1 --branch=v1.13.1 git@github.com:pytorch/pytorch.git
213213
cd pytorch
214214
git submodule sync
215215
git submodule update --init --recursive
@@ -265,7 +265,7 @@ set USE_QNNPACK=OFF
265265
set USE_PYTORCH_QNNPACK=OFF
266266
set USE_XNNPACK=OFF
267267
set MSVC_Z7_OVERRIDE=OFF
268-
set PYTORCH_BUILD_VERSION=2.1.2
268+
set PYTORCH_BUILD_VERSION=1.13.1
269269
set PYTORCH_BUILD_NUMBER=1
270270
python setup.py install
271271
```

Diff for: dev-tools/docker/build_linux_aarch64_cross_build_image.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
HOST=docker.elastic.co
2323
ACCOUNT=ml-dev
2424
REPOSITORY=ml-linux-aarch64-cross-build
25-
VERSION=12
25+
VERSION=11
2626

2727
set -e
2828

Diff for: dev-tools/docker/build_linux_aarch64_native_build_image.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ sleep 5
3434
HOST=docker.elastic.co
3535
ACCOUNT=ml-dev
3636
REPOSITORY=ml-linux-aarch64-native-build
37-
VERSION=12
37+
VERSION=11
3838

3939
set -e
4040

Diff for: dev-tools/docker/build_linux_build_image.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ sleep 5
3434
HOST=docker.elastic.co
3535
ACCOUNT=ml-dev
3636
REPOSITORY=ml-linux-build
37-
VERSION=28
37+
VERSION=27
3838

3939
set -e
4040

Diff for: dev-tools/docker/build_macosx_build_image.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
HOST=docker.elastic.co
2323
ACCOUNT=ml-dev
2424
REPOSITORY=ml-macosx-build
25-
VERSION=18
25+
VERSION=17
2626

2727
set -e
2828

Diff for: dev-tools/docker/linux_aarch64_cross_builder/Dockerfile

+1-1
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
#
1111

1212
# Increment the version here when a new tools/3rd party components image is built
13-
FROM docker.elastic.co/ml-dev/ml-linux-aarch64-cross-build:12
13+
FROM docker.elastic.co/ml-dev/ml-linux-aarch64-cross-build:11
1414

1515
MAINTAINER David Roberts <dave.roberts@elastic.co>
1616

Diff for: dev-tools/docker/linux_aarch64_cross_image/Dockerfile

+1-1
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ RUN \
2424
RUN \
2525
mkdir -p /usr/local/sysroot-aarch64-linux-gnu/usr && \
2626
cd /usr/local/sysroot-aarch64-linux-gnu/usr && \
27-
wget --quiet -O - https://s3-eu-west-1.amazonaws.com/prelert-artifacts/dependencies/usr-aarch64-linux-gnu-12.tar.bz2 | tar jxf - && \
27+
wget --quiet -O - https://s3-eu-west-1.amazonaws.com/prelert-artifacts/dependencies/usr-aarch64-linux-gnu-11.tar.bz2 | tar jxf - && \
2828
cd .. && \
2929
ln -s usr/lib lib && \
3030
ln -s usr/lib64 lib64

Diff for: dev-tools/docker/linux_aarch64_native_builder/Dockerfile

+1-1
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
#
1111

1212
# Increment the version here when a new tools/3rd party components image is built
13-
FROM docker.elastic.co/ml-dev/ml-linux-aarch64-native-build:12
13+
FROM docker.elastic.co/ml-dev/ml-linux-aarch64-native-build:11
1414

1515
MAINTAINER David Roberts <dave.roberts@elastic.co>
1616

Diff for: dev-tools/docker/linux_aarch64_native_image/Dockerfile

+2-2
Original file line numberDiff line numberDiff line change
@@ -138,7 +138,7 @@ RUN \
138138
# If the PyTorch branch is changed also update PYTORCH_BUILD_VERSION
139139
RUN \
140140
cd ${build_dir} && \
141-
git -c advice.detachedHead=false clone --depth=1 --branch=v2.1.2 https://github.com/pytorch/pytorch.git && \
141+
git -c advice.detachedHead=false clone --depth=1 --branch=v1.13.1 https://github.com/pytorch/pytorch.git && \
142142
cd pytorch && \
143143
git submodule sync && \
144144
git submodule update --init --recursive && \
@@ -151,7 +151,7 @@ RUN \
151151
export USE_MKLDNN=ON && \
152152
export USE_QNNPACK=OFF && \
153153
export USE_PYTORCH_QNNPACK=OFF && \
154-
export PYTORCH_BUILD_VERSION=2.1.2 && \
154+
export PYTORCH_BUILD_VERSION=1.13.1 && \
155155
export PYTORCH_BUILD_NUMBER=1 && \
156156
/usr/local/bin/python3.10 setup.py install && \
157157
mkdir /usr/local/gcc103/include/pytorch && \

Diff for: dev-tools/docker/linux_aarch64_native_tester/Dockerfile

+1-1
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
#
1111

1212
# Increment the version here when a new tools/3rd party components image is built
13-
FROM docker.elastic.co/ml-dev/ml-linux-aarch64-native-build:12
13+
FROM docker.elastic.co/ml-dev/ml-linux-aarch64-native-build:11
1414

1515
MAINTAINER David Roberts <dave.roberts@elastic.co>
1616

Diff for: dev-tools/docker/linux_builder/Dockerfile

+1-1
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
#
1111

1212
# Increment the version here when a new tools/3rd party components image is built
13-
FROM docker.elastic.co/ml-dev/ml-linux-build:28
13+
FROM docker.elastic.co/ml-dev/ml-linux-build:27
1414

1515
MAINTAINER David Roberts <dave.roberts@elastic.co>
1616

Diff for: dev-tools/docker/linux_image/Dockerfile

+25-3
Original file line numberDiff line numberDiff line change
@@ -145,11 +145,12 @@ RUN \
145145
# If the PyTorch branch is changed also update PYTORCH_BUILD_VERSION
146146
RUN \
147147
cd ${build_dir} && \
148-
git -c advice.detachedHead=false clone --depth=1 --branch=v2.1.2 https://github.com/pytorch/pytorch.git && \
148+
git -c advice.detachedHead=false clone --depth=1 --branch=v1.13.1 https://github.com/pytorch/pytorch.git && \
149149
cd pytorch && \
150150
git submodule sync && \
151151
git submodule update --init --recursive && \
152152
sed -i -e 's/system(/strlen(/' torch/csrc/jit/codegen/fuser/cpu/fused_kernel.cpp && \
153+
cd ${build_dir}/pytorch && \
153154
export BLAS=MKL && \
154155
export BUILD_TEST=OFF && \
155156
export BUILD_CAFFE2=OFF && \
@@ -159,16 +160,37 @@ RUN \
159160
export USE_QNNPACK=OFF && \
160161
export USE_PYTORCH_QNNPACK=OFF && \
161162
export USE_XNNPACK=OFF && \
162-
export PYTORCH_BUILD_VERSION=2.1.2 && \
163+
export PYTORCH_BUILD_VERSION=1.13.1 && \
163164
export PYTORCH_BUILD_NUMBER=1 && \
164165
export MAX_JOBS=10 && \
165166
/usr/local/bin/python3.10 setup.py install && \
167+
cd ${build_dir} && \
168+
git clone https://github.com/intel/intel-extension-for-pytorch.git && \
169+
cd intel-extension-for-pytorch && \
170+
git fetch --all && \
171+
git checkout v1.13.100+cpu && \
172+
git submodule sync && \
173+
git submodule update --init --recursive && \
174+
git config --global --add safe.directory `pwd` && \
175+
/usr/local/bin/python3.10 -m pip install -r requirements.txt && \
176+
/usr/local/bin/python3.10 setup.py clean && \
177+
echo "1.13.1+cpu\n" > ${build_dir}/pytorch/torch/build-version && \
178+
export CC=/usr/local/gcc103/bin/gcc && \
179+
export TORCH_VERSION="v1.13.1" && \
180+
export TORCH_IPEX_VERSION="1.13.100+cpu" && \
181+
# building ipex is memory hungry, so we need to limit the number of jobs
182+
sed -i 's/multiprocessing.cpu_count()/1/g' setup.py && \
183+
/usr/local/bin/python3.10 setup.py build_clib ${build_dir}/pytorch/torch && \
184+
cd ${build_dir}/pytorch && \
166185
mkdir /usr/local/gcc103/include/pytorch && \
167186
cp -r torch/include/* /usr/local/gcc103/include/pytorch/ && \
168187
cp torch/lib/libtorch_cpu.so /usr/local/gcc103/lib && \
169188
cp torch/lib/libc10.so /usr/local/gcc103/lib && \
189+
cp ${build_dir}/intel-extension-for-pytorch/build/Release/packages/intel_extension_for_pytorch/lib/libintel-ext-pt-cpu.so /usr/local/gcc103/lib && \
170190
cd .. && \
171-
rm -rf pytorch
191+
rm -rf pytorch && \
192+
rm -rf intel-extension-for-pytorch
193+
172194

173195
FROM centos:7
174196
COPY --from=builder /usr/local/gcc103 /usr/local/gcc103

Diff for: dev-tools/docker/linux_tester/Dockerfile

+1-1
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
#
1111

1212
# Increment the version here when a new tools/3rd party components image is built
13-
FROM docker.elastic.co/ml-dev/ml-linux-build:28
13+
FROM docker.elastic.co/ml-dev/ml-linux-build:27
1414

1515
MAINTAINER David Roberts <dave.roberts@elastic.co>
1616

0 commit comments

Comments
 (0)