[ML-41] Revert to old oneCCL and Prepare for OAP 1.1 (#40)

* Revert "[ML-33] Optimize oneCCL port detecting (#34)"

This reverts commit c07d70c.

* Revert "[ML-12] Improve CI and add pseudo cluster testing (#20)"

This reverts commit 6fe5d3e.

* Revert "Print time duration for each PCA step (#32)"

This reverts commit e1c33d9.

* Revert "[ML-13] Add ALS with new oneCCL APIs (#14)"

This reverts commit 3252ae9.

* Revert "[ML-25] Fix oneCCL KVS port auto detect and improve logging (#24)"

This reverts commit 9080d02.

* Revert "[ML-18]  Auto detect KVS port for oneCCL to avoid port conflict (#19)"

This reverts commit ac20823.

* Revert "Merge pull request #11 from xwu99/new-oneccl"

This reverts commit b60158b, reversing
changes made to db2414c.

* Revert "use spark.oap.mllib.oneccl.kvs.ip to workaround KVS IP hang issue"

This reverts commit 318cae1.

* Revert "Add IP Port kvs_attr and Kmeans, PCA test cases validated"

This reverts commit 99b9551.

* Revert "Port oneccl to new C++ apis"

This reverts commit ac216f4.

* Fix Makefile
Add build.sh
Fix oap-mllib-ci.yml
Fix OneCCL port for multiple test run

* Remove kmeans-hibench example

* modify README

* modify README

* modify README

.github/workflows/oap-mllib-ci.yml

@@ -38,5 +38,4 @@ jobs:
           source /opt/intel/oneapi/dal/latest/env/vars.sh
           source /opt/intel/oneapi/tbb/latest/env/vars.sh
           source /tmp/oneCCL/build/_install/env/setvars.sh
-          # temp disable and will enable for new release of oneCCL
-          #./build.sh
+          ./test.sh

.gitignore

@@ -1,7 +1,6 @@
 *.o
 *.log
 .vscode
-*.iml
 target/
 .idea/
 .idea_modules/

README.md

@@ -17,7 +17,7 @@ You can find the all the OAP MLlib documents on the [project web page](https://o
 
 ### Java/Scala Users Preferred
 
-Use a pre-built OAP MLlib JAR to get started. You can firstly download OAP package from [OAP-JARs-Tarball](https://github.com/Intel-bigdata/OAP/releases/download/v1.0.0-spark-3.0.0/oap-1.0.0-bin-spark-3.0.0.tar.gz) and extract this Tarball to get `oap-mllib-x.x.x-with-spark-x.x.x.jar` under `oap-1.0.0-bin-spark-3.0.0/jars`.
+Use a pre-built OAP MLlib JAR to get started. You can firstly download OAP package from [OAP-JARs-Tarball](https://github.com/oap-mllib/releases/download/v1.1.0-spark-3.0.0/oap-1.1.0-bin-spark-3.0.0.tar.gz) and extract this Tarball to get `oap-mllib-x.x.x-with-spark-x.x.x.jar` under `oap-1.1.0-bin-spark-3.0.0/jars`.
 
 Then you can refer to the following [Running](#running) section to try out.
 
@@ -65,14 +65,6 @@ To use K-means example for sanity check, you need to upload a data file to your
     $ ./run.sh
 ```
 
-### Benchmark with HiBench
-Use [Hibench](https://github.com/Intel-bigdata/HiBench) to generate dataset with various profiles, and change related variables in `run-XXX.sh` script when applicable.  Then run the following commands:
-```
-    $ cd oap-mllib/examples/kmeans-hibench
-    $ ./build.sh
-    $ ./run-hibench-oap-mllib.sh
-```
-
 ### PySpark Support
 
 As PySpark-based applications call their Scala couterparts, they shall be supported out-of-box. An example can be found in the [Examples](#examples) section.
@@ -95,7 +87,7 @@ Intel® oneAPI Toolkits and its components can be downloaded and install from [h
 
 More details about oneAPI can be found [here](https://software.intel.com/content/www/us/en/develop/tools/oneapi.html).
 
-You can also refer to [this script and comments in it](https://github.com/Intel-bigdata/OAP/blob/branch-1.0-spark-3.x/oap-mllib/dev/install-build-deps-centos.sh) to install correct oneAPI version and manually setup the environments.
+You can also refer to [this script and comments in it](https://github.com/oap-project/oap-mllib/blob/branch-1.1-spark-3.x/dev/install-build-deps-centos.sh) to install correct oneAPI version and manually setup the environments.
 
 Scala and Java dependency descriptions are already included in Maven POM file. 
 
@@ -138,7 +130,7 @@ CCL_ROOT    | Path to oneCCL home directory
 We suggest you to source `setvars.sh` script into current shell to setup building environments as following:
 
 ```
-	$ source /opt/intel/inteloneapi/setvars.sh
+	$ source /opt/intel/oneapi/setvars.sh
 	$ source /your/oneCCL_source_code/build/_install/env/setvars.sh
 ```
 
@@ -160,8 +152,11 @@ Example         |  Description
 ----------------|---------------------------
 kmeans          |  K-means example for Scala
 kmeans-pyspark  |  K-means example for PySpark
-kmeans-hibench  |  Use HiBench-generated input dataset to benchmark K-means performance
+pca             |  PCA example for Scala
+pca-pyspark     |  PCA example for PySpark
 
 ## List of Accelerated Algorithms
 
 * K-Means (CPU, Experimental)
+* PCA (CPU, Experimental)
+

dev/install-build-deps-centos.sh

@@ -23,7 +23,7 @@ cd /tmp
 rm -rf oneCCL
 git clone https://github.com/oneapi-src/oneCCL
 cd oneCCL
-git checkout 2021.1
+git checkout beta08
 mkdir -p build && cd build
 cmake ..
 make -j 2 install

dev/install-build-deps-ubuntu.sh

@@ -17,7 +17,7 @@ echo "Building oneCCL ..."
 cd /tmp
 git clone https://github.com/oneapi-src/oneCCL
 cd oneCCL
-git checkout 2021.1
+git checkout beta08
 mkdir build && cd build
 cmake ..
 make -j 2 install