Skip to content

Commit

Permalink
[ML-224] Optimize table merge performance issues (#225)
Browse files Browse the repository at this point in the history 
* 1. add JNI wrapper
2. modify compiler config
3. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add ci test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* java code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* cpp code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update prepare-build-deps.sh

* add convert array, vector to homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. rollback ci config
2. remove CI test becaue homogentable only support GPU

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. update prepare-build-deps.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* remove ci GPU test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. remove GPU CI test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. fix ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update header

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update header

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-cluster-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci conf

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update oneapi to 2022.1.2

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci conf

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comment

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update build-jni.sh

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update test.sh

* Update test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update test.sh

* Update test.sh

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix link libze_loader.so.1

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci-local-dpc-test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update oneapi to 2022.1.2

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* Update prepare-build-deps-gpu.sh

* Update prepare-build-deps-gpu.sh

* Update prepare-build-deps.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update install-build-level-zero-deps-ubuntun.sh

* Update install-build-level-zero-deps-ubuntun.sh

* Update install-build-level-zero-deps-ubuntun.sh

* Update ci-local-dpc-test.sh

* Update GPU.cpp

* Update GPU.cpp

* Update ci-build-test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update HomogenTableImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* Update ci-local-dpc-test.sh

* Update ci-local-dpc-test.sh

* update HomogenTable API

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update install-build-level-zero-deps-ubuntu.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update run.sh

* Update build-jni.sh

* 1. Merge branch 'make_homogen_table' into convert_homogentable
2. update ci
3. add a new parameters for HomogenTable class

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update pom.xml

* Update test.sh

* Update Makefile

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ConvertHomogenTableSuite.scala

* Update install-level-zero-ubuntu.sh

* Update install-level-zero-ubuntu.sh

* Update build.sh

* Update test.sh

* Update test.sh

* Update HomogenTableImpl.cpp

* Update install-build-deps-ubuntu.sh

* Update HomogenTableImpl.cpp

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update install-build-deps-ubuntu.sh

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update list-compute-devices.cpp

* Convert homogentable to array/vector/matrix (#4)

* 1. convert homogenTable to array/vector/matrix
2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Merge branch 'make_homogen_table' into convert_homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Convert RDD[Vector]/RDD[LabeledPoint] to HomogenTable (#8)

* 1. convert rdd to HomogenTable
* 2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add empty homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Mirgate kmeans daal to DPC++ (#7)

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. Mirgate kmeans daal to DPC++
* 2. add unit test

* 1. move local host to distributed host
2. add cluster test
3. code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-cluster-test.sh

* Update ci-cluster-test.sh

* Update build.sh

* Update build.sh

* Update OneDAL.cpp

* Update OneDAL.cpp

* Update OneDAL.cpp

* Update com_intel_oap_mllib_OneDAL__.h

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update NaiveBayesResult.java

* 1. update code style
2. When using the Host/CPU device throws a fail

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback checkstyle.xml

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback checkstyle.xml

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update OneDAL.scala

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* revert ALS.scala

* revert checkstyle.xml

* :Revert "revert checkstyle.xml"

This reverts commit 4f74bac.

* disable cluster test for gpu

* disable cluster test for gpu

* nit

* Migrate KMeans  daal to DPC++ (#209)

* 1. add JNI wrapper
2. modify compiler config
3. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add ci test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* java code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* cpp code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update prepare-build-deps.sh

* add convert array, vector to homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. rollback ci config
2. remove CI test becaue homogentable only support GPU

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. update prepare-build-deps.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* remove ci GPU test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. remove GPU CI test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. fix ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update header

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update header

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-cluster-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci conf

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update oneapi to 2022.1.2

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci conf

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comment

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update build-jni.sh

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update test.sh

* Update test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update test.sh

* Update test.sh

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix link libze_loader.so.1

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci-local-dpc-test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update oneapi to 2022.1.2

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* Update prepare-build-deps-gpu.sh

* Update prepare-build-deps-gpu.sh

* Update prepare-build-deps.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update install-build-level-zero-deps-ubuntun.sh

* Update install-build-level-zero-deps-ubuntun.sh

* Update install-build-level-zero-deps-ubuntun.sh

* Update ci-local-dpc-test.sh

* Update GPU.cpp

* Update GPU.cpp

* Update ci-build-test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update HomogenTableImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* Update ci-local-dpc-test.sh

* Update ci-local-dpc-test.sh

* update HomogenTable API

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update install-build-level-zero-deps-ubuntu.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update run.sh

* Update build-jni.sh

* 1. Merge branch 'make_homogen_table' into convert_homogentable
2. update ci
3. add a new parameters for HomogenTable class

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update pom.xml

* Update test.sh

* Update Makefile

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ConvertHomogenTableSuite.scala

* Update install-level-zero-ubuntu.sh

* Update install-level-zero-ubuntu.sh

* Update build.sh

* Update test.sh

* Update test.sh

* Update HomogenTableImpl.cpp

* Update install-build-deps-ubuntu.sh

* Update HomogenTableImpl.cpp

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update install-build-deps-ubuntu.sh

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update list-compute-devices.cpp

* Convert homogentable to array/vector/matrix (#4)

* 1. convert homogenTable to array/vector/matrix
2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Merge branch 'make_homogen_table' into convert_homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Convert RDD[Vector]/RDD[LabeledPoint] to HomogenTable (#8)

* 1. convert rdd to HomogenTable
* 2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add empty homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Mirgate kmeans daal to DPC++ (#7)

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. Mirgate kmeans daal to DPC++
* 2. add unit test

* 1. move local host to distributed host
2. add cluster test
3. code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-cluster-test.sh

* Update ci-cluster-test.sh

* Update build.sh

* Update build.sh

* Update OneDAL.cpp

* Update OneDAL.cpp

* Update OneDAL.cpp

* Update com_intel_oap_mllib_OneDAL__.h

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update NaiveBayesResult.java

* 1. update code style
2. When using the Host/CPU device throws a fail

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback checkstyle.xml

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback checkstyle.xml

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update OneDAL.scala

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* revert ALS.scala

* revert checkstyle.xml

* :Revert "revert checkstyle.xml"

This reverts commit 4f74bac.

* disable cluster test for gpu

* disable cluster test for gpu

* nit

Co-authored-by: minmingz <minming.zhu@inte.com>
Co-authored-by: Wu, Xiaochang <xiaochang.wu@intel.com>

* fix merge

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update KMeansOneAPIImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* use std::copy to copy data

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add debug info

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Migrate pca daal to DPC++ (#223)

* 1. add JNI wrapper
2. modify compiler config
3. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add ci test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* java code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* cpp code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update prepare-build-deps.sh

* add convert array, vector to homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. rollback ci config
2. remove CI test becaue homogentable only support GPU

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. update prepare-build-deps.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* remove ci GPU test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. remove GPU CI test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. fix ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update header

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update header

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-cluster-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci conf

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update oneapi to 2022.1.2

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci conf

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comment

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update build-jni.sh

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update test.sh

* Update test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update test.sh

* Update test.sh

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix link libze_loader.so.1

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci-local-dpc-test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update oneapi to 2022.1.2

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* Update prepare-build-deps-gpu.sh

* Update prepare-build-deps-gpu.sh

* Update prepare-build-deps.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update install-build-level-zero-deps-ubuntun.sh

* Update install-build-level-zero-deps-ubuntun.sh

* Update install-build-level-zero-deps-ubuntun.sh

* Update ci-local-dpc-test.sh

* Update GPU.cpp

* Update GPU.cpp

* Update ci-build-test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update HomogenTableImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* Update ci-local-dpc-test.sh

* Update ci-local-dpc-test.sh

* update HomogenTable API

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update install-build-level-zero-deps-ubuntu.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update run.sh

* Update build-jni.sh

* 1. Merge branch 'make_homogen_table' into convert_homogentable
2. update ci
3. add a new parameters for HomogenTable class

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update pom.xml

* Update test.sh

* Update Makefile

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ConvertHomogenTableSuite.scala

* Update install-level-zero-ubuntu.sh

* Update install-level-zero-ubuntu.sh

* Update build.sh

* Update test.sh

* Update test.sh

* Update HomogenTableImpl.cpp

* Update install-build-deps-ubuntu.sh

* Update HomogenTableImpl.cpp

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update install-build-deps-ubuntu.sh

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update list-compute-devices.cpp

* Convert homogentable to array/vector/matrix (#4)

* 1. convert homogenTable to array/vector/matrix
2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Merge branch 'make_homogen_table' into convert_homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Convert RDD[Vector]/RDD[LabeledPoint] to HomogenTable (#8)

* 1. convert rdd to HomogenTable
* 2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add empty homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Mirgate kmeans daal to DPC++ (#7)

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. Mirgate kmeans daal to DPC++
* 2. add unit test

* 1. move local host to distributed host
2. add cluster test
3. code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-cluster-test.sh

* Update ci-cluster-test.sh

* Update build.sh

* Update build.sh

* Update OneDAL.cpp

* Update OneDAL.cpp

* Update OneDAL.cpp

* Update com_intel_oap_mllib_OneDAL__.h

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update NaiveBayesResult.java

* 1. update code style
2. When using the Host/CPU device throws a fail

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback checkstyle.xml

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback checkstyle.xml

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update OneDAL.scala

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* revert ALS.scala

* revert checkstyle.xml

* :Revert "revert checkstyle.xml"

This reverts commit 4f74bac.

* disable cluster test for gpu

* disable cluster test for gpu

* nit

* migrate cpa to DPC++

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update OutputHelpers.hpp

* update com_intel_oap_mllib_feature_PCADALImpl.h

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update PCADALImpl.scala

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. update function name and variable name

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update stayHomogenTablePtrToVector to saveHomogenTablePtrToVector

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* change compute_device to ComputeDevice

Signed-off-by: minmingzhu <minming.zhu@intel.com>

Co-authored-by: minmingz <minming.zhu@inte.com>
Co-authored-by: Wu, Xiaochang <xiaochang.wu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add std::cout code

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* [ML-226] Migrate correlation daal to DPC++  (#215)

* 1. add JNI wrapper
2. modify compiler config
3. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add ci test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* java code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* cpp code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update prepare-build-deps.sh

* add convert array, vector to homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. rollback ci config
2. remove CI test becaue homogentable only support GPU

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. update prepare-build-deps.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* remove ci GPU test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. remove GPU CI test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. fix ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update header

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update header

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-cluster-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci conf

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update oneapi to 2022.1.2

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci conf

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comment

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update build-jni.sh

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update test.sh

* Update test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update test.sh

* Update test.sh

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix link libze_loader.so.1

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci-local-dpc-test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update oneapi to 2022.1.2

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* Update prepare-build-deps-gpu.sh

* Update prepare-build-deps-gpu.sh

* Update prepare-build-deps.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update install-build-level-zero-deps-ubuntun.sh

* Update install-build-level-zero-deps-ubuntun.sh

* Update install-build-level-zero-deps-ubuntun.sh

* Update ci-local-dpc-test.sh

* Update GPU.cpp

* Update GPU.cpp

* Update ci-build-test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update HomogenTableImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* Update ci-local-dpc-test.sh

* Update ci-local-dpc-test.sh

* update HomogenTable API

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update install-build-level-zero-deps-ubuntu.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update run.sh

* Update build-jni.sh

* 1. Merge branch 'make_homogen_table' into convert_homogentable
2. update ci
3. add a new parameters for HomogenTable class

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update pom.xml

* Update test.sh

* Update Makefile

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ConvertHomogenTableSuite.scala

* Update install-level-zero-ubuntu.sh

* Update install-level-zero-ubuntu.sh

* Update build.sh

* Update test.sh

* Update test.sh

* Update HomogenTableImpl.cpp

* Update install-build-deps-ubuntu.sh

* Update HomogenTableImpl.cpp

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update install-build-deps-ubuntu.sh

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update list-compute-devices.cpp

* Convert homogentable to array/vector/matrix (#4)

* 1. convert homogenTable to array/vector/matrix
2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Merge branch 'make_homogen_table' into convert_homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Convert RDD[Vector]/RDD[LabeledPoint] to HomogenTable (#8)

* 1. convert rdd to HomogenTable
* 2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add empty homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Mirgate kmeans daal to DPC++ (#7)

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. Mirgate kmeans daal to DPC++
* 2. add unit test

* 1. migrate pca to DPC++
2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* normalize data

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. migrate correlation to DPC++ interface
2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. move local host to distributed host
2. add cluster test
3. code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-cluster-test.sh

* Update ci-cluster-test.sh

* Update build.sh

* Update build.sh

* Update OneDAL.cpp

* Update OneDAL.cpp

* Update OneDAL.cpp

* Update com_intel_oap_mllib_OneDAL__.h

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update NaiveBayesResult.java

* 1. update code style
2. When using the Host/CPU device throws a fail

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback checkstyle.xml

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback checkstyle.xml

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update OneDAL.scala

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* revert ALS.scala

* revert checkstyle.xml

* :Revert "revert checkstyle.xml"

This reverts commit 4f74bac.

* disable cluster test for gpu

* disable cluster test for gpu

* nit

* migrate cpa to DPC++

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update OutputHelpers.hpp

* update com_intel_oap_mllib_feature_PCADALImpl.h

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update PCADALImpl.scala

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. update function name and variable name

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update stayHomogenTablePtrToVector to saveHomogenTablePtrToVector

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* change compute_device to ComputeDevice

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. migrate pca to DPC++
2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* normalize data

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. migrate correlation to DPC++ interface
2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update prepare-build-deps-gpu.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update Makefile

* Update OutputHelpers.hpp

* Update PCAOneAPIImpl.cpp

* Update service.h

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update OneDAL.scala

* Update PCADALImpl.scala

* Update KmeansHomogenTableSuite.scala

* Update TestCommon.scala

* Update test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

Signed-off-by: minmingzhu <minming.zhu@intel.com>
Co-authored-by: minmingz <minming.zhu@inte.com>
Co-authored-by: Wu, Xiaochang <xiaochang.wu@intel.com>

* [ML-227] Migrate moments daal to DPC++  (#229)

* add low order moments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rename unit filename

Signed-off-by: minmingzhu <minming.zhu@intel.com>

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update prepare-build-deps.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* remove extra code

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update PCADALImpl.scala

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update PCADALImpl.scala

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update PCADALImpl.scala

* retrigger checks

* update ci-tests.yml

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* merge rdd to table

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update GPU.cpp

* Update mllib-dal/src/main/scala/com/intel/oap/mllib/OneDAL.scala

* Update mllib-dal/src/main/scala/com/intel/oap/mllib/OneDAL.scala

---------

Signed-off-by: minmingzhu <minming.zhu@intel.com>
Co-authored-by: minmingz <minming.zhu@inte.com>
Co-authored-by: Wu, Xiaochang <xiaochang.wu@intel.com>
  • Loading branch information
@minmingzhu @xwu99
3 people authored Apr 13, 2023
1 parent e60a768 commit 2f6b35c
Show file tree
Hide file tree
Showing 6 changed files with 34 additions and 8 deletions.
22 changes: 22 additions & 0 deletions mllib-dal/src/main/scala/com/intel/oap/mllib/OneDAL.scala
View file
Original file line number Diff line number Diff line change
Expand Up @@ -668,6 +668,28 @@ object OneDAL {
}
}

def coalesceToHomogenTables(data: RDD[Vector], executorNum: Int,
device: Common.ComputeDevice): RDD[Long] = {
// Coalesce partitions belonging to the same executor
val coalescedRdd = data
.coalesce(executorNum,
partitionCoalescer = Some(new ExecutorInProcessCoalescePartitioner()))
.setName("coalescedRdd")
.cache()

// convert RDD to HomogenTable
val coalescedTables = coalescedRdd.mapPartitionsWithIndex { (index: Int, it: Iterator[Vector]) =>
val table = makeHomogenTable(it.toArray, device)
Iterator(table.getcObejct())
}.setName("coalescedTables").cache()
coalescedTables.count()
// Unpersist instances RDD
if (data.getStorageLevel != StorageLevel.NONE) {
data.unpersist()
}
coalescedTables
}

def makeNumericTable(arrayVectors: Array[Vector]): NumericTable = {

val numCols = arrayVectors.head.size
Expand Down
2 changes: 1 addition & 1 deletion mllib-dal/src/main/scala/com/intel/oap/mllib/clustering/KMeansDALImpl.scala
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ class KMeansDALImpl(var nClusters: Int,
val useDevice = sparkContext.getConf.get("spark.oap.mllib.device", Utils.DefaultComputeDevice)
val computeDevice = Common.ComputeDevice.getDeviceByName(useDevice)
val coalescedTables = if (useDevice == "GPU") {
OneDAL.rddVectorToMergedHomogenTables(data, executorNum, computeDevice)
OneDAL.coalesceToHomogenTables(data, executorNum, computeDevice)
} else {
OneDAL.rddVectorToMergedTables(data, executorNum)
}
Expand Down
2 changes: 1 addition & 1 deletion mllib-dal/src/main/scala/com/intel/oap/mllib/feature/PCADALImpl.scala
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ class PCADALImpl(val k: Int,
val useDevice = sparkContext.getConf.get("spark.oap.mllib.device", Utils.DefaultComputeDevice)
val computeDevice = Common.ComputeDevice.getDeviceByName(useDevice)
val coalescedTables = if (useDevice == "GPU") {
OneDAL.rddVectorToMergedHomogenTables(normalizedData, executorNum,
OneDAL.coalesceToHomogenTables(normalizedData, executorNum,
computeDevice)
} else {
OneDAL.rddVectorToMergedTables(normalizedData, executorNum)
Expand Down
2 changes: 1 addition & 1 deletion mllib-dal/src/main/scala/com/intel/oap/mllib/stat/CorrelationDALImpl.scala
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ class CorrelationDALImpl(
val useDevice = sparkContext.getConf.get("spark.oap.mllib.device", Utils.DefaultComputeDevice)
val computeDevice = Common.ComputeDevice.getDeviceByName(useDevice)
val coalescedTables = if (useDevice == "GPU") {
OneDAL.rddVectorToMergedHomogenTables(data, executorNum,
OneDAL.coalesceToHomogenTables(data, executorNum,
computeDevice)
} else {
OneDAL.rddVectorToMergedTables(data, executorNum)
Expand Down
2 changes: 1 addition & 1 deletion mllib-dal/src/main/scala/com/intel/oap/mllib/stat/SummarizerDALImpl.scala
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ class SummarizerDALImpl(val executorNum: Int,
val useDevice = sparkContext.getConf.get("spark.oap.mllib.device", Utils.DefaultComputeDevice)
val computeDevice = Common.ComputeDevice.getDeviceByName(useDevice)
val coalescedTables = if (useDevice == "GPU") {
OneDAL.rddVectorToMergedHomogenTables(data, executorNum,
OneDAL.coalesceToHomogenTables(data, executorNum,
computeDevice)
} else {
OneDAL.rddVectorToMergedTables(data, executorNum)
Expand Down
12 changes: 8 additions & 4 deletions mllib-dal/src/main/scala/org/apache/spark/ml/clustering/spark321/KMeans.scala
View file
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,10 @@ class KMeans @Since("1.5.0") (

logInfo(s"KMeansDAL fit using $executor_num Executors")

if (handlePersistence) {
instances.persist(StorageLevel.MEMORY_AND_DISK)
}

val initStartTime = System.nanoTime()

val distanceMeasureInstance = DistanceMeasure.decodeFromString($(distanceMeasure))
Expand Down Expand Up @@ -143,14 +147,14 @@ class KMeans @Since("1.5.0") (
val strInitMode = $(initMode)
logInfo(f"Initialization with $strInitMode took $initTimeInSeconds%.3f seconds.")

if (handlePersistence) {
instances.persist(StorageLevel.MEMORY_AND_DISK)
}

val inputData = instances.map {
case (point: Vector, weight: Double) => point
}

if (handlePersistence) {
inputData.persist(StorageLevel.MEMORY_AND_DISK)
}

val kmeansDAL = new KMeansDALImpl(getK, getMaxIter, getTol,
DistanceMeasure.EUCLIDEAN, centers, executor_num, executor_cores)

Expand Down

0 comments on commit 2f6b35c

Please sign in to comment.