Skip to content

Commit

Permalink
[ML-251] [Random forest] Add Random forest GPU Classifier (oap-projec…
Browse files Browse the repository at this point in the history 
…t#257)

* Migrate KMeans  daal to DPC++ (oap-project#209)

* 1. add JNI wrapper
2. modify compiler config
3. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add ci test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* java code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* cpp code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update prepare-build-deps.sh

* add convert array, vector to homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. rollback ci config
2. remove CI test becaue homogentable only support GPU

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. update prepare-build-deps.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* remove ci GPU test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. remove GPU CI test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. fix ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update header

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update header

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-cluster-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci conf

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update oneapi to 2022.1.2

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci conf

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comment

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update build-jni.sh

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update test.sh

* Update test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update test.sh

* Update test.sh

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix link libze_loader.so.1

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci-local-dpc-test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update oneapi to 2022.1.2

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* Update prepare-build-deps-gpu.sh

* Update prepare-build-deps-gpu.sh

* Update prepare-build-deps.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update install-build-level-zero-deps-ubuntun.sh

* Update install-build-level-zero-deps-ubuntun.sh

* Update install-build-level-zero-deps-ubuntun.sh

* Update ci-local-dpc-test.sh

* Update GPU.cpp

* Update GPU.cpp

* Update ci-build-test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update HomogenTableImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* Update ci-local-dpc-test.sh

* Update ci-local-dpc-test.sh

* update HomogenTable API

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update install-build-level-zero-deps-ubuntu.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update run.sh

* Update build-jni.sh

* 1. Merge branch 'make_homogen_table' into convert_homogentable
2. update ci
3. add a new parameters for HomogenTable class

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update pom.xml

* Update test.sh

* Update Makefile

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ConvertHomogenTableSuite.scala

* Update install-level-zero-ubuntu.sh

* Update install-level-zero-ubuntu.sh

* Update build.sh

* Update test.sh

* Update test.sh

* Update HomogenTableImpl.cpp

* Update install-build-deps-ubuntu.sh

* Update HomogenTableImpl.cpp

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update install-build-deps-ubuntu.sh

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update list-compute-devices.cpp

* Convert homogentable to array/vector/matrix (#4)

* 1. convert homogenTable to array/vector/matrix
2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Merge branch 'make_homogen_table' into convert_homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Convert RDD[Vector]/RDD[LabeledPoint] to HomogenTable (#8)

* 1. convert rdd to HomogenTable
* 2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add empty homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Mirgate kmeans daal to DPC++ (#7)

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. Mirgate kmeans daal to DPC++
* 2. add unit test

* 1. move local host to distributed host
2. add cluster test
3. code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-cluster-test.sh

* Update ci-cluster-test.sh

* Update build.sh

* Update build.sh

* Update OneDAL.cpp

* Update OneDAL.cpp

* Update OneDAL.cpp

* Update com_intel_oap_mllib_OneDAL__.h

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update NaiveBayesResult.java

* 1. update code style
2. When using the Host/CPU device throws a fail

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback checkstyle.xml

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback checkstyle.xml

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update OneDAL.scala

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* revert ALS.scala

* revert checkstyle.xml

* :Revert "revert checkstyle.xml"

This reverts commit 4f74bac.

* disable cluster test for gpu

* disable cluster test for gpu

* nit

Co-authored-by: minmingz <minming.zhu@inte.com>
Co-authored-by: Wu, Xiaochang <xiaochang.wu@intel.com>

* Migrate pca daal to DPC++ (oap-project#223)

* 1. add JNI wrapper
2. modify compiler config
3. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add ci test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* java code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* cpp code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update prepare-build-deps.sh

* add convert array, vector to homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. rollback ci config
2. remove CI test becaue homogentable only support GPU

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. update prepare-build-deps.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* remove ci GPU test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. remove GPU CI test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. fix ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update header

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update header

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-cluster-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci conf

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update oneapi to 2022.1.2

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci conf

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comment

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update build-jni.sh

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update test.sh

* Update test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update test.sh

* Update test.sh

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix link libze_loader.so.1

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci-local-dpc-test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update oneapi to 2022.1.2

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* Update prepare-build-deps-gpu.sh

* Update prepare-build-deps-gpu.sh

* Update prepare-build-deps.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update install-build-level-zero-deps-ubuntun.sh

* Update install-build-level-zero-deps-ubuntun.sh

* Update install-build-level-zero-deps-ubuntun.sh

* Update ci-local-dpc-test.sh

* Update GPU.cpp

* Update GPU.cpp

* Update ci-build-test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update HomogenTableImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* Update ci-local-dpc-test.sh

* Update ci-local-dpc-test.sh

* update HomogenTable API

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update install-build-level-zero-deps-ubuntu.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update run.sh

* Update build-jni.sh

* 1. Merge branch 'make_homogen_table' into convert_homogentable
2. update ci
3. add a new parameters for HomogenTable class

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update pom.xml

* Update test.sh

* Update Makefile

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ConvertHomogenTableSuite.scala

* Update install-level-zero-ubuntu.sh

* Update install-level-zero-ubuntu.sh

* Update build.sh

* Update test.sh

* Update test.sh

* Update HomogenTableImpl.cpp

* Update install-build-deps-ubuntu.sh

* Update HomogenTableImpl.cpp

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update install-build-deps-ubuntu.sh

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update list-compute-devices.cpp

* Convert homogentable to array/vector/matrix (#4)

* 1. convert homogenTable to array/vector/matrix
2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Merge branch 'make_homogen_table' into convert_homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Convert RDD[Vector]/RDD[LabeledPoint] to HomogenTable (#8)

* 1. convert rdd to HomogenTable
* 2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add empty homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Mirgate kmeans daal to DPC++ (#7)

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. Mirgate kmeans daal to DPC++
* 2. add unit test

* 1. move local host to distributed host
2. add cluster test
3. code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-cluster-test.sh

* Update ci-cluster-test.sh

* Update build.sh

* Update build.sh

* Update OneDAL.cpp

* Update OneDAL.cpp

* Update OneDAL.cpp

* Update com_intel_oap_mllib_OneDAL__.h

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update NaiveBayesResult.java

* 1. update code style
2. When using the Host/CPU device throws a fail

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback checkstyle.xml

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback checkstyle.xml

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update OneDAL.scala

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* revert ALS.scala

* revert checkstyle.xml

* :Revert "revert checkstyle.xml"

This reverts commit 4f74bac.

* disable cluster test for gpu

* disable cluster test for gpu

* nit

* migrate cpa to DPC++

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update OutputHelpers.hpp

* update com_intel_oap_mllib_feature_PCADALImpl.h

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update PCADALImpl.scala

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. update function name and variable name

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update stayHomogenTablePtrToVector to saveHomogenTablePtrToVector

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* change compute_device to ComputeDevice

Signed-off-by: minmingzhu <minming.zhu@intel.com>

Co-authored-by: minmingz <minming.zhu@inte.com>
Co-authored-by: Wu, Xiaochang <xiaochang.wu@intel.com>

* [ML-226] Migrate correlation daal to DPC++  (oap-project#215)

* 1. add JNI wrapper
2. modify compiler config
3. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add ci test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* java code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* cpp code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update ci-local-test.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update install-build-deps-ubuntu.sh

* Update prepare-build-deps.sh

* add convert array, vector to homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. rollback ci config
2. remove CI test becaue homogentable only support GPU

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. update prepare-build-deps.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* remove ci GPU test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. remove GPU CI test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. fix ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update header

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update header

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-cluster-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci conf

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update oneapi to 2022.1.2

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci conf

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comment

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update build-jni.sh

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update test.sh

* Update test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update test.sh

* Update test.sh

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix link libze_loader.so.1

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci-local-dpc-test.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update oneapi to 2022.1.2

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* Update prepare-build-deps-gpu.sh

* Update prepare-build-deps-gpu.sh

* Update prepare-build-deps.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update install-build-level-zero-deps-ubuntun.sh

* Update install-build-level-zero-deps-ubuntun.sh

* Update install-build-level-zero-deps-ubuntun.sh

* Update ci-local-dpc-test.sh

* Update GPU.cpp

* Update GPU.cpp

* Update ci-build-test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update HomogenTableImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* Update ci-local-dpc-test.sh

* Update ci-local-dpc-test.sh

* update HomogenTable API

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update install-build-level-zero-deps-ubuntu.sh

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-local-dpc-test.sh

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update ci

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update run.sh

* Update build-jni.sh

* 1. Merge branch 'make_homogen_table' into convert_homogentable
2. update ci
3. add a new parameters for HomogenTable class

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update pom.xml

* Update test.sh

* Update Makefile

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ConvertHomogenTableSuite.scala

* Update install-level-zero-ubuntu.sh

* Update install-level-zero-ubuntu.sh

* Update build.sh

* Update test.sh

* Update test.sh

* Update HomogenTableImpl.cpp

* Update install-build-deps-ubuntu.sh

* Update HomogenTableImpl.cpp

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update install-build-deps-ubuntu.sh

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update list-compute-devices.cpp

* Convert homogentable to array/vector/matrix (#4)

* 1. convert homogenTable to array/vector/matrix
2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Merge branch 'make_homogen_table' into convert_homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Convert RDD[Vector]/RDD[LabeledPoint] to HomogenTable (#8)

* 1. convert rdd to HomogenTable
* 2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add empty homogentable

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Mirgate kmeans daal to DPC++ (#7)

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. Mirgate kmeans daal to DPC++
* 2. add unit test

* 1. migrate pca to DPC++
2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* normalize data

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. migrate correlation to DPC++ interface
2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. move local host to distributed host
2. add cluster test
3. code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update ci-cluster-test.sh

* Update ci-cluster-test.sh

* Update build.sh

* Update build.sh

* Update OneDAL.cpp

* Update OneDAL.cpp

* Update OneDAL.cpp

* Update com_intel_oap_mllib_OneDAL__.h

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update NaiveBayesResult.java

* 1. update code style
2. When using the Host/CPU device throws a fail

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback checkstyle.xml

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rollback checkstyle.xml

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update OneDAL.scala

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* revert ALS.scala

* revert checkstyle.xml

* :Revert "revert checkstyle.xml"

This reverts commit 4f74bac.

* disable cluster test for gpu

* disable cluster test for gpu

* nit

* migrate cpa to DPC++

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update OutputHelpers.hpp

* update com_intel_oap_mllib_feature_PCADALImpl.h

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update PCADALImpl.scala

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. update function name and variable name

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update stayHomogenTablePtrToVector to saveHomogenTablePtrToVector

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* change compute_device to ComputeDevice

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. migrate pca to DPC++
2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* normalize data

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. migrate correlation to DPC++ interface
2. add unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update prepare-build-deps-gpu.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update Makefile

* Update OutputHelpers.hpp

* Update PCAOneAPIImpl.cpp

* Update service.h

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update OneDAL.scala

* Update PCADALImpl.scala

* Update KmeansHomogenTableSuite.scala

* Update TestCommon.scala

* Update test.sh

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* fix comments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

Signed-off-by: minmingzhu <minming.zhu@intel.com>
Co-authored-by: minmingz <minming.zhu@inte.com>
Co-authored-by: Wu, Xiaochang <xiaochang.wu@intel.com>

* [ML-227] Migrate moments daal to DPC++  (oap-project#229)

* add low order moments

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* rename unit filename

Signed-off-by: minmingzhu <minming.zhu@intel.com>

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add RF

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* RF example pom.xml

* update Makefile

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add RandomForestClassifier JNI head file

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update DecisionForestOneAPIImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update DecisionForestOneAPIImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update DecisionForestOneAPIImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update DecisionForestOneAPIImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update DecisionForestOneAPIImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update DecisionForestOneAPIImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update DecisionForestOneAPIImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update DecisionForestOneAPIImpl.cpp

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update debug log

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* add RF classifier unit test

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. update code style

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update  RandomForestClassifier.scala

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* 1. remove CSR Table
2. remove scala example instead of PYSpark example
3. refactor convert c++ map to JavaMap

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

* update

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* update RF example

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update service.h

* update dataset

* update

Signed-off-by: minmingzhu <minming.zhu@intel.com>

* Update random_forest_classifier_example.py

* Update random_forest_classifier_example.py

---------

Signed-off-by: minmingzhu <minming.zhu@intel.com>
Co-authored-by: minmingz <minming.zhu@inte.com>
Co-authored-by: Wu, Xiaochang <xiaochang.wu@intel.com>
  • Loading branch information
@minmingzhu @xwu99
3 people authored May 11, 2023
1 parent e8ee55f commit 14a5a4b
Show file tree
Hide file tree
Showing 18 changed files with 2,145 additions and 93 deletions.
1 change: 1 addition & 0 deletions examples/random-forest-pyspark/IntelGpuResourceFile.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[{"id":{"componentName": "spark.worker","resourceName":"gpu"},"addresses":["0","1","2","3"]}]
90 changes: 90 additions & 0 deletions examples/random-forest-pyspark/random_forest_classifier_example.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

"""
Random Forest Classifier Example.
"""
from __future__ import print_function
import sys

from pyspark import Row
from pyspark.ml import Pipeline
from pyspark.ml.functions import vector_to_array
from pyspark.ml.classification import RandomForestClassifier
from pyspark.ml.feature import IndexToString, StringIndexer, VectorIndexer
from pyspark.ml.evaluation import MulticlassClassificationEvaluator
from pyspark.ml.linalg import DenseVector
from pyspark.sql import SparkSession
from pyspark.sql.functions import col, split

if __name__ == "__main__":
spark = SparkSession\
.builder\
.appName("RandomForestClassifierExample")\
.getOrCreate()

if (len(sys.argv) != 2) :
print("Require data file path as input parameter")
sys.exit(1)

# Load and parse the data file, converting it to a DataFrame.
data_sparse = spark.read.format("libsvm").load(sys.argv[1]).toDF("label", "features_sparse")
data = data_sparse.rdd.map(lambda x: Row(label=x[0], features=DenseVector(x[1].toArray()))).toDF()
data.printSchema()
data.show()

# Index labels, adding metadata to the label column.
# Fit on whole dataset to include all labels in index.
labelIndexer = StringIndexer(inputCol="label", outputCol="indexedLabel").fit(data)

# Automatically identify categorical features, and index them.
# Set maxCategories so features with > 4 distinct values are treated as continuous.
featureIndexer = \
VectorIndexer(inputCol="features", outputCol="indexedFeatures", maxCategories=4).fit(data)

# Split the data into training and test sets (30% held out for testing)
(trainingData, testData) = data.randomSplit([0.7, 0.3])

# Train a RandomForest model.
rf = RandomForestClassifier(labelCol="indexedLabel", featuresCol="indexedFeatures", numTrees=10)

# Convert indexed labels back to original labels.
labelConverter = IndexToString(inputCol="prediction", outputCol="predictedLabel",
labels=labelIndexer.labels)

# Chain indexers and forest in a Pipeline
pipeline = Pipeline(stages=[labelIndexer, featureIndexer, rf, labelConverter])

# Train model. This also runs the indexers.
model = pipeline.fit(trainingData)

# Make predictions.
predictions = model.transform(testData)

# Select example rows to display.
predictions.select("predictedLabel", "label", "features").show(5)

# Select (prediction, true label) and compute test error
evaluator = MulticlassClassificationEvaluator(
labelCol="indexedLabel", predictionCol="prediction", metricName="accuracy")
accuracy = evaluator.evaluate(predictions)
print("Test Error = %g" % (1.0 - accuracy))

rfModel = model.stages[2]
print(rfModel.toDebugString) # summary only

spark.stop()
38 changes: 38 additions & 0 deletions examples/random-forest-pyspark/run-gpu-standalone.sh
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
#!/usr/bin/env bash

source ../../conf/env.sh

# CSV data is the same as in Spark example "ml/pca_example.py"
# The data file should be copied to $HDFS_ROOT before running examples
DATA_FILE=$HDFS_ROOT/data/sample_libsvm_data.txt

DEVICE=GPU
RESOURCE_FILE=$PWD/IntelGpuResourceFile.json
WORKER_GPU_AMOUNT=4
EXECUTOR_GPU_AMOUNT=1
TASK_GPU_AMOUNT=1
APP_PY=random_forest_classifier_example.py


time $SPARK_HOME/bin/spark-submit --master $SPARK_MASTER \
--num-executors $SPARK_NUM_EXECUTORS \
--executor-cores $SPARK_EXECUTOR_CORES \
--total-executor-cores $SPARK_TOTAL_CORES \
--driver-memory $SPARK_DRIVER_MEMORY \
--executor-memory $SPARK_EXECUTOR_MEMORY \
--conf "spark.serializer=org.apache.spark.serializer.KryoSerializer" \
--conf "spark.default.parallelism=$SPARK_DEFAULT_PARALLELISM" \
--conf "spark.sql.shuffle.partitions=$SPARK_DEFAULT_PARALLELISM" \
--conf "spark.driver.extraClassPath=$SPARK_DRIVER_CLASSPATH" \
--conf "spark.executor.extraClassPath=$SPARK_EXECUTOR_CLASSPATH" \
--conf "spark.oap.mllib.device=$DEVICE" \
--conf "spark.worker.resourcesFile=$RESOURCE_FILE" \
--conf "spark.worker.resource.gpu.amount=$WORKER_GPU_AMOUNT" \
--conf "spark.executor.resource.gpu.amount=$EXECUTOR_GPU_AMOUNT" \
--conf "spark.task.resource.gpu.amount=$TASK_GPU_AMOUNT" \
--conf "spark.shuffle.reduceLocality.enabled=false" \
--conf "spark.network.timeout=1200s" \
--conf "spark.task.maxFailures=1" \
--jars $OAP_MLLIB_JAR \
$APP_PY DATA_FILE \
2>&1 | tee random_forest_classifier-$(date +%m%d_%H_%M_%S).log
32 changes: 32 additions & 0 deletions mllib-dal/src/main/java/com/intel/oap/mllib/classification/LearningNode.java
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
package com.intel.oap.mllib.classification;

import java.io.Serializable;

public class LearningNode implements Serializable {
public int level;
public double impurity;
public int splitIndex;
public double splitValue;
public boolean isLeaf;
public double[] probability;
public int sampleCount ;

public String toString() {
String str = String.format("level is %s; " +
"impurity is %s; " +
"splitIndex is %s; " +
"splitValue is %s; " +
"isLeaf is %s; " +
"probability size is %s;" +
"sampleCount is %s;",
new Integer(level),
new Double(impurity),
new Integer(splitIndex),
new Double(splitValue),
new Boolean(isLeaf),
probability == null ? "null" : probability.length,
new Integer(sampleCount));

return str;
}
}
6 changes: 6 additions & 0 deletions mllib-dal/src/main/java/com/intel/oap/mllib/classification/RandomForestResult.java
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package com.intel.oap.mllib.classification;

public class RandomForestResult {
public long predictionNumericTable;
public long probabilitiesNumericTable;
}
2 changes: 1 addition & 1 deletion mllib-dal/src/main/java/com/intel/oneapi/dal/table/HomogenTable.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ public double[] getDoubleData() {
public void addHomogenTable(long homogenTableAddr ) {
impl.addHomogenTable(homogenTableAddr);
}

@Override
public long getcObejct() {
return impl.getcObject();
}
Expand Down
1 change: 1 addition & 0 deletions mllib-dal/src/main/java/com/intel/oneapi/dal/table/Table.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,5 @@ public Table() {}
protected abstract boolean hasData();
protected abstract Long getKind();
protected abstract TableMetadata getMetaData();
public abstract long getcObejct();
}
Loading

0 comments on commit 14a5a4b

Please sign in to comment.