Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert RDD[Vector]/RDD[LabeledPoint] to HomogenTable #8

Merged
merged 185 commits into from
May 31, 2022
Merged
Show file tree
Hide file tree
Changes from 178 commits
Commits
Show all changes
185 commits
Select commit Hold shift + click to select a range
08352ab
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
a88614f
update ConvertHomogenTableSuite.scala
minmingzhu Apr 17, 2022
aaca1ea
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 17, 2022
5aa8430
update test.sh
minmingzhu Apr 17, 2022
44e00c5
Update build-jni.sh
minmingzhu Apr 17, 2022
0ee4023
convert rdd to HomogenTable
minmingzhu Apr 18, 2022
0fafa13
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
8b2386c
add empty homogentable
minmingzhu Apr 22, 2022
244d33d
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 22, 2022
962aefd
merge convert homogentable
minmingzhu Apr 22, 2022
711ba0c
update
minmingzhu Apr 22, 2022
672b89c
update
minmingzhu Apr 23, 2022
1dfc511
update ColumnAccessorImpl.cpp RowAccessorImpl.cpp
minmingzhu Apr 24, 2022
c0e54a6
merge
minmingzhu Apr 24, 2022
77a46a9
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 25, 2022
08c70e2
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 25, 2022
ba99cb6
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 28, 2022
7843ca3
Merge remote-tracking branch 'origin/convert_homogentable' into conve…
minmingzhu Apr 28, 2022
e0b782f
Update ci-local-dpc-test.sh
minmingzhu Apr 28, 2022
0306b8c
Merge branch 'convert_homogentable' into rdd_to_homogenTable
minmingzhu Apr 28, 2022
2ad859d
update
minmingzhu Apr 28, 2022
6c32fca
merge make_homogen_table
minmingzhu May 9, 2022
f3a72e5
rollback to edb9f3d
minmingzhu May 9, 2022
cc5352c
update
minmingzhu May 9, 2022
884c0b4
Merge remote-tracking branch 'origin/convert_homogentable' into conve…
minmingzhu May 9, 2022
c3b214e
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
47a7171
update ConvertHomogenTableSuite.scala
minmingzhu Apr 17, 2022
4556a5d
update ColumnAccessorImpl.cpp RowAccessorImpl.cpp
minmingzhu Apr 24, 2022
7f9a768
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 25, 2022
0ea2448
update
minmingzhu May 9, 2022
35af201
update
minmingzhu May 9, 2022
065d0d3
Merge remote-tracking branch 'origin/convert_homogentable' into conve…
minmingzhu May 9, 2022
8ab660d
update
minmingzhu May 9, 2022
0cb1f19
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
354d4a9
update ConvertHomogenTableSuite.scala
minmingzhu Apr 17, 2022
f6c228b
update ColumnAccessorImpl.cpp RowAccessorImpl.cpp
minmingzhu Apr 24, 2022
e1cba0d
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 25, 2022
fcf8314
rollback to edb9f3d
minmingzhu May 9, 2022
0c4bd73
update
minmingzhu May 9, 2022
2226da9
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
73f99fa
update ColumnAccessorImpl.cpp RowAccessorImpl.cpp
minmingzhu Apr 24, 2022
800845b
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 25, 2022
e13e27a
update
minmingzhu May 9, 2022
177e486
update
minmingzhu May 9, 2022
e4778fe
Merge remote-tracking branch 'origin/convert_homogentable' into conve…
minmingzhu May 9, 2022
7114c7c
update
minmingzhu May 9, 2022
ef2a7d7
update cpp style
minmingzhu May 9, 2022
45bf666
Update build.sh
minmingzhu May 10, 2022
5b86e6c
fix comments
minmingzhu May 10, 2022
8f31387
Merge remote-tracking branch 'origin/convert_homogentable' into conve…
minmingzhu May 10, 2022
f060188
update code style
minmingzhu May 10, 2022
d853874
fix comments
minmingzhu May 16, 2022
d84ab04
update
minmingzhu May 16, 2022
961f55f
update
minmingzhu May 16, 2022
9e694f2
update
minmingzhu May 16, 2022
6f1bf5a
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
311caa6
Update build-jni.sh
minmingzhu Apr 17, 2022
c68e84c
convert rdd to HomogenTable
minmingzhu Apr 18, 2022
5127322
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
0a1d158
add empty homogentable
minmingzhu Apr 22, 2022
9e64cca
update
minmingzhu Apr 23, 2022
6b690b7
update ColumnAccessorImpl.cpp RowAccessorImpl.cpp
minmingzhu Apr 24, 2022
7b22863
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 25, 2022
e53a3c5
update
minmingzhu Apr 28, 2022
2dcf1c6
Merge remote-tracking branch 'origin/rdd_to_homogenTable' into rdd_to…
minmingzhu May 17, 2022
b7fb088
update
minmingzhu May 17, 2022
fca1db7
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
08565ff
update ConvertHomogenTableSuite.scala
minmingzhu Apr 17, 2022
6efc1db
convert rdd to HomogenTable
minmingzhu Apr 18, 2022
685baf9
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
d952d0c
add empty homogentable
minmingzhu Apr 22, 2022
17ad654
update
minmingzhu Apr 23, 2022
8c3a85a
update ColumnAccessorImpl.cpp RowAccessorImpl.cpp
minmingzhu Apr 24, 2022
a769f4b
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 25, 2022
faa1057
update
minmingzhu Apr 28, 2022
0fb6eef
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
c2e3d34
convert rdd to HomogenTable
minmingzhu Apr 18, 2022
7e13554
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
1b90460
add empty homogentable
minmingzhu Apr 22, 2022
7f65eef
update
minmingzhu Apr 23, 2022
ffdbe87
update ColumnAccessorImpl.cpp RowAccessorImpl.cpp
minmingzhu Apr 24, 2022
2cce077
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 25, 2022
9e81f76
update
minmingzhu Apr 28, 2022
9c29701
update
minmingzhu May 17, 2022
2356c54
Merge remote-tracking branch 'origin/rdd_to_homogenTable' into rdd_to…
minmingzhu May 17, 2022
3d03c75
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
cb3c621
convert rdd to HomogenTable
minmingzhu Apr 18, 2022
fff9e09
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
219421e
add empty homogentable
minmingzhu Apr 22, 2022
4853c30
update
minmingzhu Apr 23, 2022
fb74ceb
update ColumnAccessorImpl.cpp RowAccessorImpl.cpp
minmingzhu Apr 24, 2022
5ef6673
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 25, 2022
f2f62d7
update
minmingzhu Apr 28, 2022
a26810d
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
b33bdda
Update build-jni.sh
minmingzhu Apr 17, 2022
a220486
convert rdd to HomogenTable
minmingzhu Apr 18, 2022
b3f748e
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
f2c73ea
add empty homogentable
minmingzhu Apr 22, 2022
e709f79
update
minmingzhu Apr 23, 2022
5682bda
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 25, 2022
1f58164
update
minmingzhu Apr 28, 2022
493b4e4
update
minmingzhu May 17, 2022
e6a0568
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
bda3e5d
update ConvertHomogenTableSuite.scala
minmingzhu Apr 17, 2022
39e3451
convert rdd to HomogenTable
minmingzhu Apr 18, 2022
92d122d
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
913dbb1
add empty homogentable
minmingzhu Apr 22, 2022
7dfeb45
update
minmingzhu Apr 23, 2022
8d6966f
update ColumnAccessorImpl.cpp RowAccessorImpl.cpp
minmingzhu Apr 24, 2022
37c7289
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 25, 2022
05d040c
update
minmingzhu Apr 28, 2022
e2d82b5
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
4878b3a
convert rdd to HomogenTable
minmingzhu Apr 18, 2022
7dcd34f
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
86a3300
add empty homogentable
minmingzhu Apr 22, 2022
9a76374
update
minmingzhu Apr 23, 2022
a8506e0
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 25, 2022
67133e5
update
minmingzhu Apr 28, 2022
995634d
update
minmingzhu May 17, 2022
5ac54bb
rebase dpcpp
minmingzhu May 17, 2022
561ccfd
Merge branch 'convert_homogentable' into rdd_to_homogenTable
minmingzhu May 17, 2022
44b8b01
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
9159a0a
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
bd6f1bb
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 25, 2022
134e53e
update
minmingzhu Apr 28, 2022
2f1e118
rollback to edb9f3d
minmingzhu May 9, 2022
0ad11d6
update
minmingzhu May 9, 2022
9bf6625
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
e57d776
Merge branch 'make_homogen_table' into convert_homogentable
minmingzhu Apr 25, 2022
c44215f
update
minmingzhu May 9, 2022
304326d
update
minmingzhu May 9, 2022
dbde288
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
eeaba6a
rollback to edb9f3d
minmingzhu May 9, 2022
c83b143
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
f491862
update
minmingzhu May 9, 2022
c5ee622
update
minmingzhu May 9, 2022
2656a8d
update
minmingzhu May 9, 2022
f7dd4a3
fix comments
minmingzhu May 10, 2022
2b87a75
fix comments
minmingzhu May 16, 2022
0b47896
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
1fd7bd3
convert rdd to HomogenTable
minmingzhu Apr 18, 2022
2926f6f
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
7b0f487
update
minmingzhu Apr 23, 2022
fed83f3
update
minmingzhu Apr 28, 2022
c5fb769
update
minmingzhu May 17, 2022
1e71601
convert rdd to HomogenTable
minmingzhu Apr 18, 2022
6061062
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
10ea8a8
add empty homogentable
minmingzhu Apr 22, 2022
3aa21db
update
minmingzhu Apr 23, 2022
7bc9782
update
minmingzhu Apr 28, 2022
72ac27e
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
7fc8771
update
minmingzhu Apr 23, 2022
45025fe
update
minmingzhu Apr 28, 2022
52d82ca
update
minmingzhu May 17, 2022
e01b283
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
27af86d
convert rdd to HomogenTable
minmingzhu Apr 18, 2022
97e511b
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
da2d606
update
minmingzhu Apr 23, 2022
42a0c3b
update ColumnAccessorImpl.cpp RowAccessorImpl.cpp
minmingzhu Apr 24, 2022
d135493
update
minmingzhu Apr 28, 2022
472bbd0
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
0c1e0c0
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
366d92c
update
minmingzhu Apr 23, 2022
fa80695
update
minmingzhu May 17, 2022
036c08c
1. convert homogenTable to array/vector/matrix
minmingzhu Apr 17, 2022
da503b7
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
8668c0e
update
minmingzhu Apr 23, 2022
ef48e29
1. convert rdd to HomogenTable
minmingzhu Apr 21, 2022
6af59e7
update
minmingzhu Apr 28, 2022
280ea13
update code style
minmingzhu May 18, 2022
56c5ba0
update HomogenTableTest.java
minmingzhu May 18, 2022
b698539
remove unused code
minmingzhu May 18, 2022
25a9498
fixed comments
minmingzhu May 25, 2022
bd1ce8f
update
minmingzhu May 25, 2022
cfa2e8b
Update com_intel_oap_mllib_OneDAL__.h
minmingzhu May 25, 2022
6cd5e86
Update OneDAL.cpp
minmingzhu May 25, 2022
c2c4834
Update HomogenTableImpl.cpp
minmingzhu May 25, 2022
e2ebc05
Update ConvertHomogenTableSuite.scala
minmingzhu May 25, 2022
d177b9c
used static_cast
minmingzhu May 26, 2022
26be02e
Update GPU.cpp
minmingzhu May 26, 2022
7fcdc46
fix comments
minmingzhu May 27, 2022
f7e47df
Merge remote-tracking branch 'origin/rdd_to_homogenTable' into rdd_to…
minmingzhu May 27, 2022
ea63e32
update
minmingzhu May 29, 2022
27262c4
update
minmingzhu May 29, 2022
0e2084c
update
minmingzhu May 31, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@
public class HomogenTable extends Table {
HomogenTableImpl impl;

public HomogenTable(Common.ComputeDevice device){
super();
impl = new HomogenTableImpl(device);
}

public HomogenTable(long cTable){
super();
impl = new HomogenTableImpl(cTable);
Expand Down Expand Up @@ -138,6 +143,10 @@ public double[] getDoubleData() {
return impl.getDoubleData();
}

public void addHomogenTable(long homogenTableAddr ) {
impl.addHomogenTable(homogenTableAddr);
}

public long getcObejct() {
return impl.getcObject();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,10 @@ public class HomogenTableImpl implements HomogenTableIface {
private TableMetadata metadata;
private Common.ComputeDevice device;

protected HomogenTableImpl() {
protected HomogenTableImpl(Common.ComputeDevice computeDevice) {
super();
this.device = computeDevice;
this.cObject = this.cEmptyTableInit();
}

public HomogenTableImpl(long cTable) {
Expand Down Expand Up @@ -138,6 +140,9 @@ public long getcObject(){
return this.cObject;
}

public void addHomogenTable(long homogenTableAddr ) {
this.cObject = cAddHomogenTable(this.cObject, homogenTableAddr, this.device.ordinal());
}
private native long iInit(long rowCount,
long colCount,
int[] data,
Expand Down Expand Up @@ -173,4 +178,8 @@ private native long lInit(long rowCount,
private native long[] cGetLongData(long cObject);
private native float[] cGetFloatData(long cObject);
private native double[] cGetDoubleData(long cObject);
private native long cAddHomogenTable(long cObject,
long homogenTableAddr,
int computeDeviceIndex);
private native long cEmptyTableInit();
}
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,9 @@
public class RowAccessor {
private long cObject;
private Common.ComputeDevice cDevice;

public RowAccessor(long cObject, Common.ComputeDevice device) {
this.cObject = cObject;
this.cDevice =device;
this.cDevice = device;
}

public double[] pullDouble(){
Expand Down Expand Up @@ -39,4 +38,5 @@ private native float[] cPullFloat(long cObject, long cRowStartIndex,
long cRowEndIndex, int computeDeviceIndex);
private native int[] cPullInt(long cObject, long cRowStartIndex,
long cRowEndIndex, int computeDeviceIndex);

}
4 changes: 3 additions & 1 deletion mllib-dal/src/main/native/OneDAL.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,10 @@

#include <cstring>
#include <iostream>
#include <set>

#include "com_intel_oap_mllib_OneDAL__.h"

#include "service.h"

using namespace daal;
Expand Down Expand Up @@ -150,4 +152,4 @@ Java_com_intel_oap_mllib_OneDAL_00024_cNewCSRNumericTableDouble(
CSRNumericTablePtr *ret = new CSRNumericTablePtr(numericTable);

return (jlong)ret;
}
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 8 additions & 6 deletions mllib-dal/src/main/native/oneapi/dal/ColumnAccessorImpl.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
#ifdef CPU_GPU_PROFILE
#include "GPU.h"
#endif

#ifndef ONEDAL_DATA_PARALLEL
#define ONEDAL_DATA_PARALLEL
#endif
Expand All @@ -38,6 +39,7 @@ using namespace std;
using namespace oneapi::dal;
typedef std::shared_ptr<homogen_table> homogenPtr;


/*
* Class: com_intel_oneapi_dal_table_ColumnAccessor
* Method: cPull
Expand Down Expand Up @@ -124,11 +126,11 @@ JNIEXPORT jfloatArray JNICALL Java_com_intel_oneapi_dal_table_ColumnAccessor_cPu
JNIEXPORT jintArray JNICALL Java_com_intel_oneapi_dal_table_ColumnAccessor_cPullInt
(JNIEnv *env, jobject, jlong cTableAddr, jlong cColumnIndex, jlong cRowStartIndex,
jlong cRowEndIndex, jint cComputeDevice) {
printf("ColumnAccessor PullInt \n");
homogen_table htable = *((homogen_table *)cTableAddr);
column_accessor<const int> acc { htable };
oneapi::dal::array<int> col_values;
jintArray newIntArray;
printf("ColumnAccessor PullInt \n");
homogen_table htable = *((homogen_table *)cTableAddr);
column_accessor<const int> acc { htable };
oneapi::dal::array<int> col_values;
jintArray newIntArray;
switch(getComputeDevice(cComputeDevice)) {
case compute_device::host:{
col_values = acc.pull(cColumnIndex, {cRowStartIndex, cRowEndIndex});
Expand All @@ -149,7 +151,7 @@ switch(getComputeDevice(cComputeDevice)) {
default: {
return newIntArray;
}
}
}
newIntArray = env->NewIntArray(col_values.get_count());
env->SetIntArrayRegion(newIntArray, 0, col_values.get_count(), col_values.get_data());
return newIntArray;
Expand Down
Loading