[SPARK-5992][ML] Locality Sensitive Hashing#15148

Closed

Yunni wants to merge 46 commits intoapache:masterfrom

Yunni:SPARK-5992-yunn-lsh

Commits on Sep 13, 2016

First Commit of LSH function implementation. Implement basic Estimator-Model class hierarchy to make RandomProjection works.
Yunni
committed
Implementation of Approximate Nearest Neighbors. Add distCol as another model parameters
Yunni
committed

Commits on Sep 19, 2016

Implement approxSimilarityJoin(). Remove modelDataset and distCol as discussed in the Design Doc.
Yunni
committed
Add test utility method to check LSH property. Tested on random projection.
Yunni
committed
Add testing utility for approximate nearest neighbor. Run the testing on random projection.
Yunni
committed
Add testing utility for approximate similarity join. Run the testing on random projection.
Yunni
committed
Code review comments. A new unit test of k nearest neighbor for large k
Yun Ni
committed
Code review comments. A new unit test of k nearest neighbor for large k
Yun Ni
committed
(1) Refactor hashDistCol for nearest neighbor search. (2) Add scaladoc for LSH along with reference papers
Yun Ni
committed

Commits on Sep 20, 2016

Commits on Sep 26, 2016

Commits on Oct 4, 2016

Commits on Oct 5, 2016

Code review comments
Yun Ni
committed

Commits on Oct 10, 2016

Commits on Oct 11, 2016

Code review comments
Yunni
committed
SignRandomProjection: LSH Classes for cosine distance metrics
Yun Ni
committed
Change hashFunctions to Arrays
Yun Ni
committed

Commits on Oct 12, 2016

BitSampling: LSH Class for Hamming Distance
Yun Ni
committed

Commits on Oct 13, 2016

Commits on Oct 17, 2016

For similarity join, expose leftCol and rightCol as parameters
Yunni
committed

Commits on Oct 22, 2016

Code Review comments: (1) Save BitSampling and SignRandomProjection for a follow-up PR (2) Use 'datasetA' and 'datasetB' as the default colNames in Similarity Join
Yunni
committed

Commits on Oct 23, 2016

(1) Reset all random seed != 0 (2) Add docstring about the output schema of Similarity Join (3) Change 'record' -> 'row' for clarity
Yunni
committed

Commits on Oct 27, 2016

Commits on Oct 28, 2016

Remove default values for outputCol
Yun Ni
committed
Remove default values for outputCol
Yun Ni
committed
Add more scaladoc
Yun Ni
committed