[SPARK-1405] [mllib] Latent Dirichlet Allocation (LDA) using EM#4047

Closed

jkbradley wants to merge 31 commits intoapache:masterfrom

jkbradley:davidhall-lda

Commits on Feb 2, 2015

Added 3 files from dlwh LDA implementation
jkbradley
committed
Prepped LDA main class for PR, but some cleanups remain
jkbradley
committed
cleanups before PR
jkbradley
committed
separated LDA models into own file. more cleanups before PR
jkbradley
committed
Unit tests and fixes for LDA, now ready for PR
jkbradley
committed
scala style fix
jkbradley
committed
fixed example name
jkbradley
committed
mapPart -> flatMap

mengxr
authored and
jkbradley
committed
use axpy

mengxr
authored and
jkbradley
committed
+ -> +=

mengxr
authored and
jkbradley
committed
optimize computePTopic

mengxr
authored and
jkbradley
committed
* -> *=

mengxr
authored and
jkbradley
committed
optimize aggregateMessages

mengxr
authored and
jkbradley
committed
reset spacing

mengxr
authored and
jkbradley
committed
minor

mengxr
authored and
jkbradley
committed
Moved auto term,topic smoothing computation to get*Smoothing methods. Changed word to term in some places. Updated LDAExample to use default smoothing amounts.
jkbradley
committed
renamed LDA LearningState.collectTopicTotals to globalTopicTotals
jkbradley
committed
small cleanup
jkbradley
committed
Added checkpointing to LDA
jkbradley
committed
* Removed Document type in favor of (Long, Vector)
jkbradley
committed
merged improvements from LDATiming into LDAExample. Will remove LDATiming after done testing
jkbradley
committed
Added Java-friendly run method to LDA.
jkbradley
committed
fixed scalastyle
jkbradley
committed
Added catch for BreakIterator exception. Improved preprocessing to reduce passes over data
jkbradley
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[SPARK-1405] [mllib] Latent Dirichlet Allocation (LDA) using EM#4047

[SPARK-1405] [mllib] Latent Dirichlet Allocation (LDA) using EM#4047
jkbradley wants to merge 31 commits intoapache:masterfrom
jkbradley:davidhall-lda

Commits on Feb 2, 2015

Added 3 files from dlwh LDA implementation

Prepped LDA main class for PR, but some cleanups remain

cleanups before PR

separated LDA models into own file. more cleanups before PR

Unit tests and fixes for LDA, now ready for PR

scala style fix

fixed example name

mapPart -> flatMap

use axpy

+ -> +=

optimize computePTopic

* -> *=

optimize aggregateMessages

reset spacing

minor

Moved auto term,topic smoothing computation to get*Smoothing methods. Changed word to term in some places. Updated LDAExample to use default smoothing amounts.

renamed LDA LearningState.collectTopicTotals to globalTopicTotals

small cleanup

Added checkpointing to LDA

* Removed Document type in favor of (Long, Vector)

merged improvements from LDATiming into LDAExample. Will remove LDATiming after done testing

Added Java-friendly run method to LDA.

fixed scalastyle

Added catch for BreakIterator exception. Improved preprocessing to reduce passes over data

Commits on Feb 3, 2015

Removed LDATiming. Added PeriodicGraphCheckpointerSuite.scala. Small LDA cleanups.

removed duplicate graphx dependency in mllib/pom.xml

Merge remote-tracking branch 'upstream/master' into davidhall-lda

cleaned up PeriodicGraphCheckpointerSuite.scala

Updates per code review. Main change was in LDAExample for faster vocab computation. Also updated PeriodicGraphCheckpointerSuite.scala to clean up checkpoint files at end

cleaned up doc based on code review

small doc fix