expose improvementTol inside FirstOrderMinimizer

[coderxiang]

When I run the LBFGS implementation of Spark on a moderate size proprietary data set (~ 5M samples), I see the resultant vector diverges from some standard solver (e.g., liblinear, sci-learn) on around half of the coefficients. The computation usually stops around 20 iterations although I set the iteration number to be 100 with a very strict tolerance variable (1e-15). The formulation is strongly convex so the solution should be unique. Configing the improvementTol variable inside FirstOrderMinimizer.scala can solve this issue.

Two ways of doing this:

1. Exposing the improvementTol variable to LBFGS, as is done in the PR
2. Change the default value of improvementTol in FirstOrderMinimizer.scala

I chose the first one here.

[dlwh]

thanks. Yeah, I need to figure out the convergence checks.

[dbtsai]

@dlwh, are you going to publish this so I can upgrade Spark 1.3 dependency before it's released? thanks.

[dlwh]

sure. what's the timeline?

On Mon, Mar 2, 2015 at 6:17 PM, DB Tsai notifications@github.com wrote:

@dlwh https://github.com/dlwh, are you going to publish this so I can
upgrade Spark 1.3 dependency before it's released? thanks.

—
Reply to this email directly or view it on GitHub
#373 (comment).

[dbtsai]

Is OWLQN affected by this issue? Spark 1.3 is RC2, so if we want to change, we need to be fast. Thanks.

[dlwh]

ok, Debasish has been working on his new optimization algorithm. I imagine
that that's post Spark 1.3 in any case?

On Mon, Mar 2, 2015 at 6:49 PM, DB Tsai notifications@github.com wrote:

Is OWLQN affected by this issue? Spark 1.3 is RC2, so if we want to
change, we need to be fast. Thanks.

—
Reply to this email directly or view it on GitHub
#373 (comment).

[dlwh]

owlqn's convergence conditions are also fixed by this.

On Mon, Mar 2, 2015 at 7:17 PM, David Hall david.lw.hall@gmail.com wrote:

ok, Debasish has been working on his new optimization algorithm. I imagine
that that's post Spark 1.3 in any case?

On Mon, Mar 2, 2015 at 6:49 PM, DB Tsai notifications@github.com wrote:

Is OWLQN affected by this issue? Spark 1.3 is RC2, so if we want to
change, we need to be fast. Thanks.

—
Reply to this email directly or view it on GitHub
#373 (comment).

[dbtsai]

Sounds great. Then let's publish the breeze with this fix, and I'll open another PR to Spark.

[dlwh]

0.11 is released. Will take a bit to propagate to maven central.