Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

infer_c in sgd_armijo: where does it come from? #6

Open
chanshing opened this issue Jun 10, 2020 · 0 comments
Open

infer_c in sgd_armijo: where does it come from? #6

chanshing opened this issue Jun 10, 2020 · 0 comments

Comments

@chanshing
Copy link

First, thanks for the nice work. I am excited to try out this optimizer.
I noticed that in get_optimizer you have the following line

sls/src/optimizers/__init__.py

Line 27 in 318e9b5

c = (1e-3) * np.sqrt(n_batches_per_epoch)

Where does this heuristic come from? Is it valid just for the experiments you're considering?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@chanshing