-
-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Type Constraints in the Rule Structs #205
Comments
Although I don't remember the specific beats of that discussion, this choice to standardize eltypes was discussed at length over multiple issues. #151 is a good jumping-off point with links. |
@mcabbott what do you think we should do here? Now we have
Should it become struct Rule{T1,T2}
eta::T1
beta::T2
Rule(eta, beta = (0.7, 0.8)) = eta < 0 ? error() : new(eta, beta)
Rule(; eta = 0.1, beta = (0.7, 0.8)) = Rule(eta, beta)
end ? (and possibly add a nice show method avoiding) |
That looks good to me, if @mcabbott is game you should go for it! Maybe it's worth making a draft PR and seeing what breaks? |
We dumped type parameters partly because it was annoying that The sketch above doesn't run, but looks like it wants to allow any different type for β... perhaps the minimal thing is just one type parameter |
So T <: Real will mess with us a bit, since we have
Unfortunately I don't know a good way in Julia to define |
I was working on getting Optimisers to work with reactant, and it (mostly) does, but one of the current issues is that
eta
is forced to be Float64 in some of the structs.But consider the following IR:
While this looks correct, if you take a closer look:
The learning rate and the other parameters get embedded into the IR as constants. So even if we do an
adjust
of the learning rate, it will still be using the old learning rateThe text was updated successfully, but these errors were encountered: